Multi-user sessions using a custom transformer-based span-cat pipeline

[ryanwesslen]

hi @coltonflowers1!

Thanks for your question and welcome to the Prodigy community

@coltonflowers1

One annotator was able to label 154 examples, but the other only received 32, with "No Tasks Available" afterward. The annotator with 154 examples still has more examples loading in their session.

After running with verbose logging we get the following logs after trying to access the user-session with only 32 examples:

Thanks for the logs -- that's very helpful.

Like this post, I suspect there's an issue with multi-processing.

Pattern only custom recipe failed with 10K annotation records

Hi! By annotation records you mean examples in the data you're loading in? 10k really isn't much data at all so I don't think it's directly related to that. The error here seems to happen when Prodigy is validating the first batch of the stream and then putting the generator back together. Apparently that error in itertools.tee is related to multiprocessing and raised because the iterator it creates isn't thread-safe...? I'm not sure why this happens in some cases for some generators and not ot…

We may need to take a deeper dive into this.

Were annotators 1 and 2 annotating at the same time? Given you were using --update, I suspect that you could have an issue if both annotators are trying to update the model at the same time. I would suggest in the interim removing --update and see if you find the same error. In the meantime, I'll let you know if I find anything else on our end.

Also - one small recommendation -- given you have a lot of labels, you can list them to a labels.txt where each label is a new line and then pass labels.txt to --labels in your prodigy command to make your command a bit easier (and less prone to forgetting one of your labels).