I’m trying to allow a few different annotators on the textcat.teach recipe. I know I can distinguish between the sessions with the session parameter.
But it looks like for each session, everyone is getting the same set of documents to annotate. Is there any way to split it up such that each annotator is getting a different subset of documents than other annotators?
This is primarily so we can crank thru a large amount of annotations.
Hi! I think what you’re looking for is the "feed_overlap" setting. You can overwrite that in your recipe config or prodigy.json.
If set to true (default), there will be a full overlap of the feeds sent out to different annotators – so everyone will get to see the same set of examples. This seems to be the behaviour you’re seeing now. If set to false, there won’t be an overlap between the questions sent out to different sessions, so everyone will get to see different examples.
(In the future, we also want to add support for customising the level of overlap – for example, you could specify that there should be a 20% overlap, or that each example should be annotated exactly twice, no matter by whom.)
Oh, super cool. I will try this out, thank you so much!
Hello @ines any news regarding the option to specify the percentage of overlap or specifying the number each example should be annotated ? Thanks