prodigy ner.manual testset /data/test.jsonl --label /data/my_labels.txt
test.jsonl has 700+ lines, i get ‘no tasks available’ msg when the ‘total’ is 604
i do not know how to deal with it
Hi! There are several possible explanations why this would be happening:
Does your data contain any duplicates? Prodigy will filter out duplicate texts from the stream automatically, so you’ll only ever see a task once. When you restart the server with the same dataset, do you see new examples, or does it show “no task available” immediately? If so, this definitely indicates that all examples in the input text have been annotated once.
Otherwise, did you or the annotator reload the app at any point? If so, it’s possible that you end up with unlabelled batches, since Prodigy has no way of knowing that it’s “not coming back”. You can find more details and an implementation example of an “infinite stream” here: "No tasks available" on page refresh
thanks for quick response, my problem is same as the link mentioned in your reply, is there any official guide or doc to solve this problem(step by step).it is hard for me to do it myself…
thanks
Here's a full example of a stream that loops until all examples are in the dataset and then stops:
In this example, it also adds options and images, so if you just want to load the data, you can leave that out. You can implement it by editing the built-in ner.manual recipe, wrapping the existing recipe (see your PRODIGY_README.html for more details) or by writing a custom loader.