Am getting the following error when running the review property on prodigy.
command:
prodigy review newset myset --label OMS,OOT,METHODS
error:
✘ Reviewing 'blocks' annotations isn't supported yet
You can vote for this feature on the forum: https://support.prodi.gy
Thanks for reaching out! The error is correct, the review recipe currently does not properly support the blocks view.
I would like to understand your use-case more though. Did you generate this dataset with a custom recipe or a built-in one? Also, do you have a screenshot of the interface you'd like to use?
I am using the python -m prodigy ner.llm.correct nannotated-text config.cfg yourdata.csv to generate the dataset first, as demonstrated in your spacy llms video.
Once I hace the annotated_text now am trying to review and see how it performed with the prodigy interface.
Ah! That's indeed using a blocks interface under the hood. But it's also something that should work out of the box because it is an internal recipe. This is something that I'll explore today!
I investigated the issue and I think I have a proposal for a patch ready. We still won't be able to render everything that a blocks UI might generate, but we may want to allow a user like yourself to be able to pass --view-id ner_manual to the review recipe to make it clear that you're only interested in re-using that UI.
It's going through review now and I'll keep you posted on any changes.
Again, thanks for reporting, and sorry for the headache! I'll try to get this patched ASAP.
Thank you, will be waiting on the developments.
Reporting back. I think you should now be able to upgrade to version 1.13.3 that comes with a patch for this issue. You will need to pass --view-id ner_manual to the review call, but that should be it!
Let me know if there's still issues with this!
Thank you. Will get back once I try it later today.
Worked okay. Many thanks Vincent!
Thanks for this - ran into the same issue after annotating with custom recipes for textcat, based on the new ones for models-as-annotators and llm.correct (sorry, forget exact names). FYI there's another related issue with this workflow, where a task might allow multiple classes, but then the review UI is unaware that multiple were allowed. The workaround of putting "choice_style": "multiple" into prodigy.json overcomes this issue, but it's likely to trip up lots of folks - and I don't believe there's currently a way to pass this config into a recipe without using the config file.
With regards to choice_style: that's correct. This information is stored in the config.json file and isn't stored on the annotated example in the database. So the review recipe needs this information in order to work.
Just to check @dansowter you're not stuck now? If you are I'll gladly see if I can help.
That's very kind of you. No, I'm not stuck, but I appreciate the offer.