Am back having some issues that I couldn't sort out using Prodigy. I am annotating a very large set of text for NER with 50 long and complex labels. I run this command:
dotenv run -- python3 -m prodigy ner.llm.correct annotated-xxx config.cfg examples.jsonl
It loaded the labels and prompted me to use the prodigy UI. On the UI I could see the labels, "Show prompt sent to the LLM" and Show responses from the LLM" and the Accept, Reject buttons. I don't see the text to be annotated - so that I can manually manipulate the annotation when I found a mistake. Looks like I'm missing something here. With very small dataset it works fine as expected, but with the large dataset like mine it couldn't show the text annotated on the UI. I appreciate any help on this.
Hi @Fantahun ,
We haven't seen that issue before. Could you share how many examples does your dataset have so that I can try to reproduce the issue.
I had the same problem for any model, but it works perfectly fine with OpenAI's model.
Thanks for your reply @magdaaniol. I'm using 262 texts (lines of text) in my examples.jsonl, fifty labels - most of which are multi-word and my ner_example.yaml is almost empty. I'm planning to use a couple tens of thousands of text in examples.jsonl. BTW I'm using OpenAI's GPT-4 LLM in the background. If possible/and required I can share screenshots as well.
Thank you again.
Thanks for the extra info. Are you sure the only factor that changes how things are rendered is the length of the input file? That in principle shouldn't be the case as the input is processed in batches.
Could you try the same input and the number of labels with the regular
ner.manual and see if you experience the same issue? (Just to exclude the potential issues with the input being corrupted/empty due to the LLM API)
I'm suspecting the labels might be covering up most of the UI, but you still should be able scroll to see all the elements. It might be helpful if you share the screenshot of what you're seeing - thanks!
Also, which Prodigy version are you on? in v1.13.2 we introduced a dedicated front end component for handling LLMs so that would help me to figure out how is your UI being rendered.