Hello, I got this error when I run the train recipe for textcat. This is my command
python -m prodigy train chat_test1_model --textcat chat_test1_train,eval:chat_test1_eval --label-stats
but I could not figure out what caused this issue and how I could fix it. Thank you for your help!
Hi jiebei,
could you share the error that you saw?
of course. As you can you, the training get started but the error appeared quickly after. Thank you!
One thing: in the future, please refrain from using screenshots when sharing logs or code. The downside of a screenshot is that you cannot copy/paste/search through it and it usually makes it harder for us to figure out what going wrong.
That said. This is interesting, I've never seen this behavior before. Are you running multiple training procedures at the same time? Is there an annotation process running that's using the same model?
Could you share the output of prodigy stats? Also, do you have a subset of the data that you can share so that I might reproduce this issue locally?
As a quick fix, could you make sure that the model directory is empty before training a new model?
Thanks for your suggestions! I deleted the model folder and re-run the command. However, I got the same error.
"PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'chat_test1_model\model-best\vocab'
PS C:\Users\luoji\Dropbox\Working\Chat\nlp experiment>"
Do you have any suggestions for fixing this issue? Thank you!
Hi Jiebei,
could you share the output of prodigy stats? That'll help us understand the versions that you're using.
One thing that I wonder too: you seem to be saving your model to a dropbox folder. Could you save it directly to disk instead? Dropbox may also be the culprit here.
The problem is solved by moving my output directory to local. Thank you so much!
Happy to hear it!
Yeah, Dropbox can still be fine as a backup service that to write into after training. But during training Dropbox will try to do updates every time that you write to a folder because it wants to sync. This doesn't always play nice with other software when it comes to locks and permissions.
