We were trying to create MongoDB wrapper for Prodigy and we've come a bit far with it.
However, when wrapping a custom recipe
terms.teach as far as I see in Prodigy source code, some of the data is being saved to SQL even before Save button functionality is being used. The end result is that some of the data ends up in an SQLite file and some of it ends up in MongoDB which is far from ideal.
Given that I am not expert, there are two different JSON objects stored under the examples table, the ones that get saved in SQLite only are the initial seed CSV values when prodigy command is ran:
And the other ones seem to be trained data that is saved upon saving:
Firstly question: What is the reason for those two seemingly different rows in the same table?
Second question: Couldn't db be passed to
terms.teach so it worked like in for instance
ner.teach so everything could be saved in a single place?