Initially we build a Named entity Recognition Model in our Local Machine. Now we are looking to use Databricks and productionize it. Is it possible to use Prodigy in Databricks?
Hi Arda.
I'm somewhat unfamiliar with Databricks deployments and what the requirements are on their end. Prodigy could be launched in a Docker container or on a VM and if Databricks allows for that, then you should be able to get it working.
You might want to think about how the data is stored though. In the default settings Prodigy will write to a local SQLite database. If you prefer the data go elsewhere you'd either need to write a cronjob that can move the annotations manually or you can also use Postgres or MySQL as a backend for Prodigy.