Custom templates with custom DB and exclude logic

[ines (Ines Montani)]

Thanks for sharing – the HTML template looks really nice! It also looks like you've been doing everything correctly so far.

@ohenrik

However it seems like it does not store the answers.

Does it not store anything in the database, or just not the "answer" key? Can you export your annotations using the db-out command, and could you post an example of an annotation task in your stream? Also, is this specific to PostgreSQL, i.e. does it work as expected using the default SQLite storage?

Another thing that's always helpful for debugging is to run your commands with the environment variable PRODIGY_LOGGING=basic. This will log everything that's going on and makes it easier to spot problems – or track down an error.

@ohenrik

It also does not keep track of what objects have already been compared, so it starts from the beginning every time.

By default, Prodigy makes as little assumptions about your stream as possible – but you can tell it to exclude examples that are already present in one or more datasets. Simply add the IDs to the 'exclude' setting returned by your recipe. In your case, this would just be the current set:

return {
    'dataset': dataset,
    'exclude': [dataset],  # exclude all annotations already in the dataset
    # etc.
}

@ohenrik

To be clear, I just want to create a annotated dataset. Not use any of the models provided by Prodigy. I find the documentation a bit lacking on custom recipes.

Yes, the way you've solved this in your custom recipe is absolutely correct In terms of documentation, did you already see the custom recipes workflow and the custom recipes API docs in the PRODIGY_README.html? And if so, is there anything in particular that you feel like currently missing? We're always keen to add more examples of specific use cases!