Broken Web App with custom ner.make-gold

, ,
1

Hi,

I wrote a custom recipe that uses the ner.make-gold recipe, except for the on_load method which I rewrite to filter out previously seen examples. I wrote it based off this forum post.

Here is my custom recipe:

import prodigy
from prodigy.recipes.ner import make_gold
from prodigy.util import INPUT_HASH_ATTR # the input hash attr constant

@prodigy.recipe("make-gold-filtered",
                dataset=prodigy.recipe_args["dataset"],
                model=prodigy.recipe_args["spacy_model"],
                source=prodigy.recipe_args["source"],
                label=prodigy.recipe_args["label"],
                unsegmented=prodigy.recipe_args["unsegmented"]
                )

def make_gold_filtered_recipe(dataset, model, source, label, unsegmented):

    components = make_gold(dataset, model, source=source, label=label, unsegmented=unsegmented)

    def on_load(controller):
        """Filter out previously seen examples."""
        input_hashes = controller.db.get_input_hashes(dataset) # Since building gold dataset, each text should be seen once so filter by the input hash rather than task hash

        components["stream"] = (eg for eg in components["stream"] if eg[INPUT_HASH_ATTR] not in input_hashes)

    components["on_load"] = on_load
    components["view"] = "ner"
    return components

When I tried using it from the command line, it ran without any errors, but when I opened the web server I got an error message saying that the web app was broken. I could even click accept or reject and the examples would show up on the left side of the web app, but there was just an error message where each task should be displayed:

image Screen Shot 2019-08-15 at 11.36.27 AM

Do you have any ideas as to what went wrong? Is there a bug in my recipe or in the web app or something else?

2

Hi! Could you check your browser's developer tools console and check if there's an error message? And what does your stream look like? Does it all work as expected when you run the regular ner.make-gold recipe with the same settings?

3

Hi,
Thanks for the quick response! I opened the developer tools console in both Safari and Chrome and both had error messages related to the same function.

Chrome:

image
image.png978×233 30.4 KB
Screen Shot 2019-08-15 at 12.24.03 PM

Safari:

image
image.png1902×626 95.4 KB
Screen Shot 2019-08-15 at 12.25.01 PM

When I run the built-in recipe ner.make-gold with my data source it works fine. My data source is a .jsonl file on my local machine.

4

Thanks for checking! I just tested it and I found the problem: In your recipe, you're passing in a string as the label argument, and not a list of labels.

The easiest way to fix it is to change the recipe argument to:

label=prodigy.recipe_args["label_set"]

This is basically the same as:

label = ('Label(s) to annotate. Comma-separated list or path to text file with one label per line', 
         'option', 'l',
          prodigy.util.split_string)

So basically, when the --label setting comes in from the command line, e.g. like "LABEL1,LABEL2", the value will be split and turned into a list, like ["LABEL1", "LABEL2"].

(Prodigy should definitely handle this more gracefully, though and not just fail – I'll udpate the app and add a fallback for cases like this.)

5

Thank you for your help!

6

Just released v1.8.4, which should fix this under the hood!