TypeError: 'str' object does not support item assignment - ner.manual

[bev.manz (bev)]

Hello, any ideas on why this is failing, I've prepared data in jsonl and am trying to run this:

prodigy ner.manual ${DATA_SET_NAME} ${MODEL} ./${DATA_FILE} --label ${LABELS} --patterns ${PATTERNS_FILE}

Using . ./prodigy_env.sh to store parameters such as:

MODEL=blank:en
LABELS=label1,label2,label3
DATA_SET_NAME=model_name

Let me know if you need any more info and appreciate any advice thank you!

Traceback (most recent call last):
  File "/opt/conda/envs/prodigy/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/envs/prodigy/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/conda/envs/prodigy/lib/python3.9/site-packages/prodigy/__main__.py", line 61, in <module>
    controller = recipe(*args, use_plac=True)
  File "cython_src/prodigy/core.pyx", line 374, in prodigy.core.recipe.recipe_decorator.recipe_proxy
  File "cython_src/prodigy/core.pyx", line 63, in prodigy.core.Controller.from_components
  File "cython_src/prodigy/core.pyx", line 160, in prodigy.core.Controller.__init__
  File "cython_src/prodigy/components/feeds.pyx", line 104, in prodigy.components.feeds.Feed.__init__
  File "cython_src/prodigy/components/feeds.pyx", line 150, in prodigy.components.feeds.Feed._init_stream
  File "cython_src/prodigy/components/stream.pyx", line 107, in prodigy.components.stream.Stream.__init__
  File "cython_src/prodigy/components/stream.pyx", line 58, in prodigy.components.stream.validate_stream
  File "cython_src/prodigy/components/preprocess.pyx", line 164, in add_tokens
  File "/opt/conda/envs/prodigy/lib/python3.9/site-packages/spacy/language.py", line 1539, in pipe
    for doc in docs:
  File "/opt/conda/envs/prodigy/lib/python3.9/site-packages/spacy/language.py", line 1583, in pipe
    for doc in docs:
  File "/opt/conda/envs/prodigy/lib/python3.9/site-packages/spacy/language.py", line 1580, in <genexpr>
    docs = (self._ensure_doc(text) for text in texts)
  File "/opt/conda/envs/prodigy/lib/python3.9/site-packages/spacy/language.py", line 1529, in <genexpr>
    docs_with_contexts = (
  File "cython_src/prodigy/components/preprocess.pyx", line 157, in genexpr
  File "/opt/conda/envs/prodigy/lib/python3.9/site-packages/prodigy/recipes/ner.py", line 166, in <genexpr>
    stream = (eg for _, eg in pattern_matcher(stream))
  File "cython_src/prodigy/models/matcher.pyx", line 235, in __call__
TypeError: 'str' object does not support item assignment

[ryanwesslen]

hi @bev.manz!

@bev.manz

  File "cython_src/prodigy/models/matcher.pyx", line 235, in __call__
TypeError: 'str' object does not support item assignment

This looks similar to this issue, where it was a problem in the data, i.e., the meta tag.

TypeError: 'str' object does not support item assignment

Hi! I just tried it and it looks like the problem here is that your data includes a simple string value as its "meta", which Prodigy normally uses as a dict: "meta":"NG" The matcher tries to add its score to it thinking it's a dict, so that's what causes the error you see. We should add a workaround for this to prevent the error and port the meta to a dict automatically, but in the meantime, you can fix this by making the meta a dictionary with keys, e.g. like this: "meta": {"id": "NG"}

Can you check?

This is very similar too:

What is the syntax for textcat.teach for multi-class?

Hello @jdewsnip, thank you for your question and welcome to the prodigy forum! Could you post an example of your .jsonl-file? I suspect that you might use the key "meta" with a string value in your data, for which Prodigy uses a dict as value. During the prediction, it tries to add a score to the meta-dictionary which could cause the error you see.

If it's not obvious, I would start to remove part of your data. For example, start with one record that you're confident is correct. Then add in 50% other data, then the remainder.

Initially I had suspect it was a problem with your pattern file too, since it seemed to hit the matcher.pyx. So if you can't find the data issue, another thing to try is run it without the pattern file.

Unfortunately, these errors are really tough for us to diagnose without a reproducible example. I understand your data may be too sensitive to share, but if you're still having issues, if you could even create one or two dummy examples that makes it a lot easier for us to reproduce.

Hope this helps!

[bev.manz (bev)]

Hello,

Thanks for this, yes looks like I had packed my meta into a string somehow, but have made sure it's a dict!
All sorted now.