texcat.correct "TypeError: Cannot read properties of undefined (reading 'reject')"

1

Hi,

I'm trying to use textcat.correct in the following way

prodigy textcat.correct <MY_DATASET> ./model/model-best data/raw.jsonl --label <MY_LABEL>

I get

"TypeError: Cannot read properties of undefined (reading 'reject')"

when I launch the web app.

The dataset has been labelled with single label. It was done with an earlier version of Prodigy.

A typical example looks like

{"text":"SOME TEXT","meta":{METADATA},"label":"<MY_LABEL>","_input_hash":-1524754645,"_task_hash":735813869,"_session_id":null,"_view_id":"classification","answer":"accept"}

Perhaps the label format has changed with the upgrade? I know for example I have to use textcat_multilabel when training now because for binary it expects LABEL,NOT_LABEL

2

Hi! This is strange and I wonder if the UI somehow gets tripped up by the pre-defined annotations in the data :thinking: I need to try and see if I can reproduce this!

If you're using textcat.correct, you'll only see the model's predictions anyway and the pre-set annotations should be ignored/discarded. So as a workaround, could you try and remove the "label" (or everything except for the "text" and meta, really) from your input data?

3

So, the JSONL file that I'm inputting doesn't have any "label" fields. The example I showed above is actually from the the dataset that already in the db. But your response made me try something else: if I use textcat.correct to create a NEW dataset it works fine, but I can't add it my existing dataset.

I actually had previously created a custom recipe for this purpose which filtered the stream based on high/low model scores to help me find positives/negatives in imbalanced data. In that case I just made it work like textcat.teach and appended to the dataset. I though using the exclude parameter here would allow me to do the same.

Anyway, if I want to use the built in version, I can create a new dataset using textcat.correct and exclude the existing dataset, and then merge them afterwards.

4

I am getting this error when I reach the last document in my dataset. The only change I made in my custom recipe was an edit to create a progress bar by making stream a list.

    "stream": list(stream),

TypeError: Cannot read properties of undefined (reading 'reject')

My input data looks like this (chat id opens a link to a full conversation):

{"chat_id":"1234","deployment":"name-of-institution","prompt":null,"text":null}
{"chat_id":"5678","deployment":"name-of-other-institution","prompt":null,"text":null}

I tried with another task with different input, and the result is the same. Here are the logs:

422 Unprocessable Entity

500 Internal Server Error

Task exception was never retrieved

Cheyanne

5

Could you share more of the custom recipe with an example so we may reproduce locally? Part of me is wondering how the example/task is hashed because I can imagine that it's having trouble with text: null there. .

6

I've been using this recipe and I haven't seen this before. I also tried another custom recipe, and I got the same result. Here is one of the custom recipes. Test data is in my comment above. There is a link and chat id, which I have masked in the recipe and the sample data. The only changes I made were changing instant submit to true to prodigy.json, upgrading to the latest version v1.11.8 (I was one version behind locally v.1.11.7), and making the stream into a list in order to add the progress bar to the UI: it was previously an infinity sign.

import json
from typing import Dict, Generator

import prodigy
from prodigy.components.loaders import JSONL
from prodigy.core import Controller
#from typing import List, Optional
#from prodigy.util import split_string

SENTIMENT_OPTIONS = ["positive", "negative", "neutral"]

MASKED = [
    "Mask1",
    "Mask2",
    "Mask3"
]

UNLIMITED_ROWS = [
    {"view_id": "html", "html_template": "<div style=\"padding: 0 10px; border: 1px solid #ddd; border-radius: 4px; text-align: left;\">" +
        "<label style=\"font-size: 12px; font-weight: bold; opacity: 0.75; margin-bottom: 10px;\">deployment</label>" +
        "<div style=\"max-height: 300px; overflow-y: scroll; margin-bottom: 10px;\">{{deployment}}</div>" +
      "</div>"
    },
    {"view_id": "html", "html_template": "<div style=\"padding: 0 10px; border: 1px solid #ddd; border-radius: 4px; text-align: left;\">" +
        "<label style=\"font-size: 12px; font-weight: bold; opacity: 0.75; margin-bottom: 10px;\">link</label>" +
        "<div style=\"max-height: 300px; overflow-y: scroll; margin-bottom: 10px;\">{{prompt}}</div>" +
        "<a href=\"MASKEDLINK={{chat_id}}\" target=\"_blank\" style=\"margin-bottom: 10px;\">View Full Chat</a>" +
      "</div>"
    },
    {"view_id": "choice"},
    {"view_id": "html", "html_template": "<div style='float:left;'>" +
        "<input type='checkbox' id='stt' name='stt' value='STT Error' style='margin-right:10px;' data-id='{{chat_id}}' onchange='updateSttError()'" +
        "<label for='stt'>STT Error</label>" +
      "</div>"
    },
    {"view_id": "text_input", "field_label": "user input field"}
]

def add_options(stream, label_field="label", choices=MASKED) -> Generator[Dict, None, None]:
    """
    Convert each line in the ``stream`` to a ``task`` with a text and an
    options field

    :param stream: the input stream
    :param label_field: key; defaults to "label"
    :yield: a task Dict with text and options
    """
    for line in stream:
        if label_field in line:
            options = json.loads(line[label_field])
        else:
            options = []
        for word in (options + choices):
            if word not in options:
                options.append(word)
        task = {
            "deployment": line["deployment"],
            "chat_id": line["chat_id"],
            "stt_error": False,
            "options": [
                {"id": o, "deployment": o, "prompt": o, "text": o} for o in options
            ]
        }

        yield task

@prodigy.recipe(
    "masked-name",
    dataset=("The dataset to save to", "positional", None, str),
    file_path=("Path to texts", "positional", None, str),
    label_field=("Label to use for the accept task", "option", "f", str),
    choice_field=("Choices to add to the input", "option", "c", str)
)
def custom_labels(dataset, file_path, label_field, choice_field):
    """
    Annotate the text with labels from the list from the ``label_field`` in
    the input file. Augmented with choices from ``choice_field``.
    """

    blocks = UNLIMITED_ROWS

    stream = JSONL(file_path)
    stream = add_options(stream)  # add options to each task

    javascript = """
      // Set stt_error to false by default
      prodigy.update({ stt_error: false });

      function updateSttError() {
        prodigy.update({ stt_error: document.getElementById('stt').checked });
      }

      document.addEventListener('prodigyanswer', (event) => {
        // Reset stt_error to false
        prodigy.update({ stt_error: false });
      });
    """

    return {
        "dataset": dataset,
        "view_id": "blocks",
        "stream": list(stream),
        "config": {
          "blocks": blocks,
          "choice_style": "multiple",
          "javascript": javascript
        }
    }
7

Interesting. I can confirm that I see this pop-up at the end.

CleanShot 2022-09-28 at 14.51.38
CleanShot 2022-09-28 at 14.51.381289×653 44.3 KB

Will investigate and will respond when I understand what is happening.

8

Found it. It's the javascript that you've attached along. I think calling prodigy.update on an empty example is causing the warning.

Just to check, is this issue blocking you from annotating data? It seems like I'm still able to hit "save" and get my annotations in the database. While I agree it's annoying, it also seems like it's harmless because it only appears once you've exhausted the examples. Or am I misunderstanding?

9

@koaning Thank you. I just confirmed with annotators that they have seen this at the end of a task, so confirmed that it has been happening for a bit and not impacting the data, but just sort of strange. I saw this happen with a dataset where the text is populated, as well. It seems to only occur at the end of the task instead of showing no tasks available, so agreed I don't think it's impacting saved annotations. I have instant submit set to true, and the annotations appear to be saved regardless of this, but just curious if there was anything in the code causing this TypeError.

10

@koaning Just another note that removing ,"prompt":null,"text":null from the sample data still resulted in the same TypeError.

{"chat_id":"5678","deployment":"name-of-other-institution","prompt":null,"text":null}
11

@koaning I've tried to remove this TypeError with try/except and suppress, but neither worked. Any suggestions on how to remove this error from popping up?

12

I think the solution is to fix it on the JS side.

document.addEventListener('prodigyanswer', (event) => {
  // In this section, you'll probably want to check if there's a task
  // before updating it
  prodigy.update({ stt_error: false });
});

When I check the docs, it seems like you might be able to use prodigy.content in there to check if it exists. Something like this?

document.addEventListener('prodigyanswer', (event) => {
  if(prodigy.content){
    prodigy.update({ stt_error: false });
  }
});
13

Thanks for the response @koaning . I tried this and it I still got the error on the last task.

14

That's curious. Could you show me the logs of this?

document.addEventListener('prodigyanswer', (event) => {
  if(prodigy.content){
    console.log(prodigy.content);
    prodigy.update({ stt_error: false });
  }
});

I'm curious to see the contents if it's empty.

15 
INFO:     Started server process [44024]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8080 (Press CTRL+C to quit)
INFO:     ::1:60927 - "GET /?session=cheyanne HTTP/1.1" 200 OK
INFO:     ::1:60927 - "GET /bundle.js HTTP/1.1" 200 OK
INFO:     ::1:60927 - "GET /project/cheyanne HTTP/1.1" 200 OK
10:13:36: POST: /get_session_questions
INFO:     ::1:60928 - "GET /favicon.ico HTTP/1.1" 200 OK
10:13:36: CONTROLLER: Getting batch of questions for session: hbm_sept_test_oct2022_v13-cheyanne
10:13:36: FEED: Finding next batch of questions in stream
10:13:36: FEED: re-adding open tasks to stream
10:13:36: FEED: Stream is empty
10:13:36: FEED: batch of questions requested for session hbm_sept_test_oct2022_v13-cheyanne: 6
10:13:36: RESPONSE: /get_session_questions (6 examples)
{'tasks': [{'deployment': 'mask', 'chat_id': '1234', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': 581470756, '_task_hash': 1411263747, '_view_id': 'blocks'}, {'deployment': 'dcu-web', 'chat_id': 'b310c9c0-e54b-4155-a094-14888197e745', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': -532194325, '_task_hash': 619672658, '_view_id': 'blocks'}, {'deployment': 'wcu-full-ivr', 'chat_id': 'b3115898-97d5-4ff8-9463-428e32ab9092', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': 1175343265, '_task_hash': 202396348, '_view_id': 'blocks'}, {'deployment': 'hudsonvalley-web', 'chat_id': 'b3169f61-2473-47f1-bbfd-05535a9c7e85', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': 1697291989, '_task_hash': 1887559219, '_view_id': 'blocks'}, {'deployment': 'Florida-IVR', 'chat_id': 'b336ef24-f6d0-4849-9cef-c1dccca5b089', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': -152270768, '_task_hash': -1758458144, '_view_id': 'blocks'}, {'deployment': 'langley-telephone-banking', 'chat_id': 'b3377ae3-2f0c-41d6-913b-f6e06c1aeb94', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': 1589927944, '_task_hash': -1174590159, '_view_id': 'blocks'}], 'total': 0, 'progress': 0.0, 'session_id': 'hbm_sept_test_oct2022_v13-cheyanne'}

INFO:     ::1:60927 - "POST /get_session_questions HTTP/1.1" 200 OK
10:13:39: POST: /give_answers (received 1, session ID 'hbm_sept_test_oct2022_v13-cheyanne')
[{'deployment': 'mask', 'chat_id': '1234', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': 581470756, '_task_hash': 1411263747, '_view_id': 'blocks', 'config': {'choice_style': 'single'}, 'accept': ['Helpful'], 'answer': 'accept'}]

10:13:40: CONTROLLER: Receiving 1 answers
10:13:41: DB: Creating dataset 'hbm_sept_test_oct2022_v13-cheyanne'
{'created': datetime.datetime(2022, 9, 30, 10, 13, 26)}

10:13:41: FEED: received answers for session hbm_sept_test_oct2022_v13-cheyanne: 1
10:13:41: DB: Getting dataset 'hbm_sept_test_oct2022_v13'
10:13:41: DB: Getting dataset 'hbm_sept_test_oct2022_v13-cheyanne'
10:13:41: DB: Added 1 examples to 2 datasets
10:13:41: CONTROLLER: Added 1 answers to dataset 'hbm_sept_test_oct2022_v13' in database PostgreSQL
10:13:41: RESPONSE: /give_answers
{'progress': 0.17}

INFO:     ::1:60927 - "POST /give_answers HTTP/1.1" 200 OK
10:14:17: POST: /give_answers (received 1, session ID 'hbm_sept_test_oct2022_v13-cheyanne')
[{'deployment': 'mask', 'chat_id': '1234', 'stt_error': False, 'options': [{'id': 'Helpful', 'deployment': 'Helpful', 'prompt': 'Helpful', 'text': 'Helpful'}, {'id': 'Not-Helpful', 'deployment': 'Not-Helpful', 'prompt': 'Not-Helpful', 'text': 'Not-Helpful'}, {'id': 'Neutral', 'deployment': 'Neutral', 'prompt': 'Neutral', 'text': 'Neutral'}], '_input_hash': -532194325, '_task_hash': 619672658, '_view_id': 'blocks', 'accept': ['Not-Helpful'], 'config': {'choice_style': 'single'}, 'answer': 'accept'}]

10:14:17: CONTROLLER: Receiving 1 answers
10:14:18: DB: Getting dataset 'hbm_sept_test_oct2022_v13-cheyanne'
10:14:18: FEED: received answers for session hbm_sept_test_oct2022_v13-cheyanne: 1
10:14:18: DB: Getting dataset 'hbm_sept_test_oct2022_v13'
10:14:18: DB: Getting dataset 'hbm_sept_test_oct2022_v13-cheyanne'
10:14:19: DB: Added 1 examples to 2 datasets
10:14:19: CONTROLLER: Added 1 answers to dataset 'hbm_sept_test_oct2022_v13' in database PostgreSQL
10:14:19: RESPONSE: /give_answers
{'progress': 0.33}

INFO:     ::1:60935 - "POST /give_answers HTTP/1.1" 200 OK
16

@cheyanneb the logs I am asking for won't appear in the server end but in the front end. When you open the browser devtools (shortcut on Chrome on Mac: CMD + alt + J, shortcut on Chrome on Windows/Linux: Ctrl + alt + J) you should see it appear in the Console tab. This tab should look something like:

CleanShot 2022-10-01 at 11.52.27
CleanShot 2022-10-01 at 11.52.271525×421 63.7 KB

17

@koaning apologies! Here is that output:

{deployment: 'masked', chat_id: '1234', stt_error: false, options: Array(3), _input_hash: 1697291989, …}
VM41:12 {deployment: 'masked', chat_id: '1234', stt_error: false, options: Array(3), _input_hash: -152270768, …}
VM41:12 {deployment: 'masked', chat_id: '1234', stt_error: false, options: Array(3), _input_hash: 1589927944, …}
VM41:12 {deployment: 'masked', chat_id: '1234', stt_error: false, options: Array(3), _input_hash: 1589927944, …}
bundle.js:1 TypeError: Cannot read properties of undefined (reading 'reject')
    at t.findInterface (bundle.js:1:23165)
    at r.getColumns (bundle.js:1:159823)
    at t.value (bundle.js:1:160351)
    at _i (bundle.js:1:367439)
    at wi (bundle.js:1:367234)
    at bundle.js:1:393463
    at Ra (bundle.js:1:396116)
    at Na (bundle.js:1:396498)
    at ys (bundle.js:1:403385)
    at gs (bundle.js:1:402765)
$i @ bundle.js:1
i.componentDidCatch.n.callback @ bundle.js:1
Ki @ bundle.js:1
Vi @ bundle.js:1
Oa @ bundle.js:1
ja @ bundle.js:1
(anonymous) @ bundle.js:1
t.unstable_runWithPriority @ bundle.js:1
ms @ bundle.js:1
ys @ bundle.js:1
gs @ bundle.js:1
cs @ bundle.js:1
za @ bundle.js:1
enqueueSetState @ bundle.js:1
_.setState @ bundle.js:1
(anonymous) @ bundle.js:1
v @ bundle.js:1
(anonymous) @ bundle.js:1
updateQuestion @ bundle.js:1
(anonymous) @ VM41:13
(anonymous) @ bundle.js:1
setTimeout (async)
n.answerQuestion @ bundle.js:1
(anonymous) @ bundle.js:1
(anonymous) @ bundle.js:1
d @ bundle.js:1
(anonymous) @ bundle.js:1
k @ bundle.js:1
T @ bundle.js:1
E @ bundle.js:1
j @ bundle.js:1
Sn @ bundle.js:1
ws @ bundle.js:1
Le @ bundle.js:1
Tn @ bundle.js:1
(anonymous) @ bundle.js:1
t.unstable_runWithPriority @ bundle.js:1
xs @ bundle.js:1
Cn @ bundle.js:1
18

Interesting. So we can confirm that on the javascript side it is having trouble reading properties of something undefined.

I guess this is the final thing I might try?

document.addEventListener('prodigyanswer', (event) => {
  if(prodigy.content === undefined){
    console.log("This is the end of the queue");
  }else{
    prodigy.update({ stt_error: false });
  }
});

That said, it's probably not worth diving into too much at the moment. I'll be logging it as a bug internally so folks will have a look later.

19

I tried this solution, but I still get the same error:

bundle.js:1 TypeError: Cannot read properties of undefined (reading 'reject')
    at t.findInterface (bundle.js:1:23165)
    at r.getColumns (bundle.js:1:159823)
    at t.value (bundle.js:1:160351)
    at _i (bundle.js:1:367439)
    at wi (bundle.js:1:367234)
    at bundle.js:1:393463
    at Ra (bundle.js:1:396116)
    at Na (bundle.js:1:396498)
    at ys (bundle.js:1:403385)
    at gs (bundle.js:1:402765)
$i @ bundle.js:1
i.componentDidCatch.n.callback @ bundle.js:1
Ki @ bundle.js:1
Vi @ bundle.js:1
Oa @ bundle.js:1
ja @ bundle.js:1
(anonymous) @ bundle.js:1
t.unstable_runWithPriority @ bundle.js:1
ms @ bundle.js:1
ys @ bundle.js:1
gs @ bundle.js:1
cs @ bundle.js:1
za @ bundle.js:1
enqueueSetState @ bundle.js:1
_.setState @ bundle.js:1
(anonymous) @ bundle.js:1
v @ bundle.js:1
(anonymous) @ bundle.js:1
updateQuestion @ bundle.js:1
(anonymous) @ VM68:14
(anonymous) @ bundle.js:1
setTimeout (async)
n.answerQuestion @ bundle.js:1
(anonymous) @ bundle.js:1
(anonymous) @ bundle.js:1
d @ bundle.js:1
(anonymous) @ bundle.js:1
k @ bundle.js:1
T @ bundle.js:1
E @ bundle.js:1
j @ bundle.js:1
Sn @ bundle.js:1
ws @ bundle.js:1
Le @ bundle.js:1
Tn @ bundle.js:1
(anonymous) @ bundle.js:1
t.unstable_runWithPriority @ bundle.js:1
xs @ bundle.js:1
Cn @ bundle.js:1
20

Then I fear I don't know what might help in the short term. The issue has been logged on our end as a bug to be fixed in a future release.

1 Like