ERROR: Can't fetch tasks. Make sure the server is running correctly.

[samoz]

Hello! Just installed Prodigy and I'm trying to get through the "Workflow first step" but I get stuck at "Named Entity Recognition". I run Windows and has installed everything in Anaconda. I have copy-pasted the prodigy.json specification from the docs to my home folder. When I try to run the command I get the below output, but when opening localhost I get "ERROR: Can't fetch tasks. Make sure the server is running correctly." What's wrong? Can be something really basic which I don't yet understand! Best, Sara

python -m prodigy ner.teach my_set en_core_web_sm news_headlines.jsonl
12:22:12 - APP: Using Hug endpoints (deprecated)
12:22:13 - RECIPE: Calling recipe 'ner.teach'
12:22:13 - RECIPE: Starting recipe ner.teach
12:22:13 - LOADER: Using file extension 'jsonl' to find loader
12:22:13 - LOADER: Loading stream from jsonl
12:22:13 - LOADER: Rehashing stream
12:22:13 - RECIPE: Creating EntityRecognizer using model en_core_web_sm
12:22:16 - MODEL: Added sentence boundary detector to model pipeline
12:22:16 - SORTER: Resort stream to prefer uncertain scores (bias 0.0)
12:22:16 - CONTROLLER: Initialising from recipe
12:22:16 - VALIDATE: Creating validator for view ID 'ner'
12:22:16 - DB: Initialising database SQLite
12:22:16 - DB: Connecting to database SQLite
12:22:16 - DB: Loading dataset 'my_set' (0 examples)
12:22:16 - DB: Creating dataset '2019-08-26_12-22-16'
12:22:16 - DatasetFilter: Getting hashes for excluded examples
12:22:16 - DatasetFilter: Excluding 0 tasks from datasets: my_set
12:22:16 - CONTROLLER: Initialising from recipe
12:22:16 - CORS: initialize wildcard "*" CORS origins

Starting the web server at http://localhost:8080 ...
Open the app in your browser and start annotating!

[ines (Ines Montani)]

Hi! That's strange – the logs look good to me. The error message in the app typically happens when the server has died for some reason and the app can't connect to it anymore.

Was there any error message below the "Starting the web server" output? And what happens when you try and start the server again? Does it produce the same problem?

[samoz]

No error message below the "Starting the web server" output. When I try and start the server again the same error is reproduced (I have also tried the classical windows trick of restarting the computer and to use another browser).

[ines (Ines Montani)]

Thanks for checking! And that's definitely strange And when the error occurs and you check back to your terminal, the server has stopped and the command line prompt is back? If so, I wonder if you're hitting a weird issue with the current REST API setup we're using.

Could you try running pip install fastapi and set the environment variable PRODIGY_FASTAPI=1? This will use FastAPI instead of Hug for the server.

[samoz]

I use Anaconda prompt, which (as far as I figured) is windows command promp but on Anaconda (sorry for the vagueness) . After it says starting web server, nothing happends: the window doesnt close and ther is no output whatsoever on it. Will do what you say with fastapi and report back soon (I honestly need to get my laundry )

In the meantime. I just changed the port to 9999 (sine I remeber some localhost port issue from before). Now The following is printed:

Starting the web server at http://localhost:9999 ...
Open the app in your browser and start annotating!

21:11:17 - GET: /project
21:11:17 - POST: /get_session_questions
21:11:17 - FEED: Finding next batch of questions in stream
21:11:17 - CONTROLLER: Validating the first batch for session: my_set-default
21:11:17 - PREPROCESS: Splitting sentences
21:11:17 - FILTER: Filtering duplicates from stream
21:11:17 - FILTER: Filtering out empty examples for key 'text'
Exception when serving /get_session_questions
Traceback (most recent call last):
File "C:\Users\sara\Anaconda3\lib\site-packages\waitress\channel.py", line 336, in service
task.service()
File "C:\Users\sara\Anaconda3\lib\site-packages\waitress\task.py", line 175, in service
self.execute()
File "C:\Users\sara\Anaconda3\lib\site-packages\waitress\task.py", line 452, in execute
app_iter = self.channel.server.application(env, start_response)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\api.py", line 451, in api_auto_instantiate
return module.hug_wsgi(*args, **kwargs)
File "C:\Users\sara\Anaconda3\lib\site-packages\falcon\api.py", line 244, in call
responder(req, resp, **params)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 789, in call
raise exception
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 762, in call
self.render_content(self.call_function(input_parameters), context, request, response, **kwargs)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 698, in call_function
return self.interface(**parameters)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 100, in call
return __hug_internal_self._function(*args, **kwargs)
File "C:\Users\sara\Anaconda3\lib\site-packages\prodigy_api\hug_app.py", line 228, in get_session_questions
tasks = controller.get_questions(session_id=session_id)
File "cython_src\prodigy\core.pyx", line 130, in prodigy.core.Controller.get_questions
File "cython_src\prodigy\components\feeds.pyx", line 58, in prodigy.components.feeds.SharedFeed.get_questions
File "cython_src\prodigy\components\feeds.pyx", line 63, in prodigy.components.feeds.SharedFeed.get_next_batch
File "cython_src\prodigy\components\feeds.pyx", line 140, in prodigy.components.feeds.SessionFeed.get_session_stream
File "C:\Users\sara\Anaconda3\lib\site-packages\toolz\itertoolz.py", line 376, in first
return next(iter(seq))
File "cython_src\prodigy\components\sorters.pyx", line 151, in iter
File "cython_src\prodigy\components\sorters.pyx", line 61, in genexpr
File "cython_src\prodigy\models\ner.pyx", line 292, in call
File "cython_src\prodigy\models\ner.pyx", line 261, in get_tasks
File "C:\Users\sara\Anaconda3\lib\site-packages\toolz\itertoolz.py", line 726, in partition_all
prev = next(it)
File "cython_src\prodigy\models\ner.pyx", line 211, in predict_spans
File "C:\Users\sara\Anaconda3\lib\site-packages\toolz\itertoolz.py", line 726, in partition_all
prev = next(it)
File "cython_src\prodigy\components\preprocess.pyx", line 39, in split_sentences
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 723, in pipe
for doc, context in izip(docs, contexts):
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 751, in pipe
for doc in docs:
File "nn_parser.pyx", line 221, in pipe
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\util.py", line 463, in minibatch
batch = list(itertools.islice(items, int(batch_size)))
File "nn_parser.pyx", line 221, in pipe
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\util.py", line 463, in minibatch
batch = list(itertools.islice(items, int(batch_size)))
File "pipes.pyx", line 397, in pipe
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\util.py", line 463, in minibatch
batch = list(itertools.islice(items, int(batch_size)))
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 954, in _pipe
for doc in docs:
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 726, in
docs = (self.make_doc(text) for text in texts)
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 715, in
texts = (tc[0] for tc in text_context1)
File "cython_src\prodigy\components\preprocess.pyx", line 38, in genexpr
File "cython_src\prodigy\components\filters.pyx", line 35, in filter_duplicates
File "cython_src\prodigy\components\filters.pyx", line 16, in filter_empty
File "cython_src\prodigy\components\loaders.pyx", line 22, in _rehash_stream
File "cython_src\prodigy\components\loaders.pyx", line 142, in JSONL
File "C:\Users\sara\Anaconda3\lib\pathlib.py", line 1183, in open
opener=self._opener)
File "C:\Users\sara\Anaconda3\lib\pathlib.py", line 1037, in _opener
return self._accessor.open(self, flags, mode)
File "C:\Users\sara\Anaconda3\lib\pathlib.py", line 387, in wrapped
return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: 'news_headlines.jsonl'

[samoz]

Oh and of course news_headlines.jsonl is in the home folder.

[ines (Ines Montani)]

Ah, so it seems like something swallowed the output? In any case, this definitely means that it can't find the file. Are you sure it's in the same directory you're executing prodigy from?

[samoz]

Nope I did not. When I do (I assume I was suppose to from the start) I get a new error message:

21:29:02 - GET: /project
21:29:02 - POST: /get_session_questions
21:29:02 - FEED: Finding next batch of questions in stream
21:29:02 - CONTROLLER: Validating the first batch for session: my_set-default
21:29:02 - PREPROCESS: Splitting sentences
21:29:02 - FILTER: Filtering duplicates from stream
21:29:02 - FILTER: Filtering out empty examples for key 'text'
Exception when serving /get_session_questions
Traceback (most recent call last):
File "C:\Users\sara\Anaconda3\lib\site-packages\waitress\channel.py", line 336, in service
task.service()
File "C:\Users\sara\Anaconda3\lib\site-packages\waitress\task.py", line 175, in service
self.execute()
File "C:\Users\sara\Anaconda3\lib\site-packages\waitress\task.py", line 452, in execute
app_iter = self.channel.server.application(env, start_response)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\api.py", line 451, in api_auto_instantiate
return module.hug_wsgi(*args, **kwargs)
File "C:\Users\sara\Anaconda3\lib\site-packages\falcon\api.py", line 244, in call
responder(req, resp, **params)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 789, in call
raise exception
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 762, in call
self.render_content(self.call_function(input_parameters), context, request, response, **kwargs)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 698, in call_function
return self.interface(**parameters)
File "C:\Users\sara\Anaconda3\lib\site-packages\hug\interface.py", line 100, in call
return __hug_internal_self._function(*args, **kwargs)
File "C:\Users\sara\Anaconda3\lib\site-packages\prodigy_api\hug_app.py", line 228, in get_session_questions
tasks = controller.get_questions(session_id=session_id)
File "cython_src\prodigy\core.pyx", line 130, in prodigy.core.Controller.get_questions
File "cython_src\prodigy\components\feeds.pyx", line 58, in prodigy.components.feeds.SharedFeed.get_questions
File "cython_src\prodigy\components\feeds.pyx", line 63, in prodigy.components.feeds.SharedFeed.get_next_batch
File "cython_src\prodigy\components\feeds.pyx", line 140, in prodigy.components.feeds.SessionFeed.get_session_stream
File "C:\Users\sara\Anaconda3\lib\site-packages\toolz\itertoolz.py", line 376, in first
return next(iter(seq))
File "cython_src\prodigy\components\sorters.pyx", line 151, in iter
File "cython_src\prodigy\components\sorters.pyx", line 61, in genexpr
File "cython_src\prodigy\models\ner.pyx", line 292, in call
File "cython_src\prodigy\models\ner.pyx", line 261, in get_tasks
File "C:\Users\sara\Anaconda3\lib\site-packages\toolz\itertoolz.py", line 726, in partition_all
prev = next(it)
File "cython_src\prodigy\models\ner.pyx", line 211, in predict_spans
File "C:\Users\sara\Anaconda3\lib\site-packages\toolz\itertoolz.py", line 726, in partition_all
prev = next(it)
File "cython_src\prodigy\components\preprocess.pyx", line 39, in split_sentences
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 723, in pipe
for doc, context in izip(docs, contexts):
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 751, in pipe
for doc in docs:
File "nn_parser.pyx", line 221, in pipe
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\util.py", line 463, in minibatch
batch = list(itertools.islice(items, int(batch_size)))
File "nn_parser.pyx", line 221, in pipe
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\util.py", line 463, in minibatch
batch = list(itertools.islice(items, int(batch_size)))
File "pipes.pyx", line 397, in pipe
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\util.py", line 463, in minibatch
batch = list(itertools.islice(items, int(batch_size)))
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 954, in _pipe
for doc in docs:
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 726, in
docs = (self.make_doc(text) for text in texts)
File "C:\Users\sara\Anaconda3\lib\site-packages\spacy\language.py", line 715, in
texts = (tc[0] for tc in text_context1)
File "cython_src\prodigy\components\preprocess.pyx", line 38, in genexpr
File "cython_src\prodigy\components\filters.pyx", line 35, in filter_duplicates
File "cython_src\prodigy\components\filters.pyx", line 16, in filter_empty
File "cython_src\prodigy\components\loaders.pyx", line 23, in _rehash_stream
File "cython_src\prodigy\util.pyx", line 194, in prodigy.util.set_hashes
File "cython_src\prodigy\util.pyx", line 224, in prodigy.util.get_hash
AttributeError: module 'murmurhash' has no attribute 'hash'

[ines (Ines Montani)]

Ah, that should be fixed by just upgrading murmurhash. See here:

AttributeError: module 'murmurhash' has no attribute 'hash'

I created a clean Anaconda environment on Windows 10 and successfully installed SpaCy and prodigy-1.8.3-cp35.cp36.cp37-cp35m.cp36m.cp37m-win_amd64.whl After web server starts..as soon as I open the localhost on the browser I get this error and log below. Anxious to try prodigy... Thx! [image] [image]

[samoz]

Now it works! Thanks for helping me out Ines!

[erijcken (Emil)]

Hi Ines,

I ran into the same error and am not sure if I placed the data in the correct location.

How I did it is:

import os
os.chdir('PATH')
!python -m prodigy ner.manual annotated_names blank:nl texts1.jsonl --label "PERSON"

Is this the correct way?

[ines (Ines Montani)]

@erijcken

I ran into the same error and am not sure if I placed the data in the correct location.

The command looks okay – but it of course depends on where the file texts1.jsonl is. If you want to be safe, you can just use the absolute path to the file instead.

The error in the UI is shown if there's an error on the server. So in that case, you should be able to go back to your terminal (or notebook) and see the traceback of the Python error. This should usually tell you what's going on.