hi @MaxB!
Thanks for the background! That makes sense how you're approaching -- definitely running on each port is best given different data/tasks but challenging if you're trying to run simultaneously. Running simultaneous processes is even harder to run without containers and/or orchestration engine like Kubernetes, which is likely out-of-scope.
Another other idea you may want is to run these multiples processes in separate terminals rather than in Python. You can do this manually (e.g., open up different terminal windows) or you could use a terminal multiplexer like screen (see comment below):
Or tmux is another common option as mentioned below.
You can run on different ports like prodigy.server() in terminal by prefixing each command:
PRODIGY_PORT=8080 prodigy ner.manual dataset1 en_core_web_sm data1.jsonl --label A,B,C
PRODIGY_PORT=1234 prodigy ner.manual dataset2 en_core_web_sm data2.jsonl --label D,E,F
Hope this helps and let me know if you have any further questions!