# Starting a Multi-user session **URL:** \ **Category:** Uncategorized\ **Tags:** solved, multi-user\ **Created:** [January 23, 2023, 7:07pm UTC](https://support.prodi.gy/t/starting-a-multi-user-session/6264 "2023-01-23T19:07:14Z")\ **Posts on this page:** 1\ **Showing post:** 6
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [January 24, 2023, 7:18pm UTC](https://support.prodi.gy/t/starting-a-multi-user-session/6264/6 "2023-01-24T19:18:20Z")
> [@NoorKhalifa](#): > > While reading, I saw that the current version of Prodigy is fine for multi-user annotations using the same port, right? Correct. An alternative to multi-user named sessions is to run each annotator on a different port/dataset. Here's a good pro/con of each: > [@Help - Multiple Annotators setup](https://support.prodi.gy/t/help-multiple-annotators-setup/3343/4): > > Ah, sorry if the terminology I used was confusing! What I meant was the session/instance/process. There are generally two ways you could set this up and it depends on your project, annotators and how you want to set up the workflow: Start one Prodigy process one a given host/port per annotator and save the annotations to separate datasets. For example, on port 8080, you start the Prodigy server with 100 documents and save to annotations\_andrew, on port 8081 you start Prodigy with the next 100 … > [@NoorKhalifa](#): > > I will be able to allow each annotator to annotate the whole dataset and I will be able to get each annotated dataset separately, right? Sounds like then you want "overlapping" annotations, i.e., you need to set `feed_overlap` to `true`. You can do this in `prodigy.json` or as an override. I just responded to a similar example; see the bottom of the difference of `feed_overlap`: > [@NER Team](https://support.prodi.gy/t/ner-team/6262/4): > > hi @kushalrsharma, When all sessions have the same data, we call that "overlapping" annotations. However, it seems that you want "non-overlapping" annotations, i.e., each example in the data once to whoever is available. As a result, your data will have each example labelled only once in total. This is exactly what setting "feed\_overlap" to false will do (this is the value by default). But it seems like you're saying that you think your "feed\_overlap" is still set to false, but it is not pr… > [@NoorKhalifa](#): > > However, when I save the first 10 then go on to annotate more records and exit or refresh, the new progress is still not being saved. Is there a way of enforcing autosave? If you want to autosave, you can either set your `batch_size` to 1 or setting `instant_submit` in [prodigy.json](https://prodi.gy/docs/install#config) (config) to true. The one downside of these approaches is that it'll remove the undo as you'll no longer be holding the batch in client before sending to DB. I can't remember the specific differences but this post seems to discuss it more: > [@Synchronous Batch Mode](https://support.prodi.gy/t/synchronous-batch-mode/3594/2): > > Hi! One of the main concerns for the web app is to make sure the queue is never running low, so it will keep asking for new batches in the background as the current batch is being annotated. The motivation here is that one individual example typically doesn't matter very much, and a single update to the model won't move the needle enough for it to be worth blocking on it. Depending on the model, blocking could easily take 10+ seconds or even a minute, so that's not really viable to do synchronou… Also, the Progress Bar is not updated real time; it's only updated when a new batch is retrieved. We've been debating modifying this in a future version but there are some unintended consequences in high latency environments with multiple annotators that can cause issues. > [@Blocks and Progress bar](https://support.prodi.gy/t/blocks-and-progress-bar/3973/2): > > Hi! Prodigy will calculate the progress automatically if the stream returned by the recipe has a length (e.g. if it's a list). If the stream is a generator, it could potentially be infinite and it will only be read one batch at a time, so Prodigy can't know how many examples are left. If you know how many examples you have, or want to implement some other custom logic to calculate the progress, you can also add a "progress" callback to the components returned by your recipe: [https://prodi.gy/d…](https://prodi.gy/docs/custom-recipes#progress) If you wanted something more realtime, you could check out the `update` callback. @koaning has a great video on it to track annotator speed: [![](https://img.youtube.com/vi/uBtOebwnTRs/maxresdefault.jpg "Using the Update callback - Prodigy Shorts") ](https://www.youtube.com/watch?v=uBtOebwnTRs) Since, you're using `feed_overlap` is true, be sure to update Prodigy today. Yesterday, [we released v1.11.9](https://support.prodi.gy/t/prodigy-v1-11-9-release/6263/2) which fixed a lingering bug of "duplicated" annotations in high latency/multiple annotators sessions. Also, it's worth mentioning the team is working very hard on releasing Prodigy v2 in a few months. This will add even greater customization to `feed_overlap` (e.g., what if you want only a certain percentage overlapping across annotators). > For the release of Prodigy `v2`, we have some exciting new features and a significant redesign to the way examples are sent to different annotators. Specifically, this redesign will eliminate the need for this tradeoff and should eliminate 100% of unwanted duplicates. > It will also bring more customization to the `feed_overlap` setting like [setting the number of annotations you want for each task](https://support.prodi.gy/t/setting-limit-per-document-for-consensus/6235/3), or configuring a percentage of examples to have overlapping annotations for. We're even working on registering custom policies to distribute work to different annotators. Also, since you're looking at having multiple annotators, be sure to check out @pmbaumgartner great Inter-annotator Agreement recipes. This can allow you to better "calibrate" how consistent annotators are (see Peter's [wonderful NormConf talk](https://youtu.be/GrQcVU-eapc?t=310) on why calibration is important). We'd love feedback as this project is evolving! > [@Inter-Annotator Agreement Recipes - Feedback Wanted](https://support.prodi.gy/t/inter-annotator-agreement-recipes-feedback-wanted/6175): > > Hey everyone - I thought I would share some recipes I made for calculating Inter-Annotator Agreement (also called [Inter-Rater Reliability](https://en.wikipedia.org/wiki/Inter-rater_reliability)) in Prodigy. It supports a variety of annotation situations so you can have any number of annotators and incomplete overlap of annotations. Currently it supports classification tasks (binary/multiclass/multilabel), but not any spans, NER, audio, or image tasks. Here's a link to the repo: link[GitHub - pmbaumgartner/prodigy-iaa](https://github.com/pmbaumgartner/prodigy-iaa) Everything is bundled as a … Thanks again for your questions (sorry about dumping a ton of resources). Hopefully you have plenty of resources and please post back if you have interesting experiments or link to blog/paper if you're successful! --- _[View the full topic](https://support.prodi.gy/t/starting-a-multi-user-session/6264)._