# textcat.teach uncertain sorter show options with score 0 **URL:** \ **Category:** Uncategorized\ **Tags:** usage, textcat\ **Created:** [May 30, 2022, 10:01pm UTC](https://support.prodi.gy/t/textcat-teach-uncertain-sorter-show-options-with-score-0/5678 "2022-05-30T22:01:44Z")\ **Posts on this page:** 1\ **Showing post:** 4
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [August 30, 2022, 5:55pm UTC](https://support.prodi.gy/t/textcat-teach-uncertain-sorter-show-options-with-score-0/5678/4 "2022-08-30T17:55:13Z")
Hi @info2000! For your three new questions, I wasn't sure if you're asking for only `textcat.teach` or Prodigy in general. The answers below should be the same for both. > [@info2000](#): > > -how are filtered the repeated samples into a dataset? I'm sorry, I don't understand the question. Can you rephrase? If you're interested in the default behavior for deduplication, you may find the Docs for the [`filter_duplicates`](https://prodi.gy/docs/api-components#filter_duplicates) function to be helpful or in the answers below. > [@info2000](#): > > -if 2 samples are the "same text" but the sample b has one space different than sample A, are threat like same sample and deduplicated? For two texts where only difference is one space, would have different values for their `input_hash`. Since they have different `input_hash` values (and also different `task_hash`), they would be treated as two different samples. > [@info2000](#): > > -if I start prodigy with 100 samples file, label them, and restart with other file with 50 samples, but 30 samples was also included on the initial file, are these 30 samples removed/avoid ? Let's assume for both sessions, you'll set your dataset to the _same_ Prodigy dataset. For annotation, by default records are deduped by `task_hash`, see [`"exclude_by"` field in Configuration](https://prodi.gy/docs/install#config). Therefore, if you're doing the same task (e.g., `textcat` recipe) you did in the first 100 samples, then only 20 samples would be shown (30 repeated tasks would be excluded). But if the 2nd annotation session used a different task (e.g., different recipe like `ner`), then all 50 samples would be shown. > [@info2000](#): > > Why could this be happening? > > Is low effective classify the same text +24 times (I had 24 labels, but should review the same tuple example-label multiple times) Sorry we haven't responded yet to this. Are you still interested in an explanation for this? As Vincent had recommended, if we have a reproducible example, it makes it a lot easier for us to debug. I've found some past posts that may explain shed some light. While uncommon, it's not impossible to see low scores when using the default `ema` algorithm: > if you get a long sequence of low-scoring examples, the probability sorter will ask you fewer questions from them, while the exponential moving average sorter will get impatient, and start asking you questions even if the scores are low. > [@Taking advantage of TONS of unlabeled data](https://support.prodi.gy/t/taking-advantage-of-tons-of-unlabeled-data/525/2): > > You might want to implement your own sorter method for this. The function just needs to take a sequence of (score, example) tuples and yield out a sequence of examples. Another simple thing you could try is to use prefer\_uncertain(stream, algorithm='probability'). The probability algorithm randomly drops examples proportional to their score. In contrast, the default sorter doesn’t trust that the scores produced by the model are going to stay well calibrated. It asks you about examples which are … As an alternative, you may want to consider using `algorithm = 'probability'` as the post you originally cited mentioned: > On the other hand, if you know the target class is rare, you want the sorter to “believe” the scores much more. In this case the probability sorter is better. I also found the discussion here to be helpful: > [@Prodigy Active Learning prefer\_uncertain mechanism](https://support.prodi.gy/t/prodigy-active-learning-prefer-uncertain-mechanism/657/7): > > The sorting mechanism is designed to account for large input data sets, so it works in a streaming fashion, sorting chunks of data. It's a tricky balancing act because we don't want to block the annotation feed while we search for the best examples. The standard deviation we refer to is a running estimate of the standard deviation. This is an approximation, the calculation is similar to making an exponential moving average estimation. It's the standard deviation of the specified "figure of meri… --- _[View the full topic](https://support.prodi.gy/t/textcat-teach-uncertain-sorter-show-options-with-score-0/5678)._