# Multi-line named entity **URL:** \ **Category:** Uncategorized\ **Tags:** usage, ner\ **Created:** [January 7, 2020, 3:09pm UTC](https://support.prodi.gy/t/multi-line-named-entity/2400 "2020-01-07T15:09:43Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![cjapes](https://avatars.discourse-cdn.com/v4/letter/c/cdc98d/32.png) [@cjapes](https://support.prodi.gy/u/cjapes)\ **Post date:** [January 7, 2020, 3:09pm UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/1 "2020-01-07T15:09:43Z")
Hi, I'm trying to annotate blocks of email as signature, disclaimer, reply-blocks but when trying to select the blocks it looks like prodigy does not allow to annotate a block that contains a line break, is there a way around this? ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [January 8, 2020, 11:21am UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/2 "2020-01-08T11:21:59Z")
Hi! > [@cjapes](#): > > prodigy does not allow to annotate a block that contains a line break, is there a way around this? Yes, that's by design – for actual named entity tasks, you typically don't want newlines in the spans, because that's pretty much always a mistake. So by default, newline tokens are unselectable. You can turn this behaviour off by setting `"allow_newline_highlight": true` in your `prodigy.json` or recipe config. You can read more about newline tokens here: [Annotation interfaces · Prodigy · An annotation tool for AI, Machine Learning & NLP](https://prodi.gy/docs/api-interfaces#ner_manual-newlines) From what you describe, highlighting whole paragraphs also seems a bit inefficient, so there's probably a better way to solve this. What's your end goal, and what are you looking to train / do with the data later on? Actual NER model implementations typically work best for short phrases with distinct start and end points (like noun phrases), so framing your problem as an NER task likely won't work very well, and a text classifier would do much better at the problem. (See [this section on NER vs. textcat](https://prodi.gy/docs/named-entity-recognition#ner-vs-textcat) for some background on this.) In that case, highlighting the exact full spans is also less important. So you could go sentence by sentence and label whether the sentence is of one of the categories you're interested in. ---
**Author:** ![cjapes](https://avatars.discourse-cdn.com/v4/letter/c/cdc98d/32.png) [@cjapes](https://support.prodi.gy/u/cjapes)\ **Post date:** [January 9, 2020, 6:08pm UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/3 "2020-01-09T18:08:30Z")
Hi Ines, thank you for your quick reply. I am trying to replicate the work of Carvalho et al. in [http://www.cs.cmu.edu/~vitor/papers/sigFilePaper\_submited.pdf](http://www.cs.cmu.edu/~vitor/papers/sigFilePaper_submited.pdf) , basically learning to extract signature and reply lines from email. It doesn't look to fit neither a NER task or a text classification task. In this particular project I just want to use prodi.gy to label the dataset. this would be an example: ``` From: wcohen@cs.cmu.edu To: Vitor Carvalho Subject: Re: Did you try to compile javadoc recently? Date: 25 Mar 2004 12:05:51 -0500 Try cvs update –dP, this removes files & directories that have been deleted from cvs. - W On Wed, 2004-03-24 at 19:58, Vitor Carvalho wrote: > I just checked-out the baseline m3 code and > "Ant dist" is working fine, but "ant javadoc" is not. > Thanks > Vitor ------------------------------------------------------------------ William W. Cohen “Would you drive a mime wcohen@cs.cmu.edunuts if you played an http://www.wcohen.com audio tape at full Associate Research Professor blast?” ---- CALD, Carnegie-Mellon University S. Wright ``` ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [January 9, 2020, 8:36pm UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/4 "2020-01-09T20:36:09Z")
Ah, cool! In that case, I definitely think by labelling spans you'd be making the task unnecessarily hard to annotate. Instead, you could go over the text line by line and use the choice interface to select the label that applies to the line? Similar to the [`textcat.manual`](https://prodi.gy/docs/text-classification#manual) recipe with multiple labels. If you want more context (e.g. previous and next line), you could write a custom recipe and set your stream up like this: ```python def get_stream(): for i, line in enumerate(lines): prev_line = lines[i - 1] if i != 0 else "" next_line = lines[i + 1] if i != len(lines) - 1 else "" yield {"html": f"{prev_line}
{line}
{next_line}", "id": i} ``` This will show the lines as HTML, with the line you're currently annotating displayed in bold. The rest of the recipe could look pretty much exactly like [this this example](https://prodi.gy/docs/custom-recipes#example-choice). ---
**Author:** ![vish](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/vish/32/2258_2.png) [@vish](https://support.prodi.gy/u/vish)\ **Post date:** [January 23, 2024, 12:45pm UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/5 "2024-01-23T12:45:59Z")
(setting the json line, does let user select it, so it solves the original problem. - thanks) with "span" classification (unlike NER) a span might be a lot longer than the typical NER (and contain a few lines, e.g. an email signature block has several lines) Question: what's the "best practice" for span length? should I try to capture the entire signature block (which may be 5-6 lines with 3-8 words on each)? or would it be better to capture the "first line in a block" ? ---
**Author:** ![magdaaniol](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/magdaaniol/32/2787_2.png) [@magdaaniol](https://support.prodi.gy/u/magdaaniol)\ **Post date:** [January 24, 2024, 10:38am UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/6 "2024-01-24T10:38:59Z")
Hey @vish , I suppose that showing lines within some local context should be enough for the annotator to make the decision. The good thing about Prodigy is that you can program the way examples are shown to the user leveraging patterns like the one you've mentioned. Not sure I understand your question about span categorization. In Prodigy you can annotate overlapping spans with [`spans.manual`](https://prodi.gy/docs/recipes#spans-manual). This data can be used to [train](https://prodi.gy/docs/span-categorization#training) spaCy [SpanCategorizer](https://spacy.io/api/spancategorizer) . ---
**Author:** ![vish](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/vish/32/2258_2.png) [@vish](https://support.prodi.gy/u/vish)\ **Post date:** [January 24, 2024, 11:23am UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/7 "2024-01-24T11:23:08Z")
editing the question I asked - to clarify ---
**Author:** ![magdaaniol](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/magdaaniol/32/2787_2.png) [@magdaaniol](https://support.prodi.gy/u/magdaaniol)\ **Post date:** [January 26, 2024, 12:30pm UTC](https://support.prodi.gy/t/multi-line-named-entity/2400/8 "2024-01-26T12:30:46Z")
Hey @vish > [@vish](#): > > Question: what's the "best practice" for span length? > should I try to capture the entire signature block (which may be 5-6 lines with 3-8 words on each)? > or would it be better to capture the "first line in a block" ? Span Categorizer can handle spans longer than NER, but spans of several lines including entire paragraphs can affect the model performance due the number of span suggestions from the n-gram suggester. Here's a relevant spaCy post on the topic: > **[What is the maximum length the annotation of a span can be? · explosion/spaCy...](https://github.com/explosion/spaCy/discussions/11450)** > > Hi, I am trying to train a span categorizer but since I'm new to it, I am a bit unsure how to annotate my data. I'm using the spancat as a parser to extract specific sections from a document. Somet... You might need to experiment with the configuration of the suggester function as explained in this thread on memory errors with long spans: [spancat out of memory - #2 by ines](https://support.prodi.gy/t/spancat-out-of-memory/5459/2) If your spans cover entire sentences and sequences of them, reframing the task as text (sentences) classification will often be easier to learn.