# Discovering associated words/phrases using NLP

**URL:** <https://support.prodi.gy/t/discovering-associated-words-phrases-using-nlp/4335>\
**Category:** Uncategorized\
**Tags:** usage, ner, textcat\
**Created:** [June 16, 2021, 7:05am UTC](https://support.prodi.gy/t/discovering-associated-words-phrases-using-nlp/4335 "2021-06-16T07:05:42Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [June 17, 2021, 1:19am UTC](https://support.prodi.gy/t/discovering-associated-words-phrases-using-nlp/4335/4 "2021-06-17T01:19:57Z")

</div>

I agree with @SofieVL that framing this as a pure span extraction task will likely be inefficient because the boundaries of the spans are very ambiguous and potentially disjoint, and it'll be very difficult to create consistent data for it. The information about the season may also be spread across multiple keywords and indicators, including how the sentence is phrased, which a text classifier will be much better suited for.

This thread might be relevant to you, and it actually deals with a very similar task (extracting season compatibility for garments from reviews):

> [@Using a text classifier instead of NER](https://support.prodi.gy/t/using-a-text-classifier-instead-of-ner/2712/5):
>
> From the example you posted above, it definitely looks like text classification is a good approach for the task. The text classifier will be able to take all words in the text into account, so if certain keywords commonly occur in summer texts, it's something that your model will be able to learn and generalise from. I think what Matt is referring to in his comment is more related to using a text classifier to decide on fine-grained categories for spans mentioned in the text. This is easier if…

I actually just did a talk where I walked through a similar problem (span extraction vs. labels over the whole text), which might be interesting as well. The example starts around ~13:00: [https://www.youtube.com/watch?v=mJqFI7vhqdA](https://www.youtube.com/watch?v=mJqFI7vhqdA)

So for your use case, you could start by classifying the texts by season, which should be pretty straightforward to label and predict. You can still use keywords to pre-select and pre-label the examples, e.g. if a text contains "warm weather", it'll likely be `SUMMER`. If you have a text classifier that gives you the attention scores for the individual tokens, this would give you insights into what the strongest signals were that led to the prediction.

However, what you do next depends on your end goal and what you're actually looking to do with those tokens/phrases. If those will be consumed by some other downstream application, you might want to consider how useful those will actually be: you'll likely end up with arbitrary tokens that don't necessarily have anything in common or represent the same type of syntactic constituents. So it might be difficult to actually do anything useful with them.

---

_[View the full topic](https://support.prodi.gy/t/discovering-associated-words-phrases-using-nlp/4335)._