# Parsing/Identifying sections in job descriptions **URL:** https://support.prodi.gy/t/parsing-identifying-sections-in-job-descriptions/1100 **Category:** Uncategorized **Tags:** usage, custom, ner **Created:** [January 3, 2019, 1:31pm UTC](https://support.prodi.gy/t/parsing-identifying-sections-in-job-descriptions/1100 "2019-01-03T13:31:53Z") **Posts on this page:** 1 **Showing post:** 2
### Author: ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines) #### Post date: [January 4, 2019, 10:51pm UTC](https://support.prodi.gy/t/parsing-identifying-sections-in-job-descriptions/1100/2 "2019-01-04T22:51:27Z")
Hi! I remember answering some questions about a job description project a while ago, so maybe this thread might be useful, too: > [@Help with building NER for job descriptions](https://support.prodi.gy/t/help-with-building-ner-for-job-descriptions/1075/4): > > Yes, exactly. The main idea here is that you want to get over the cold-start problem (where the model knows nothing), pre-train it so it predicts something and then use the existing model's prediction to collect better annotations to improve it, update the improved model with more examples, and so on. You might have to experiment with a few different approaches to find out what works best. Maybe it makes sense to start off with annotating a few hundred examples by hand to give the model someth… > [@azfar](#): > > I’m not sure how well NER will work if the entities are several lines or a paragraph long? > [...] > Is there any other strategy besides NER that I can use to parse these job descriptions as I’ve mentioned? By definition, a named entitiy is usually a "real world object" – like a person, an organisation, a product or other distinct names like that. This is also what the underlying statistical model is optimised for. If your goal is to label longer phrases or even paragraphs, this is not typically an end-to-end problem for named entity recognition. Instead, your task might be a better fit for a combined approach using the entity recognizer to label more generic entity types, the dependency parse to select the whole phrase you're interested and maybe the text classifier to assign top-level topics to the extracted paragraphs. I explain this idea in more detail in this thread, which should be pretty relevant to your use case as well: > [@Fact extraction for earnings news](https://support.prodi.gy/t/fact-extraction-for-earnings-news/1023/2): > > Hi and welcome! smiley This is an interesting project and definitely sounds like something that can be solved by NLP. Large-scale information extraction (including things like “populate a database from free-form text”) is a use case where NLP really shines and something that is already working very well across research and production. I think what it really comes down to is breaking the larger, abstract goal down into smaller machine learning tasks, and finding out what works best for each in… > [@azfar](#): > > The idea is that as I annotate data for NER, the model will keep learning alongside and try to make the annotating easier as the process goes. Is this how it works? Yes, that's one workflow we've built in. You can also use Prodigy to label your data [100% from scratch](http://prodi.gy/demo?view_id=ner_manual) and without a model in the loop. The data can be exported in a straightforward JSONL file, so you don't have to use spaCy and can use the annotations with any other tool or process. In general, Prodigy's philosophy is to make annotation faster and more efficient by breaking larger tasks down into smaller decisions and automating/scripting as much as possible. Another thing we advocate for is running smaller experiments and iterating on your data. NLP is pretty experimental so you just need to try lots of stuff. For example, you might want to try out different labelling strategies to see which one is most promising. Can you teach the named entity recognizer your new entity definition, or does it make more sense for a text classification task? Do fine-grained or more generic categories work better? Is the data suitable and does the model improve if you add more data from source X? If you can just try these things quickly by labelling a few examples and running a few experiments, you'll ideally spend less time shooting in the dark and are able to focus on the most promising solution quicker. Btw, if you haven't seen it yet, you might also want to check out our `prodigy-recipes` repo, which shows some examples of how you can script Prodigy to build different annotation workflows and do your own automation: > **[GitHub - explosion/prodigy-recipes: 🍳 Recipes for the Prodigy, our fully...](https://github.com/explosion/prodigy-recipes)** > > 🍳 Recipes for the Prodigy, our fully scriptable annotation tool - GitHub - explosion/prodigy-recipes: 🍳 Recipes for the Prodigy, our fully scriptable annotation tool That said, if you do find that Prodigy just isn't the right tool for you, we're happy to issue a refund. It's a pretty specific developer tool for a pretty specific use case, and we believe in keeping the scope focused. So Prodigy can't be the right tool for everything and everyone – and that's okay 🙂 --- _[View the full topic](https://support.prodi.gy/t/parsing-identifying-sections-in-job-descriptions/1100)._