# Extracting current and prior company affiliations from bios **URL:** \ **Category:** Uncategorized\ **Tags:** usage, ner, best-practices\ **Created:** [February 1, 2019, 2:25pm UTC](https://support.prodi.gy/t/extracting-current-and-prior-company-affiliations-from-bios/1176 "2019-02-01T14:25:51Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [February 1, 2019, 4:58pm UTC](https://support.prodi.gy/t/extracting-current-and-prior-company-affiliations-from-bios/1176/2 "2019-02-01T16:58:42Z")
Hi! First, if you haven't seen it yet, you might find @honnibal's talk on this topic helpful. It discusses strategies for breaking a larger NLP problem down into smaller tasks and designing label schemes, which all sounds very relevant to you. https://www.youtube.com/embed/jpWqz85F_4Y?feature=oembed&wmode=opaque&list=PLBmcuObd5An7oE73XacDNJ3ZxO82ogc6j If you're trying to predict categories like `PRIOR_COMPANY` and `CURRENT_COMPANY`, you might actually run into a similar problem as the "crime location" and "victim" example in the video: Whether "ACME Corp" is a prior company or current company isn't really inherent to the entity "ACME Corp" itself. It depends on very subtle signals and on the surrounding entities and their relationships to each other. As you've already noticed, this can be very difficult for the model to learn if you're treating it as a pure entity recognition task. You might see better results if you take one step back and start with the generic categories first that are easier to learn: "ACME Corp" is an `ORG` and "Dave" is a `PERSON`. Even out-of-the-box, the pre-trained models probably give you a decent accuracy on this, and you can then use Prodigy to improve those further on your data, until it's very solid. You could also introduce a new category, `ROLE` or `JOB` (for "CFO"). Once you have a model that can accurately predict those general entities, you could try and resolve the relationships between them. In your examples, the syntax seems to hold most of the clues you need and that's something you can usually predict quite accurately. [Here's an example](https://explosion.ai/demos/displacy?text=Dave%20worked%20at%20ACME%20Corp%20as%20CEO%20and%20now%20serves%20as%20CFO%20of%20Wallmart&model=en_core_web_sm&cpu=0&cph=0) of the sentence in the displaCy visualizer. ![34](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/1X/1944e21c7eedddb3172a3573adb682874995e1fb.jpg) The above visualization only shows the coarse-grained tags like `VERB`, i.e. `token.pos_`. [See here](https://spacy.io/api/annotation#pos-tagging) for the full part-of-speech tags predicted as `token.tag_`. For example, `token.tag_ == 'VBD'` lets you check for verbs in past tense. "Dave" is the nominal subject attached to a past-tense verb with the lemma "work". From that verb, you can resolve two prepositional phrases: "at ACME Corp" (preposition "at" plus `ORG` entity) and "as CEO" (preposition as plus `JOB` entity) – these are all things you can extract programmatically if you have the part-of-speech tags, the dependency parse and relevant entity types. You might have to write a few different rules to cover the possible constructions but it'll also give you a lot more fine-grained control. You can also keep improving the tagger and parser on your specific data, focusing on the labels you care about the most (e.g. using Prodigy's `dep.teach` and `pos.teach`). For example, making sure verbs are correctly tagged as past tense or that subjects are correctly attached. (This is also fairly easy to annotate, even without an extensive linguistic background.) Of course, it always depends on your data and problem, so you'd have to experiment. But I do think combining more general entity types with syntax-based rules could potentially help a lot with solving your problem. For more details and inspiration, here's another thread on the concept of combining predictions and rules to solve different information extraction problems: > [@Fact extraction for earnings news](https://support.prodi.gy/t/fact-extraction-for-earnings-news/1023/2): > > Hi and welcome! smiley This is an interesting project and definitely sounds like something that can be solved by NLP. Large-scale information extraction (including things like “populate a database from free-form text”) is a use case where NLP really shines and something that is already working very well across research and production. I think what it really comes down to is breaking the larger, abstract goal down into smaller machine learning tasks, and finding out what works best for each in… --- _[View the full topic](https://support.prodi.gy/t/extracting-current-and-prior-company-affiliations-from-bios/1176)._