Prodigy + spaCy for negation extraction and a link between the entities

First, many thanks for your hard work on Prodigy and spaCy, they are absolutely amazing products for rapid text mining and NLP modelling!

I have two questions:

I am wondering, whether you can advise on more efficient implementation with Prodigy and spaCy of identifying negations in texts. More specifically, I work on medical text mining and interested in information extraction (drug names, doses, diagnoses, etc). I have a large list (a vocabulary) of drug names and use simple rules (with regex) to extract drugs from patients’ notes as prescribed drugs. It works well so far. However, some drugs are mentioned in the texts as non-tolerable and thus should not be extracted. For example, consider a short text snippet:

text_0 = ‘John has developed sides effects while using trazodone 75 mg, therefore I will prescribe a combination of Citalopram 50mg and 25 mg of Fluoxetine’

Question 1: I need to extract only actual drugs -> {‘Citalopram’, ‘Fluoxetine’} and to label ‘Trazodone’ as ‘problematic’. Is it possible to directly annotate with Prodigy these drugs and train a model? (‘prescribed’ and ‘problematic’ tags).

Question 2: Also, I need to extract the doses of drugs, however, their pattern is not standardised. It may appear right after the drug name, or it could be mentioned anywhere near the drug name.

While being very excited about NERing with Prodigy, I’ve been thinking about tagging drug names and their doses. However, it is unclear to me, how to specify a link between doses and drug names (for example, 25mg corresponds to Fluoxetine and 50 mg to Citalopram and not vise versa).

Thanks in advance!

1 Like

Hi Andrey,

I think the dependency parser will probably be useful for your task. Have a look at the analysis produced for the example you gave:

The parse has some errors: especially, the dosage “75 mg” is attached incorrectly. The parser has attached it to “using”, when the correct attachment is to “trazodone”. These errors make extracting the correct relations a little more complicated, as the rules become a bit more hacky – it’s still useful though, and generally better than working from the raw text. You can also try to improve the parser on your domain using the dep.teach recipe, although this is still a bit experimental.

Often a good solution is to develop a list of template rules that activate a relation under some construction. For instance, you might have a rule for trigger verbs like “prescribe”, that says “If there are any DRUG entities as dobj relations of this verb, add them as actual drugs”. Another rule could negate trigger verbs. You would attach it to a word like “not”, and it would say “if this word attaches to a trigger verb, block extraction of its arguments”. You’d then have another template for noun phrase cases, like “prescription of Trazodone”.

There will probably only be a few of these templates, each only triggered by a handful of words. If the template is too ambiguous, you can learn whether it should apply to a given context. For instance, let’s say you have an example like “Trazodone 75mg. Symptoms persist.”. It’s hard to get this right with rules, so you might want to have an additional model that learns a tag like PRESCRIBED here, which is only trained on DRUG examples.

The general strategy is to use the machine learning to add annotations that give you better properties to write rules against. There’s a balance between how easy the rules are to write, and how easy the annotations are to learn. If you have a simple rule that depends on long-range relations, that’s probably difficult for a model to learn — while the rule will be easy to write. On the other hand, statistical models are good at accumulating lots of small pieces of evidence, which are difficult to put together into effective rules.

In order to do this effectively, I would definitely recommend making an evaluation set you can test the whole process against. The ner.manual recipe is probably best for this. For relations, one notation you could use is to annotate the whole relation, starting from the beginning of the first entity and ending at the last word of the second entity. You would then make a quick second pass to annotate the DRUG and DOSE spans within this parent span.

Hi Matt,

Brilliant, a massive thank you for your detailed answer and the ideas! I am reading the Prodigy manual and experimenting with various recipes. I’ve been thinking about the dependency parser, but was not sure how to best apply it in this context. I will experiment with your suggestions.