Rel training

, ,
2

Hi Pantelis,

In general your workflow is exactly how we'd recommend it. The REL component was developed as a tutorial and is not really robust and generic enough to cover any use-case, but I hope the provided example code can help you hit the ground running.

First, the extracted annotations from my rel.manual also have the Named Entities, but i think i have seen mentioned elsewhere that it still better to separately train the two tasks. So i should keep my already trained NER model and train a REL model on top of that?

Yes, it would be better to train the two tasks separately. Imagine having a REL component that links addresses to people names. Your NER model would recognize ADDRESS and PERSON labels. If you train your NER model on the REL data, it will receive as training data only entities that are actually in a relationship, which is highly confusing to the NER. Instead, you want your NER to be trained on consistent data: all names and addresses in your text, whether they are in a relationship or not. Then leave that second part up to the REL model.

Second, given the above scenario how to actually do that in terms of pipeline training, changing the config file appropriately is enough or does it require something more?

Let's assume you have a trained NER model on disk as part of a spaCy pipeline called ner_trained. Then you'll have a pipeline defined like this (minimally):

[nlp]
pipeline = ["ner","relation_extractor"]

[components.ner]
source = "ner_trained"

[components.relation_extractor]
factory = "relation_extractor"

[training]
frozen_components = ["ner"]

Then, there are two ways in which you link the training together.

  1. you use the gold entities from your training data in the REL training. This ensures that the training process receives "clean" instances, allowing it to learn "better", but it may also result in a REL model that is less robust against mistakes by the NER. This is what the tutorial code does. It does it sneakily by setting the gold entities on the predicted doc in the reader that is used during training: https://github.com/explosion/projects/blob/v3/tutorials/rel_component/scripts/custom_functions.py#L31

  2. you use the predicted entities (from your previously trained NER model) during REL training. This will make the training data slightly more noisy, but could result in more robust performance on realistic data, because the REL will have to deal with wrong entities at some point (you'll never have 100% accuracy). The example code is set up to do that, because it defines the instances from eg.predicted. You do need one more addition to your config file for this:

[training]
frozen_components = ["annotating_components"]

Which ensures that even though the NER is frozen (i.e. not being updated), it will still run (before the REL) and set its predictions on eg.predicted, so the REL model can use those.

Third, is there a generic use case alteration in the code of projects/tutorials/rel_component at v3 · explosion/projects · GitHub
to run on my extracted annotations?

We've got in our roadmap to work on a more robust & generic version of the REL model, but I'm afraid this is not currently available and it won't be in the near future.

Hope this helps!

1 Like