Transfer Learning for French NER

[ines (Ines Montani)]

Hi! I answered your transfer learning question in the other thread I hope this explains some of the terminology and what transfer learning usually refers to:

Transfer Learning for NER

Hi! If you're using a pre-trained entity recognizer or other component and are then updating it with more examples, that's not what's typically referred to as "transfer learning" – it's more like "domain adaptation". The idea of transfer learning is that you initialise your model with weights trained using a different objective – for instance, a language model trained to predict the next word. These representations encode a lot of knowledge about the language and the world that can be transferr…

The idea is that you can often get better results by initialising your model with better representations that encode some knowledge about the language and the world – for example, language model weights that were trained by predicting the next word. For example, one way to do this is by initialising the model with transformer embeddings and using those for the token-to-vector embedding. Your tokens end up with "better" and more representative vectors this way, and when those are used as features in the component models (e.g. NER), you often see better results. What's "pretrained" here is the token-to-vector embedding layer – not the actual task-specific component like NER.