false positives in Spacy NER

,
1

I have trained a Spacy NER model with following training data
TRAIN_DATA = [
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),
("LIBOR Interest rates", {"entities": [(0, 5, "LIBOR_WRD")]}),

]

When I? tested with this sentence "CPIMG is not a technology" , I get "CPIMG" getting detected as LIBOR_WRD . What has made space to detect this as LIBOR_WRD . I do not see the context or neighbour words are same as training data . The only this common is all "CAPITALS" how can i avoide this problem ?

2

You can avoid the issue by annotating more data.

The problem here is that you haven't really provided any negative examples --- you've just repeated the same positive example over and over. The model can make any number of generalisations that are compatible with your data: all of the following theories match the examples you've given it:

  • Every word at the start of a sentence is a LIBOR_WRD
  • Every word beginning with an L is a LIBOR_WRD
  • Every word ending in BOR is a LIBOR_WRD
  • Every word before Interest is a LIBOR_WRD
  • Every word two words before a word ending in tes is a LIBOR_WRD

etc. Some training tricks such as dropout can help the model generalise slightly, but the model has so little evidence to work with, it's unlikely to come up with a policy that's useful for what you want to do.

If you really just want to recognize the word LIBOR, you can use a patterns dictionary. If you need to infer a more subtle behaviour, you need to give the model enough evidence to work with, by annotating more examples with Prodigy.