Just released a new nightly that includes improvements around the ner.teach workflow. Could you try re-running your experiments and see how you go? You'll now also be able to mix binary with manual annotations and ner.teach will ask about sentences with no entities in order to improve performance.