# Format neuralcoref inferences for use with prodigy relations recipe **URL:** \ **Category:** Uncategorized\ **Tags:** solved, relations, coref\ **Created:** [October 24, 2022, 3:51pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041 "2022-10-24T15:51:17Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![BenGriffithsPEP](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/bengriffithspep/32/3361_2.png) [@BenGriffithsPEP](https://support.prodi.gy/u/BenGriffithsPEP)\ **Post date:** [October 24, 2022, 3:51pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041/1 "2022-10-24T15:51:17Z")
Hi. I'm trying to create a custom co-reference model and I'd like to use [neuralcoref](https://github.com/huggingface/neuralcoref/tree/master/neuralcoref) to give me a headstart manually annotating my data with co-reference relationships. Is there an easy way to format neuralcoref inferences for [prodigy rel recipes](https://prodi.gy/docs/api-interfaces#relations)? Thanks! ---
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [October 24, 2022, 4:52pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041/2 "2022-10-24T16:52:42Z")
hi @BenGriffithsPEP! Thanks for your message and welcome to the Prodigy community ๐Ÿ‘‹ Unfortunately, we don't have an off-the-shelf "correct" recipe for `neuralcoref`. As the ticket below mentions, what would be important is to understand the format required (which I think is part of your question): > [@How to convert the rel.manual jsonl annoatated output or export relations annotations for training the neuralcoref model](https://support.prodi.gy/t/how-to-convert-the-rel-manual-jsonl-annoatated-output-or-export-relations-annotations-for-training-the-neuralcoref-model/5720/2): > > I've never worked with the neuralcoref library, partially because it's a 3rd party project maintained by Huggingface. You'd need to check their documentation to understand what format they required. Have you seen the [project documentation](https://github.com/huggingface/neuralcoref/blob/master/neuralcoref/train/training.md) and [this blogpost](https://medium.com/huggingface/how-to-train-a-neural-coreference-model-neuralcoref-2-7bb30c1abdfe)? Once you understand their format, you could create a [custom recipe](https://prodi.gy/docs/custom-recipes#writing). And as you found, it may be helpful to look over the [docs for the relations recipes](https://prodi.gy/docs/dependencies-relations#custom). Also, you may want to look at the [`dep.correct` recipe](https://prodi.gy/docs/recipes#dep-correct) to see how a "correct" recipe with the [`"relations"` user interface](https://prodi.gy/docs/api-interfaces#relations) (note that link shows what is the format for the `"relations"` UI which may be helpful too). To view this recipe, run: ```bash python -m prodigy stats ``` Where you should then see the `Location:` where Prodigy has been installed. In that location folder, look for the file `recipes/dep.py` where you'll see the `dep.correct` recipe. Similarly, you may find the [`coref.manual`](https://prodi.gy/docs/recipes#coref-manual) recipe to be helpful too, which you can find in the `recipes/coref.py`. Essentially, you'd want to combine the idea of "correcting" the `neuralcoref` model into the `coref.manual` recipe. If you're able to get a recipe, feel free to post back and/or post it as a GitHub gist! We would greatly appreciate it. Hope this helps! ---
**Author:** ![BenGriffithsPEP](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/bengriffithspep/32/3361_2.png) [@BenGriffithsPEP](https://support.prodi.gy/u/BenGriffithsPEP)\ **Post date:** [October 25, 2022, 3:32pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041/3 "2022-10-25T15:32:53Z")
Hey @ryanwesslen thanks for getting back to me and answering my question. If I do end up creating a custom recipe I will 100% share it here. I have a few additional follow up questions. When using either the coref or rel recipe to annotate coreferences, I'm a little confused which direction the relationship should go. I read somewhere in the prodigy documentation that it's not so important with coref because they are simply pairs of references, but wouldn't this have an impact when it comes to training a model? Also, how should you deal with pronouns that link to multiples entities e.g. "The doctor and nurse saw the patient. They did a great job." Should you assign a reference from 'doctor' to 'they' and 'nurse' to 'they'. In this case isn't the head / child of the relationship important? I'm not sure which should be the head and which should be the child? ---
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [October 26, 2022, 5:20pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041/4 "2022-10-26T17:20:00Z")
> [@BenGriffithsPEP](#): > > I read somewhere in the prodigy documentation that it's not so important with coref because they are simply pairs of references, but wouldn't this have an impact when it comes to training a model? It's a subtle point, but the scoring - including the loss calculation - only cares about whether two mentions are in the same cluster, it doesn't have a notion of direction in relationships. You can think of this as undirected graph. For more details, I would highly recommend our recent deep dive post on Neural Coreference Resolution in spaCy: > **[End-to-end Neural Coreference Resolution in spaCy ยท Explosion](https://explosion.ai/blog/coref/)** > > Coreference resolution is the problem of resolving entities in texts to references such as pronouns. Even if you've never heard of it, it's something we all do constantly every day, and is a key to understanding natural language. We recently added an... > [@BenGriffithsPEP](#): > > Also, how should you deal with pronouns that link to multiples entities e.g. "The doctor and nurse saw the patient. They did a great job." Should you assign a reference from 'doctor' to 'they' and 'nurse' to 'they'. In this case isn't the head / child of the relationship important? I'm not sure which should be the head and which should be the child? You would link "the doctor and the nurse" as one mention to "they". But in this case, this means that "the nurse" cannot be linked separately (it can in data but the model can't find it). This is the "split heads" referenced in the blog post. > More precisely the problem is treated as a clustering problem over non-overlapping spans in a document. The non-overlapping constraint renders the system incapable of handling the "split antecedent" problem. For example in "Alice and Bob said they like cheese, but he prefers sushi." The pronoun "they" refers to "Alice and Bob" and "he" refers to "Bob". However, the span "Bob" is inside "Alice and Bob" so we have to choose to either resolve "they" to "Alice and Bob" or "he" to "Bob". The lack of split antecedent handling is a limitation of many coreference resolutions systems including ours. As the post outlines, the spaCy core team's work on `coref` component is still experimental but likely could be very helpful for you. Since it's experimental, we haven't fully integrated it into Prodigy yet but there's a lot of opportunity with a custom recipe. If you have more questions, I would suggest posting on the [spaCy discussions forum](https://github.com/explosion/spacy/discussions) as that's where the spaCy core team answers spaCy specific questions (this forum is for Prodigy-specific questions). ---
**Author:** ![BenGriffithsPEP](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/bengriffithspep/32/3361_2.png) [@BenGriffithsPEP](https://support.prodi.gy/u/BenGriffithsPEP)\ **Post date:** [October 27, 2022, 1:34pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041/5 "2022-10-27T13:34:42Z")
Thanks for this information, it's really helpful! ---
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [October 27, 2022, 1:40pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041/6 "2022-10-27T13:40:35Z")
Also, be on the look out very soon for an accompanying spaCy coref video tutorial (with code too!). I just saw a sneak preview and it's an excellent summary of the post! ---
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [November 2, 2022, 3:26pm UTC](https://support.prodi.gy/t/format-neuralcoref-inferences-for-use-with-prodigy-relations-recipe/6041/7 "2022-11-02T15:26:33Z")
Just released the new experimental `coref` video by Edward and team: [![](https://img.youtube.com/vi/fio3BejnRsM/maxresdefault.jpg "Coreference Resolution in spaCy") ](https://www.youtube.com/watch?v=fio3BejnRsM) Here's the [GitHub code](https://github.com/explosion/projects/tree/v3/experimental/coref) too. Enjoy! ๐Ÿš€