Well, I think it's important to consider the following questions separately:
- What should the data look like during annotation?
- What should the data look like when training the different models?
- What should the data look like for evaluation?
For evaluation, you'll probably want something like:
You might want that during annotation as well -- that's one option. Another option is to have two annotation tasks, one where you have left elbow injury NEG_ILLNESS and another where you have negation predicted over a span of text.
The best way to set up the various representations is ultimately an empirical question --- it comes down to whatever works best. My general advice is to prefer text classification where possible, and try to factorise the decisions so that distinct pieces of information aren't all packed together into one label. You especially want to avoid labelling schemes where the decision depends on information in lots of different places, which is what happens when you have the negation case. In the negation case, you have the two decisions (is it an illness, is it negated), and the illness decision is made at the start of the phrase, while the negation decision depends on some word somewhere else. It's much better to have a lot of transforms to compose multiple models, if that lets you get an easier set of problems.
If you do decide to make the negation a text classification task, here's one way you can handle multiple illnesses. You could split the sentence up into multiple examples, so that you're predicting over spans of text that only have one illness. For instance, in:
You might split this into Tom had flu but, on the other hand, Tom did not suffer any and but, on the other hand, Tom did not suffer any left elbow injury because he did not fall. You could also use a windowed approach.
By the way, once you're dealing with only the negation case, you might find that a rule-based approach actually performs well. You could use the dependency parse to find which illness the negation word has a shorter path to, for instance. Even if there's 20 different ways people say the negation, that's really not that many rules to handle all the possibilities.