# Sub-label the existing labels

**URL:** <https://support.prodi.gy/t/sub-label-the-existing-labels/3548>\
**Category:** Uncategorized\
**Tags:** usage, ner, solved\
**Created:** [October 20, 2020, 9:50am UTC](https://support.prodi.gy/t/sub-label-the-existing-labels/3548 "2020-10-20T09:50:19Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [October 21, 2020, 9:36am UTC](https://support.prodi.gy/t/sub-label-the-existing-labels/3548/2 "2020-10-21T09:36:18Z")

</div>

Hi! By sub-label, do you mean, hierarchical categories? For example, if you have the label `LOCATION`, annotate whether the entity is `LOCATION_CITY` or `LOCATION_COUNTRY`, etc.? If so, one workflow could be to stream in your examples again with one entity at a time, and add multiple-choice options for the sub-labels. Then, all the annotator has to focus on is a single mention and a subset of sub-labels, so it should be really quick to annotate (and easy to evaluate, in case there are conflicts and disagreements).

To implement this, you could use a custom interface with [two blocks](https://prodi.gy/docs/custom-interfaces): `ner` (to render the entity) and `choice` (for the options). The stream could look something like this:

```python
options = [{"id": "LOCATION_CITY", "text": "LOCATION > CITY"}] # etc.

def get_stream(stream):
    for eg in stream:
        for span in eg.get("spans", []): # one example by annotated span
            yield {"text": eg["text"], "spans": [span], "options": options}

```

And then your blocks could look like this:

```python
blocks = [
    {"view_id": "ner_manual"},
    {"view_id": "choice", "text": None} # prevent text from being shown in both UIs
]

```

---

_[View the full topic](https://support.prodi.gy/t/sub-label-the-existing-labels/3548)._