# Extracting useful information from Job description
**URL:** https://support.prodi.gy/t/extracting-useful-information-from-job-description/6255
**Category:** Uncategorized
**Tags:** textcat, spancat, ner
**Created:** [January 20, 2023, 3:23pm UTC](https://support.prodi.gy/t/extracting-useful-information-from-job-description/6255 "2023-01-20T15:23:29Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Aditya\_varma\_10](https://support.prodi.gy/u/Aditya_varma_10)
#### Post date: [January 20, 2023, 3:23pm UTC](https://support.prodi.gy/t/extracting-useful-information-from-job-description/6255/1 "2023-01-20T15:23:29Z")
Hey,
I am trying to extract some information from job descriptions such as company name, location, responsibilities, required skills etc.
I have gone through the post
[https://support.prodi.gy/t/parsing-identifying-sections-in-job-descriptions/1100](https://support.prodi.gy/t/parsing-identifying-sections-in-job-descriptions/1100)
Where it is suggested to use a text classifier to classify sentences or paragraphs and ner on top of that.
I have few doubts regarding this approach.
1. In order to classify sentences should I write seperate script to divide the whole document into sentences and create the dataset where each sentence in the document is single input for text classification to annotate using prodigy .
2. Is it better to use spancat instead of above approach to annotate sentence as both spans and entities.
I tried training a spancat model on whole document and the spans extend from 1 word to 20 words per span. I tried training the model and observed that training is taking too long to complete even after gpu.
Is there a better way to speed up the training process?
If the training is taking too long is there a way to resume the training from last checkpoint in case the training is stopped before completion.
---
### Author:  [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)
#### Post date: [January 24, 2023, 6:52pm UTC](https://support.prodi.gy/t/extracting-useful-information-from-job-description/6255/2 "2023-01-24T18:52:12Z")
hi @Aditya_varma_10!
Thanks for your question and welcome to the Prodigy community 👋
> [@Aditya\_varma\_10](#):
>
> In order to classify sentences should I write seperate script to divide the whole document into sentences and create the dataset where each sentence in the document is single input for text classification to annotate using prodigy .
Typically, yes. The [docs](https://prodi.gy/docs/text-classification#whole-document) cover this:
> If your documents are longer than a few hundred words each, we recommend applying the annotations to smaller sections of the document. Often paragraphs work well. Breaking large documents up into chunks lets the annotator focus on smaller pieces of text at a time, which helps them move through the data more consistently. It also gives you **finer-grained labels** : you get to see which paragraphs were marked as indicating a label, which makes it much easier to review the decisions later.
It's important to know that manual recipes do **not** do sentence segmentation be default, `teach` and `correct` recipes will do sentence segmentation by default. You can also use [`split_sentences`](https://prodi.gy/docs/api-components#split_sentences) too.
> [@Aditya\_varma\_10](#):
>
> I tried training a spancat model on whole document and the spans extend from 1 word to 20 words per span. I tried training the model and observed that training is taking too long to complete even after gpu.
>
> Is there a better way to speed up the training process?
What [suggester function](https://prodi.gy/docs/span-categorization#suggesters) are you using? If you're using the default `ngrams` then yes, this can drastically slow down training, especially on very long text.
Also this post suggests ways to break down the problem:
> [@spancat with really large spans? (Identify sections in text)](https://support.prodi.gy/t/spancat-with-really-large-spans-identify-sections-in-text/6094/5):
>
> I found a dialogue on the forum that might be inspirational here. It's a different problem, but it highlights another two-step approach to rethinking spans. That said, reading your reply still makes me think that textcat might be the simplest way forward, albeit on paragraphs instead of sentences. While I like your idea of using NER to detect the start of a section, I wonder if you might be able to leverage that this always starts on a newline, which suggests a heuristic might be better than…
Also, try running `data debug` (e.g., after running `data-to-spacy`) to get stats about your spans lengths:
> **[What is the maximum length the annotation of a span can be? · explosion/spaCy...](https://github.com/explosion/spaCy/discussions/11450#discussioncomment-3592377)**
>
> Hi, I am trying to train a span categorizer but since I'm new to it, I am a bit unsure how to annotate my data. I'm using the spancat as a parser to extract specific sections from a document. Somet...
> [@Aditya\_varma\_10](#):
>
> Is it better to use spancat instead of above approach to annotate sentence as both spans and entities.
Sorry, I don't understand this comment. Are you asking whether it is better to use `spancat` instead of `ner`? If so, the answer is it "depends".
(Also, that post was originally from 2019 and was before `spancat` was introduced.)
There's no hard rule but the [Prodigy docs](https://prodi.gy/docs/span-categorization#ner-vs-spancat) give you some idea:
| Named Entity Recognition | Span Categorization |
| --- | --- |
| spans are non-overlapping syntactic units like proper nouns (e.g. persons, organizations, products) | spans are potentially overlapping units like noun phrases or sentence fragments |
| model predicts single token-based tags like `B-PERSON` with one tag per token | model predicts scores and labels for suggested spans |
| takes advantage of clear token boundaries | less sensitive to exact token boundaries |
A good rule of thumb is if the meaning of your spans changes if you were to rearrange the words, then use `ner`. Alternatively, if you have more flexibility that you can rearrange the words within a span and it can have generally the same meaning, then `spancat` may work better.
Also, if you haven't seen, we have a [case study project](https://github.com/explosion/projects/tree/v3/experimental/ner_spancat_compare) where we compared `ner` vs. `spancat` performance.
> [@Aditya\_varma\_10](#):
>
> If the training is taking too long is there a way to resume the training from last checkpoint in case the training is stopped before completion.
Yes, see this post:
> **[Is It Possible to Resume Training Via CLI in Spacy v3 (transformers)? ·...](https://github.com/explosion/spaCy/discussions/8176)**
>
> I was training a ner model with a config file and train and validation set , I trained for 10 epochs and model-best and model-last got saved and I use those model for inference. python -m spacy tra...
FYI, for spaCy specific questions (e.g., training, compute, GPU), I recommend searching/using the [spaCy GitHub discussions forum](https://github.com/explosion/spaCy/discussions).