What is the meaning of the long_text flag?

usage
textcat
#1

Hi there,

I couldn’t really grep what is the meaning of the long_text flag I see around the model - does it affect the UI, sentence splitting, or the model itself?

Thanks,
Beka

(Matthew Honnibal) #2

In full hindsight, I think it might have been better to handle this in the pre-processing, instead of having it as an argument in the model.

The long_text argument applied sentence splitting, and then asks questions about each sentence. For evaluation, it runs the model over each sentence in the document, and sums the scores.

You can achieve the same thing easily enough by modifying the data or the recipe, which I would probably recommend.

#3

Thanks for the quick reply!