I would like to use the last layer of textcat and use that as a high dimensional representation of the text (and then apply TSNE/UMAP). How can I extract it? Thanks, you guys are the best!
The code in this thread might be useful, especially my reply about getting the attention weights: Highlighting the matching words for text classfication
The general approach is to attach a wrapper around some layer of the text classifier’s model. In the wrapper, you can capture inputs and outputs and write them somewhere, before communicating through the wrapped layer. This is a general solution for getting access to any of the internal representations in one of the models.