Hi @jrfernandez,
Your approach does sound reasonable, although I can think of other ways to do it too. For instance, you might try going back in time a bit further to get the more bearish-sentiment comments. You could also collect into one set, and then do the rebalancing separately. This might be good if you can get the model helping you to select a more balanced sample. Specifically, the active learning is often pretty good in this type of text classification task, especially if you use a terminology list to help bootstrap it.
The Insults Classifier video was one of the first videos we put up, so it refers to an earlier version of Prodigy -- so the specific commands might be slightly different now. You might still find it helpful to get an overview of the workflow, though.