Isn't the thing with transformers is that they work really well, but only if you have a lot of training data? If data is sparse, you need more specialized models