Interesting idea. If a person sits down and writes a bunch of essays, that usually leads to improved language skills. Maybe hallucinating additional training data could be something somewhat similar for transformer networks.