Maybe move the utility methods out of TextTrainer?

[matt-gardner]

You could make the argument that the way data is handled and the way we build models are too tightly coupled, and should be decomposed. That would mean, basically, making a cleaner separation between the objects that read and process data and TextTrainer, and perhaps also splitting out the _embed_input, _get_encoder, and _get_seq2seq_encoder methods into a separate model utility class.

I'm not totally sold that this is necessary, though. In order to make the handling of word / word+character tokenizers transparent to the model class, you have to have a tight coupling between the data generator and the _embed_input method. I think it would be pretty difficult to make this work without the way that it's currently structured.