Questions regarding the pre-trained models

[SargamMenghani]

Hello, we had a few questions regarding the model weights that have been provided:

1. We tried to evaluate the Lakh400kPretrainOnly model (using the reproduce_paper_eval.sh script) and got the following results (because we ran this using reproduce_paper_eval.sh, the validation and test sets that we are using are from the nesmdb dataset):

valid loss  1.86 | valid ppl  6.448 | test loss  1.71 | test ppl  5.541 

Can you please confirm that these match with what you were seeing?

2. We also tried to generate a few samples using the Lakh400kPretrainOnly model. Out of the 25 samples that we generated, 15 samples only had the "WT" and "NO" channel notes and no "P1"/"P2"/"TR" channel notes. Do you have an intuition as to why this might be happening?

3. We are also having trouble in generating chiptunes from the LakhNES model. The chiptunes don't sound correct to us and we think that we might be missing some steps in the generation process or performing a step incorrectly. Here are the steps that we followed for generating the chiptunes (if you can point to us anything that we are missing/doing incorrectly in this process, it will help us a lot.) :
   a) we downloaded the LakhNES model from the link provided in the repo
   b) we ran python generate.py model/pretrained/LakhNES/ --out_dir ./generated/LakhNES --num 25 (this ran without any errors)
   c) we used tx1_to_midi function (in tx1_midi.py script) to convert the generated tx1 files to midi (we then used the timidity software to listen to these midi files)

If you are able to provide the 775k LakhNES data (by LakhNES data we mean the LakhMIDI examples mapped to NES channels) that you used for pre-training, it will help us a lot. We have generated our own 775k LakhNES examples using your scripts but because there is some randomness involved in mapping the instruments, we are not entirely sure if what we have matches with what you had used.

Thank you in advance!

[ruohoruotsi]

Hi there,
using the Lakh400kPretrainOnly, I also got similar results to you (using the ./reproduce....sh script)

====================================================================================================
| valid loss  1.86 | valid ppl     6.438 | test loss  1.72 | test ppl     5.601 
====================================================================================================

I'm about to use it to generate some some samples, so I don't have responses to 2. or 3. yet.