Reduction of Sudoku output

[SpinTensor]

All BMs give only a single number as output for validating the computational result, except the Sudoku one, which spits out the solved sudokus several times. This might slow down the program, as the computations might wait on the return of the print functions. Maybe this should be reduced to one of the Sudokus only, to better measure the computational time?

[attractivechaos]

Yes, good point. Sudoku is the only benchmark here that repeats the same input. I worry compiler might optimize key functions out if there is no outcome, so force the programs to output. Perhaps we may sum over the middle number (i.e. out[41]). I may try this at some point.

[nsajko]

I worry compiler might optimize key functions out if there is no outcome, so force the programs to output.

Writing to /dev/null seems like it would be a better option than printing.

[SpinTensor]

Yes /dev/null helps because the frame-buffer of the terminal, or the filesystem does not need to answer to the program. I agree that the some output must be written, and preferably in a way that does not allow optimizations. I think the sum of a number from the sudoku would be good. Basically sum over out[41] for all 200 Sudokus. This way nothing can be dropped.