Missing Generator::score_batch-like function for perplexity calculation in ctranslate2-rs

[gn64]

Hello,
First of all, thank you so much for providing this wonderful library. It has been incredibly helpful for various use cases, and I appreciate all the work that went into creating and maintaining it.

I’m currently trying to calculate perplexity with ctranslate2-rs for a GPT-like model by using the Generator class. However, I cannot find any function equivalent to the score_batch method from the original CTranslate2 library (specifically for the Generator). The ctranslate2-rs crate seems to focus primarily on translation tasks, and I can’t seem to find any scoring or perplexity-related methods for the Generator.

Is there currently any way to compute perplexity in ctranslate2-rs, or is this functionality missing? If so, are there plans to add a score_batch-like function to the Generator class? Any guidance or workarounds for GPT-like perplexity calculations would be greatly appreciated.

[jkawamoto]

Currently, scoring functions haven't been implemented yet. However, it seems like adding score_batch to Generator wouldn't be difficult, and I'll add them soon.

[jkawamoto]

Generator::score_batch is now available as of v0.9.6.

Here is an example of its usage:

ctranslate2-rs/examples/gpt-2.rs

Lines 101 to 103 in 578de58

	// Scoring the prompts.
	let scores = g.score_batch(&[prompts], &ScoringOptions::default())?;
	println!("{:?}", scores[0]);