Joining chunks

[cubuspl42]

I have a utility GPT solution based on the non-streaming API (as it's simpler). At the same time, I wanted to preview the response as it's created, for multiple reasons. The easiest way to achieve that, without rewriting the upper layer, was creating this utility:

private fun joinChatCompletionChunks(
    chunks: Iterable<ChatCompletionChunk>,
): ChatCompletion {
    val firstChatCompletionChunk = chunks.first()
    val firstChatChunk = firstChatCompletionChunk.choices.first()
    val firstDelta = firstChatChunk.delta
    val firstFunctionCall = firstDelta.functionCall

    val lastChatCompletionChunk = chunks.last()
    val lastChatChunk = lastChatCompletionChunk.choices.first()

    val content = chunks.mapNotNull { it.choices.firstOrNull()?.delta?.content }.joinToString(separator = "")
    val arguments =
        chunks.mapNotNull { it.choices.firstOrNull()?.delta?.functionCall?.arguments }.joinToString(separator = "")

    return ChatCompletion(
        id = firstChatCompletionChunk.id,
        created = firstChatCompletionChunk.created,
        model = firstChatCompletionChunk.model,
        choices = listOf(
            ChatChoice(
                index = 0,
                message = ChatMessage(
                    role = ChatRole.Assistant,
                    content = content,
                    functionCall = firstFunctionCall?.copy(
                        argumentsOrNull = arguments,
                    ),
                ),
                finishReason = lastChatChunk.finishReason ?: FinishReason("unknown"),
            )
        ),
    )
}

It may be buggy, but it was implemented in a minute and works for my case.

Wouldn't something like this belong in openai-kotlin (in a bug-less version, preferably)?

[aallam]

This can be useful indeed!