Flow utilities

[ahyatt]

Hi all,

I've been thinking of improving the kind of interactions we can have with emacs and LLMs. I've written llm-flows to demonstrate the possibilities. Right now there's just one possible thing, but it's pretty nice and general: a way to replace content in a buffer, but be able to look at the diff before replacing it, and optionally revise by sending commands to the LLM asking for certain changes.

I'm curious what others think (especially @s-kostyaev). I'm also toying with the idea of making this more of a state machine-based approach. And I don't know where to put this - in this llm repository, or in a new repository? But I need to experiment more with different uses of this. Any feedback or thoughts would be welcome.

Here's an example of how I use it:

;; A test example

(defun ash/llm-correct-text (start end)
  (interactive "r")
  (llm-flows-replace-region ash/llm-openai
                         (make-llm-chat-prompt
                          :context
                          "You are an experienced editor that specializes in clear and correct communication.  You will be given a passage of text.  Return only a rewrite of that passage,  correct in both grammar and spelling, and clearer in meaning.  Do not return any other commentary other than the rewrite of the passage, which will be replacing the original text.  Keep the formatting style of the original text.  For example, if the text is in org-mode format, keep the formatting in the org-mode style instead of using markdown."
                          :interactions (list (make-llm-chat-prompt-interaction :role 'user
                                                                                :content (buffer-substring-no-properties start end))))
                         "Text fixer"))

[s-kostyaev]

Hi @ahyatt. I will try it later, but idea to generate diffs that user can approve is great. But heavy local models can do it for a long time. Can we to improve user experience show generation stream in overlay or in child frame for example? This should be optional. After generation done it should hide and user should see the diff. It just ideas now, no need to rush implementation now, but we can discuss it.

About repositories - as you wish. It doesn't matter to me. While it's experiment I'd leave it in separate repository.

[ahyatt]

Yeah, one of the issues is that I couldn't think of a good way to have the UI show the streaming output. Thanks for getting me thinking about it, let me consider your ideas and see what I can come up with.

[r0man]

Hi @ahyatt,

what's the state of the flow branch? It looks very interesting. I read your posts on Emacs devel/Reddit about this. Are you happy with the fsm based approach? I'm thinking of giving this a try for an extension to Magit that suggests a message when committing files.

[ahyatt]

The approach looks like a good one. I'd like to wrap what is there in macros, though, to make it easier to work with. It also makes sense to integrate function calling into it somehow, but I haven't done so. If you want to experiment, go ahead and experiment against the flow branch and let me know how it goes, that can help me refine it to a releasable version. I think it may take many weeks before I get it to that point, though.

[r0man]

Ok, will do. Thanks for your answer.