Generic mcp.el for Emacs (Model Context Protocol)

[karthink]

@ahyatt, I wanted to check if you're working on a Model Context Protocol (MCP) adapter for llm. It was described by the author as "LSP but for LLM assistants". If support for it catches on, it should make adding context to LLM queries easier in a semi-standardized way.

Last time around, we ended up reinventing the wheel with llm/gptel/chatgpt-shell etc. I'm hoping we can write a generic mcp.el library this time that has no (or few) dependencies, and use it as appropriate in different Emacs LLM client UIs. Of course, this is only relevant if the open protocol catches on, since it's only supported by Anthropic and a few editors like Zed right now.

I was planning to look into writing something early next year, assuming people continue to write MCP servers. I haven't read the full spec yet, just have a rough idea so far. But if you already have some plans or are writing something like mcp.el, I'd be happy to help.

CC'ing @xenodium and @s-kostyaev for thoughts and input.

[xenodium]

Thanks for cc'ing me @karthink!

I've only just caught up adding multi-llm support to chatgpt-shell and did indeed re-invent the wheel :/

I'm hoping we can write a generic mcp.el library this time that has no (or few) dependencies

This sounds great.

assuming people continue to write MCP servers

Looks like it's moving fast already. Looking forward to seeing usages in practice.

I may give it a little time to settle and see how the general user experiences develops. For example, I've yet to add things like function calling... which I'm guessing gets obsoleted if MCP catches on? In the case of chatgpt-shell, it took a few iterations to get a comfortable user experience. I did add (and discard) plenty of things. I'm guessing this may have been the case for other folks building Emacs packages.

Either way, I'll be following this closely :)

[karthink]

I don't think function calling is obsoleted by MCP -- but I'm not sure yet. I'll try to make some time to read the spec soon, and come back with some concrete ideas.

I did add (and discard) plenty of things. I'm guessing this may have been the case for other folks building Emacs packages.

Yeah, gptel has a long list of abandoned branches. Things usually don't work how I imagine they will, unfortunately.

[ahyatt]

Thanks for starting this conversation! Is the idea to use Emacs as a LSP host, or an LSP server? An LSP server could be interesting, but emacs has too many "capabilities", and it isn't clear to me how to best use it as a server, aside from maybe basic buffer listing and buffer content commands. As an LSP host, it's more intersting - Emacs could discover and use various sources of data and presumably pass them to . Right now it's similar to what you can do with function calling, but should be easier with service discovery (instead of having to write your own).

This is something I'm interesting in, but don't have any plans to work on. I have been thinking of pursuing a similar sort of standardization for context building in the llm package, but that's purely internal - not following the LSP spec but getting some inspiration from it. I'll mention here if I start doing something like that.

I do think that if this happens, and it needs to call out to llms, that should be done via the llm package, since it really is just a unopinionated package for abstracting amonst llms, without any UI components. It's also in GNU ELPA, which may be important to some, but unfortunately it also means that unlike other packages the llm package has to warn users about non-free software.

At any rate, let's keep this thread alive so we can coordinate and either work together or at least not do duplicate work!

[karthink]

Thanks for starting this conversation! Is the idea to use Emacs as a LSP host, or an LSP server?

LSP host, I think.

I do think that if this happens, and it needs to call out to llms, that should be done via the llm package, since it really is just a unopinionated package for abstracting amonst llms, without any UI components.

I don't think MCP involves any LLM calls at all. It returns partial or full prompts to the host (Emacs), which sends them to the LLM. There's more to it than that, I'll read the spec when I have more time and come back here with my findings. If you get around to reading it, please let us know as well!

[xenodium]

I have been thinking of pursuing a similar sort of standardization for context building in the llm package

While I've added some helpers to facilitate manually building context (usually feeding into M-x chatgpt-shell-prompt-compose), this is still an area of friction.

I've been tempted to prototype a smart context core of sorts, yet it feels like an entirely new/separate project, which I may not have the time for. Possibly run as a separate process/daemon. Maybe if I wait long enough, it gets obsoleted/solved by mcp 😅

[ahyatt]

Yes, a unified way to create context would be useful! What I did in the llm package (in llm-prompt) is to allow context generators to be either strings, lists, or generators for context, and it also allows filling context both forward and backward. It may require even more, such as filling inside-out (think of feeding in an org tree surrounding a target headline). I think all that is necessary, and I could separate that out into another project that isn't tied to the llm package.

My hope was to also bundle that with standard contexts that are useful, like one for org agenda, the org tree of the currently clocked in item, the current file (inside-out would help here too).

One other thing I'd like to see happen is to have a way to build up more complicated flows. I wrote something for this using state machine (the fsm package), and it does work, but I need to use it for more things and refine it before I'm confident enough to release it. I'd be happy not doing that and using a new module if someone writes one. One thing that would be useful is to also use it for context building, such as building up context by feeding it a project's file tree and let the LLM choose what is the best file to fill in the context. I saw a post yesterday on a similar technique: https://arcturus-labs.com/blog/2024/11/21/roaming-rag--make-_the-model_-find-the-answers/, but in the context of a single document.

OK, yet a third thing I'd like to see happen: a unified way of getting an LLM to interact with a buffer in a way that is natural to emacs. That means having it comment on text via overlays in a way the user can see and dismiss or act on. Even better, having a way to show its suggested changes. I know that gptel can do this via ediffs, but IIUC, no one, including gptel has developed a way to do this that isn't based on feeding a whole buffer or part of a buffer in and getting the altered text back. A solution might be to allow the LLM to actually perform edits (which could be real edits or show up as suggestions) via function calling, but I haven't figured out a good way to do that yet.

[karthink]

allow context generators to be either strings, lists, or generators for context, and it also allows filling context both forward and backward. It may require even more, such as filling inside-out (think of feeding in an org tree surrounding a target headline).

This sounds interesting. I'll check out how llm collects context.

This is also well ahead of what gptel does here, which is nothing at all. You can always write bespoke functions to do these things and give them to gptel, and it will run them and add the context. But there is no formal or informal spec for adding context chunks.

My hope was to also bundle that with standard contexts that are useful, like one for org agenda, the org tree of the currently clocked in item, the current file (inside-out would help here too).

My experience here has been that no matter what I think is a standard set, I find users asking to use it, or using it, in ways I would have never considered. For now I've given up on the idea of providing standard "context generators" with the package.

One other thing I'd like to see happen is to have a way to build up more complicated flows. I wrote something for this using state machine (the fsm package), and it does work, but I need to use it for more things and refine it before I'm confident enough to release it

Similar situation here -- I have a state machine driving the requests in a private branch in gptel, but it doesn't work well and the design isn't clear yet. I don't currently have the time for anything beyond bug fixes on gptel.

One thing that would be useful is to also use it for context building, such as building up context by feeding it a project's file tree and let the LLM choose what is the best file to fill in the context

Yup, chaining LLM requests was my primary reason for the fsm approach as well, but integration with other sync/async processes has suggested itself as a use case since.

I know that gptel can do this via ediffs, but IIUC, no one, including gptel has developed a way to do this that isn't based on feeding a whole buffer or part of a buffer in and getting the altered text back.

This is correct. gptel's in-place suggestion/rewrite UI got a little facelift recently though, there are some demos in this reddit thread.

A solution might be to allow the LLM to actually perform edits (which could be real edits or show up as suggestions) via function calling, but I haven't figured out a good way to do that yet.

I have a basic question about how this would work. I'm assuming you have to do one of the following:

1. Send the whole buffer + additional context, and get it to output location-specific suggestions/annotations, like a linter/editor does.
2. Make separate requests in parallel, each involving small chunks, and collect suggestions.

The former option will probably produce better quality results. But are LLMs any good at providing a list of locations and suggested changes in a structured format? I imagine that only the highest tier models can probably manage it. This seems like the kind of workflow that would break often and unexpectedly.

[xenodium]

My hope was to also bundle that with standard contexts that are useful, like one for org agenda, the org tree of the currently clocked in item, the current file (inside-out would help here too).

The likes of flymake and friends (spelling, grammar? ) can have neat integration points.

I have a very basic integration for fixing errors. It's smerged-based, with styling/tweaks.

a unified way of getting an LLM to interact with a buffer in a way that is natural to emacs. That means having it comment on text via overlays in a way the user can see and dismiss or act on.

There may be an opportunity here to build separate packages, applicable beyond the LLM space.

1. Generic infrastructure to interact/mutate buffers (UX-agnostic).
2. UX that plugs into 1. to present mutations to users, requiring intervention. User can plug in their favorite UX.

Both relate to each other how completing-read relates to vertico, helm, ivy...

I like what @karthink did with this interactive merge/insertion

I can see this being applicable beyond LLM interactions, and handy for package developers if a simple API for interactively applying changes is exposed.

[ahyatt]

Great points, I completely agree that breaking down into individually useful packages like this makes a lot of sense. I also really like that @karthink UI you show, that's pretty ideal IMHO, for many things much easier than a merge-driven UI. If think if we want to go in that direction, we should be submitting those kinds of packages to GNU ELPA, and probably discussing the general thinking in emacs-devel@.

[karthink]

I also really like that @karthink UI you show, that's pretty ideal IMHO, for many things much easier than a merge-driven UI. If think if we want to go in that direction, we should be submitting those kinds of packages to GNU ELPA, and probably discussing the general thinking in emacs-devel@.

This UI is not part of gptel -- it is provided by the work-in-progress inline-diff package, which is generic. It actually implements the Myers diff algorithm in elisp, and does not call out to diff.

[karthink]

Tagging @lizqwerscott, who has written mcp.el. So far it supports providing tools via MCP.

@lizqwerscott: Thanks for writing mcp.el. In this thread we are trying to coordinate so we can use a generic MCP Emacs package from different LLM clients: mainly chatgpt-shell, the llm library + ellama, and gptel. We can discuss integration aspects here. Your work looks like a great start!

I'm looking forward to trying it, hopefully in the coming week.

[lizqwerscott]

Yes, @karthink you're correct. mcp.el is essentially a jsonrpc client similar to eglot.

I agree with this plan.

[lizqwerscott]

The MCP server supports some asynchronous tool invocations.

For example, the longRunningOperation tool in MCP test server sends a request to invoke the tool and provides a progress update after some time while waiting for the operation to complete.

I believe we first need to define an interface to implement the following actions:

1. Allow gptel or llm to access the MCP server's resources, prompts, and tools.
2. Enable synchronous or asynchronous tool invocation.

Example of a Tool Invocation

For implementing a tool to create a file, we can define an Elisp function like this:

(lambda (path filename content)
  (let ((full-path (expand-file-name filename path)))
    (with-temp-buffer
      (insert content)
      (write-file full-path))
    (format "Created file %s in %s" filename path)))

Connecting to the MCP Server

For the current mcp.el, first connect to the MCP server (MCP File Server):

(mcp-connect-server "filesystem"
                    "npx"
                    '("-y"
                      "@modelcontextprotocol/server-filesystem"
                      "/home/lizqwer/Downloads/"))

Invoking the Tool

Then, invoke the tool:

(let ((connection (gethash "filesystem" mcp-server-connections)))
  (mcp-call-tool "write_file" '(:path "filename or file path" :content "the file content")))

[ahyatt]

If mcp needs to discover tools, that's something only it will be able to do. But if it needs to then use those tools, it shouldn't be providing the integrations to do tool use on its own. Would the idea be that it doesn't call to the various providers itself for tool use, it depends on some client to make those calls using the client's preferred library? That's possible, but I'd guess that it'd likely be useful for mcp to be involved in the tool use as well. But it'd be bad to do it itself, tool use is complicated, and the less clients that have to implement this themselves, the better.

Ideally, I don't think that gptel and llm should be independent; gtpel should depend on llm, as should chatgpt-shell. It's not a good use of our time to each have to deal with all these servers, and the more everyone can standardize on one library to handle the work of abstracting these capabilities, the better. That's why I made the llm library. Users shouldn't have to reason about, for example, which servers are supported and which ways of specifying those models are needed by ellama, GPTel, chatgpt-shell, or, now mcp.el, it should be a unified interface, something that feels like part of emacs. I'm comfortable with the fact that we aren't there now, but I'd like us to get there eventually.

[karthink]

Would the idea be that it doesn't call to the various providers itself for tool use, it depends on some client to make those calls using the client's preferred library? That's possible, but I'd guess that it'd likely be useful for mcp to be involved in the tool use as well. But it'd be bad to do it itself, tool use is complicated, and the less clients that have to implement this themselves, the better.

Going by this chart, it looks like there's a pretty good separation of concerns?

llm handles the LLM <--> Client interactions, mcp handles the Client <--> Server interactions, and llm includes some glue code to interface with mcp:

• If the use of mcp in llm is enabled, then llm uses mcp to obtain the list of tools to send (or appends it to the list of llm-specific tools)
• llm handles the interaction with the LLM, which may reply with a tool call.
• llm calls an mcp function (mcp-call-tool?) with the arguments provided by the LLM. If this interaction is asynchronous, it uses the callback method we use in Standard function-call data spec for Emacs LLM clients #124.
• llm sends the results to the LLM if required.

Some shim code is required in llm to (i) obtain the list of available tools from mcp and (ii) call the tool via mcp. At no point does mcp communicate with the LLM, so it doesn't need to (re)implement any of the LLM API code that llm does.

---

Ideally, I don't think that gptel and llm should be independent; gtpel should depend on llm, as should chatgpt-shell. It's not a good use of our time to each have to deal with all these servers...
[...]
Users shouldn't have to reason about, for example, which servers are supported and which ways of specifying those models are needed by ellama, GPTel, chatgpt-shell, or, now mcp.el

It's too late to replace gptel's LLM API code with llm's right now, in that it would be a lot of effort with no short term benefits, with the short term detriment of creating a new collection of bugs, and a long term detriment of losing the hard-earned fixes to edge cases over the past two years. I agree with the long term goal though -- at some point the maintenance burden will make this option more palatable, and we can come back to it then.

Right now, my hope is just that chatgpt-shell, llm and gptel can share the design of future features, and re-use modules like mcp as much as possible. This will make users' lives easier, and also make the eventual task of switching the LLM API backend code less onerous.

it should be a unified interface, something that feels like part of emacs

👍 I like how eglot integrates with many of Emacs' capabilities. I don't know what the equivalent of that is for LLM usage, because there's no equivalent to a well-defined language server protocol, but this is what I'm experimenting with on the UI side in gptel.

[lizqwerscott]

I added asynchronous tool calls Add Asynchronous Calls and no longer directly call gptel-make-tool.

Instead, it generates a plist. Like #124

You can refer to the documentation.