Brand new AI provider Feedback | 全新 AI Provider 管理模块反馈

[arvinxx]

We released a brand new AI Provider management module in version v1.44.0, bringing a completely new experience for managing AI service providers. Based on extensive user feedback, we introduced the following in this new Provider version:

• Create custom service providers (supporting different SDK formats, such as multiple OpenAI format providers), create custom models;
• Capabilities and parameter customization for all models (currently only FC and Vision are available, more will be opened in the future based on community demand);
• Custom sorting by service provider, enable custom sorting of models;
• Supported full-type models at the framework level (language models, embedding models, TTS/STT models, text-to-image models). This framework enables us to support richer modal AI models in the future, such as text-to-video and music models.

As the new feature for the beginning of 2025, we expect this new module to encompass all possibilities for the coming year, making LobeChat the most versatile client in the AI Chat field~

Important

Note: This AI Provider module upgrade is only available under the server DB. Future iterations will support pglite mode. The dexie db will maintain its original implementation and will be completely removed in v2.0.

At the same time, the AI Provider page has undergone a complete rewrite from data storage to front-end implementation in order to achieve better performance and interaction experience. Therefore, at this stage, the logic related to the original environment variables may not be implemented or may have flaws. If you encounter any scenarios that do not meet your expectations, please provide feedback below, and I will strive to complete the fix within T+1 days.

Therefore, when this post is closed, if there are still no feedback on the environment variable issue, it will be removed as a deprecated feature in V2.

---

我们在 v1.44.0 版本发布了全新的 AI Provider 管理模块，带来了全新的 AI 服务商管理体验。结合诸多用户反馈，我们在这一全新的 Provider 版本中，带来了：

• 创建自定义服务商（支持不同 SDK 格式，例如多个 OpenAI 格式的服务商）、创建自定义模型；
• 所有模型的能力、参数自定义（目前仅开放 FC 和 Vision，未来将会根据社区诉求开放更多参数，例如模型是否支持流式等）；
• 服务商自定义排序、启用模型的自定义排序；
• 在框架层支持了全量类型的模型（语言模型、Embedding模型、TTS/STT 模型、文生图模型。这一框架使得我们未来也将能支持上文生视频、音乐模型等更丰富模态的 AI 模型。

Important

注意事项：本次 AI Provider 模块升级，只在服务端 DB 下可用，后续迭代会支持 pglite 模式。而 dexie db 仍保持原有实现，后续将在 v2.0 中完全移除。

由于AI Provider 这个页面从数据存储到前端实现是一次完全的重写，以期获得更好的操作性能和交互体验。因此数据上是完全独立的两套，我们不做任何迁移，升级后你需要自行重新配置对应的 Provider。同时现阶段的实现中，和原有 环境变量 相关的部分的逻辑可能没有实现或者存在瑕疵。如果遇到不符合你的预期场景，请在下方反馈，我会争取在当天评估是否对齐原有逻辑，如需对齐， T+1 天内完成对齐。

待本帖关闭时，如果仍然没有反馈的环境变量问题，也将作为废弃功能在 V2 中移除。

作为 2025 年的开年新特性，我们期望这个新的模块能承载未来一年的全部可能性，让 LobeChat 变成 AI Chat 领域最全面的 AI 客户端 🚀

[arvinxx]

环境变量中 _MODEL_LIST -all 功能失效，无法仅启用需要的模型
是否可以在环境变量里模型排序，之前是可以通过_MODEL_LIST里添加模型的先后顺序来排

@SpeedupMaster MODEL_LIST 环境变量会保持原样，今天会兼容上，之前什么样，现在也应该是什么样；

是否能在环境变量里添加自定义服务商？

不考虑在环境变量中添加自定义服务商，能配置的东西太少了，未来可能会考虑加一个全局范围的服务商设定，面向管理员来控制。

还有就是tts和embedding这种非聊天模型也会展示在聊天框的下拉模型列表里

@coulsontl 这个是 by design ，后续这个 ai provider 的页面还会加入文生图、文生视频等各类模型，provider 也不再仅局限于 chat 类的 provider。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

The *_MODEL_LIST -all function in the environment variable is invalid and cannot enable only the required models.
Is it possible to sort the models in the environment variables? Previously, it was possible to sort the models by adding them in *_MODEL_LIST.

@SpeedupMaster MODEL_LIST environment variables will remain the same and will be compatible today. What was before should be what it is now;

Is it possible to add a custom service provider in environment variables?

I don’t consider adding a custom service provider in the environment variables. There are too few things that can be configured. In the future, I may consider adding a global service provider setting for administrators to control.

In addition, non-chat models such as tts and embedding will also be displayed in the drop-down model list of the chat box.

@coulsontl This is by design. In the future, various models such as Vincent pictures and Vincent videos will be added to the AI ​​provider page. The provider is no longer limited to chat providers.

[gemnioo]

the cross-region inference in aws bedrock still makes user confused; the naming issues need to be addressed

Ref: #4659

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Looking forward to whether the function of environment variable preconfiguration and custom service provider will be provided in the future ❤️

[TonyBlur]

当前版本(1.45.0)还存在获取模型列表后，仍然会显示当前API默认的部分模型，但是实际/v1/models中并没有获取到的模型，且无法删除，如下图所示

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In the current version (1.45.0), after obtaining the model list, some of the default models of the current API will still be displayed, but the actual models are not obtained in /v1/models and cannot be deleted, as shown in the figure below

[ysicing]

@arvinxx Azure 好像也有问题

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

@arvinxx Azure seems to have a problem too

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

All the original model configuration information is gone. Can it still be restored? Lobech does not provide configuration export or backup functions. It is too unscientific to clear the configuration information every time there is a major upgrade.

[arvinxx]

原先的模型配置信息全都没有了，还能恢复吗？

你可以进入 /settings/llm 拿回之前的配置，也可以降级到之前版本继续使用。

lobech并不提供配置导出或备份功能，每次大升级就清空配置信息，这太不科学了

目前没有哪次是完全清空配置的，总会有让你恢复配置的手段。既然选择自部署了，就请接受自部署可能带来的不稳定因素。如果你要追求稳定性，完全可以用我们的Cloud版，我们会确保我们的付费用户配置不会清空。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

All the original model configuration information is gone. Can it still be restored? Lobech does not provide configuration export or backup functions. It is too unscientific to clear the configuration information every time there is a major upgrade.

You can enter /settings/llm to get back the previous configuration

[coulsontl]

在 v1.45.3 中，已经按照批量插入，不做已有配置项更新的逻辑来处理了。

那需要提供一个重置按钮了，我后面对接了一个OneAPI 经常会增加删除模型，现在只增不删了，一个一个手动删除很麻烦。如果能提供多选批量删除就更好了。建议删除可以使用软删除，这样如果不小心删了，下次点击刷新的时候可以再把之前的配置找回来不需要重新添加。

而且模型列表的加载好慢，大概270多个模型需要8秒，电脑是Mac M1性能也不差

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In v1.45.3, it has been processed according to the logic of batch insertion and no update of existing configuration items.

Then we need to provide a reset button. I connected to a OneAPI later and often added and deleted models. Now I can only add but not delete. It is very troublesome to delete manually.

Moreover, the model list is very slow to load. It takes about 8 seconds for more than 270 models. The computer is a Mac M1 and the performance is not bad.

[W3XA56Y3RDUKBS]

环境：

部署时间： 2025.01.09, 20:01 北京时间
使用docker部署数据库版
compose.yaml:

  lobe:
    image: lobehub/lobe-chat-database:latest
    container_name: lobe-chat
    ports:
      - '3210:3210'
    depends_on:
      - postgresql
      - minio
      - logto
    env_file:
      - .env
    restart: always

.env:

ENABLED_OLLAMA=0
OPENAI_MODEL_LIST=-all,+gpt-4o-mini=gpt-4o-mini,+gpt-4o-2024-11-20=gpt-4o,+claude-3-5-haiku-20241022=claude-3-5-haiku,+claude-3-5-sonnet-20241022=claude-3-5-sonnet,+deepseek-chat,+gemini-2.0-flash-exp,+flux-schnell=Flux 1 Schnell [画图]

版本显示为：

问题：

但是用户登录后，还是显示默认的几个openai的模型，我的OPENAI_MODEL_LIST没有生效。并且用户可以通过点击fetch获取全部的列表。请问如何操作可以使模型列表默认展示我在环境变量中设置好的列表？以及如何能禁止用户使用fetch models的功能？

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

environment:

Deployment time: 2025.01.09, 20:01 Beijing time
Deploy the database version using docker
compose.yaml:

  lobe:
    image: lobehub/lobe-chat-database:latest
    container_name: lobe-chat
    ports:
      - '3210:3210'
    depends_on:
      -postgresql
      - minio
      -logto
    env_file:
      - .env
    restart: always

.env:

ENABLED_OLLAMA=0
OPENAI_MODEL_LIST=-all,+gpt-4o-mini=gpt-4o-mini,+gpt-4o-2024-11-20=gpt-4o,+claude-3-5-haiku-20241022=claude-3 -5-haiku,+claude-3-5-sonnet-20241022=claude-3-5-sonnet,+deepseek-chat,+gemini-2.0-flash-exp,+flux-schnell=Flux 1 Schnell [Drawing]

The version is shown as:

question:

However, after the user logs in, several default openai models are still displayed, and my OPENAI_MODEL_LIST does not take effect. And users can get the entire list by clicking fetch. How can I make the model list display the list I set in the environment variable by default? And how to prohibit users from using the fetch models function?

[arvinxx]

So does hugging face.

Hf在1.45.4也修复了

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

So does hugging face.

Hf is also fixed in 1.45.4