[Bug] 插件无法使用、无法语音输入！

[JoeChen2me]

📦 Environment

Vercel, Docker

📌 Version

v.1.4.5.0

💻 Operating System

Ubuntu

🌐 Browser

Edge

🐛 Bug Description

插件问题

我在阿里云服务器通过docker compose的方式部署了 数据库版本的LobeChat。
部署之后，我通过服务器的公网IP：3210端口的方式进行访问，访问是正常的。可以进行对话和文件上传、向量化等操作。
但是无法使用插件功能。存在两种情况的报错

• LLM的回复出现报错,提示信息为

{
  "error": {
    "message": "[GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:streamGenerateContent?alt=sse: [400 ] Invalid JSON payload received. Unknown name \"default\" at 'tools[0].function_declarations[0].parameters.properties[1].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"default\" at 'tools[0].function_declarations[0].parameters.properties[3].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"default\" at 'tools[0].function_declarations[0].parameters.properties[4].value': Cannot find field. [{\"@type\":\"type.googleapis.com/google.rpc.BadRequest\",\"fieldViolations\":[{\"field\":\"tools[0].function_declarations[0].parameters.properties[1].value\",\"description\":\"Invalid JSON payload received. Unknown name \\\"default\\\" at 'tools[0].function_declarations[0].parameters.properties[1].value': Cannot find field.\"},{\"field\":\"tools[0].function_declarations[0].parameters.properties[3].value\",\"description\":\"Invalid JSON payload received. Unknown name \\\"default\\\" at 'tools[0].function_declarations[0].parameters.properties[3].value': Cannot find field.\"},{\"field\":\"tools[0].function_declarations[0].parameters.properties[4].value\",\"description\":\"Invalid JSON payload received. Unknown name \\\"default\\\" at 'tools[0].function_declarations[0].parameters.properties[4].value': Cannot find field.\"}]}]"
  },
  "provider": "google"
}

• 另外一种报错信息，是提示manifest请求失败，需要配置跨域访问

语音输入问题

打开语音输入一直提示润色中，但是没有后续反应。请问如何解决

但是在我Vercel上所部署的客户端版本，可以实现插件+语音输入。
是否是我阿里云服务器的端口设置有问题？导致端口被拦截了呢。

📷 Recurrence Steps

1. 安装插件 出现安装失败。提示配置manifest
2. 使用已预装的插件会提示如下报错

{
  "error": {
    "message": "[GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:streamGenerateContent?alt=sse: [400 ] Invalid JSON payload received. Unknown name \"default\" at 'tools[0].function_declarations[0].parameters.properties[1].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"default\" at 'tools[0].function_declarations[0].parameters.properties[3].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"default\" at 'tools[0].function_declarations[0].parameters.properties[4].value': Cannot find field. [{\"@type\":\"type.googleapis.com/google.rpc.BadRequest\",\"fieldViolations\":[{\"field\":\"tools[0].function_declarations[0].parameters.properties[1].value\",\"description\":\"Invalid JSON payload received. Unknown name \\\"default\\\" at 'tools[0].function_declarations[0].parameters.properties[1].value': Cannot find field.\"},{\"field\":\"tools[0].function_declarations[0].parameters.properties[3].value\",\"description\":\"Invalid JSON payload received. Unknown name \\\"default\\\" at 'tools[0].function_declarations[0].parameters.properties[3].value': Cannot find field.\"},{\"field\":\"tools[0].function_declarations[0].parameters.properties[4].value\",\"description\":\"Invalid JSON payload received. Unknown name \\\"default\\\" at 'tools[0].function_declarations[0].parameters.properties[4].value': Cannot find field.\"}]}]"
  },
  "provider": "google"
}

3. 使用语音输入会卡在润色中之后无反应。

上述情况目前百分百复现。

🚦 Expected Behavior

1. 可以正常安装和使用插件
2. 可以正常使用语音输入服务

📝 Additional Information

请问是否是端口配置的问题呢？
我目前开放了3210、8000、9000端口。是否还需要开放其他端口？

[lobehubbot]

👀 @JoeChen2me

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[dosubot]

@JoeChen2me 你好！我是Dosu，一个能够帮助解决bug、回答问题并成为贡献者的机器人。我会在等待人类维护者的过程中帮你解决问题。

The issue you're experiencing with the 'Invalid JSON payload received' error when using GoogleGenerativeAI is likely due to unsupported JSON schema properties such as 'default' in the request payload. These fields are not supported by the Google Gemini model, leading to the error. To resolve this, you can disable the plugins/function calls feature, which will prevent the problematic function declarations causing the 400 Bad Request error. Additionally, it is recommended not to enable plugins for the Gemini model due to its poor Tools Calling capabilities ^[1].

Regarding the CORS issue for manifest requests, you may need to configure your server to allow cross-origin requests. Unfortunately, specific CORS settings were not found in the repository, so you might need to check your server configuration or consult additional resources for setting up CORS.

For the voice input issue, ensure that all necessary ports are open. Besides the ports you've already opened (3210, 8000, 9000), you should also open the following ports for full functionality, including plugins and voice input:

• MINIO_CONSOLE_PORT: 9001
• ZITADEL_PORT: 8080
• POSTGRESQL_PORT: 5432

These ports are necessary for the operation of LobeChat and its components ^[2][3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[SpeedupMaster]

• 插件安装报错是由于被墙了，开代理可以解决
• 插件使用报错是gemini的function call的问题

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• The plug-in installation error is caused by being blocked, which can be solved by opening a proxy.
• The error reported when using the plug-in is due to the function call of gemini.

[Richardallis411]

Em

…

-- Richard Allison

On January 8, 2025 at 5:06:49 PM, LobeHub Bot ***@***.***) wrote: Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿 ------------------------------ - The plug-in installation error is caused by being blocked, which can be solved by opening a proxy. - The error reported when using the plug-in is due to the function call of gemini. — Reply to this email directly, view it on GitHub <#5353 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BM72DGHVDKJVZTX2IMIR5Q32JXDSTAVCNFSM6AAAAABU3CZEKKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZYHE4TCOBSG4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[JoeChen2me]

• 插件安装报错是由于被墙了，开代理可以解决
• 插件使用报错是gemini的function call的问题

你好，请问描述的开代理可以解决指的是 给我的阿里云服务器上代理还是我访问的机器本身开代理呢? 是否有具体的走代理的Domain规则，我想通过添加rules的方式来为代理软件指定使用代理服务。 @SpeedupMaster

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• The plug-in installation error is caused by being blocked, which can be solved by opening a proxy.
• The error reported when using the plug-in is a problem of gemini’s function call.

Hello, may I ask if the description of "opening a proxy can solve the problem" refers to "a proxy for my Alibaba Cloud server" or "a proxy for the machine I visit"? Is there a specific Domain rule for proxying? I would like to pass Add rules to specify the use of proxy services for the proxy software. @SpeedupMaster

[JoeChen2me]

@JoeChen2me 你好！我是Dosu，一个能够帮助解决bug、回答问题并成为贡献者的机器人。我会在等待人类维护者的过程中帮你解决问题。

The issue you're experiencing with the 'Invalid JSON payload received' error when using GoogleGenerativeAI is likely due to unsupported JSON schema properties such as 'default' in the request payload. These fields are not supported by the Google Gemini model, leading to the error. To resolve this, you can disable the plugins/function calls feature, which will prevent the problematic function declarations causing the 400 Bad Request error. Additionally, it is recommended not to enable plugins for the Gemini model due to its poor Tools Calling capabilities [1].您在使用 GoogleGenerativeAI 时遇到的“收到无效 JSON 负载”错误的问题可能是由于请求负载中不支持的 JSON 架构属性（例如“默认”）造成的。 Google Gemini 模型不支持这些字段，从而导致错误。要解决此问题，您可以禁用插件/函数调用功能，这将防止有问题的函数声明导致 400 Bad Request 错误。另外，由于Gemini模型的工具调用能力较差，建议不要启用插件[1] 。

Regarding the CORS issue for manifest requests, you may need to configure your server to allow cross-origin requests. Unfortunately, specific CORS settings were not found in the repository, so you might need to check your server configuration or consult additional resources for setting up CORS.关于清单请求的 CORS 问题，您可能需要配置服务器以允许跨域请求。不幸的是，在存储库中找不到特定的 CORS 设置，因此您可能需要检查服务器配置或查阅其他资源来设置 CORS。

For the voice input issue, ensure that all necessary ports are open. Besides the ports you've already opened (3210, 8000, 9000), you should also open the following ports for full functionality, including plugins and voice input:对于语音输入问题，请确保所有必要的端口均已打开。除了您已经打开的端口（3210、8000、9000）之外，您还应该打开以下端口以获得完整功能，包括插件和语音输入：

• MINIO_CONSOLE_PORT: 9001  MINIO_CONSOLE_端口：9001
• ZITADEL_PORT: 8080  CITADEL_端口：8080
• POSTGRESQL_PORT: 5432  POSTGRESQL_端口：5432

These ports are necessary for the operation of LobeChat and its components [2][3].这些端口对于 LobeChat 及其组件的运行是必需的[2] [3] 。

To continue talking to Dosu, mention @dosu.继续跟Dosu说话，提一下。

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other帮助Dosu学习！给予反馈：反应很好|不相关的答案|错误来源|太冗长|幻觉|其他

已经开放了9001 8080 5432端口。能否详细说明CROS跨域配置的问题。
我目前部署在云服务器上，只有公网IP，没有域名。目前只能通过IP进行访问。跟这个有关系吗。
关于模型调用的问题，目前GPT-4o模型和deepseek-V3的工具调用能力如何？ @dosu.

[dosubot]

关于CORS跨域配置的问题，如果您在云服务器上部署并且只有公网IP，您需要确保服务器的CORS设置允许从您的客户端IP进行访问。具体的CORS配置可能需要在您的服务器软件（如Nginx、Apache等）中进行设置，通常包括允许的来源、方法和头信息。

至于模型调用能力，关于GPT-4o和DeepSeek-V3模型在LobeChat中的工具调用能力，目前没有找到相关的具体信息。您可能需要进行一些测试以评估它们在您的特定用例中的表现。

如果您需要进一步的帮助，可能需要查看服务器的具体配置文档或联系相关的技术支持。

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[SpeedupMaster]

• 插件安装报错是由于被墙了，开代理可以解决
• 插件使用报错是gemini的function call的问题

你好，请问描述的开代理可以解决指的是 给我的阿里云服务器上代理还是我访问的机器本身开代理呢? 是否有具体的走代理的Domain规则，我想通过添加rules的方式来为代理软件指定使用代理服务。 @SpeedupMaster

不知道你是所有插件都无法安装还是个别的无法安装。有几个插件好像是被墙了，翻墙后可以安装，安装后不翻墙好像也能用，只是会弹报错信息。Domain规则不太清楚

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• The plug-in installation error is due to being blocked, which can be solved by opening a proxy.
• The error reported when using the plug-in is a problem of gemini’s function call.

Hello, may I ask if the description of "opening a proxy can solve the problem" refers to "a proxy for my Alibaba Cloud server" or "a proxy for the machine I visit"? Is there a specific Domain rule for proxying? I want to Specify the use of proxy services for the proxy software by adding rules. @SpeedupMaster

I don’t know if you can’t install all plug-ins or just some of them. Some plug-ins seem to be blocked, and they can be installed after circumventing the wall. After installation, they seem to work without circumventing the wall, but an error message will pop up. Domain rules are not clear