Release 0.0.35

Update the Doc

Dockerfile

@@ -4,7 +4,7 @@ WORKDIR /app
 
 COPY . /app
 
-RUN yarn install --registry https://registry.npmmirror.com/ && yarn run build
+RUN yarn install --registry https://registry.npmmirror.com/ --ignore-engines && yarn run build
 
 FROM node:lts-alpine
 

README.md

@@ -1,15 +1,19 @@
 # GLM AI Free 服务
 
-![](https://img.shields.io/github/license/llm-red-team/glm-free-api.svg)
+<hr>
+
+<span>[ 中文 | <a href="README_EN.md">English</a> ]</span>
+
+[![](https://img.shields.io/github/license/llm-red-team/glm-free-api.svg)](LICENSE)
 ![](https://img.shields.io/github/stars/llm-red-team/glm-free-api.svg)
 ![](https://img.shields.io/github/forks/llm-red-team/glm-free-api.svg)
 ![](https://img.shields.io/docker/pulls/vinlic/glm-free-api.svg)
 
-支持高速流式输出、支持多轮对话、支持智能体对话、支持AI绘图、支持联网搜索、支持长文档解读、支持图像解析，零配置部署，多路token支持，自动清理会话痕迹。
+支持GLM-4-Plus高速流式输出、支持多轮对话、支持智能体对话、支持Zero思考推理模型、支持视频生成、支持AI绘图、支持联网搜索、支持长文档解读、支持图像解析，零配置部署，多路token支持，自动清理会话痕迹。
 
 与ChatGPT接口完全兼容。
 
-还有以下四个free-api欢迎关注：
+还有以下十个free-api欢迎关注：
 
 Moonshot AI（Kimi.ai）接口转API [kimi-free-api](https://github.com/LLM-Red-Team/kimi-free-api)
 
@@ -17,57 +21,56 @@ Moonshot AI（Kimi.ai）接口转API [kimi-free-api](https://github.com/LLM-Red-
 
 阿里通义 (Qwen) 接口转API [qwen-free-api](https://github.com/LLM-Red-Team/qwen-free-api)
 
-聆心智能 (Emohaa) 接口转API [emohaa-free-api](https://github.com/LLM-Red-Team/emohaa-free-api)
+秘塔AI (Metaso) 接口转API [metaso-free-api](https://github.com/LLM-Red-Team/metaso-free-api)
 
-## 声明
+字节跳动（豆包）接口转API [doubao-free-api](https://github.com/LLM-Red-Team/doubao-free-api)
 
-仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！
+字节跳动（即梦AI）接口转API [jimeng-free-api](https://github.com/LLM-Red-Team/jimeng-free-api)
 
-仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！
+讯飞星火（Spark）接口转API [spark-free-api](https://github.com/LLM-Red-Team/spark-free-api)
 
-仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！
+MiniMax（海螺AI）接口转API [hailuo-free-api](https://github.com/LLM-Red-Team/hailuo-free-api)
+
+深度求索（DeepSeek）接口转API [deepseek-free-api](https://github.com/LLM-Red-Team/deepseek-free-api)
+
+聆心智能 (Emohaa) 接口转API [emohaa-free-api](https://github.com/LLM-Red-Team/emohaa-free-api)（当前不可用）
 
 ## 目录
 
-* [声明](#声明)
+* [免责声明](#免责声明)
-* [在线体验](#在线体验)
 * [效果示例](#效果示例)
 * [接入准备](#接入准备)
   * [智能体接入](#智能体接入)
   * [多账号接入](#多账号接入)
 * [Docker部署](#Docker部署)
   * [Docker-compose部署](#Docker-compose部署)
-* [Render部署](#Render部署)
+  * [Render部署](#Render部署)
-* [Vercel部署](#Vercel部署)
+  * [Vercel部署](#Vercel部署)
 * [原生部署](#原生部署)
+* [推荐使用客户端](#推荐使用客户端)
 * [接口列表](#接口列表)
   * [对话补全](#对话补全)
+  * [视频生成](#视频生成)
   * [AI绘图](#AI绘图)
   * [文档解读](#文档解读)
   * [图像解析](#图像解析)
   * [refresh_token存活检测](#refresh_token存活检测)
 * [注意事项](#注意事项)
   * [Nginx反代优化](#Nginx反代优化)
+  * [Token统计](#Token统计)
+* [Star History](#star-history)
+  
+## 免责声明
 
-## 声明
+**逆向API是不稳定的，建议前往智谱AI官方 https://open.bigmodel.cn/ 付费使用API，避免封禁的风险。**
 
-仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！
+**本组织和个人不接受任何资金捐助和交易，此项目是纯粹研究交流学习性质！**
 
-仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！
+**仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！**
 
-仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！
+**仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！**
 
-## 在线体验
+**仅限自用，禁止对外提供服务或商用，避免对官方造成服务压力，否则风险自担！**
-
-此链接仅临时测试功能，只有一路并发，如果遇到异常请稍后重试，建议自行部署使用。
-
-https://udify.app/chat/Pe89TtaX3rKXM8NS
-
-## 测试接口
-
-此接口实例部署在[Render](#Render部署)上面，遇到容器回收可能导致响应速度较慢，仅供测试，建议自行部署。
-
-https://glm-free-api-ij4o.onrender.com
 
 ## 效果示例
 
@@ -81,10 +84,20 @@ https://glm-free-api-ij4o.onrender.com
 
 ![智能体对话](./doc/example-9.png)
 
+### 结合Dify工作流Demo
+
+体验地址：https://udify.app/chat/m46YgeVLNzFh4zRs
+
+<img width="390" alt="image" src="https://github.com/LLM-Red-Team/glm-free-api/assets/20235341/4773b9f6-b1ca-460c-b3a7-c56bdb1f0659">
+
 ### 多轮对话Demo
 
 ![多轮对话](./doc/example-6.png)
 
+### 视频生成Demo
+
+[点击预览](https://sfile.chatglm.cn/testpath/video/c1f59468-32fa-58c3-bd9d-ab4230cfe3ca_0.mp4)
+
 ### AI绘图Demo
 
 ![AI绘图](./doc/example-10.png)
@@ -174,7 +187,7 @@ services:
 ### Render部署
 
 **注意：部分部署区域可能无法连接glm，如容器日志出现请求超时或无法连接，请切换其他区域部署！**
-**注意：免费账户的容器实例将在一段时间不活动时自动停止运行，这会导致下次请求时遇到50秒或更长的延迟，建议通过 GET `/ping` 请求来维持容器活动**
+**注意：免费账户的容器实例将在一段时间不活动时自动停止运行，这会导致下次请求时遇到50秒或更长的延迟，建议查看[Render容器保活](https://github.com/LLM-Red-Team/free-api-hub/#Render%E5%AE%B9%E5%99%A8%E4%BF%9D%E6%B4%BB)**
 
 1. fork本项目到你的github账号下。
 
@@ -246,6 +259,14 @@ pm2 reload glm-free-api
 pm2 stop glm-free-api
 ```
 
+## 推荐使用客户端
+
+使用以下二次开发客户端接入free-api系列项目更快更简单，支持文档/图像上传！
+
+由 [Clivia](https://github.com/Yanyutin753/lobe-chat) 二次开发的LobeChat [https://github.com/Yanyutin753/lobe-chat](https://github.com/Yanyutin753/lobe-chat)
+
+由 [时光@](https://github.com/SuYxh) 二次开发的ChatGPT Web [https://github.com/SuYxh/chatgpt-web-sea](https://github.com/SuYxh/chatgpt-web-sea)
+
 ## 接口列表
 
 目前支持与openai兼容的 `/v1/chat/completions` 接口，可自行使用与openai或其他兼容的客户端接入接口，或者使用 [dify](https://dify.ai/) 等线上服务接入使用。
@@ -265,8 +286,13 @@ Authorization: Bearer [refresh_token]
 请求数据：
 ```json
 {
-    // 如果使用智能体请填写智能体ID到此处，否则可以乱填
+    // 默认模型：glm-4-plus
-    "model": "glm4",
+    // zero思考推理模型：glm-4-zero / glm-4-think
+    // 如果使用智能体请填写智能体ID到此处
+    "model": "glm-4-plus",
+    // 目前多轮对话基于消息合并实现，某些场景可能导致能力下降且受单轮最大token数限制
+    // 如果您想获得原生的多轮对话体验，可以传入首轮消息获得的id，来接续上下文
+    // "conversation_id": "65f6c28546bae1f0fbb532de",
     "messages": [
         {
             "role": "user",
@@ -281,8 +307,9 @@ Authorization: Bearer [refresh_token]
 响应数据：
 ```json
 {
+    // 如果想获得原生多轮对话体验，此id，你可以传入到下一轮对话的conversation_id来接续上下文
     "id": "65f6c28546bae1f0fbb532de",
-    "model": "glm4",
+    "model": "glm-4",
     "object": "chat.completion",
     "choices": [
         {
@@ -303,9 +330,64 @@ Authorization: Bearer [refresh_token]
 }
 ```
 
+### 视频生成
+
+视频生成接口
+
+**如果您的账号未开通VIP，可能会因排队导致生成耗时较久**
+
+**POST /v1/videos/generations**
+
+header 需要设置 Authorization 头部：
+
+```
+Authorization: Bearer [refresh_token]
+```
+
+请求数据：
+```json
+{
+    // 模型名称
+    // cogvideox：默认官方视频模型
+    // cogvideox-pro：先生成图像再作为参考图像生成视频，作为视频首帧引导视频效果，但耗时更长
+    "model": "cogvideox",
+    // 视频生成提示词
+    "prompt": "一只可爱的猫走在花丛中",
+    // 支持使用图像URL或者BASE64_URL作为视频首帧参考图像（如果使用cogvideox-pro则会忽略此参数）
+    // "image_url": "https://sfile.chatglm.cn/testpath/b5341945-3839-522c-b4ab-a6268cb131d5_0.png",
+    // 支持设置视频风格：卡通3D/黑白老照片/油画/电影感
+    // "video_style": "油画",
+    // 支持设置情感氛围：温馨和谐/生动活泼/紧张刺激/凄凉寂寞
+    // "emotional_atmosphere": "生动活泼",
+    // 支持设置运镜方式：水平/垂直/推近/拉远
+    // "mirror_mode": "水平"
+}
+```
+
+响应数据：
+```json
+{
+    "created": 1722103836,
+    "data": [
+        {
+            // 对话ID，目前没啥用
+            "conversation_id": "66a537ec0603e53bccb8900a",
+            // 封面URL
+            "cover_url": "https://sfile.chatglm.cn/testpath/video_cover/c1f59468-32fa-58c3-bd9d-ab4230cfe3ca_cover_0.png",
+            // 视频URL
+            "video_url": "https://sfile.chatglm.cn/testpath/video/c1f59468-32fa-58c3-bd9d-ab4230cfe3ca_0.mp4",
+            // 视频时长
+            "video_duration": "6s",
+            // 视频分辨率
+            "resolution": "1440 × 960"
+        }
+    ]
+}
+```
+
 ### AI绘图
 
-对话补全接口，与openai的 [images-create-api](https://platform.openai.com/docs/api-reference/images/create) 兼容。
+图像生成接口，与openai的 [images-create-api](https://platform.openai.com/docs/api-reference/images/create) 兼容。
 
 **POST /v1/images/generations**
 
@@ -352,7 +434,7 @@ Authorization: Bearer [refresh_token]
 ```json
 {
     // 如果使用智能体请填写智能体ID到此处，否则可以乱填
-    "model": "glm4",
+    "model": "glm-4",
     "messages": [
         {
             "role": "user",
@@ -379,7 +461,7 @@ Authorization: Bearer [refresh_token]
 ```json
 {
     "id": "cnmuo7mcp7f9hjcmihn0",
-    "model": "glm4",
+    "model": "glm-4",
     "object": "chat.completion",
     "choices": [
         {
@@ -509,4 +591,4 @@ keepalive_timeout 120;
 
 ## Star History
 
-[![Star History Chart](https://api.star-history.com/svg?repos=LLM-Red-Team/glm-free-api&type=Date)](https://star-history.com/#LLM-Red-Team/glm-free-api&Date)
+[![Star History Chart](https://api.star-history.com/svg?repos=LLM-Red-Team/glm-free-api&type=Date)](https://star-history.com/#LLM-Red-Team/glm-free-api&Date)

README_EN.md

@@ -0,0 +1,596 @@
+# GLM AI Free Service
+
+[![](https://img.shields.io/github/license/llm-red-team/glm-free-api.svg)](LICENSE)
+![](https://img.shields.io/github/stars/llm-red-team/glm-free-api.svg)
+![](https://img.shields.io/github/forks/llm-red-team/glm-free-api.svg)
+![](https://img.shields.io/docker/pulls/vinlic/glm-free-api.svg)
+
+Supports high-speed streaming output, multi-turn dialogues, internet search, long document reading, image analysis, zero-configuration deployment, multi-token support, and automatic session trace cleanup.
+
+Fully compatible with the ChatGPT interface.
+
+Also, the following free APIs are available for your attention:
+
+Moonshot AI (Kimi.ai) API to API [kimi-free-api](https://github.com/LLM-Red-Team/kimi-free-api/tree/master)
+
+StepFun (StepChat) API to API [step-free-api](https://github.com/LLM-Red-Team/step-free-api)
+
+Ali Tongyi (Qwen) API to API [qwen-free-api](https://github.com/LLM-Red-Team/qwen-free-api)
+
+ZhipuAI (ChatGLM) API to API [glm-free-api](https://github.com/LLM-Red-Team/glm-free-api)
+
+ByteDance (Doubao) API to API [doubao-free-api](https://github.com/LLM-Red-Team/doubao-free-api)
+
+Meta Sota (metaso) API to API [metaso-free-api](https://github.com/LLM-Red-Team/metaso-free-api)
+
+Iflytek Spark (Spark) API to API [spark-free-api](https://github.com/LLM-Red-Team/spark-free-api)
+
+MiniMax（Hailuo）API to API [hailuo-free-api](https://github.com/LLM-Red-Team/hailuo-free-api)
+
+DeepSeek（DeepSeek）API to API [deepseek-free-api](https://github.com/LLM-Red-Team/deepseek-free-api)
+
+Lingxin Intelligence (Emohaa) API to API [emohaa-free-api](https://github.com/LLM-Red-Team/emohaa-free-api) (OUT OF ORDER)
+
+## Table of Contents
+
+* [Announcement](#Announcement)
+* [Online Experience](#Online-Experience)
+* [Effect Examples](#Effect-Examples)
+* [Access Preparation](#Access-Preparation)
+    * [Agent Access](#Agent-Access)
+    * [Multiple Account Access](#Multiple-Account-Access)
+* [Docker Deployment](#Docker-Deployment)
+    * [Docker-compose Deployment](#Docker-compose-Deployment)
+    * [Render Deployment](#Render-Deployment)
+    * [Vercel Deployment](#Vercel-Deployment)
+* [Native Deployment](#Native-Deployment)
+* [Recommended Clients](#Recommended-Clients)
+* [Interface List](#Interface-List)
+    * [Conversation Completion](#Conversation-Completion)
+    * [Video Generation](#Video-Generation)
+    * [AI Drawing](#AI-Drawing)
+    * [Document Interpretation](#Document-Interpretation)
+    * [Image Analysis](#Image-Analysis)
+    * [Refresh_token Survival Detection](#Refresh_token-Survival-Detection)
+* [Notification](#Notification)
+    * [Nginx Anti-generation Optimization](#Nginx-Anti-generation-Optimization)
+    * [Token Statistics](#Token-Statistics)
+* [Star History](#star-history)
+  
+## Announcement
+
+**This API is unstable. So we highly recommend you go to the [Zhipu](https://open.bigmodel.cn/) use the offical API, avoiding banned.**
+
+**This organization and individuals do not accept any financial donations and transactions. This project is purely for research, communication, and learning purposes!**
+
+**For personal use only, it is forbidden to provide services or commercial use externally to avoid causing service pressure on the official, otherwise, bear the risk yourself!**
+
+**For personal use only, it is forbidden to provide services or commercial use externally to avoid causing service pressure on the official, otherwise, bear the risk yourself!**
+
+**For personal use only, it is forbidden to provide services or commercial use externally to avoid causing service pressure on the official, otherwise, bear the risk yourself!**
+
+## Online Experience
+This link is only for temporary testing of functions and cannot be used for a long time. For long-term use, please deploy by yourself.
+
+https://udify.app/chat/Pe89TtaX3rKXM8NS
+
+## Effect Examples
+
+### Identity Verification
+
+![Identity Verification](./doc/example-1.png)
+
+### AI-Agent
+
+Agent link：[Comments Generator](https://chatglm.cn/main/gdetail/65c046a531d3fcb034918abe)
+
+![AI-Agent](./doc/example-9.png)
+
+### Combined with Dify workflow
+
+Experience link：https://udify.app/chat/m46YgeVLNzFh4zRs
+
+<img width="390" alt="image" src="https://github.com/LLM-Red-Team/glm-free-api/assets/20235341/4773b9f6-b1ca-460c-b3a7-c56bdb1f0659">
+
+### Multi-turn Dialogue
+
+![Multi-turn Dialogue](./doc/example-6.png)
+
+### Video Generation
+
+[View](https://sfile.chatglm.cn/testpath/video/c1f59468-32fa-58c3-bd9d-ab4230cfe3ca_0.mp4)
+
+### AI Drawing
+
+![AI Drawing](./doc/example-10.png)
+
+### Internet Search
+
+![Internet Search](./doc/example-2.png)
+
+### Long Document Reading
+
+![Long Document Reading](./doc/example-5.png)
+
+### Using Code
+
+![Using Code](./doc/example-12.png)
+
+### Image Analysis
+
+![Image Analysis](./doc/example-3.png)
+
+## Access Preparation
+
+Obtain `refresh_token` from [Zhipu](https://chatglm.cn/)
+
+Enter Zhipu Qingyan and start a random conversation, then press F12 to open the developer tools. Find the value of `tongyi_sso_ticket` in Application > Cookies, which will be used as the Bearer Token value for Authorization: `Authorization: Bearer TOKEN`
+
+![example0](./doc/example-0.png)
+
+### Agent Access
+
+Open a window of Agent Chat, the ID in the url is the ID of the Agent, which is the parameter of `model`.
+
+![example11](./doc/example-11.png)
+
+### Multiple Account Access
+
+You can provide multiple account chatglm_refresh_tokens and use `,` to join them:
+
+`Authorization: Bearer TOKEN1,TOKEN2,TOKEN3`
+
+The service will pick one each time a request is made.
+
+## Docker Deployment
+
+Please prepare a server with a public IP and open port 8000.
+
+Pull the image and start the service
+
+```shell
+docker run -it -d --init --name step-free-api -p 8000:8000 -e TZ=Asia/Shanghai vinlic/step-free-api:latest
+```
+
+check real-time service logs
+
+```shell
+docker logs -f glm-free-api
+```
+
+Restart service
+
+```shell
+docker restart glm-free-api
+```
+
+Shut down service
+
+```shell
+docker stop glm-free-api
+```
+
+### Docker-compose Deployment
+
+```yaml
+version: '3'
+
+services:
+  glm-free-api:
+    container_name: glm-free-api
+    image: vinlic/glm-free-api:latest
+    restart: always
+    ports:
+      - "8000:8000"
+    environment:
+      - TZ=Asia/Shanghai
+```
+
+### Render Deployment
+
+**Attention: Some deployment regions may not be able to connect to Kimi. If container logs show request timeouts or connection failures (Singapore has been tested and found unavailable), please switch to another deployment region!**
+
+**Attention: Container instances for free accounts will automatically stop after a period of inactivity, which may result in a 50-second or longer delay during the next request. It is recommended to check [Render Container Keepalive](https://github.com/LLM-Red-Team/free-api-hub/#Render%E5%AE%B9%E5%99%A8%E4%BF%9D%E6%B4%BB)**
+
+1. Fork this project to your GitHub account.
+
+2. Visit [Render](https://dashboard.render.com/) and log in with your GitHub account.
+
+3. Build your Web Service (`New+` -> `Build and deploy from a Git repository` -> `Connect your forked project` -> `Select deployment region` -> `Choose instance type as Free` -> `Create Web Service`).
+
+4. After the build is complete, copy the assigned domain and append the URL to access it.
+
+### Vercel Deployment
+
+**Note: Vercel free accounts have a request response timeout of 10 seconds, but interface responses are usually longer, which may result in a 504 timeout error from Vercel!**
+
+Please ensure that Node.js environment is installed first.
+
+```shell
+npm i -g vercel --registry http://registry.npmmirror.com
+vercel login
+git clone https://github.com/LLM-Red-Team/glm-free-api
+cd glm-free-api
+vercel --prod
+```
+
+## Native Deployment
+
+Please prepare a server with a public IP and open port 8000.
+
+Please install the Node.js environment and configure the environment variables first, and confirm that the node command is available.
+
+Install dependencies
+
+```shell
+npm i
+```
+
+Install PM2 for process guarding
+
+```shell
+npm i -g pm2
+```
+
+Compile and build. When you see the dist directory, the build is complete.
+
+```shell
+npm run build
+```
+
+Start service
+
+```shell
+pm2 start dist/index.js --name "glm-free-api"
+```
+
+View real-time service logs
+
+```shell
+pm2 logs glm-free-api
+```
+
+Restart service
+
+```shell
+pm2 reload glm-free-api
+```
+
+Shut down service
+
+```shell
+pm2 stop glm-free-api
+```
+
+## Recommended Clients
+
+Using the following second-developed clients for free-api series projects is faster and easier, and supports document/image uploads!
+
+[Clivia](https://github.com/Yanyutin753/lobe-chat)'s modified LobeChat [https://github.com/Yanyutin753/lobe-chat](https://github.com/Yanyutin753/lobe-chat)
+
+[Time@](https://github.com/SuYxh)'s modified ChatGPT Web [https://github.com/SuYxh/chatgpt-web-sea](https://github.com/SuYxh/chatgpt-web-sea)
+
+## interface List
+
+Currently, the `/v1/chat/completions` interface compatible with openai is supported. You can use the client access interface compatible with openai or other clients, or use online services such as [dify](https://dify.ai/) Access and use.
+
+### Conversation Completion
+
+Conversation completion interface, compatible with openai's [chat-completions-api](https://platform.openai.com/docs/guides/text-generation/chat-completions-api).
+
+**POST /v1/chat/completions**
+
+The header needs to set the Authorization header:
+
+```
+Authorization: Bearer [refresh_token]
+```
+
+Request data:
+```json
+{
+    // Default model: glm-4-plus
+    // zero thinking model: glm-4-zero / glm-4-think
+    // If using the Agent, fill in the Agent ID here
+    "model": "glm-4",
+    // Currently, multi-round conversations are realized based on message merging, which in some scenarios may lead to capacity degradation and is limited by the maximum number of tokens in a single round.
+    // If you want a native multi-round dialog experience, you can pass in the ids obtained from the last round of messages to pick up the context
+    // "conversation_id": "65f6c28546bae1f0fbb532de",
+    "messages": [
+        {
+            "role": "user",
+            "content": "Who RU？"
+        }
+    ],
+    // If using SSE stream, please set it to true, the default is false
+    "stream": false
+}
+```
+
+Response data：
+```json
+{
+    "id": "65f6c28546bae1f0fbb532de",
+    "model": "glm-4",
+    "object": "chat.completion",
+    "choices": [
+        {
+            "index": 0,
+            "message": {
+                "role": "assistant",
+                "content": "My name is Zhipu Qingyan."
+            },
+            "finish_reason": "stop"
+        }
+    ],
+    "usage": {
+        "prompt_tokens": 1,
+        "completion_tokens": 1,
+        "total_tokens": 2
+    },
+    "created": 1710152062
+}
+```
+
+### Video Generation
+
+Video API
+
+**If you're not VIP, you will wait in line for a long time.**
+
+**POST /v1/videos/generations**
+
+The header needs to set the Authorization header:
+
+```
+Authorization: Bearer [refresh_token]
+```
+
+Request data:
+```json
+{
+    // 模型名称
+    // cogvideox：默认官方视频模型
+    // cogvideox-pro：先生成图像再作为参考图像生成视频，作为视频首帧引导视频效果，但耗时更长
+    "model": "cogvideox",
+    // 视频生成提示词
+    "prompt": "一只可爱的猫走在花丛中",
+    // 支持使用图像URL或者BASE64_URL作为视频首帧参考图像（如果使用cogvideox-pro则会忽略此参数）
+    // "image_url": "https://sfile.chatglm.cn/testpath/b5341945-3839-522c-b4ab-a6268cb131d5_0.png",
+    // 支持设置视频风格：卡通3D/黑白老照片/油画/电影感
+    // "video_style": "油画",
+    // 支持设置情感氛围：温馨和谐/生动活泼/紧张刺激/凄凉寂寞
+    // "emotional_atmosphere": "生动活泼",
+    // 支持设置运镜方式：水平/垂直/推近/拉远
+    // "mirror_mode": "水平"
+}
+```
+
+Response data:
+```json
+{
+    "created": 1722103836,
+    "data": [
+        {
+            // 对话ID，目前没啥用
+            "conversation_id": "66a537ec0603e53bccb8900a",
+            // 封面URL
+            "cover_url": "https://sfile.chatglm.cn/testpath/video_cover/c1f59468-32fa-58c3-bd9d-ab4230cfe3ca_cover_0.png",
+            // 视频URL
+            "video_url": "https://sfile.chatglm.cn/testpath/video/c1f59468-32fa-58c3-bd9d-ab4230cfe3ca_0.mp4",
+            // 视频时长
+            "video_duration": "6s",
+            // 视频分辨率
+            "resolution": "1440 × 960"
+        }
+    ]
+}
+```
+
+### AI Drawing
+
+This format is compatible with the [gpt-4-vision-preview](https://platform.openai.com/docs/guides/vision) API format.
+
+**POST /v1/images/generations**
+
+The header needs to set the Authorization header:
+
+```
+Authorization: Bearer [refresh_token]
+```
+
+Request data:
+```json
+{
+    // 如果使用智能体请填写智能体ID到此处，否则可以乱填
+    "model": "cogview-3",
+    "prompt": "A cute cat"
+}
+```
+
+Response data:
+```json
+{
+    "created": 1711507449,
+    "data": [
+        {
+            "url": "https://sfile.chatglm.cn/testpath/5e56234b-34ae-593c-ba4e-3f7ba77b5768_0.png"
+        }
+    ]
+}
+```
+
+### Document Interpretation
+
+Provide an accessible file URL or BASE64_URL to parse.
+
+**POST /v1/chat/completions**
+
+The header needs to set the Authorization header:
+
+```
+Authorization: Bearer [refresh_token]
+```
+
+Request data:
+```json
+{
+    // 如果使用智能体请填写智能体ID到此处，否则可以乱填
+    "model": "glm-4",
+    "messages": [
+        {
+            "role": "user",
+            "content": [
+                {
+                    "type": "file",
+                    "file_url": {
+                        "url": "https://mj101-1317487292.cos.ap-shanghai.myqcloud.com/ai/test.pdf"
+                    }
+                },
+                {
+                    "type": "text",
+                    "text": "文档里说了什么？"
+                }
+            ]
+        }
+    ],
+    // 如果使用SSE流请设置为true，默认false
+    "stream": false
+}
+```
+
+Response data:
+```json
+{
+    "id": "cnmuo7mcp7f9hjcmihn0",
+    "model": "glm-4",
+    "object": "chat.completion",
+    "choices": [
+        {
+            "index": 0,
+            "message": {
+                "role": "assistant",
+                "content": "根据文档内容，我总结如下：\n\n这是一份关于希腊罗马时期的魔法咒语和仪式的文本，包含几个魔法仪式：\n\n1. 一个涉及面包、仪式场所和特定咒语的仪式，用于使某人爱上你。\n\n2. 一个针对女神赫卡忒的召唤仪式，用来折磨某人直到她自愿来到你身边。\n\n3. 一个通过念诵爱神阿芙罗狄蒂的秘密名字，连续七天进行仪式，来赢得一个美丽女子的心。\n\n4. 一个通过燃烧没药并念诵咒语，让一个女子对你产生强烈欲望的仪式。\n\n这些仪式都带有魔法和迷信色彩，使用各种咒语和象征性行为来影响人的感情和意愿。"
+            },
+            "finish_reason": "stop"
+        }
+    ],
+    "usage": {
+        "prompt_tokens": 1,
+        "completion_tokens": 1,
+        "total_tokens": 2
+    },
+    "created": 100920
+}
+```
+
+### Image Analysis
+
+Provide an accessible image URL or BASE64_URL to parse.
+
+This format is compatible with the [gpt-4-vision-preview](https://platform.openai.com/docs/guides/vision) API format. You can also use this format to transmit documents for parsing.
+
+**POST /v1/chat/completions**
+
+The header needs to set the Authorization header:
+
+```
+Authorization: Bearer [refresh_token]
+```
+
+Request data:
+```json
+{
+    "model": "65c046a531d3fcb034918abe",
+    "messages": [
+        {
+            "role": "user",
+            "content": [
+                {
+                    "type": "image_url",
+                    "image_url": {
+                        "url": "http://1255881664.vod2.myqcloud.com/6a0cd388vodbj1255881664/7b97ce1d3270835009240537095/uSfDwh6ZpB0A.png"
+                    }
+                },
+                {
+                    "type": "text",
+                    "text": "图像描述了什么？"
+                }
+            ]
+        }
+    ],
+    "stream": false
+}
+```
+
+Response data:
+```json
+{
+    "id": "65f6c28546bae1f0fbb532de",
+    "model": "glm",
+    "object": "chat.completion",
+    "choices": [
+        {
+            "index": 0,
+            "message": {
+                "role": "assistant",
+                "content": "图片中展示的是一个蓝色背景下的logo，具体地，左边是一个由多个蓝色的圆点组成的圆形图案，右边是“智谱·AI”四个字，字体颜色为蓝色。"
+            },
+            "finish_reason": "stop"
+        }
+    ],
+    "usage": {
+        "prompt_tokens": 1,
+        "completion_tokens": 1,
+        "total_tokens": 2
+    },
+    "created": 1710670469
+}
+```
+
+### Refresh_token Survival Detection
+
+Check whether refresh_token is alive. If live is not true, otherwise it is false. Please do not call this interface frequently (less than 10 minutes).
+
+**POST /token/check**
+
+Request data:
+```json
+{
+    "token": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9..."
+}
+```
+
+Response data:
+```json
+{
+    "live": true
+}
+```
+
+## Notification
+
+### Nginx Anti-generation Optimization
+
+If you are using Nginx reverse proxy `glm-free-api`, please add the following configuration items to optimize the output effect of the stream and optimize the experience.
+
+```nginx
+# Turn off proxy buffering. When set to off, Nginx will immediately send client requests to the backend server and immediately send responses received from the backend server back to the client.
+proxy_buffering off;
+# Enable chunked transfer encoding. Chunked transfer encoding allows servers to send data in chunks for dynamically generated content without knowing the size of the content in advance.
+chunked_transfer_encoding on;
+# Turn on TCP_NOPUSH, which tells Nginx to send as much data as possible before sending the packet to the client. This is usually used in conjunction with sendfile to improve network efficiency.
+tcp_nopush on;
+# Turn on TCP_NODELAY, which tells Nginx not to delay sending data and to send small data packets immediately. In some cases, this can reduce network latency.
+tcp_nodelay on;
+#Set the timeout to keep the connection, here it is set to 120 seconds. If there is no further communication between client and server during this time, the connection will be closed.
+keepalive_timeout 120;
+```
+
+### Token Statistics
+
+Since the inference side is not in glm-free-api, the token cannot be counted and will be returned as a fixed number!!!!!
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=LLM-Red-Team/glm-free-api&type=Date)](https://star-history.com/#LLM-Red-Team/glm-free-api&Date)

package.json

@@ -1,6 +1,6 @@
 {
   "name": "glm-free-api",
-  "version": "0.0.22",
+  "version": "0.0.35",
   "description": "GLM Free API Server",
   "type": "module",
   "main": "dist/index.js",
@@ -13,8 +13,8 @@
     "dist/"
   ],
   "scripts": {
-    "dev": "tsup src/index.ts --format cjs,esm --sourcemap --dts --publicDir public --watch --onSuccess \"node dist/index.js\"",
+    "dev": "tsup src/index.ts --format cjs,esm --sourcemap --dts --publicDir public --watch --onSuccess \"node --enable-source-maps dist/index.js\"",
-    "start": "node dist/index.js",
+    "start": "node --enable-source-maps dist/index.js",
     "build": "tsup src/index.ts --format cjs,esm --sourcemap --dts --clean --publicDir public"
   },
   "author": "Vinlic",
@@ -38,6 +38,7 @@
     "mime": "^4.0.1",
     "minimist": "^1.2.8",
     "randomstring": "^1.3.0",
+    "sharp": "^0.33.4",
     "uuid": "^9.0.1",
     "yaml": "^2.3.4"
   },

src/api/consts/exceptions.ts

@@ -7,5 +7,6 @@ export default {
     API_FILE_EXECEEDS_SIZE: [-2004, '远程文件超出大小'],
     API_CHAT_STREAM_PUSHING: [-2005, '已有对话流正在输出'],
     API_CONTENT_FILTERED: [-2006, '内容由于合规问题已被阻止生成'],
-    API_IMAGE_GENERATION_FAILED: [-2007, '图像生成失败']
+    API_IMAGE_GENERATION_FAILED: [-2007, '图像生成失败'],
+    API_VIDEO_GENERATION_FAILED: [-2008, '视频生成失败'],
 }

src/api/controllers/chat.ts

@@ -2,6 +2,8 @@ import { PassThrough } from "stream";
 import path from "path";
 import _ from "lodash";
 import mime from "mime";
+import sharp from "sharp";
+import fs from "fs-extra";
 import FormData from "form-data";
 import axios, { AxiosResponse } from "axios";
 
@@ -15,6 +17,8 @@ import util from "@/lib/util.ts";
 const MODEL_NAME = "glm";
 // 默认的智能体ID，GLM4
 const DEFAULT_ASSISTANT_ID = "65940acff94777010aa6b796";
+// zero推理模型智能体ID
+const ZERO_ASSISTANT_ID = "676411c38945bbc58a905d31";
 // access_token有效期
 const ACCESS_TOKEN_EXPIRES = 3600;
 // 最大重试次数
@@ -163,13 +167,14 @@ async function removeConversation(
  *
  * @param messages 参考gpt系列消息格式，多轮对话请完整提供上下文
  * @param refreshToken 用于刷新access_token的refresh_token
- * @param assistantId 智能体ID，默认使用GLM4原版
+ * @param model 智能体ID，默认使用GLM4原版
  * @param retryCount 重试次数
  */
 async function createCompletion(
   messages: any[],
   refreshToken: string,
-  assistantId = DEFAULT_ASSISTANT_ID,
+  model = MODEL_NAME,
+  refConvId = "",
   retryCount = 0
 ) {
   return (async () => {
@@ -179,23 +184,36 @@ async function createCompletion(
     const refFileUrls = extractRefFileUrls(messages);
     const refs = refFileUrls.length
       ? await Promise.all(
-        refFileUrls.map((fileUrl) => uploadFile(fileUrl, refreshToken))
+          refFileUrls.map((fileUrl) => uploadFile(fileUrl, refreshToken))
-      )
+        )
       : [];
 
+    // 如果引用对话ID不正确则重置引用
+    if (!/[0-9a-zA-Z]{24}/.test(refConvId)) refConvId = "";
+
+    let assistantId = /^[a-z0-9]{24,}$/.test(model) ? model : DEFAULT_ASSISTANT_ID;
+
+    if(model.indexOf('think') != -1 || model.indexOf('zero') != -1) {
+      assistantId = ZERO_ASSISTANT_ID;
+      logger.info('使用思考模型');
+    }
+
     // 请求流
     const token = await acquireToken(refreshToken);
     const result = await axios.post(
       "https://chatglm.cn/chatglm/backend-api/assistant/stream",
       {
         assistant_id: assistantId,
-        conversation_id: "",
+        conversation_id: refConvId,
-        messages: messagesPrepare(messages, refs),
+        messages: messagesPrepare(messages, refs, !!refConvId),
         meta_data: {
           channel: "",
           draft_id: "",
+          if_plus_model: true,
           input_question_type: "xxxx",
           is_test: false,
+          platform: "pc",
+          quote_log_id: ""
         },
       },
       {
@@ -216,7 +234,7 @@ async function createCompletion(
       }
     );
     if (result.headers["content-type"].indexOf("text/event-stream") == -1) {
-      result.data.on("data", buffer => logger.error(buffer.toString()));
+      result.data.on("data", (buffer) => logger.error(buffer.toString()));
       throw new APIException(
         EX.API_REQUEST_FAILED,
         `Stream response Content-Type invalid: ${result.headers["content-type"]}`
@@ -225,14 +243,14 @@ async function createCompletion(
 
     const streamStartTime = util.timestamp();
     // 接收流为输出文本
-    const answer = await receiveStream(result.data);
+    const answer = await receiveStream(model, result.data);
     logger.success(
       `Stream has completed transfer ${util.timestamp() - streamStartTime}ms`
     );
 
     // 异步移除会话
-    removeConversation(answer.id, refreshToken, assistantId).catch((err) =>
+    removeConversation(answer.id, refreshToken, assistantId).catch(
-      console.error(err)
+      (err) => !refConvId && console.error(err)
     );
 
     return answer;
@@ -245,7 +263,8 @@ async function createCompletion(
         return createCompletion(
           messages,
           refreshToken,
-          assistantId,
+          model,
+          refConvId,
           retryCount + 1
         );
       })();
@@ -259,13 +278,14 @@ async function createCompletion(
  *
  * @param messages 参考gpt系列消息格式，多轮对话请完整提供上下文
  * @param refreshToken 用于刷新access_token的refresh_token
- * @param assistantId 智能体ID，默认使用GLM4原版
+ * @param model 智能体ID，默认使用GLM4原版
  * @param retryCount 重试次数
  */
 async function createCompletionStream(
   messages: any[],
   refreshToken: string,
-  assistantId = DEFAULT_ASSISTANT_ID,
+  model = MODEL_NAME,
+  refConvId = "",
   retryCount = 0
 ) {
   return (async () => {
@@ -275,23 +295,36 @@ async function createCompletionStream(
     const refFileUrls = extractRefFileUrls(messages);
     const refs = refFileUrls.length
       ? await Promise.all(
-        refFileUrls.map((fileUrl) => uploadFile(fileUrl, refreshToken))
+          refFileUrls.map((fileUrl) => uploadFile(fileUrl, refreshToken))
-      )
+        )
       : [];
 
+    // 如果引用对话ID不正确则重置引用
+    if (!/[0-9a-zA-Z]{24}/.test(refConvId)) refConvId = "";
+
+    let assistantId = /^[a-z0-9]{24,}$/.test(model) ? model : DEFAULT_ASSISTANT_ID;
+
+    if(model.indexOf('think') != -1 || model.indexOf('zero') != -1) {
+      assistantId = ZERO_ASSISTANT_ID;
+      logger.info('使用思考模型');
+    }
+
     // 请求流
     const token = await acquireToken(refreshToken);
     const result = await axios.post(
       `https://chatglm.cn/chatglm/backend-api/assistant/stream`,
       {
         assistant_id: assistantId,
-        conversation_id: "",
+        conversation_id: refConvId,
-        messages: messagesPrepare(messages, refs),
+        messages: messagesPrepare(messages, refs, !!refConvId),
         meta_data: {
           channel: "",
           draft_id: "",
+          if_plus_model: true,
           input_question_type: "xxxx",
           is_test: false,
+          platform: "pc",
+          quote_log_id: ""
         },
       },
       {
@@ -317,7 +350,7 @@ async function createCompletionStream(
         `Invalid response Content-Type:`,
         result.headers["content-type"]
       );
-      result.data.on("data", buffer => logger.error(buffer.toString()));
+      result.data.on("data", (buffer) => logger.error(buffer.toString()));
       const transStream = new PassThrough();
       transStream.end(
         `data: ${JSON.stringify({
@@ -343,13 +376,13 @@ async function createCompletionStream(
 
     const streamStartTime = util.timestamp();
     // 创建转换流将消息格式转换为gpt兼容格式
-    return createTransStream(result.data, (convId: string) => {
+    return createTransStream(model, result.data, (convId: string) => {
       logger.success(
         `Stream has completed transfer ${util.timestamp() - streamStartTime}ms`
       );
       // 流传输结束后异步移除会话
-      removeConversation(convId, refreshToken, assistantId).catch((err) =>
+      removeConversation(convId, refreshToken, assistantId).catch(
-        console.error(err)
+        (err) => !refConvId && console.error(err)
       );
     });
   })().catch((err) => {
@@ -361,7 +394,8 @@ async function createCompletionStream(
         return createCompletionStream(
           messages,
           refreshToken,
-          assistantId,
+          model,
+          refConvId,
           retryCount + 1
         );
       })();
@@ -379,8 +413,10 @@ async function generateImages(
   return (async () => {
     logger.info(prompt);
     const messages = [
-
+      {
-      { role: "user", content: prompt },
+        role: "user",
+        content: prompt.indexOf("画") == -1 ? `请画：${prompt}` : prompt,
+      },
     ];
     // 请求流
     const token = await acquireToken(refreshToken);
@@ -393,8 +429,11 @@ async function generateImages(
         meta_data: {
           channel: "",
           draft_id: "",
+          if_plus_model: true,
           input_question_type: "xxxx",
           is_test: false,
+          platform: "pc",
+          quote_log_id: ""
         },
       },
       {
@@ -447,6 +486,180 @@ async function generateImages(
   });
 }
 
+async function generateVideos(
+  model = "cogvideox",
+  prompt: string,
+  refreshToken: string,
+  options: {
+    imageUrl: string;
+    videoStyle: string;
+    emotionalAtmosphere: string;
+    mirrorMode: string;
+    audioId: string;
+  },
+  refConvId = "",
+  retryCount = 0
+) {
+  return (async () => {
+    logger.info(prompt);
+
+    // 如果引用对话ID不正确则重置引用
+    if (!/[0-9a-zA-Z]{24}/.test(refConvId)) refConvId = "";
+
+    const sourceList = [];
+    if (model == "cogvideox-pro") {
+      const imageUrls = await generateImages(undefined, prompt, refreshToken);
+      options.imageUrl = imageUrls[0];
+    }
+    if (options.imageUrl) {
+      const { source_id: sourceId } = await uploadFile(
+        options.imageUrl,
+        refreshToken,
+        true
+      );
+      sourceList.push(sourceId);
+    }
+
+    // 发起生成请求
+    let token = await acquireToken(refreshToken);
+    const result = await axios.post(
+      `https://chatglm.cn/chatglm/video-api/v1/chat`,
+      {
+        conversation_id: refConvId,
+        prompt,
+        source_list: sourceList.length > 0 ? sourceList : undefined,
+        advanced_parameter_extra: {
+          emotional_atmosphere: options.emotionalAtmosphere,
+          mirror_mode: options.mirrorMode,
+          video_style: options.videoStyle,
+        },
+      },
+      {
+        headers: {
+          Authorization: `Bearer ${token}`,
+          Referer: "https://chatglm.cn/video",
+          "X-Device-Id": util.uuid(false),
+          "X-Request-Id": util.uuid(false),
+          ...FAKE_HEADERS,
+        },
+        // 30秒超时
+        timeout: 30000,
+        validateStatus: () => true,
+      }
+    );
+    const { result: _result } = checkResult(result, refreshToken);
+    const { chat_id: chatId, conversation_id: convId } = _result;
+
+    // 轮询生成进度
+    const startTime = util.unixTimestamp();
+    const results = [];
+    while (true) {
+      if (util.unixTimestamp() - startTime > 600)
+        throw new APIException(EX.API_VIDEO_GENERATION_FAILED);
+      const token = await acquireToken(refreshToken);
+      const result = await axios.get(
+        `https://chatglm.cn/chatglm/video-api/v1/chat/status/${chatId}`,
+        {
+          headers: {
+            Authorization: `Bearer ${token}`,
+            Referer: "https://chatglm.cn/video",
+            "X-Device-Id": util.uuid(false),
+            "X-Request-Id": util.uuid(false),
+            ...FAKE_HEADERS,
+          },
+          // 30秒超时
+          timeout: 30000,
+          validateStatus: () => true,
+        }
+      );
+      const { result: _result } = checkResult(result, refreshToken);
+      const {
+        status,
+        msg,
+        plan,
+        cover_url,
+        video_url,
+        video_duration,
+        resolution,
+      } = _result;
+      if (status != "init" && status != "processing") {
+        if (status != "finished")
+          throw new APIException(EX.API_VIDEO_GENERATION_FAILED);
+        let videoUrl = video_url;
+        if (options.audioId) {
+          const [key, id] = options.audioId.split("-");
+          const token = await acquireToken(refreshToken);
+          const result = await axios.post(
+            `https://chatglm.cn/chatglm/video-api/v1/static/composite_video`,
+            {
+              chat_id: chatId,
+              key,
+              audio_id: id,
+            },
+            {
+              headers: {
+                Authorization: `Bearer ${token}`,
+                Referer: "https://chatglm.cn/video",
+                "X-Device-Id": util.uuid(false),
+                "X-Request-Id": util.uuid(false),
+                ...FAKE_HEADERS,
+              },
+              // 30秒超时
+              timeout: 30000,
+              validateStatus: () => true,
+            }
+          );
+          const { result: _result } = checkResult(result, refreshToken);
+          videoUrl = _result.url;
+        }
+        results.push({
+          conversation_id: convId,
+          cover_url,
+          video_url: videoUrl,
+          video_duration,
+          resolution,
+        });
+        break;
+      }
+      await new Promise((resolve) => setTimeout(resolve, 1000));
+    }
+
+    //https://chatglm.cn/chatglm/video-api/v1/reference/audio_group
+
+    axios
+      .delete(`https://chatglm.cn/chatglm/video-api/v1/chat/${chatId}`, {
+        headers: {
+          Authorization: `Bearer ${token}`,
+          Referer: "https://chatglm.cn/video",
+          "X-Device-Id": util.uuid(false),
+          "X-Request-Id": util.uuid(false),
+          ...FAKE_HEADERS,
+        },
+        validateStatus: () => true,
+      })
+      .catch((err) => logger.error("移除视频生成记录失败：", err));
+
+    return results;
+  })().catch((err) => {
+    if (retryCount < MAX_RETRY_COUNT) {
+      logger.error(`Video generation error: ${err.message}`);
+      logger.warn(`Try again after ${RETRY_DELAY / 1000}s...`);
+      return (async () => {
+        await new Promise((resolve) => setTimeout(resolve, RETRY_DELAY));
+        return generateVideos(
+          model,
+          prompt,
+          refreshToken,
+          options,
+          refConvId,
+          retryCount + 1
+        );
+      })();
+    }
+    throw err;
+  });
+}
+
 /**
  * 提取消息中引用的文件URL
  *
@@ -487,54 +700,71 @@ function extractRefFileUrls(messages: any[]) {
  * 消息预处理
  *
  * 由于接口只取第一条消息，此处会将多条消息合并为一条，实现多轮对话效果
- * 使用”你“这个角色回复”我“这个角色，以第一人称对话\n
- * 我:旧消息1
- * 你:旧消息2
- * 我:新消息
  *
  * @param messages 参考gpt系列消息格式，多轮对话请完整提供上下文
+ * @param refs 参考文件列表
+ * @param isRefConv 是否为引用会话
  */
-function messagesPrepare(messages: any[], refs: any[]) {
+function messagesPrepare(messages: any[], refs: any[], isRefConv = false) {
-  // 检查最新消息是否含有"type": "image_url"或"type": "file",如果有则注入消息
+  let content;
-  let latestMessage = messages[messages.length - 1];
+  if (isRefConv || messages.length < 2) {
-  let hasFileOrImage =
+    content = messages.reduce((content, message) => {
-    Array.isArray(latestMessage.content) &&
+      if (_.isArray(message.content)) {
-    latestMessage.content.some(
+        return message.content.reduce((_content, v) => {
-      (v) => typeof v === "object" && ["file", "image_url"].includes(v["type"])
+          if (!_.isObject(v) || v["type"] != "text") return _content;
-    );
+          return _content + (v["text"] || "") + "\n";
-  if (hasFileOrImage) {
+        }, content);
-    let newFileMessage = {
+      }
-      content: "关注用户最新发送文件和消息",
+      return content + `${message.content}\n`;
-      role: "system",
+    }, "");
-    };
+    logger.info("\n透传内容：\n" + content);
-    messages.splice(messages.length - 1, 0, newFileMessage);
-    logger.info("注入提升尾部文件注意力system prompt");
   } else {
-    // 由于注入会导致设定污染，暂时注释
+    // 检查最新消息是否含有"type": "image_url"或"type": "file",如果有则注入消息
-    // let newTextMessage = {
+    let latestMessage = messages[messages.length - 1];
-    //   content: "关注用户最新的消息",
+    let hasFileOrImage =
-    //   role: "system",
+      Array.isArray(latestMessage.content) &&
-    // };
+      latestMessage.content.some(
-    // messages.splice(messages.length - 1, 0, newTextMessage);
+        (v) =>
-    // logger.info("注入提升尾部消息注意力system prompt");
+          typeof v === "object" && ["file", "image_url"].includes(v["type"])
+      );
+    if (hasFileOrImage) {
+      let newFileMessage = {
+        content: "关注用户最新发送文件和消息",
+        role: "system",
+      };
+      messages.splice(messages.length - 1, 0, newFileMessage);
+      logger.info("注入提升尾部文件注意力system prompt");
+    } else {
+      // 由于注入会导致设定污染，暂时注释
+      // let newTextMessage = {
+      //   content: "关注用户最新的消息",
+      //   role: "system",
+      // };
+      // messages.splice(messages.length - 1, 0, newTextMessage);
+      // logger.info("注入提升尾部消息注意力system prompt");
+    }
+    content = (
+      messages.reduce((content, message) => {
+        const role = message.role
+          .replace("system", "<|sytstem|>")
+          .replace("assistant", "<|assistant|>")
+          .replace("user", "<|user|>");
+        if (_.isArray(message.content)) {
+          return message.content.reduce((_content, v) => {
+            if (!_.isObject(v) || v["type"] != "text") return _content;
+            return _content + (`${role}\n` + v["text"] || "") + "\n";
+          }, content);
+        }
+        return (content += `${role}\n${message.content}\n`);
+      }, "") + "<|assistant|>\n"
+    )
+      // 移除MD图像URL避免幻觉
+      .replace(/\!\[.+\]\(.+\)/g, "")
+      // 移除临时路径避免在新会话引发幻觉
+      .replace(/\/mnt\/data\/.+/g, "");
+    logger.info("\n对话合并：\n" + content);
   }
 
-  const content = (
-    messages.reduce((content, message) => {
-      if (_.isArray(message.content)) {
-        return (
-          message.content.reduce((_content, v) => {
-            if (!_.isObject(v) || v["type"] != "text") return _content;
-            return _content + ("<|user|>\n" + v["text"] || "") + "\n";
-          }, content)
-        );
-      }
-      return (content += `${message.role
-        .replace("system", "<|sytstem|>")
-        .replace("assistant", "<|assistant|>")
-        .replace("user", "<|user|>")}\n${message.content}\n`);
-    }, "") + "<|assistant|>\n"
-  ).replace(/\!\[.+\]\(.+\)/g, "");
   const fileRefs = refs.filter((ref) => !ref.width && !ref.height);
   const imageRefs = refs
     .filter((ref) => ref.width || ref.height)
@@ -542,7 +772,6 @@ function messagesPrepare(messages: any[], refs: any[]) {
       ref.image_url = ref.file_url;
       return ref;
     });
-  logger.info("\n对话合并：\n" + content);
   return [
     {
       role: "user",
@@ -551,19 +780,19 @@ function messagesPrepare(messages: any[], refs: any[]) {
         ...(fileRefs.length == 0
           ? []
           : [
-            {
+              {
-              type: "file",
+                type: "file",
-              file: fileRefs,
+                file: fileRefs,
-            },
+              },
-          ]),
+            ]),
         ...(imageRefs.length == 0
           ? []
           : [
-            {
+              {
-              type: "image",
+                type: "image",
-              image: imageRefs,
+                image: imageRefs,
-            },
+              },
-          ]),
+            ]),
       ],
     },
   ];
@@ -601,8 +830,13 @@ async function checkFileUrl(fileUrl: string) {
  *
  * @param fileUrl 文件URL
  * @param refreshToken 用于刷新access_token的refresh_token
+ * @param isVideoImage 是否是用于视频图像
  */
-async function uploadFile(fileUrl: string, refreshToken: string) {
+async function uploadFile(
+  fileUrl: string,
+  refreshToken: string,
+  isVideoImage: boolean = false
+) {
   // 预检查远程文件URL可用性
   await checkFileUrl(fileUrl);
 
@@ -629,6 +863,22 @@ async function uploadFile(fileUrl: string, refreshToken: string) {
   // 获取文件的MIME类型
   mimeType = mimeType || mime.getType(filename);
 
+  if (isVideoImage) {
+    const im = sharp(fileData).resize(1440, null, {
+      fit: "inside", // 保持宽高比
+    });
+    const metadata = await im.metadata();
+    const cropHeight = metadata.height > 960 ? 960 : metadata.height;
+    fileData = await im
+      .extract({
+        width: 1440,
+        height: cropHeight,
+        left: 0,
+        top: (metadata.height - cropHeight) / 2,
+      })
+      .toBuffer();
+  }
+
   const formData = new FormData();
   formData.append("file", fileData, {
     filename,
@@ -639,7 +889,9 @@ async function uploadFile(fileUrl: string, refreshToken: string) {
   const token = await acquireToken(refreshToken);
   let result = await axios.request({
     method: "POST",
-    url: "https://chatglm.cn/chatglm/backend-api/assistant/file_upload",
+    url: isVideoImage
+      ? "https://chatglm.cn/chatglm/video-api/v1/static/upload"
+      : "https://chatglm.cn/chatglm/backend-api/assistant/file_upload",
     data: formData,
     // 100M限制
     maxBodyLength: FILE_MAX_SIZE,
@@ -647,7 +899,9 @@ async function uploadFile(fileUrl: string, refreshToken: string) {
     timeout: 60000,
     headers: {
       Authorization: `Bearer ${token}`,
-      Referer: `https://chatglm.cn/`,
+      Referer: isVideoImage
+        ? "https://chatglm.cn/video"
+        : "https://chatglm.cn/",
       ...FAKE_HEADERS,
       ...formData.getHeaders(),
     },
@@ -675,14 +929,15 @@ function checkResult(result: AxiosResponse, refreshToken: string) {
 /**
  * 从流接收完整的消息内容
  *
+ * @param model 模型
  * @param stream 消息流
  */
-async function receiveStream(stream: any): Promise<any> {
+async function receiveStream(model: string, stream: any): Promise<any> {
   return new Promise((resolve, reject) => {
     // 消息初始化
     const data = {
       id: "",
-      model: MODEL_NAME,
+      model,
       object: "chat.completion",
       choices: [
         {
@@ -694,12 +949,16 @@ async function receiveStream(stream: any): Promise<any> {
       usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 },
       created: util.unixTimestamp(),
     };
+    const isSilentModel = model.indexOf('silent') != -1;
+    let thinkingText = "";
     let toolCall = false;
     let codeGenerating = false;
     let textChunkLength = 0;
     let codeTemp = "";
     let lastExecutionOutput = "";
     let textOffset = 0;
+    let refContent = "";
+    logger.info(`是否静默模型: ${isSilentModel}`);
     const parser = createParser((event) => {
       try {
         if (event.type !== "event") return;
@@ -727,6 +986,7 @@ async function receiveStream(stream: any): Promise<any> {
                 textChunkLength = 0;
                 innerStr += "\n";
               }
+
               if (type == "text") {
                 if (toolCall) {
                   innerStr += "\n";
@@ -735,20 +995,24 @@ async function receiveStream(stream: any): Promise<any> {
                 }
                 if (partStatus == "finish") textChunkLength = text.length;
                 return innerStr + text;
-              } else if (
+              } else if (type == "text_thinking" && !isSilentModel) {
+                if (toolCall) {
+                  innerStr += "\n";
+                  textOffset++;
+                  toolCall = false;
+                }
+                thinkingText = text;
+                return innerStr;
+              }else if (
                 type == "quote_result" &&
                 status == "finish" &&
                 meta_data &&
-                _.isArray(meta_data.metadata_list)
+                _.isArray(meta_data.metadata_list) &&
+                !isSilentModel
               ) {
-                const searchText =
+                refContent = meta_data.metadata_list.reduce((meta, v) => {
-                  meta_data.metadata_list.reduce(
+                  return meta + `${v.title} - ${v.url}\n`;
-                    (meta, v) => meta + `检索 ${v.title}(${v.url}) ...`,
+                }, refContent);
-                    ""
-                  ) + "\n";
-                textOffset += searchText.length;
-                toolCall = true;
-                return innerStr + searchText;
               } else if (
                 type == "image" &&
                 _.isArray(image) &&
@@ -766,7 +1030,7 @@ async function receiveStream(stream: any): Promise<any> {
                 textOffset += imageText.length;
                 toolCall = true;
                 return innerStr + imageText;
-              } else if (type == "code" && partStatus == "init") {
+              } else if (type == "code" && status == "init") {
                 let codeHead = "";
                 if (!codeGenerating) {
                   codeGenerating = true;
@@ -778,7 +1042,7 @@ async function receiveStream(stream: any): Promise<any> {
                 return innerStr + codeHead + chunk;
               } else if (
                 type == "code" &&
-                partStatus == "finish" &&
+                status == "finish" &&
                 codeGenerating
               ) {
                 const codeFooter = "\n```\n";
@@ -789,7 +1053,7 @@ async function receiveStream(stream: any): Promise<any> {
               } else if (
                 type == "execution_output" &&
                 _.isString(content) &&
-                partStatus == "done" &&
+                status == "finish" &&
                 lastExecutionOutput != content
               ) {
                 lastExecutionOutput = content;
@@ -807,8 +1071,16 @@ async function receiveStream(stream: any): Promise<any> {
           );
           data.choices[0].message.content += chunk;
         } else {
+          if(thinkingText)
+            data.choices[0].message.content = `[思考开始]\n${thinkingText}[思考结束]\n\n${data.choices[0].message.content}`;
           data.choices[0].message.content =
-            data.choices[0].message.content.replace(/【\d+†source】/g, "");
+            data.choices[0].message.content.replace(
+              /【\d+†(来源|源|source)】/g,
+              ""
+            ) +
+            (refContent
+              ? `\n\n搜索结果来自：\n${refContent.replace(/\n$/, "")}`
+              : "");
           resolve(data);
         }
       } catch (err) {
@@ -828,18 +1100,22 @@ async function receiveStream(stream: any): Promise<any> {
  *
  * 将流格式转换为gpt兼容流格式
  *
+ * @param model 模型
  * @param stream 消息流
  * @param endCallback 传输结束回调
  */
-function createTransStream(stream: any, endCallback?: Function) {
+function createTransStream(model: string, stream: any, endCallback?: Function) {
   // 消息创建时间
   const created = util.unixTimestamp();
   // 创建转换流
   const transStream = new PassThrough();
+  const isSilentModel = model.indexOf('silent') != -1;
   let content = "";
+  let thinking = false;
   let toolCall = false;
   let codeGenerating = false;
   let textChunkLength = 0;
+  let thinkingText = "";
   let codeTemp = "";
   let lastExecutionOutput = "";
   let textOffset = 0;
@@ -847,7 +1123,7 @@ function createTransStream(stream: any, endCallback?: Function) {
     transStream.write(
       `data: ${JSON.stringify({
         id: "",
-        model: MODEL_NAME,
+        model,
         object: "chat.completion.chunk",
         choices: [
           {
@@ -885,6 +1161,11 @@ function createTransStream(stream: any, endCallback?: Function) {
               innerStr += "\n";
             }
             if (type == "text") {
+              if(thinking) {
+                innerStr += "[思考结束]\n\n"
+                textOffset = thinkingText.length + 8;
+                thinking = false;
+              }
               if (toolCall) {
                 innerStr += "\n";
                 textOffset++;
@@ -892,11 +1173,26 @@ function createTransStream(stream: any, endCallback?: Function) {
               }
               if (partStatus == "finish") textChunkLength = text.length;
               return innerStr + text;
+            } else if (type == "text_thinking" && !isSilentModel) {
+              if(!thinking) {
+                innerStr += "[思考开始]\n";
+                textOffset = 7;
+                thinking = true;
+              }
+              if (toolCall) {
+                innerStr += "\n";
+                textOffset++;
+                toolCall = false;
+              }
+              if (partStatus == "finish") textChunkLength = text.length;
+              thinkingText += text.substring(thinkingText.length, text.length);
+              return innerStr + text;
             } else if (
               type == "quote_result" &&
               status == "finish" &&
               meta_data &&
-              _.isArray(meta_data.metadata_list)
+              _.isArray(meta_data.metadata_list) &&
+              !isSilentModel
             ) {
               const searchText =
                 meta_data.metadata_list.reduce(
@@ -923,7 +1219,7 @@ function createTransStream(stream: any, endCallback?: Function) {
               textOffset += imageText.length;
               toolCall = true;
               return innerStr + imageText;
-            } else if (type == "code" && partStatus == "init") {
+            } else if (type == "code" && status == "init") {
               let codeHead = "";
               if (!codeGenerating) {
                 codeGenerating = true;
@@ -935,7 +1231,7 @@ function createTransStream(stream: any, endCallback?: Function) {
               return innerStr + codeHead + chunk;
             } else if (
               type == "code" &&
-              partStatus == "finish" &&
+              status == "finish" &&
               codeGenerating
             ) {
               const codeFooter = "\n```\n";
@@ -946,7 +1242,7 @@ function createTransStream(stream: any, endCallback?: Function) {
             } else if (
               type == "execution_output" &&
               _.isString(content) &&
-              partStatus == "done" &&
+              status == "finish" &&
               lastExecutionOutput != content
             ) {
               lastExecutionOutput = content;
@@ -981,8 +1277,8 @@ function createTransStream(stream: any, endCallback?: Function) {
               index: 0,
               delta:
                 result.status == "intervene" &&
-                  result.last_error &&
+                result.last_error &&
-                  result.last_error.intervene_text
+                result.last_error.intervene_text
                   ? { content: `\n\n${result.last_error.intervene_text}` }
                   : {},
               finish_reason: "stop",
@@ -1037,10 +1333,10 @@ async function receiveImages(
           throw new APIException(EX.API_CONTENT_FILTERED);
         if (result.status != "finish") {
           result.parts.forEach((part) => {
-            const { content } = part;
+            const { status: partStatus, content } = part;
             if (!_.isArray(content)) return;
             content.forEach((value) => {
-              const { status: partStatus, type, image, text } = value;
+              const { type, image, text } = value;
               if (
                 type == "image" &&
                 _.isArray(image) &&
@@ -1055,16 +1351,12 @@ async function receiveImages(
                   imageUrls.push(value.image_url);
                 });
               }
-              if (
+              if (type == "text" && partStatus == "finish") {
-                type == "text" &&
-                partStatus == "finish"
-              ) {
                 const urlPattern = /\((https?:\/\/\S+)\)/g;
                 let match;
                 while ((match = urlPattern.exec(text)) !== null) {
                   const url = match[1];
-                  if(imageUrls.indexOf(url) == -1)
+                  if (imageUrls.indexOf(url) == -1) imageUrls.push(url);
-                    imageUrls.push(url);
                 }
               }
             });
@@ -1141,8 +1433,7 @@ async function getTokenLiveStatus(refreshToken: string) {
     const { result: _result } = checkResult(result, refreshToken);
     const { accessToken } = _result;
     return !!accessToken;
-  }
+  } catch (err) {
-  catch(err) {
     return false;
   }
 }
@@ -1151,6 +1442,7 @@ export default {
   createCompletion,
   createCompletionStream,
   generateImages,
+  generateVideos,
   getTokenLiveStatus,
   tokenSplit,
 };

src/api/routes/chat.ts

@@ -5,6 +5,9 @@ import Response from '@/lib/response/Response.ts';
 import chat from '@/api/controllers/chat.ts';
 import logger from '@/lib/logger.ts';
 
+// zero推理模型智能体ID
+const ZERO_ASSISTANT_ID = "676411c38945bbc58a905d31";
+
 export default {
 
     prefix: '/v1/chat',
@@ -13,22 +16,23 @@ export default {
 
         '/completions': async (request: Request) => {
             request
+                .validate('body.conversation_id', v => _.isUndefined(v) || _.isString(v))
                 .validate('body.messages', _.isArray)
                 .validate('headers.authorization', _.isString)
             // refresh_token切分
             const tokens = chat.tokenSplit(request.headers.authorization);
             // 随机挑选一个refresh_token
             const token = _.sample(tokens);
-            const messages =  request.body.messages;
+            const { model, conversation_id: convId, messages, stream } = request.body;
-            const assistantId = /^[a-z0-9]{24,}$/.test(request.body.model) ? request.body.model : undefined
+
-            if (request.body.stream) {
+            if (stream) {
-                const stream = await chat.createCompletionStream(request.body.messages, token, assistantId);
+                const stream = await chat.createCompletionStream(messages, token, model, convId);
                 return new Response(stream, {
                     type: "text/event-stream"
                 });
             }
             else
-                return await chat.createCompletion(messages, token, assistantId);
+                return await chat.createCompletion(messages, token, model, convId);
         }
 
     }

src/api/routes/index.ts

@@ -3,8 +3,10 @@ import fs from 'fs-extra';
 import Response from '@/lib/response/Response.ts';
 import chat from "./chat.ts";
 import images from "./images.ts";
+import videos from './videos.ts';
 import ping from "./ping.ts";
 import token from './token.js';
+import models from './models.ts';
 
 export default [
     {
@@ -22,6 +24,8 @@ export default [
     },
     chat,
     images,
+    videos,
     ping,
-    token
+    token,
+    models
 ];

src/api/routes/models.ts

@@ -0,0 +1,46 @@
+import _ from 'lodash';
+
+export default {
+
+    prefix: '/v1',
+
+    get: {
+        '/models': async () => {
+            return {
+                "data": [
+                    {
+                        "id": "glm-3-turbo",
+                        "object": "model",
+                        "owned_by": "glm-free-api"
+                    },
+                    {
+                        "id": "glm-4",
+                        "object": "model",
+                        "owned_by": "glm-free-api"
+                    },
+                    {
+                        "id": "glm-4-plus",
+                        "object": "model",
+                        "owned_by": "glm-free-api"
+                    },
+                    {
+                        "id": "glm-4v",
+                        "object": "model",
+                        "owned_by": "glm-free-api"
+                    },
+                    {
+                        "id": "glm-v1",
+                        "object": "model",
+                        "owned_by": "glm-free-api"
+                    },
+                    {
+                        "id": "glm-v1-vision",
+                        "object": "model",
+                        "owned_by": "glm-free-api"
+                    }
+                ]
+            };
+        }
+
+    }
+}

src/api/routes/videos.ts

@@ -0,0 +1,78 @@
+import _ from "lodash";
+
+import Request from "@/lib/request/Request.ts";
+import chat from "@/api/controllers/chat.ts";
+import util from "@/lib/util.ts";
+
+export default {
+
+  prefix: "/v1/videos",
+
+  post: {
+
+    "/generations": async (request: Request) => {
+      request
+        .validate(
+          "body.conversation_id",
+          (v) => _.isUndefined(v) || _.isString(v)
+        )
+        .validate("body.model", (v) => _.isUndefined(v) || _.isString(v))
+        .validate("body.prompt", _.isString)
+        .validate("body.audio_id", (v) => _.isUndefined(v) || _.isString(v))
+        .validate("body.image_url", (v) => _.isUndefined(v) || _.isString(v))
+        .validate(
+          "body.video_style",
+          (v) =>
+            _.isUndefined(v) ||
+            ["卡通3D", "黑白老照片", "油画", "电影感"].includes(v),
+          "video_style must be one of 卡通3D/黑白老照片/油画/电影感"
+        )
+        .validate(
+          "body.emotional_atmosphere",
+          (v) =>
+            _.isUndefined(v) ||
+            ["温馨和谐", "生动活泼", "紧张刺激", "凄凉寂寞"].includes(v),
+          "emotional_atmosphere must be one of 温馨和谐/生动活泼/紧张刺激/凄凉寂寞"
+        )
+        .validate(
+          "body.mirror_mode",
+          (v) =>
+            _.isUndefined(v) || ["水平", "垂直", "推近", "拉远"].includes(v),
+          "mirror_mode must be one of 水平/垂直/推近/拉远"
+        )
+        .validate("headers.authorization", _.isString);
+      // refresh_token切分
+      const tokens = chat.tokenSplit(request.headers.authorization);
+      // 随机挑选一个refresh_token
+      const token = _.sample(tokens);
+      const {
+        model,
+        conversation_id: convId,
+        prompt,
+        image_url: imageUrl,
+        video_style: videoStyle = "",
+        emotional_atmosphere: emotionalAtmosphere = "",
+        mirror_mode: mirrorMode = "",
+        audio_id: audioId,
+      } = request.body;
+      const data = await chat.generateVideos(
+        model,
+        prompt,
+        token,
+        {
+          imageUrl,
+          videoStyle,
+          emotionalAtmosphere,
+          mirrorMode,
+          audioId,
+        },
+        convId
+      );
+      return {
+        created: util.unixTimestamp(),
+        data,
+      };
+    },
+  },
+
+};

src/lib/request/Request.ts

@@ -52,7 +52,7 @@ export default class Request {
         this.time = Number(_.defaultTo(time, util.timestamp()));
     }
 
-    validate(key: string, fn?: Function) {
+    validate(key: string, fn?: Function, message?: string) {
         try {
             const value = _.get(this, key);
             if (fn) {
@@ -64,7 +64,7 @@ export default class Request {
         }
         catch (err) {
             logger.warn(`Params ${key} invalid:`, err);
-            throw new APIException(EX.API_REQUEST_PARAMS_INVALID, `Params ${key} invalid`);
+            throw new APIException(EX.API_REQUEST_PARAMS_INVALID, message || `Params ${key} invalid`);
         }
         return this;
     }