add api server #417
Open
add api server #417
+136
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tuhahaha
reviewed
Dec 26, 2024
| @@ -0,0 +1,71 @@ | |||
|
|
|||
There was a problem hiding this comment.
这个功能适合注册为一个llm class，参考这个，即当在agent中指定使用这个llm时，会自动启动这个llm server，之后进行交互。
There was a problem hiding this comment.
LLM_REGISTRY 是增加支持的后端llm接口把，我这个是接入qwen-agent后提供给前端使用的
There was a problem hiding this comment.
和WebUI是平级的，WebUI是提供一个webui的供用户使用的界面，我的pr提供的是http api形式的供用户调用的api。
具体是实现 #247 (comment) 中提到的
（推荐，但qwen-agent还没自带这个功能，开发中）用户自行学习下FastAPI，通过FastAPI把qwen-agent封装成http api服务（如果有遇到阻塞问题，可以开多线程解决）。
There was a problem hiding this comment.
Get了，我尽快测试下，没问题的话会merge这个PR
|
您好，感谢贡献！如上面评论的，希望可以把这个接口规范为一个llm class，这样可以用原始的方式调用，而不引入新的接口 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
抛砖引玉，增加一个自用的非openai格式的apiserver（openai貌似没看到单次请求返回多条结果的格式）
#247