Support tool calling For Llama 3.1

[ruifengma]

System Info / 系統信息

CentOS7
python 3.10.14
torch 2.4.0
cuda 12.2

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

0.14.0.post1

The command used to start Xinference / 用以启动 xinference 的命令

xinference-local -H 0.0.0.0 -port 8989

Reproduction / 复现过程

1. launch xinference
2. launch llama3.1
3. request with function call

Expected behavior / 期待表现

function call suppport

[qinxuye]

Thanks, @codingl2k1 Could you look at this issue?

[github-actions]

This issue is stale because it has been open for 7 days with no activity.

[yebanliuying]

docker 运行也是不支持function call和tool call

[codingl2k1]

这个暂时还不支持，很快会支持。

[github-actions]

This issue is stale because it has been open for 7 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 5 days since being marked as stale.

[ruifengma]

not planned?

[qinxuye]

We have plan to support this.

[github-actions]

This issue is stale because it has been open for 7 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 5 days since being marked as stale.

[ruifengma]

any progress?

[github-actions]

This issue is stale because it has been open for 7 days with no activity.