该接口用于获取当前设备已安装的模型列表。
llm-model-name 格式命名的为模型包。apt list | grep llm-model- apt 指令安装软件包, 如安装 llm-model-qwen2.5-0.5b-p256-ax630c 包。这个需要根据平台选择对应的模型。ModuleLLM/LLM630 Compute Kit 平台模型后缀为 -ax630c,AI Pyramid 平台后缀为 -ax650,LLM8850 平台后缀为 -axcl,具体参考模型介绍章节apt install llm-model-qwen2.5-0.5b-p256-ax630c 安装好的模型可直接通过 OpenAI API 查询当前设备可用的模型列表。程序执行前需将下方 base_url 的IP部分修改为设备实际IP地址。
curl http://127.0.0.1:8000/v1/models \
-H "Content-Type: application/json" from openai import OpenAI
client = OpenAI(
api_key="sk-",
base_url="http://192.168.20.186:8000/v1"
)
client.models.list()
print(client.models.list()) SyncPage[Model](data=[
Model(id='melotts_zh-cn', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='qwen2.5-0.5B-prefill-20e', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='sherpa-ncnn-streaming-zipformer-20M-2023-02-17', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='single_speaker_english_fast', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='single_speaker_fast', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='qwen2.5-0.5B-p256-ax630c', created=0, object='model', owned_by='user', permission=[], root='')
],
object='list')