List Models

The models endpoint lists the models installed on the device and available through the OpenAI-compatible API.

Install a Model

Before use, please refer to the software update guide for the corresponding device to complete the addition of M5Stack apt software source information and update the model list.

View the list of available llm deb packages. Packages named in the format llm-model-name are model packages.

apt list | grep llm-model-

Select and install a model package compatible with the device platform. Module LLM and LLM630 Compute Kit use AX630C models, AI Pyramid uses AX650 models, and LLM-8850 Card uses AXCL models. Refer to the model library for supported platforms and installation commands.

apt install llm-model-qwen2.5-0.5b-p256-ax630c

View Available Models

After installation, query the OpenAI-compatible API for the models available on the device. For remote access, replace the service address in the example with the device's actual IP address.

Note

After installing a new model each time, you need to manually execute systemctl restart llm-openai-api to update the model list.

Curl Call

curl http://127.0.0.1:8000/v1/models \
  -H "Content-Type: application/json"

Python Call

from openai import OpenAI
client = OpenAI(
    api_key="sk-",
    base_url="http://192.168.20.186:8000/v1"
)

client.models.list()
print(client.models.list()) 

Response Example

SyncPage[Model](data=[
Model(id='melotts_zh-cn', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='qwen2.5-0.5B-prefill-20e', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='sherpa-ncnn-streaming-zipformer-20M-2023-02-17', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='sherpa-ncnn-streaming-zipformer-zh-14M-2023-02-23', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='single_speaker_english_fast', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='single_speaker_fast', created=0, object='model', owned_by='user', permission=[], root=''),
Model(id='qwen2.5-0.5B-p256-ax630c', created=0, object='model', owned_by='user', permission=[], root='')
],
object='list') 

Next Overview

Page Tools

PDF

Model Library

Module LLM

Multimodal Models

Computer Vision

Speech Recognition & Audio

Text to Speech

Development Examples

Module LLM

LLM630 Compute Kit

AI Pyramid

OpenAI-Compatible API