StackFlow AI Platform
Model Library
Speech Recognition & Audio
Development Examples
OpenAI-Compatible API
OpenAI-Compatible API Overview and Deployment
StackFlow can provide a local HTTP interface compatible with the OpenAI API format. Existing OpenAI SDKs, Chatbox, and other compatible clients can access local models through the device IP address for chat generation, speech recognition, and speech synthesis.
“OpenAI-compatible” refers only to API-format compatibility. The models and inference services run on the local device, and requests are not sent to the OpenAI cloud.
Before Deployment
Follow the software update guide for your device to add the M5Stack APT repository and update its package index.
- Module LLM package update
- LLM630 Compute Kit package update
- AI Pyramid package update
- Install StackFlow on LLM-8850 Card
Install the OpenAI-Compatible API
- Access the device terminal through ADB, UART, or SSH, then update the APT package index.
apt update- Install the StackFlow base runtime and OpenAI-compatible API service.
apt install lib-llm llm-sys llm-openai-api- Install the services required by your application.
| AI Capability | Service | Installation Command |
|---|---|---|
| Large language model | llm-llm | apt install llm-llm |
| Multimodal model | llm-vlm | apt install llm-vlm |
| Whisper speech recognition | llm-whisper | apt install llm-whisper |
| MeloTTS speech synthesis | llm-melotts | apt install llm-melotts |
Each service also requires a model package compatible with the device platform. After installing a model, refresh the model list exposed by the OpenAI-compatible API:
systemctl restart llm-openai-apiPython SDK
Prepare a Python environment on the PC and install the official OpenAI Python library with pip. The library can then access the OpenAI-compatible API provided by the device and invoke its local models.
pip install openai