V-Training currently offers two model training modes: "Classification" (identifying objects and returning their corresponding categories) and "Detection" (using the Yolov3 algorithm, identifying object locations in the image and drawing wireframes). Users can freely choose the mode according to their usage scenarios. Below, we will introduce the model training methods for these two modes.
Firmware Burning
EasyLoader
EasyLoader is a simple and fast program burner. The EasyLoader provided on each product page comes with a case program related to the product. For users who do not need to customize the firmware or perform other operations, using EasyLoader to burn firmware to UnitV is the simplest solution (EasyLoader is currently only suitable for the Windows operating system).
After downloading the software, double-click to run the application, connect the M5 device to the computer via a data cable, select the port parameters, and click "Burn" to start burning.
Kflash_GUI
For users who need to specify the burning file, Kflash can also be used for firmware burning.
Click to download the firmware file.
Click below to download the Kflash_GUI burning tool for your operating system.
| Software Version | Download Link |
|---|---|
| Kflash_GUI_Windows | Download |
| Kflash_GUI_MacOS | Download |
| Kflash_GUI_Linux | Download |
Connect the device to the computer via a Tpye-C data cable, double-click to open the Kflash_GUI application, select the corresponding device port, development board type (M5StickV), firmware program, baud rate. Click download to start burning.
Kflash
For users accustomed to command line operations, Kflash can also be selected as a firmware burning tool. Click here for details .
Classification Mode
Model training requires a large number of training material images. Due to the differences in hardware between M5StickV and UnitV, the methods of material shooting also differ. Please refer to the instructions below according to the actual hardware used for shooting.
M5StickV-Material Shooting
Shooting training materials requires an SD card. Users need to download the boot-M5StickV program package and unzip all the files in the package to the SD card (M5StickV has requirements for the selection of SD cards, click here to view supported types ).
Click to download the boot-M5StickV program.
Insert the SD card before turning on the device for storing image materials. Press and hold the left power key to turn on the device. When the Training word appears on the screen, it indicates successful entry into the shooting program.
The navigation bar at the top of the screen will display the current Class number and the number of pictures taken in real-time. Press the HOME key to take pictures, and the button on the right side of the body is used to switch the Class number.
The program currently provides 10 groups of Classes for users to shoot training materials, each group of Class represents a recognition object. To achieve better training effects, users must shoot more than 3 groups of Classes (three or more recognition objects). To ensure the accuracy of recognition, the number of shots per group of Classes needs to exceed 35, otherwise, it will not be approved during cloud training. The more materials, the better the training effect and the higher the recognition rate.
UnitV-Material Shooting
The boot-UnitV program is a picture material shooting program suitable for UnitV, used for material collection in the early stage of model training.
Click to download the boot-UnitV program.
The training materials shot will be stored in the SD card by default, so please insert the SD card into the card slot of UnitV before running the boot program. (Note: UnitV has requirements for the selection of SD cards, click here to view supported types ).
Run MaixPy IDE, connect to the UnitV device. Click "Open File", open the downloaded boot.py file, and click the run button. After successful operation, the camera screen will be monitored in real-time in the upper right corner of the MaixPy IDE.
Take shooting operations with the camera screen on the IDE, press the A key to take pictures, and the B key to switch the
Class number. The output log will correspond to the Class number and the number of pictures taken each time. Click "Serial Monitor" below to view the log output.
The program currently provides 10 groups of Classes for users to shoot training materials, each group of Class represents a recognition object. To achieve better training effects, users must shoot more than 3 groups of Classes (three or more recognition objects). To ensure the accuracy of recognition, the number of shots per group of Classes needs to exceed 35, otherwise, it will not be approved during cloud training. The more materials, the better the training effect and the higher the recognition rate.
When shooting training materials, try to keep the environmental lighting conditions of the material shooting consistent with the actual recognition application scenario as much as possible. It is recommended to fill the screen with the recognition object completely and ensure that there are no other miscellaneous items in the background.
Note: To ensure the accuracy of recognition, the number of shots per group of Classes needs to exceed 35, otherwise, it will not be approved during cloud training. The more materials, the better the training effect and the higher the recognition rate.
Material Inspection and Compression
After completing the material shooting, turn off the M5StickV, remove the SD card, and copy the image materials ("train" and "vaild" two folders) from the SD to the computer through a card reader.
The Class number folder directory in the "train" and "vaild" folders is consistent. When switching Class and shooting materials, the program will simultaneously create a Class number folder in "train" and "vaild", and store the shot pictures into the Class folder in the respective directory of "train" and "vaild" according to the storage rules.
Before compressing and packaging, in addition to checking the correctness of the picture content, it is necessary to ensure that the total number of material pictures in the same Class number directory in the "train" and "vaild" folders exceeds 35. Class number directories with a total number of less than 35 should be deleted or backed up and processed by themselves. After completing the inspection work, the next step is to compress the material files. Compress the "train" and "vaild" folders into a "zip" format compressed package using a compression tool.
Data Upload to the Cloud
Click here to visit the data upload page , fill in your personal email according to the information prompt, click upload file (the uploaded file size is controlled within 200MB, and must be in zip format).
After the upload is successful, the file will enter the request queue, and the table in the lower left corner of the page will display the current queue situation.
Download the Recognition Model
Wait for the training to complete, the program file download address will be sent to the email reserved when uploading the file. Copy the download address in the email, download the program file to the local computer, unzip it, and copy it to the SD card.
Run the Recognition Program
Finally, insert the SD card into the M5StickV, turn on the device, and the program will run automatically.
The default program will identify objects according to the Class number and display them on the screen. Users can modify the displayed information by modifying the boot.py file.
Program Modification
Since UnitV does not integrate a screen, users can modify the existing program according to their needs to achieve recognition data output or the corresponding execution function after successful recognition. For example, print the recognition information through the serial port.
The following is a boot program with a serial port printing program added, only part of the content is commented, not the complete program. Please modify based on the boot program file returned by the training.
...
task = kpu.load("/sd/c33723fb62faf1be_mbnet10_quant.kmodel")//Load the model file
labels=["1","2","3","4","5","6"] #This list corresponds to the order of Classes during training materials shooting, corresponding to each recognized object. You can also modify the elements in the list to other string fields.
while(True):
img = sensor.snapshot()
fmap = kpu.forward(task, img)
plist=fmap[:]
pmax=max(plist)
max_index=plist.index(pmax)
a = lcd.display(img)
if pmax > 0.95://Determine if the recognition rate of the object is greater than 95%
lcd.draw_string(40, 60, "Accu:%.2f Type:%s"%(pmax, labels[max_index].strip()))
print(labels[max_index])//Print the name of the recognized object through the serial port.
....
Detection Mode (Yolov3)
Material Shooting
Similar to the training method in classification mode, in the detection mode (Yolov3), we still need to use a camera to shoot materials (here, the shooting program in classification mode can continue to be used). The difference is that in this training mode, multiple recognition objects are allowed to appear in the same image scene. Therefore, there is no need to shoot multiple objects in groups. The total amount of materials shot needs to be more than 100.
LabelIMG Material Marking
After the shooting is completed, we use the annotation tool LabelIMG to mark the recognition objects in the materials and generate annotation files. Users can install the Python environment themselves and run the following command in the command line to install LabelIMG through its pip package management tool.
pip install LabelIMG
After installation, running "LabelIMG" in the command line will open the annotation tool.
LabelIMG
- Switch the annotation tool to Yolo mode ---> 2. Open the image storage directory ---> 3. Choose the annotation file storage directory ---> Set to auto-save mode.
Press the "W" key to start drawing object bounding boxes, and name the object. (After adding a name, it will be recorded to the list, and can be directly selected for use in later annotations without repeated input), click the next button, switch the picture, until all the materials are completely marked.
In addition to adding annotation files, we also need to manually add a v-training.config configuration file to tell the training service how many recognition objects are included in this training. (For example, if two recognition objects are marked in the pattern example above, we need to fill in the number of categories as 2 in the configuration file, as follows)
{
"classes":2
}
After completing the above operations, place all materials in the same folder, referring to the directory structure below. Select all files, compress them into a zip format package, for upload to training.
folder----------------------
|--v-training.config
|--1.jpg
|--1.txt
|--2.jpg
|--2.txt
.....
After completing the above operations, place all materials in the same folder, referring to the directory structure below. Select all files, compress them into a zip format package, for upload to training.
The material compression package upload and model download method can refer to the operation of the classification mode training above. After the detection mode training is completed, a boot.py, and a xxxx.model file will be returned. Copy them to the SD card, then insert the SD card into the device, and turn it on to run the recognition program.
Operational Suggestions
- If your loss curve looks like the following, it indicates there is a problem with your dataset, you need to clean it up or add more pictures. If everything is normal, and adding more pictures does not help, then our network structure might not be able to solve your problem.
- If your loss curve looks like the following, it seems that the convolutional neural network has not converged at all. There might be some serious problems in your dataset, please check the dataset. If everything is normal, and adding more pictures does not help, then our network structure might not be able to solve your problem.
- If your loss curve looks like the following, but the recognition result is still not good, you may need to add more pictures and then retry; if everything is normal, and adding more pictures does not help, then our network structure might not be able to solve your problem.
- If the results you get are not very ideal, you might need to try adding a few more pictures and train again. The network might converge better this time.
Frequently Asked Questions (FAQ):
- “CONTENT: Number of Classes presented in Train and Valid
dataset is not equal.” Or “CONTENT: Train or Valid folder not found. If you are using the M5StickV software, make sure you reach enough image counts of 35 per class.” Or “CONTENT: Number of Classes presented in Train and Valid dataset is not equal.”
Answer: Your zip compressed file should only contain two folders, named train and valid (vaild is also acceptable)
In each folder, you have several folders named after classes (classes can only be numbers between 1 to 10). The number of folders inside the train and valid folders should be the same. The names of the folders correspond to each other, starting from 1.
- “CONTENT: Lake of Enough Valid Dataset, Only 16 pictures found, but you need 20 in total.” Or “CONTENT: Lake of Enough Train Dataset, Only 43 pictures found, but you need 45 in total.”
Answer: You do not have enough photos in your train or valid folder. You need to add more photos. Or you might have accidentally added an extra class.
- “CONTENT: Number of Classes should larger or equal to two.”
Answer: Sorry, a single class is not supported, your train and valid folders need to have at least two or more class folders.
- “CONTENT: Unexpected error happened during checking dataset, cannot identify image file 'dataset_tmp/xxxxxxxx_dataset/train/2/1.webp'”
Answer: The system could not read the image while processing it. You might need to replace this picture.