DeepSeek, as a representative of AI Large Language Models, has attracted extensive attention in the global artificial intelligence field with its excellent reasoning ability and efficient text-generation technology. As the latest iteration of this series, DeepSeek-R1 has achieved breakthroughs in technological dimensions such as the leap in long-text processing efficiency, multi-modal expansion planning, and embedded adaptation.

RK3588, as the flagship chip launched by Rockchip, has become an ideal platform for embedded AI applications with its multi-core heterogeneous computing power and powerful CPU, GPU, and NPU performance. The in-depth integration of DeepSeek-R1 and the OK3588-C development board marks the extension of large AI models from the cloud to the edge. This collaborative model of "advanced algorithm + customized chip" not only meets key requirements such as real-time performance and privacy protection on the edge side but also builds a complete value chain from technology R & D to industrial empowerment, providing a reusable innovation paradigm for the intelligent transformation of various industries. Next, let's delve into how this process is specifically implemented.

01 Transplantation Process

(1) Download the DeepSeek-R1 Source Code

Download the DeepSeek-R1-Distill-Qwen-1.5B weight file from the official website of DeepSeek-R1 on the Ubuntu virtual machine.

(2) Install the Conversion Tool

Create a virtual environment on Ubuntu and install RKLLM-Toolkit to convert the DeepSeek-R1 large language model into the RKLLM model format and compile the executable program for board-side inference.

(3) Model Conversion

Use RKLLM-Toolkit to convert the model. RKLLM-Toolkit provides model conversion and quantization functions. As one of the core functions of RKLLM-Toolkit, it allows users to convert large language models in Hugging Face or GGUF format into RKLLM models, enabling the RKLLM models to be loaded and run on the Rockchip NPU.

(4) Compile the DeepSeek-R1 Program

Install the cross-compilation toolchain to compile the RKLLM Runtime executable file. This program includes all processes such as model initialization, model inference, callback function processing output, and model resource release.

(5) Model Deployment

Upload the compiled RKLLM model and executable file to the board for execution. Then, you can have a conversation with DeepSeek-R1 on the debugging serial port of the OK3588-C development board without an internet connection.

02 Demo

DeepSeek-R1 is a multi-functional artificial intelligence assistant that can provide efficient and comprehensive support in multiple fields. Even the local offline version can give accurate and practical suggestions based on its powerful data-processing ability and extensive knowledge repository, whether it's for daily information retrieval needs, maintenance guidance for professional equipment, solutions to complex mathematical problems, or assistance in completing programming tasks. It has become a reliable partner for users in exploring various fields.

(1) General Information Search

DeepSeek-R1 can quickly retrieve and provide accurate information. For example, when asked about "Forlinx Embedded Technology Co., Ltd.", DeepSeek-R1 can introduce the company's background, main business, product features, etc. in detail, helping users understand the company comprehensively.

(2) Maintenance Advice for Professional Equipment Problems

For professional equipment problems, DeepSeek-R1 can provide detailed fault analysis and solutions. For instance, regarding the problem of the PLC reporting error code E01, R1 analyzes the possible causes of the fault, such as power supply problems, wiring errors, or hardware failures, and provides corresponding solution steps to help users quickly troubleshoot the fault.

(3) Solving Math Problems

DeepSeek-R1 has excellent mathematical operation ability and is good at solving various mathematical problems. For example, when facing the problem of purchasing red and blue pencils, it can skillfully construct equations and solve them quickly, accurately calculating the number of red and blue pencils to be purchased, providing users with immediate and accurate solutions. Moreover, DeepSeek-R1 also includes detailed verification steps to ensure the accuracy of the results.

(4) Programming Tasks

DeepSeek-R1 performs excellently in programming and can write code according to user requirements. For example, for the serial communication requirements of the OK3588-C development board, R1 can provide a complete C-language example program, including functions such as serial port initialization, data receiving, and transmission, to help users achieve serial communication.

DeepSeek-R1 has demonstrated its excellent practical value and high-efficiency performance in many fields and has become an indispensable intelligent partner in users' work.

03 Performance Evaluation

After the transplantation, we conducted a comprehensive performance evaluation to verify the running effect of DeepSeek-R1 on the OK3588-C development board. After detailed testing and comparison, the following key performance indicators are summarized:

Real-Time Performance: The answers output by DeepSeek-R1 are clear and fluent, without any delay or stuttering.

CPU Usage: The CPU usage of DeepSeek-R1 running on the OK3588-C development board is 12%-17%. This performance proves the efficiency of the framework, enabling it to run smoothly even on resource-constrained devices and expanding its application scenarios and commercial potential.

Memory Usage: During the above-mentioned function tests, the memory usage rate of DeepSeek-R1 is about 825MB. This ensures the smooth operation of the system, avoiding performance problems caused by insufficient memory and making the user's application experience more seamless.

NPU Usage: As shown in the following figure, when DeepSeek-R1 runs on the OK3588-C development board, it can allocate computing resources more efficiently, and the loads of the three cores of its NPU (Neural Processing Unit) all reach 83%.

In this demonstration, it comprehensively showed the actual application results of DeepSeek-R1, and its powerful functions and high-efficiency performance were strongly proven. The subsequent articles will introduce in detail the transplantation details of DeepSeek-R1 to the OK3588-C development board, including various transplantation methods and operation steps.

If you are interested in this process, please feel free to contact us. Forlinx Embedded will provide you with comprehensive technical support and detailed guidance. We look forward to exploring more possibilities with you.

---

Dear friends, we have created an exclusive embedded technical exchange group on Facebook, where our experts share the latest technological trends and practical skills. Join us and grow together!