Renesas RA8P1 MCU: Powering High-Performance Edge AI Applications

There has been a significant shift in the AI market. Previously, AI processing was primarily done in the cloud. Endpoint devices gathered data from sensors and sent it to the cloud for inference processing and decision-making, with results sent back to the endpoint devices. This approach required substantial bandwidth for transmitting large amounts of data to the cloud. According to the International Data Corporation (IDC), 79.4ZB of data will be sent from IoT devices to the cloud in 2025.

However, there is a growing trend towards AI inference at the edge. This approach enables fast real-time responses and increased data privacy and security, while avoiding the latency and costs associated with cloud connections. It also lowers power consumption, making it suitable for battery-powered IoT applications. AI at the edge provides advantages such as autonomy, lower latency, lower power, lower costs, and higher security, making it attractive for new and emerging applications.

MCUs are increasingly being used for Edge AI. They offer better real-time response, lower power consumption, and lower costs compared to MPUs. High-performance MCUs with integrated hardware accelerators can handle linear algebra operations like dot products and fast, parallel matrix multiplications, convolutions, and transpositions, which are essential for neural network processing. Smaller neural network models, software libraries, and ecosystem solutions optimized for resource-constrained MCUs are also available.

Build Power Efficient AI Applications with RA8P1 AI-Accelerated MCU

The RA8P1 MCUs from Renesas are the company's first AI-accelerated single and dual-core MCUs. They deliver significant performance improvements and lower power consumption, featuring high-performance Arm Cortex-M85 and Cortex-M33 CPU cores with the Arm Ethos-U55 neural network processor (NPU). Built on the advanced TSMC 22nm ultra-low leakage (22ULL) process, RA8P1 MCUs provide an unprecedented 7300+ CoreMark raw performance and 256GOPS of AI performance, making them ideal for Edge AI and IoT applications.

Together with large memory and a rich peripheral set, these devices enable demanding Voice, Vision AI, and Real-Time Analytics applications directly on the device. Dual-core RA8P1 MCUs offer high processing power, efficient task partitioning between the two cores, and improved real-time performance. Advanced security features, including immutable memory and TrustZone, ensure truly secure AI applications.

The Ethos-U55 NPU embedded in the RA8P1 is a dedicated processor optimized for executing core operations of neural network models, such as matrix multiplications and convolutions, more efficiently and with lower power consumption than the CPU core. It is optimized for lower precision arithmetic (8-bit integer) used in AI models, reducing complexity, memory usage, and power consumption without compromising inference accuracy.

Renesas has demonstrated the performance uplift of the RA8P1 MCUs using Ethos-U55 for inference processing. Use cases like Image Classification, Keyword Spotting, Visual Wake Words, Object Detection, and Anomaly Detection show a significant performance boost with the Ethos-U55 NPU compared to the CPU core.

Enable Faster Application Development with RUHMI Framework

The RA8P1 AI solution features the Robust Unified Heterogeneous Model Integration (RUHMI) Framework, which provides AI developers with all the tools necessary for faster and more efficient AI development. This is Renesas' first comprehensive AI framework for MCUs and MPUs, integrated into the e2 studio IDE to generate and deploy highly optimized neural network models in a framework-agnostic manner. RUHMI supports commonly used ML frameworks like TensorFlow Lite, Pytorch, and ONNX, along with ready-to-use application examples and models optimized for RA8P1.

The typical AI workflow with the RUHMI Framework includes model optimization and compilation (offline), data input and pre-processing, execution on the NPU, and output and post-processing. A pre-trained AI model is input via commonly used frameworks, quantized to an INT8 intermediate format, and optimized. The model is then compiled to an MCU-friendly format. Raw input data is captured by the RA8P1 MCU and pre-processed by the high-performance Cortex-M85 core. The CPU core sends the pre-processed input data and the compiled AI model's command stream to the Ethos-U55 NPU for execution. Once the NPU processes all layers of the neural network, it outputs the inference results back to the main CPU for post-processing and action.

AI Applications Enabled by RA8P1

The RA8P1 MCU, with its high inference performance, low power consumption, and real-time processing capability, is ideal for a wide range of AI applications across various market segments. These include:

• Voice AI**: Keyword spotting, voice recognition, speech recognition, noise reduction, and speaker identification.
• Vision AI**: Object detection, image classification, gesture recognition, face recognition, image analysis, and driver/vehicle monitoring.
• Real-Time Analytics**: Anomaly detection, vibration analysis, and predictive maintenance.
• Multimodal Applications**: Smart HMI with voice and vision capability, enhanced surveillance cameras, and robotics with visual and auditory inputs.

#### Application Example 1: Image Classification on RA8P1

The RA8P1 integrates the CPU cores, NPU, memory, and peripherals needed to build a vision AI application on a single chip. The application analyzes an input image and assigns it a pre-assigned label or category. The neural network model is trained on a vast dataset of images and deployed on the RA8P1 MCU. For inferencing, a new input image is fed into the model and processed through the layers of the trained network. The output layer provides the probabilistic distribution across all categories, and the category with the highest probability is assigned as the image's label. This output data can be sent to a display or cloud. In our implementation, we see a 33x improvement in inference speed with the Ethos-U55 compared to using the CPU core.

Image classification can be used in diverse applications such as security (identifying weapons and people), retail (creating product catalogs and inventory management), agriculture (identifying crop disease), smart cities (identifying traffic lights and pedestrians), and smart appliances (identifying objects inside a refrigerator).

#### Application Example 2: Driver Monitoring System on RA8P1

The Nota-AI Driver Monitoring System (DMS) is an in-cabin safety solution that enhances road safety. Using the RA8P1, the Nota-AI DMS detects unregistered drivers, driver drowsiness, cell phone usage, and driver distractions like smoking. With the higher performance of the RA8P1, we see a 4x-24x increase in inference performance for the four models used in this application—face detection, face landmark, eye landmark, and phone detection. The DMS finds applications in dashboard cameras, vehicle traveling data recorders, and driver monitoring systems.

Both these Vision AI applications make optimal use of the resources on the RA8P1, showcasing its capabilities in real-world scenarios.

By leveraging the RA8P1 MCU, developers can create powerful, efficient, and secure Edge AI applications that meet the demands of today's connected world.