Introduction to MLIR-AIE 1.3

Meanwhile, the recent debut of Lemonade 10.0 server and FastFlowLM 0.9.35 has enabled AMD Ryzen AI NPUs to run LLMs on Linux. Additionally, AMD engineers have been developing MLIR-AIE, a compiler toolchain for AMD AI Engine devices, including Ryzen AI NPUs. Therefore, the latest MLIR-AIE v1.3 release brings notable new features to the table.

What is MLIR-AIE?

However, before diving into the new features, let’s explore what MLIR-AIE is all about. For example, MLIR-AIE provides Python APIs and more for leveraging AMD NPUs as an alternative to traditional Ryzen AI software workflows focused on AI inferencing. Furthermore, the hope has been to open up the Ryzen AI NPUs to digital signal processing and other non-AI/ML workloads thanks to the versatility of LLVM and MLIR.

Key Features of MLIR-AIE

Moreover, MLIR-AIE ties into AMD’s Peano code and benefits other AMD-Xilinx accelerator products. Meanwhile, the broad support for MLIR in the software ecosystem and even across hardware vendors allows versatility too for targeting other AMD wares such as Radeon/Instinct products via ROCm, CPUs via LLVM, etc. Some key features include:

• Python APIs for leveraging AMD NPUs
• Support for digital signal processing and non-AI/ML workloads
• Versatility for targeting other AMD products

New AIECC C++ Compiler

Additionally, the new MLIR-AIE 1.3 release introduces a new C++ aiecc compiler, which serves as an alternative to the project’s existing Python-based tooling. However, this C++ implementation provides similar functionality to the Python aiecc.py tool with the following architecture:

1. Command-line argument parsing using LLVM CommandLine library
2. MLIR module loading and parsing
3. MLIR transformation pipeline execution
4. Core compilation (xchesscc/peano)
5. NPU instruction generation
6. CDO/PDI/xclbin generation
7. Multi-device support

Benefits of the New AIECC C++ Compiler

Meanwhile, the C++ aiecc compiler delivers better performance, supports C++17, and can handle MLIR module loading and parsing, NPU instruction generation, ELF instruction generation, and other features. Furthermore, this new compiler is a significant improvement over the existing Python-based tooling.

Other Improvements in MLIR-AIE 1.3

However, the new release also brings a number of AIE2 vector improvements around BF16, FMA, reductions, and tanh. Additionally, there is more robust runtime support, more IR features, and other improvements. Meanwhile, this release brings early native Windows support as an alternative to Linux use.

Conclusion

In conclusion, the MLIR-AIE 1.3 release is a significant step forward for AMD Ryzen AI NPUs. Therefore, developers and users can now take advantage of the new AIECC C++ compiler and other improvements to unlock the full potential of these devices. Finally, with the growing support for MLIR in the software ecosystem, the future of AMD Ryzen AI NPUs looks bright.