LLM Inference on Android with KleidiAI, MediaPipe, and XNNPACK

Arm has worked with the Google AI Edge team to integrate KleidiAI into the MediaPipe framework through XNNPACK. These improvements increase the throughput of quantized LLMs running on Arm chips that contain the i8mm feature. This presentation will share new techniques for Android developers who want to efficiently run LLMs on-device.

Schedule