As AI technology evolves, the latest large-scale language models, LLMs, running in any environment from the cloud to the edge, are essential to maximizing the potential and opportunities of AI.
The challenge, however, is that they are computationally resource- and energy-intensive.

To address this issue, Meta has released the latest version of its open source LLM (Llama 3.2), which improves efficiency to quickly provide users with an unprecedentedly fast AI experience.
Running the latest LLM on Arm CPUs has achieved a 5X improvement in prompt processing and a 3X improvement in token generation, achieving 19.92 tokens/second in the generation phase, according to the company.
Latency improvements have been seen especially when processing AI workloads on devices, allowing for more efficient AI processing. Scaling AI processing at the edge also reduces energy and cost by reducing power consumption from data traveling to and from the cloud.

AI performance on Arm CPUs has improved dramatically, and more than 100 billion Arm-based devices are expected to be AI-enabled in the future. This is expected to lead to further use of AI in everyday life and business.
Learn more about the latest version of open source jointly developed by Arm and Meta and how rapidly accelerating AI technologies, especially tools such as “Kleidi” and “PyTorch,” have contributed to improved AI performance. It is a subject of great interest.