Indian startup unveils system to run AI without advanced chips

Palo Alto and Bengaluru-based Ziroh Labs exemplifies the democratizing potential of Indian innovation.

What is the news?

Ziroh Labs has developed Kompact AI in partnership with IIT Madras, enabling large AI models to run on everyday CPUs instead of expensive GPU hardware.
In a recent demonstration, the team showed their framework running models like Meta’s Llama 2 and Alibaba’s Qwen2.5 on a standard laptop with an Intel Xeon processor. This technology that has been tested by US chipmakers Intel and AMD.

Why is it interesting?

This breakthrough directly addresses the ‘AI divide’ highlighted by IIT Madras Director V. Kamakoti: “Only those with high-end expensive GPU-powered resources can access, develop and deploy powerful AI.”
Ziroh Labs represents India’s unique approach to AI innovation, creating frugal yet powerful solutions that could democratize access globally. As former Sun Microsystems CSO William Raduchel notes, this will have “a profound market impact in the years ahead.”
At GIA, we’re particularly excited by startups solving fundamental infrastructure challenges in AI. By enabling inference on common hardware, Ziroh Labs is creating a bridge for developers worldwide who struggle with GPU shortages and costs, showcasing how Indian innovation can make advanced technology more inclusive.