Invisible Architect: How Anat Heilper Builds the Foundations AI Depends On

Given the significant daily tech growth, it’s easy to forget that living, breathing humans are behind each new AI breakthrough or algorithm programming. Technologist Anat Heilper is responsible for breathing life into the architectural complexity that makes AI possible.

As the director of AI and Advanced Technologies at Intel, Heilper’s sweet spot is the intersection of hardware and software. Simply put? She’s responsible for translating software’s operational goals into efficient hardware implementations, ensuring that the theoretical mechanics of software are optimized and executed. She has amassed three patents on compiler optimization for AI accelerators and received the highest award at Intel to be awarded to an individual, the IAA award.

Of course, Heilper doesn’t do it all alone. As Chief Software Architect on Intel’s NNP-I (Neural Network Processor for Inference), she helmed a team of over 80 engineers across continents, all while coordinating a layered tech stack that helped create next-generation inference. The NNP-I Spring Hill, Intel's Data Center Inference Chip, where her leadership and deep technical expertise drove the chip’s success. Her team’s efforts led to NNP-I achieving best-in-class performance and power efficiency, with a groundbreaking 4.8 TOPS/W for major data center inference workloads. This performance milestone was presented at HotChips 2019 and published in IEEE, solidifying the chip’s status as a leader in the field.

Heilper’s career stands out, given its versatility and variety. That makes perfect sense when you think about the job she’s tasked with doing. As an example, her leadership and contributions to Gaudi2 development and execution, Gaudi2 execution on Renset-50 achieved 3.2× the output of Gaudi1, 1.9× that of an 80GB Nvidia A100, and 4.1× that of an Nvidia V100. It takes significant adaptability and training to understand the complexities of tech and its physical characteristics and limitations.

Between AI and image processing to real-time systems, Heilper has done it — whether it’s designing kernel-level instructions or working with product teams. This deep and varied experience has given her the tools to understand system behavior at every level.

Now that Heilper has made a home at Intel’s CTO office, she plays a significant role in developing AI technology architecture that doesn’t exist. A key part of her job is balancing her technical know-how with an architectural understanding that can piece together what a system needs to look like before it’s even built.

Not only is Heilper’s experience and technical training well-rounded, but she’s remained fluid throughout every step of her career. She has worked in different fields, teams, and stacks — be it writing low-level device drivers or designing AI compilation flows.

Yet that’s precisely why her professional building blocks are just as layered as the work Heilper produces to bring tech to the physical plane. She embraces change and never backs down from a question she can’t answer until she can. Heilper has broken barriers as a woman and in various fields because of that drive and zest for knowledge.

Over time, Heilper has earned authority through her mastery of complex technical domains. This expertise has allowed her to bridge the gap between technical and cultural architecture, enabling her to lead large-scale projects with confidence.

Now that Heilper is working on the future of AI, she has to take scalability, sustainability, and performance into account in a way that hasn’t been done before. As a result, she views the future of AI as far more than an engineering quandary.

According to Heilper, it’s a design problem that requires creativity and imagination just as much as precision and technical know-how.

There are still many unknowns about where AI is headed next. But whatever that future holds, it’s safe to say that Anat Heilper will be behind the scenes building those blocks piece by piece.