Microsoft CEO Satya Nadella says he is “very optimistic” that technology has sufficiently advanced to support more complex, next-gen capabilities such as multi-agent AI orchestration, and open source is a key component.

Chips are getting better, cycle times are faster, and system software, model architecture, and kernels are constantly being optimized, resulting in what he described as a 10x performance boost every six to 12 months.

“We are in some crazy sort of hyperdrive Moore’s Law,” Nadella said during a fireside chat with Meta CEO Mark Zuckerberg at Meta’s inaugural LlamaCon developer event. “Any one of these tech platform shifts has not been about one S curve, it’s been multiple S curves that compound.”

Distillation is ‘like magic’

Microsoft’s ideal, Nadella said, is an orchestration layer that will offer the ability to mix and match AI models, with users pulling different aspects of intelligence from different models in areas where they excel. Open source “absolutely has a massive, massive role to play” in the building out of such platforms.

“I’m not dogmatic about closed source or open source, both of them are needed in the world,” he said. “And customers will demand them, right?”

Whether it be interplay between SQL, MySQL, Postgres, Linux, Windows, or other products, having a posture that allows interoperability is incredibly important, Nadella noted. Enterprise customers want to be able to distill custom models built with their intellectual property (IP), and open weight models will have a “huge structural advantage” in supporting that, compared to closed models.

“Taking a large model and being able to distill it into a smaller model that has even that same model shape is a big use case,” he said.

The challenge is making that available to those unable to build their own infrastructure, or those less technically sophisticated, he noted. The goal for a company like Microsoft is to build the tooling for these models as a service. Hyperscalers can deploy this infrastructure as a cloud service and build tools around it.

So, Nadella pointed out, every tenant of Microsoft 365 could have a distilled, task-specific model that they could create as an agent or a workflow and invoke from within Copilot. “That, to me, is a breakthrough scenario,” he said.

Zuckerberg agreed: “That just seems like one of the coolest things that I think is going to get built.” He also called model distillation “like magic.”

“You basically can make it so that you can get 90% or 95% of the intelligence of something that is 20 times larger in a form factor that is so much cheaper and more efficient to use,” he said.

As powerful as it is, though, there are concerns about doing distillation in a safe and secure way, he conceded, particularly when open source models are coming from different countries (such as DeepSeek, from China). He noted: “How do you make sure that you’re not inheriting security vulnerabilities or different values that are kind of problematic?”

AI takes a greater role in coding

Nadella and Zuckerberg also agreed on the dramatic possibilities for AI in coding. Nadella said that AI generates “fantastic” Python code, and estimated that up to 30% of the code currently in Microsoft’s repositories has been written by software.

Zuckerberg said Meta’s bet is that, in the next year, “maybe half” of software development will be completed by AI. Eventually, he said, “every engineer is effectively going to end up being more like a tech lead with their own little army of triggering agents.”

But successful integration is the sweet spot; AI must work seamlessly into current repos and developer workflows, Nadella emphasized. “It’s one thing to build a new greenfield app, but none of us get to work on complete greenfield all the time,” he said.

Instead, devs are working in large code bases with many workflows, so effective integration within the tool chain is paramount. This is the systems work that any engineering team needs to be doing, said Nadella, and it’s where teams will see the greatest productivity gains.

AI is an ‘existential priority’

Ultimately, Nadella described AI as “a pretty existential priority” and said its success “will come down to developers being able to go at it fearlessly.”

The world requires productivity gains in every function, from healthcare, to retail, to broad knowledge work, he said, and AI has the promise to deliver that. “Software in this new form of AI is the most malleable resource we have to solve hard problems.”

Still, it’s not enough that the technology is just there, he noted; it also requires a management and cultural change. Case in point: Electricity was around for 50 years before people figured out that it could revolutionize factories.

Databricks: Everything’s going towards open source

Ali Ghodsi, Databricks co-founder and CEO, also sat down with Zuck at LlamaCon; one of his big takeaways was that, in the long run, everything is going to move towards open source.

“You get this sort of open research and open sharing,” he said. “You get a whole world working on these models, and you just have much, much more rapid progress.”

Like Nadella, Ghodsi described a “cross pollination” between different models. People are slicing, mixing and matching, and cobbling different elements together, tasks that would be “completely impossible” without open source.

Zuckerberg added that it’s “really gratifying” to move more towards open source, pointing out that just a year ago, Llama was the main major open source model, which, of course, is now no longer the case.

This kind of ongoing exploration led Ghodsi to conclude, “It’s Day Zero of the AI era. The most amazing applications are yet to be invented. This is a complete white space.”

The “data advantage” will be the underpinning of success, Ghodsi said. Builders need to collect the right data, refine it, pass it through models — and repeat — to create a ‘flywheel’ or network effect, continually iterating on and improving products with use.

He urged devs: “Try it out, use it, explore those crazy new ideas. It’s going to be exciting times ahead.” 

Explore IT Tech News for the latest advancements in Information Technology & insightful updates from industry experts! 

Source: https://www.infoworld.com/article/3975472/open-source-has-a-massive-role-to-play-in-ai-orchestration-platforms-says-microsoft-ceo.html