Why big tech is fussing over open-source AI models

History is repeating itself, but this time around, it’s about the total cost of ownership (TCO), reliability, accountability, privacy, and security of open-source artificial intelligence (AI) models.

Prominent open-source AI models include Meta’s Llama, Stability AI’s Stable Diffusion, Eleuther AI’s Pythia suite of large language models (LLMs) and its smaller GPT-NeoX language model, Hugging Face’s BigScience Large Open-science Open-access Multilingual Language Model (BLOOM), and Databricks’ Dolly LLM. 

But there is much confusion and disagreement over what exactly constitutes an open-source AI model.

Here’s one reason. While companies Google, Meta, Microsoft, and Elon Musk’s xAI claim that they promote the use of open-source AI models, others including OpenAI, Apple, and Nvidia are perceived to be relying on closed-source models, thus keeping their AI technologies proprietary for strategic advantages. 

Yet, OpenAI, which began its journey as an open-source company, is now closed source. Google’s large Gemini model, too, is closed source but its smaller Gemma model is open. Even Musk’s Grok LLM does not qualify as a fully open-source AI model, according to the Open Source Initiative (OSI), a body that defines what open source means.

Further, Apple, which is typically known for its proprietary ecosystem, now has its OpenELM family of small language models, ranging from 270 million to 3 billion parameters, which is open and tailored for mobile devices and computers. And Nvidia too has begun embracing open-sourcing the code of some of its graphic processing unit (GPU) drivers, a move that will benefit Linux developers.

New definition

Last week, the OSI said that to qualify as open source, an AI system must allow its free use for any purpose without needing permission. Users should also be able to study and examine the AI system’s components, modify it to alter its output, and share the system with others, whether in its original form or with modifications, for any purpose.

Mark Zuckerberg, founder and CEO of Meta, insists that his company is committed to open-source AI. In a July 23 note, he asserted that while many big tech companies are developing “closed” models, “open source is quickly closing the gap”. Zuckerberg cited the example of how major tech companies developed proprietary versions of Unix, believing that closed systems were essential for advanced software. Yet, open-source Linux gradually gained popularity due to its flexibility, affordability, and growing capabilities. 

Over time, noted Zuckerberg in his note, Linux surpassed Unix in security and functionality, becoming the industry standard for cloud computing and mobile operating systems. Zuckerberg said he believes AI will follow a similar path. 

Llama 3 is already competitive with the most advanced models and leading in some areas, according to him. Llama, he adds, is already “leading on openness, modifiability, and cost efficiency”. 

While his claim is moot, the fact is that Stanford’s AI Index, released in April, reveals that organisations released 149 foundation models of which 65.7% were open source compared with only 44.4% in 2022 and 33.3% in 2021.

Salesforce, too, recently released a new suite of open-source large multimodal AI models this month, named xGen-MM (also known as BLIP-3).

“We open-source our models, curated large-scale datasets, and our fine-tuning codebase to facilitate further advancements in LMM (large multimodal models) research,” the authors said in a paper published on arXiv. 

India bullish

Closer home, Bengaluru-based startup Sarvam AI released what it touted as India’s “first foundational, open source” small Indic language model in mid-August. Called Sarvam2B, it is a 2 billion parameter model that has been “trained from scratch” on an internal dataset of 4 trillion tokens and covers 10 Indian languages. Sarvam AI simultaneously released Shuka 1.0, an open-source AudioLM that is an audio extension on the Llama 8B model to support Indian language voice-to-text.

While these announcements were part of the launch of Sarvam AI’s Generative AI (GenAI) platform, which includes voice-enabled, multilingual AI agents, an application programming interface (API) platform to help developers use these models, and a GenAI workbench designed for lawyers, the emphasis was on the platform and AI models being “open” as opposed to “closed, or proprietary.”

In June, Tech Mahindra announced its partnership with Dell Technologies for its LLM Project Indus in a bid to “leverage AI-optimised technologies with an open ecosystem of partners…” 

Likewise, AI4Bharat, a research lab at the Indian Institute of Technology, Madras, collaborates with Bhashini for dataset creation and CDAC Pune’s ParamSiddhi for model training, the emphasis being on fostering “open-source tools and models.”

Challenges remain

Unlike proprietary models, which can be restrictive and expensive, open-source models are freely available for modification and integration. This flexibility allows businesses to experiment with cutting-edge AI technologies without being locked into vendor-specific ecosystems. For example, companies such as Tesla have used open-source AI tools to build their autonomous driving technology, allowing them to iterate and improve rapidly.

Open-source AI also fosters innovation by enabling collaboration among a global community of developers. For startups and smaller companies with limited budgets, open-source AI provides access to powerful tools that would otherwise be out of reach. 

But open-source AI comes with its own set of challenges, particularly around total cost of ownership (TCO), security, and the need for skilled talent. Further, open-source AI models, while highly customisable, may not always meet the stringent security standards required by enterprises, a point often highlighted by big tech companies that promote closed-source AI.

In its report on ‘Dual-Use Foundation Models with Widely Available Model Weights’, released on 30 July, the Department of Commerce’s National Telecommunications and Information Administration (NTIA) recommends that the US government develop new capabilities to monitor for potential risks, but refrain from immediately restricting the wide availability of open model weights in the largest AI systems. “Open weight” models in AI refer to models where the trained parameters, or “weights,” are made publicly available. 

This transparency allows researchers and developers to examine, modify, or build upon the model’s internal structure, while allowing developers to build upon and adapt previous work. It, thus, makes AI tools more accessible to small companies, researchers, nonprofits, and individuals, according to the NTIA report.

However, as the Electronic Privacy Information Center (EPIC) noted in its 27 March comments to the NTIA request for comments, while making model weights widely available “may foster more independent evaluation of AI systems and greater competition compared to closed systems,” closed AI systems are less vulnerable to adversarial attacks and enable easier enforcement than open systems.

In its note, EPIC urged the NTIA, among other things, “to grapple with the nuanced advantages, disadvantages, and regulatory hurdles that emerge within AI models along the entire gradient of openness—and how benefits, risks, and effective oversight mechanisms shift as models move along the gradient.”

However, this approach, while sensible and balanced, is easier said than implemented.