GTBank Ad
HeadlinesTech

Meta’s AI Team Focused On Beating GPT-4, Court Documents Reveal

Court-unsealed messages from a Meta AI copyright case reveal the company’s AI leaders were fixated on outperforming OpenAI’s GPT-4 while developing Llama 3. 

Meta’s VP of Generative AI, Ahmad Al-Dahle, emphasised the need to “win this race,” citing Meta’s extensive GPU resources. Internal discussions also highlighted competition with Anthropic’s Claude, dismissing rivals like French startup Mistral as insignificant.

“Honestly… Our goal needs to be GPT-4,” said Meta’s VP of Generative AI, Ahmad Al-Dahle, in an October 2023 message to Meta researcher Hugo Touvron. “We have 64k GPUs coming! We need to learn how to build frontier and win this race.”

Though Meta releases open AI models, the company’s AI leaders were far more focused on beating competitors that don’t typically release their model’s weights, like Anthropic and OpenAI, and instead gate them behind an API. Meta’s executives and researchers held up Anthropic’s Claude and OpenAI’s GPT-4 as a gold standard to work toward.

The French AI startup Mistral, one of Meta’s biggest open competitors, was mentioned several times in the internal messages, but the tone was dismissive.

“Mistral is peanuts for us,” Al-Dahle said in a message. “We should be able to do better,” he said later.

READ MORE:

Tech companies are racing to upstage each other with cutting-edge AI models these days, but these court filings reveal just how competitive Meta’s AI leaders truly were — and seemingly still are. At several points in the message exchanges, Meta’s AI leads talked about how they were “very aggressive” in obtaining the right data to train Llama; at one point, an executive even said that “Llama 3 is literally all I care about,” in a message to coworkers.

Prosecutors in this case allege that Meta’s executives occasionally cut corners in their mad race to ship AI models, training on copyrighted books in the process.

Touvron noted in a message that the mix of datasets used for Llama 2 “was bad,” and talked about how Meta could use a better mix of data sources to improve Llama 3. Touvron and Al-Dahle then talked about clearing the path to use the LibGen dataset, which contains copyrighted works from Cengage Learning, Macmillan Learning, McGraw Hill, and Pearson Education.

“Do we have the right datasets in there[?]” said Al-Dahle. “Is there anything you wanted to use but couldn’t for some stupid reason?”

Meta CEO Mark Zuckerberg has previously said he’s trying to close the performance gap between Llama’s AI models and closed models from OpenAI, Google, and others. The internal messages reveal the intense pressure within the company to do so.

“This year, Llama 3 is competitive with the most advanced models and leading in some areas,” said Zuckerberg in a letter from July 2024. “Starting next year, we expect future Llama models to become the most advanced in the industry.”

When Meta ultimately released Llama 3 in April 2024, the open AI model was competitive with leading closed models from Google, OpenAI, and Anthropic, and outperformed open options from Mistral. However, the data Meta used to train its models — data Zuckerberg reportedly gave the green light to use, despite its copyright status — are facing scrutiny in several ongoing lawsuits.

Credit: TechCrunch

Join Our Community. 👋

Sign up to receive our Daily News Round-Up in your inbox.

We don’t spam! Read our privacy policy for more info.

Join our 👋
email list

Sign up to receive news updates in your inbox.

We don’t spam! Read our privacy policy for more info.

Follow Lagos Post Online Channel on WhatsApp:

Back to top button

Adblock Detected

Please consider supporting us by disabling your ad blocker