Meta has launched Meta AI assistant Llama 3, the newest addition to their Llama family of free generative AI models. Or, more precisely, the business has presented two models from its new Llama 3 family, with the remaining variants to be released at an unannounced later date.
Meta Release the new Meta AI Assistant models — Llama3 8B, 8 billion parameters are used, and Llama3 70B, there are 70 billion parameters used by this as a “major leap” in performance and efficiency over the previous generation Llama models, Llama3 8B, and Llama2 70B. (Parameters fundamentally describe an AI model’s competence on a subject, such as text analysis and generation; models with a larger parameter count are often more capable than models with fewer parameters.) According to Meta, Llama 3 8B and Llama 3 70B, which were trained on two custom-built 24,000 GPU clusters, are among the best-performing generative AI models available today.
Meta AI Llama 3 8B outperforms other open models, including Mistral’s Mistral 7B and Google’s Gemma 7B, both of which have 7 billion parameters, on at least nine benchmarks: MMLU, ARC, DROP, GPQA (a set of physics, biology, and chemistry related questions and queries), HumanEval (a code generation test), GSM-8K (math word problems and quesions), MATH (another mathematics benchmark), AGIEval (a problem-solving test set), and BIG-Bench Hard.
Mistral 7B and Gemma 7B aren’t exactly cutting-edge (Mistral 7B was introduced in September), and in a couple of the benchmarks Meta mentions, Llama 3 8B scores only a few percentage points higher than either. However, Meta claims that the Llama 3 70B model, which has a higher parameter count, is competitive with flagship generative AI models such as Gemini 1.5 Pro, the most recent addition to Google’s Gemini series.
Llama 3 70B outperforms Gemini 1.5 Pro in MMLU, HumanEval, and GSM-8K, and, while it does not compete with Anthropic’s most performant model, Claude 3 Opus, it outperforms the second-less power model in the Claude 3 series, Claude 3 Sonnet, on five benchmarks (MMLU, GPQA, HumanEval, GSM-8K, and MATH).
For what it value, Meta also created its own test set which included use cases ranging from coding and creative writing to reasoning and summarization, and surprise — Llama3 70B outperformed Mistral’s Mistral Medium model, OpenAI’s GPT-3.5, and Claude Sonnet. Meta claims that it barred its modeling teams from seeing the set to ensure neutrality, but considering that Meta created the test, the results must be taken with a grain of salt.
Meta Founder/CEO Mark Zuckerberg acknowledged the competition between OpenAI and Meta AI in a video that he released on his Facebook account, calling Meta AI “the most intelligent AI assistant that you can freely use.
Zuckerberg stated that the two smaller versions of Llama 3 currently available, with 8 billion and 70 billion parameters, outperformed rival free models on performance measures usually used to judge model quality. He stated that the largest version of Llama 3 is still being trained, with 400 billion parameters.
According to Nathan Benaich, the founder of AI-focused investment company Air Street Capital, the results were “undoubtedly impressive,” but they also showed a rising performance disparity between free and proprietary models.
Developers have reported that the prior Llama 2 version of the model failed to recognize simple context, conflating “kill” queries with murder orders. Rival Google has had similar issues, and it recently halted usage of its Gemini AI picture-generating tool after receiving criticism for producing erroneous portrayals of historical personalities.
Meta says it reduced such issues in Llama 3 by utilizing “high-quality data” to train the machine to discern subtlety. It did not expand on the datasets utilized, but it did say that Llama 3 received seven times more data than Llama 2.