Merged LLMs are the future, and we’re exploring how with Mark McQuade and Charles Goddard from Arcee AI on this episode with Jon Krohn. Learn how to combine multiple LLMs without adding bulk, train more efficiently, and dive into different expert approaches. Discover how smaller models can outperform larger ones and leverage open-source projects for big enterprise wins. This episode is packed with must-know insights for data scientists and ML engineers. Don’t miss out!

Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.

In this episode you will learn:
• Explanation of Charles' job title: Chief of Frontier Research [03:31]
• Model Merging Technology combining multiple LLMs without increasing size [04:43]
• Using MergeKit for model merging [14:49]
• Evolutionary Model Merging using evolutionary algorithms [22:55]
• Commercial applications and success stories [28:10]
• Comparison of Mixture of Experts (MoE) vs. Mixture of Agents [37:57]
• Spectrum Project for efficient training by targeting specific modules [54:28]
• Future of Small Language Models (SLMs) and their advantages [01:01:22]

Additional materials: www.superdatascience.com/801

The Super Data Science Podcast: Machine Learning | AI | Success

801: Merged LLMs Are Smaller And More Capable, with Arcee AI's Mark McQuade and Charles Goddard

The latest machine learning, A.I., and data career topics from across both academia and industry are brought to you by host Dr. Jon Krohn on the Super Data Science Podcast.

As the quantity of data on our planet doubles every couple of years and with this trend set to continue for decades to come, there's an unprecedented opportunity for you to make a meaningful impact in your lifetime. In conversation with the biggest names in the data science industry, Jon cuts through hype to fuel that professional impact.

Whether you're curious about getting started in a data career or you're a deep technical expert, whether you'd like to understand what A.I. is or you'd like to integrate more data-driven processes into your business, we have inspiring guests and lighthearted conversation for you to enjoy.

We cover tools, techniques, and implementation tricks across data collection, databases, analytics, predictive modeling, visualization, software engineering, real-world applications, commercialization, and entrepreneurship − everything you need to crush it with data science.

New tracks tagged #Model Merging