Fresh out of the studio, Craig Wiley, Senior Director of Product Management at Databricks who leads Mosaic AI, joins us to discuss the forefront of enterprise AI from model development to deployment at scale. Beginning with his career journey in ML operations, Craig explained how he recognized the critical connection between data and AI layers that could deliver order-of-magnitude acceleration in development cycles. Emphasizing the transition from classical ML operations to LLM operations, he showcased how Databricks' unified platform eliminates training-serving skew through data lineage capabilities and supports both fine-tuning and RAG approaches depending on industrial use case requirements. Highlighting compelling customer success stories including Suncorp's employee productivity platform and AstraZeneca's transformation of 400,000 clinical trial documents into queryable insights, Craig revealed a striking reality about enterprise AI evaluation - that 85% of AI use cases are being evaluated only by the engineers who built them, reinforcing that proper evaluation frameworks remain foundational for trustworthy AI implementation. He concluded by introducing Agent Bricks as Databricks' evaluation-centric approach to building production agents, emphasizing that model flexibility and rigorous testing are essential for enterprises moving from experimentation to production, while sharing his vision that the industry must evolve from the "year of agents" to the "year of evaluation and quality."

"85% of AI use cases are being evaluated by the engineer who built it saying, 'yep, seemed to work pretty well.' If you're gonna build a system that's going to be critical to the business, that's going to be important that it gets it right, then you can't do that without evaluations." - Craig Wiley

Episode Highlights: 
[00:00] Quote of the Day by Craig Wiley
[01:21] How Craig Wiley started his work in ML Ops that led him to Databricks
[02:43] Data and AI layer connection creates order-of-magnitude acceleration
[03:47] Mosaic AI acquisition expanded Gen AI solution capabilities
[04:38] Classical ML statistics versus Gen AI evaluation challenges
[05:48] Mosaic AI covers end-to-end from data ingestion
[07:12] Training-serving skew eliminated through unified platform lineage
[08:51] Fine tuning versus RAG depends on use case
[10:49] Industrial agents benefit from fine-tuned smaller models
[13:52] Agent Bricks prioritizes accuracy over simplicity alone
[15:44] Model flexibility crucial for speed and accuracy optimization
[17:59] Suncorp and AstraZeneca demonstrate diverse AI applications
[19:37] Asia Pacific shows aggressive AI adoption strategies
[22:00] 85% of AI cases evaluated only by building engineer
[24:12] Industry terminology evolves rapidly from RAG to agents
[25:39] Customer creativity with governance capabilities inspires product development

Profile: Craig Wiley, Senior Director of Product Management at Databricks and Mosaic AI
LinkedIn: https://www.linkedin.com/in/craigwiley/

Podcast Information: Bernard Leong hosts and produces the show. The proper credits for the intro and end music are "Energetic Sports Drive." G. Thomas Craig mixed and edited the episode in both video and audio format. 

Here are the links to watch or listen to our podcast.
Analyse Asia Main Site: https://analyse.asia
Analyse Asia Spotify: https://open.spotify.com/show/1kkRwzRZa4JCICr2vm0vGl 
Analyse Asia Apple Podcasts: https://podcasts.apple.com/us/podcast/analyse-asia-with-bernard-leong/id914868245 
Analyse Asia LinkedIn: https://www.linkedin.com/company/analyse-asia/


#dataaiworldtour #databricks #enterpriseai #retrievalaugmentedgeneration #agentbricks #evalsAI #evaluationAI #MLOps

Agent Bricks and How Data-AI Integration Changes Everything with Craig Wiley

A weekly podcast dissecting the pulse of business, technology & media in Asia. Hear our host interviewing leading journalists, senior executives, respectable authors & thinkers, entrepreneurs & thought leaders to discuss the most important issues that move and shake the business landscape in the Asia Pacific. The show is originally created and hosted by Bernard Leong.

New tracks tagged #ML Ops