Data Lake

O’Reilly’s Mac Slocum talks with Edd Dumbbell, VP of strategy at Silicon Valley Data Science, about the data lake, the opportunities the model presents, and the driving forces behind the concept. In the second segment, O’Reilly’s big data guru Ben Lorica chats with Rajiv Maheswaran, CEO of SecondSpectrum, about spatial-temporal pattern recognition — what Maheswaran calls “the science of moving dots.”

The data lake model is a powerhouse for invention

Guest 1: Gaurav Dhillon
Title: Chairman, and Chief Executive Officer, Snaplogic

Guest 2: Jeff Chapman
Title: Head of Technology, ING Direct

Most organizations use some form of SaaS. In the brave new world of cloud computing, what are the solutions and challenges to integrating two disparate data sets? How do we marry "data from somewhere else" with "our data here"?

Cloud and Data Integration: An unsolved problem?

Every organization these days is clear about the need to get its data act together. But that doesn’t mean the path toward data bliss is clear. Data has gravity. It resides in different places at different organizations -- on premise, in the cloud, and flowing from external sources. And the rate of change within organizations is always different. So an approach towards handling data that works for one company may be the exact wrong thing for yours.

Steven Sinofsky leads a conversation with three founders -- Prat Moghe, from Cazena; Gaurav Dhillon from SnapLogic, and Roman Stanek from GoodData – about the opportunity and variety of ways forward for companies looking to make the most of the data that matters.

---

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.
 
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

a16z Podcast: Making the Most of the Data That Matters

In this episode, Ravi talks with Rakesh Raghavan, Director of Snap Engineering at SnapLogic. Rakesh describes his first-hand experiences developing data lakes for leading online retailers and consumer-facing websites.

They'll discuss the challenges of architecting and implementing a data lake that supports both traditional reporting and ad hoc data exploration plus operational considerations before diving into a data lake project.

SnapTalk: Navigating the Data Lake - Tips From a Practitioner with Rakesh Raghavan

From the silver age of on-prem software companies like SAP and Siebel Systems to the golden age of enterprise software-as-a-service, we're now seeing an explosion of data. All types, all sizes, and all over the place. And much of it is a sort of industrial "data exhaust", where companies aren't quite sure what question to ask of the data but are being bombarded with data due to the variety of data sources available today -- from websites to sensors (and therefore data capture) everywhere. Before there is even a signal in the noise.

So how do you solve a problem like this-Data? Beyond requiring new types of plumbing and integrations, enterprises now expect -- given the age of mobile, web, cloud, and heck, let's add millennials to the mix too -- self service. To be able to ask, get, fit (curve-fit), predict. To take back the enterprise from the patchwork of integration and number of vendors we all have to deal with -- the scope of which most companies in fact are not truly aware of.

It's about the lifecycle of data in the enterprise, argues Snaplogic founder and CEO Gaurav Dhillon in this episode of the a16z Podcast, in conversation with Scott Kupor. It's in fact about the evolution of data overall -- from data warehouses to "data lakes": in stages, from purification (like wrangling data) to bottling (prepping for consumption by data scientists) to making sense of streams and streams of data!

---

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.
 
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

a16z Podcast: From Data Warehouses to Data Lakes

Building a Data Lake in AWS - Mick Bass and Carla Cole - 47Lining

In this podcast episode, O’Reilly’s Shannon Cutt talks with Damon Feldman, solutions director at MarkLogic, a company that has developed an operational and transactional NoSQL database that integrates data silos to give customers a single view of their data. They discuss data lakes, data hubs, integrating data in a lake format, data governance related to security, and more.

Damon Feldman on Combining a Data Hub and Data Lake

In this episode of the O’Reilly podcast, I speak with Han Yang, senior product manager at Cisco, working on analytics solutions. We discuss the impact of data analytics across industries, building data lifecycles for Internet of Things (IoT) applications, and advice for establishing successful analytics and machine learning projects.

Building Data Lifecycles for IoT Projects With Edge Analytics

In this podcast, Wayne Eckerson and James Serra discuss myths of modern data management. Some of the myths discussed include 'all you need is a data lake', 'the data warehouse is dead', 'we don’t need OLAP cubes anymore', 'cloud is too expensive and latency is too slow', 'you should always use a NoSQL product over a RDBMS.'

Serra is big data and data warehousing solutions architect at Microsoft with over thirty years of IT experience. He is a popular blogger and speaker and has presented at dozens of Microsoft PASS and other events. Prior to Microsoft, Serra was an independent data warehousing and business intelligence architect and developer.

James Serra: Myths of Modern Data Management

The Enterprise Integration Cloud: Built for the Cloud, Social, Mobile, and Big Data Era

In a world that has never been more connected, how do you ensure that your business data isn't fragmented? Salesforce, ServiceNow, Workday — are they streamlined or SaaS silos? Whether you need real-time application integration, business process automation, or better business analytics, SnapLogic delivers an integration platform as a service (iPaaS) that connects all of your cloud applications, APIs, and disparate data sources with the rest of the enterprise.

SnapTalk: Navigating the Data Lake - Tips From a Practitioner with Rakesh Raghavan

SnapLogic

Related tracks