podcast techno

"Incremental change may be good theory, but in practice you have to have a big enough stick to hit everybody with to make everything move at once". So shares Adrian Cockcroft, who helped lead Netflix's migration from datacenter to the cloud -- and from monolithic to microservices architecture -- when their streaming business (the "stick"!) was exploding.

So how did they -- and how can other companies -- make such big, bet-the-company kind of moves, without getting mired in fanatical internal debates? Does organizational structure need to change, especially if moving from a more product-, than project-based, approach? What happens to security? And finally, what happens to the role of CIOs; what can/should they do?

Most interestingly: How will the entire industry be affected as companies not only adopt, but essentially offer, microservices or narrow cloud APIs? How do the trends of microservices, containers, devops, cloud, as-a-service/ on-demand, serverless -- all moves towards more and more ephemerality -- change the future of computing and even work? Cockcroft (who is now a technology fellow at Battery Ventures) joins this episode of the a16z Podcast, in conversation with Frank Chen and Martin Casado (and Sonal Chokshi) to discuss these shifts and more.

---

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.
 
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

a16z Podcast: All About Microservices

Docker Distributed Systems Summit Opening Words by Solomon Hykes

In this week's podcast QCon chair Wesley Reisz talks to Randy Shoup. Shoup is the vice president of engineering at Stitch Fix. Prior to Stitch Fix, he worked for Google as the director of engineering and cloud computing, CTO and co-founder of Shopilly, and chief engineer at Ebay.

Why listen to this podcast:

 - Stitch Fix's business is a combination of art and science. Humans are much better with the machines, and the machines are much better with the humans.
 - Stitch Fix has 60 engineers, with 80 data scientists and algorithm developers. This ratio of data science to engineering is unique.
 - With Ruby-on-Rails on top of Postgres, the company maintains about 30 different applications on the same stack.
 - The practice of Test Driven Development makes Continuous Delivery work, and the practice of having the same people build the code as those who operate the code makes both of these things much more powerful.
 - Microservices gives feature velocity, the ability for individual teams to move quickly and independently of each other, and independent deployments.
 - Microservices solve a scaling problem. They solve an organisational scaling problem, and a technological scaling problem. These are not the problems that you have early on in the startup.
 - In the monolithic world, if you're not able to continue to vertically scale the application or the database or whatever your monolith is. And so for scaling reasons alone you might consider breaking it up into what we call microservices.
 
 Notes and links can be found on http://bit.ly/randy-shoup-podcast
 
 Data Science and Stitch Fix
 
 1m:57s - Stitch Fix re-imagines retail, particularly for clothing. When you sign up, you fill out survey of the kinds of things that you like and you don't like, and we choose what we think you're going to enjoy based on the millions of customers that we have already. And we use a ton of data science in that process.
 3m:00s - That goes into our algorithms and then our algorithms make personalised recommendations based on all the things we know about our other customers... there's a human element as well: we have 3,200 human stylists that are all around the United States and they choose the five items that go into the box [of clothing].
 3m:29s - What we like is that this is a combination of art and science. Modern companies combine what machines are really good at, such as chugging through the 60 to 70 questions times the millions of customers, and combining that with the human element of the stylists, figuring out what things go together, what things are trending, what things are appropriate... Humans are much better with the machines, and the machines are much better with the humans.
[...]
 
 More on this:
 
Quick scan our curated show notes on InfoQ. http://bit.ly/randy-shoup-podcast
 
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

InfoQ Podcast

Show Notes

Randy Shoup on Stitch Fix's Technology Stack,  Data Science and Microservices

In this episode of the O’Reilly Bots Podcast, I speak with Dennis Mortensen, founder and CEO of X.ai, a personal assistant bot that handles meeting scheduling through email.

Dennis Mortensen on Email Bots

In this week’s podcast, Thomas Betts talks with Chris Richardson, a developer, architect, Java Champion and author of POJOs in Action. Before his workshop on Microservices w/ Spring Boot and Docker at QCon San Francisco 2016, Richardson took time to discuss his ideas on how to use DDD and CQRS concepts as a guide for implementing a robust microservices architecture.

Why listen to this podcast:

 - "Microservice architecture" is a better term than "microservices".   The latter suggests that a single microservice is somehow interesting
 - The concepts discussed in Domain-Driven Design are an excellent guide for how to implement a microservices architecture
 - Bounded Contexts correspond well to individual microservices
 - Event sourcing and CQRS provide patterns for how to implement loosely coupled services
 - When converting a monolith to microservices, avoid a big bang rewrite, in favor of an iterative approach

Notes and links can be found on: http://bit.ly/2hZ8TM1

11m:51s - Microservices must be loosely coupled, usually creating a model with one database per service.
12m:45s - There is a business requirement to maintain data consistency across services, and using an event driven architecture is a good way to achieve that.
13m:38s - Event sourcing is specific technique for persisting domain objects as a series of events.
14m:11s - Just as transactions don’t like to be split across microservices, queries cannot simply join across multiple data sources. CQRS provides a solution that accommodates querying via microservices and materialized views.

More on this:
Quick scan our curated show notes on InfoQ. http://bit.ly/2hZ8TM1

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Chris Richardson on Domain-Driven Microservices Design

Jonas Bonér, CTO of LightBend and creator Akka, discusses using Akka when developing distributed systems. He talks about the Actor Model, and how every Microservice needs to be viewed as a system to be successful. 

Why listen to this podcast:

 - Akka is JVM-based framework design for developing distributed systems leveraging the Actor Model - an approach for writing concurrent systems that treat actors as universal primitives and the most successful model with abstraction has been streaming
 - Circuit breakers in Akka are a backup and retry policy; they protect you by capturing failure data and allow you to roll back 
 - Every Microservice needs to be viewed as a system, it needs to have multiple parts that run on different machines in order to function and be fully resilient - thus is a Microsystem
 - Two different trends have emerged when it comes to hardware and environments: one is the trend toward Multi-core, the is a movement toward virtualized environments and the cloud
 - Saga pattern of managing long running transactions in a distributed system fits very well with messaging style architectures

Notes and links can be found on: http://bit.ly/2kwB2eB

Akka
The Actor Model
When Akka and the Actor Model is the perfect choice
Circuit breakers patterns in distributed systems
Two trends toward Multi-core
Reactive Manifesto
Event Driven vs. Message Driven
Reactive Programming and Streams
Microliths to Microsystems
What do you have to get right before you start trying to deploy a distributed systems?
Working with ML / AI at Lightbend to understand tracing through distributed system
Saga Pattern

More on this:
Quick scan our curated show notes on InfoQ http://bit.ly/2kwB2eB

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Jonas Bonér on the Actor Model, Akka, Reactive Programming, Microservices and Distributed Systems

In this episode of the O’Reilly Bots Podcast, I speak with Tom Coates, co-founder of Thington, a service layer for the Internet of Things. Thington provides a conversational, messaging-like interface for controlling devices like lights and thermostats, but it’s also conversational at a deeper level: its very architecture treats the interactions between different devices like a conversation, allowing devices to make announcements to any other device that cares to listen.

Tom Coates on Conversational Devices

In this episode of the Data Show, I spoke with Anima Anandkumar, a leading machine learning researcher, and currently a principal research scientist at Amazon. I took the opportunity to get an update on the latest developments on the use of tensors in machine learning. Most of our conversation centered around MXNet—an open source, efficient, scalable deep learning framework. I’ve been a fan of MXNet dating back to when it was a research project out of CMU and UW, and I wanted to hear Anandkumar’s perspective on its recent progress as a framework for enterprises and practicing data scientists.

Deep Learning That's Easy to Implement and Easy to Scale

In this episode of the O’Reilly podcast, Jon Bruner sat down with Rahul Kamdar, director of product management and strategy at TIBCO Software. They discussed the shift from the centralized enterprise service bus (ESB) to a distributed data architecture based on microservices, APIs, and cloud-native applications.

From Enterprise Service Bus to Microservices

In this episode of the Data Show, I spoke with Karthik Ramasamy, adjunct faculty member at UC Berkeley, former engineering manager at Twitter, and co-founder of Streamlio. Ramasamy managed the team that built Heron, an open source, distributed stream processing engine, compatible with Apache Storm.  While Ramasamy has seen firsthand what it takes to build and deploy large-scale distributed systems (within Twitter, he worked closely with the team that built DistributedLog), he is first and foremost interested in designing and building end-to-end applications. As someone who organizes many conferences, I’m all too familiar with the vast array of popular big data frameworks available. But, I also know that engineers and architects are most interested in content and material that helps them cut through the options and decisions.

Architecting and Building End-to-End Streaming Applications

In this episode of the Bots Podcast, we peer into the giant companies that are beginning to adopt messaging and bots. My guest is Tom Hadfield, founder of Message.io, a service that syndicates bots across many different messaging platforms.

Tom Hadfield on Bots in the Enterprise

In this episode of the Data Show, I spoke with Reza Zadeh, adjunct professor at Stanford University, co-organizer of ScaledML, and co-founder of Matroid, a startup focused on commercial applications of deep learning and computer vision. Zadeh also is the co-author of the forthcoming book TensorFlow for Deep Learning (now in early release). Our conversation took place on the eve of the recent ScaledML conference, and much of our conversation was focused on practical and real-world strategies for scaling machine learning. In particular, we spoke about the rise of deep learning, hardware/software interfaces for machine learning, and the many commercial applications of computer vision.

Scaling Machine Learning

In this episode of the Data Show, I spoke with Lukas Biewald, co-founder and chief data scientist at CrowdFlower. In a previous episode we covered how the rise of deep learning is fueling the need for large labeled data sets and high-performance computing systems. CrowdFlower has a service that many leading companies have come to rely on to provide them with labeled data sets to train machine learning models. As deep learning models get larger and more complex, they require training data sets that are bigger than those required by other machine learning techniques.

podcast techno

daveinspace