tech

Summary: For our inaugural podcast QCon chair Wesley Reisz talks to Adrian Cockcroft, who works for Battery Ventures where he advises the firm and its portfolio companies about technology issues and also assists with deal sourcing and due diligence.
Why listen to this podcast
•	Over the last year a large number of frameworks and libraries for building microservices have emerged and we're seeing a lot of rapid change.
•	The stack you choose will often be based on the main language you use, so for example Netflix’s stack is language agnostic but the tooling is very Java-centric.
•	Architectural choices can have a profound impact on success, so some-thing as simple as an overly long timeout with retries can cause your system to suffer from a congestion collapse problem.  Have a large timeout at the edge, and progressively smaller and smaller timeouts as you get deeper into the system.
•	Start with a monolith and move to a distributed architecture when you need to because of team size or to get better separation of concerns.
•	Other disruptive trends include “serverless architecture” like AWS Lambda, and open source itself which is perhaps the true definition of disruptive.
More on this
•	You can access our in-depth coverage on microservices http://bit.ly/1SRnFz3.
•	You can also subscribe to our newsletter to receive weekly updates on the hottest topics from professional software development. http://bit.ly/24x3IVq
•	Attend Adrian Cockcroft 's session at QCon New York 2016, Jun 13-17. http://bit.ly/1TvzlDQ 


Show Notes

Adrian Cockcroft on Microservices, Terraservices and Serverless Computing

In this week’s podcast, Thomas Betts talks with Haley Tucker, a Senior Software Engineer on the Playback Features team at Netflix. While at QCon San Francisco 2016, Tucker told some production war stories about trying to deliver content to 65 million members.

Why listen to this podcast:

 - Distributed systems fail regularly, often due to unexpected reasons
 - Data canaries can identify invalid metadata before it can enter and corrupt the production environment
 - ChAP, the Chaos Automation Platform, can test failure conditions alongside the success conditions
 - Fallbacks are an important component of system stability, but the fallbacks must be fast and light to not cause secondary failures
 - Distributed systems are fundamentally social systems, and require a blameless culture to be successful

Notes and links can be found on: http://bit.ly/2hqzQ6K

2m:05s - The Video Metadata Service aggregates several sources into a consistent API consumed by other Netflix services.

2m:43s - Several checks and validations were in place within the video metadata service, but it is impossible to predict every way consumers will be using the data.

3m:30s - The access pattern used by the playback service was different than that used in the checks, and led to unexpected results in production.

3m:58s - Now, the services consuming the data are also responsible for testing and verifying the data before it rolls out to production. The Video Metadata Service can orchestrate the testing process.

More on this:
Quick scan our curated show notes on InfoQ. http://bit.ly/2hqzQ6K

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

InfoQ Podcast

Haley Tucker on Responding to Failures in Playback Features at Netflix

Jonas Bonér, CTO of LightBend and creator Akka, discusses using Akka when developing distributed systems. He talks about the Actor Model, and how every Microservice needs to be viewed as a system to be successful. 

Why listen to this podcast:

 - Akka is JVM-based framework design for developing distributed systems leveraging the Actor Model - an approach for writing concurrent systems that treat actors as universal primitives and the most successful model with abstraction has been streaming
 - Circuit breakers in Akka are a backup and retry policy; they protect you by capturing failure data and allow you to roll back 
 - Every Microservice needs to be viewed as a system, it needs to have multiple parts that run on different machines in order to function and be fully resilient - thus is a Microsystem
 - Two different trends have emerged when it comes to hardware and environments: one is the trend toward Multi-core, the is a movement toward virtualized environments and the cloud
 - Saga pattern of managing long running transactions in a distributed system fits very well with messaging style architectures

Notes and links can be found on: http://bit.ly/2kwB2eB

Akka
The Actor Model
When Akka and the Actor Model is the perfect choice
Circuit breakers patterns in distributed systems
Two trends toward Multi-core
Reactive Manifesto
Event Driven vs. Message Driven
Reactive Programming and Streams
Microliths to Microsystems
What do you have to get right before you start trying to deploy a distributed systems?
Working with ML / AI at Lightbend to understand tracing through distributed system
Saga Pattern

More on this:
Quick scan our curated show notes on InfoQ http://bit.ly/2kwB2eB

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Jonas Bonér on the Actor Model, Akka, Reactive Programming, Microservices and Distributed Systems

Wesley Reisz talks to Daniel Bryant on moving from monoliths to micro-services, covering bounded contexts, when to break up micro-services, event storming, practices like observability and tracing, and more.

Why listen to this podcast:

- Migrating a monolith to micro-services is best done by breaking off a valuable but not critical part first.
 - Designing a greenfield application as micro-services requires a strong understanding of the domain.
 - When a request enters the system, it needs to be tagged with a correlation id that flows down to all fan-out service requests.
 - Observability and metrics are essential parts to include when moving micro-services to production.
 - A service mesh allows you to scale services and permit binary transports without losing observability.

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2pFYBiT

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2pFYBiT

Daniel Bryant on Microservices and Domain Driven Design

Wesley Reisz talks to Sid Anand, a data architect at cybersecurity company Agari, about building cloud-native data pipelines. The focus of their discussion is around a solution Agari uses that is built from Amazon Kinesis Streams, serverless functions, and auto scaling groups.

Sid Anand is an architect at Agari, and a former technical architect at eBay, Netflix, and LinkedIn.  He has 15 years of data infrastructure experience at scale, is a PMC for Apache Airflow, and is also a program committee chair for QCon San Francisco and QCon London.

Why listen to this podcast

 - Real-time data pipeline processing is very latency sensitive
 - Micro-batching allows much smaller amounts of data to be processed
 - Use the appropriate data store (or stores) to support the use of the dataIngesting data quickly into a clean database with minimal indexes can be fast
 - Communicate using a messaging system that supports schema evolution

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2rJU9nB

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2rJU9nB

Sid Anand on Building Agari’s Cloud-native Data Pipelines with AWS Kinesis and Serverless

Nora Jones, a senior software engineer on Netflix’ Chaos Team, talks with Wesley Reisz about what Chaos Engineering means today. She covers what it takes to build a practice, how to establish a strategy, defines cost of impact, and covers key technical considerations when leveraging chaos engineering.

Why listen to this podcast:

- Chaos engineering is a discipline where you formulate hypotheses, perform experiments, and evaluate the results afterwards.
- Injecting a bit of failure over time is going to make your system more resilient in the end.
- Start with Tier 2 or non-critical services first, and build up success stories to grow chaos further.
- As systems become more and more distributed, there becomes a higher need for chaos engineering.
- If you’re running your first experiment, get your service owners in a war room and get them to monitor the results of the test as it is running.

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2vJoimw

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extended shownotes? Check the landing page on InfoQ: http://bit.ly/2vJoimw

Nora Jones on Establishing, Growing, and Maturing a Chaos Engineering Practice

In this podcast Shane Hastie, Lead Editor for Culture & Methods, spoke to Josh Evans, former engineering manager at Netflix on how Netflix does DevOps and the freedom and responsibility culture that undermines their way of working. 

Why listen to this podcast:

•	There are many interpretations of the term DevOps, it is a useful shorthand for a wide variety of technologies and approaches 
•	“You build it, you run it” is the concrete application of the freedom and responsibility culture
•	When building a platform tool make it so easy to use that the product teams are not tempted to try and build something for themselves
•	Product teams are free to experiment and learn, which can feel chaotic and is a valuable part of the freedom and responsibility culture
•	The value of blameless and safe incident reviews – the goal is to learn and find patterns and use that information to present whole classes of failure from happening in the future
•	Don’t view the value stream in a fragmented way – see the whole end to end system with all its interactions and dependencies and optimize the system as a cohesive whole rather than different tools and domains

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2mtCIr1

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2mtCIr1

Josh Evans on DevOps at Netflix

Software engineers, architects and team leads have found inspiration to drive change and innovation in their team by listening to the weekly InfoQ Podcast. They have received essential information that helped them validate their software development map. We have achieved that by interviewing some of the top CTOs, engineers and technology directors from companies like Uber, Netflix and more. Over 1,200,000 downloads in the last 3 years.

Haley Tucker on Responding to Failures in Playback Features at Netflix

InfoQ

Related tracks