Designing, building, and operating a microservices-based system requires changes to more than just the architecture of the system. Also impacted are the developer experience, the deployments platforms, and the management of data. InfoQ sat down with experts in related domains and learned the best practices and hidden antipatterns relevant for engineering teams adopting microservices.

Five Essential Software Architecture Podcasts 2020: Moving to Microservices

infoq

DataPodcast

techie

Vaughn Vernon is thought-leader in the space of reactive software and Domain Driven Design (DDD). Vaughn has recently released a new open source project called vlingo. The platform is designed to support DDD at the framework and toolkit level. On today’s podcast, Vaughn discusses what the framework is all about, why he felt it was needed, and some of the design decisions made in developing the platform, including things like the architecture, actor model decisions, clustering algorithm, and how DDD is realized with the framework.

Why listen to this podcast:

 - Vlingo is an open source system for building distributed, concurrent, event-driven, reactive microservices that supports (at the framework level) Domain Driven Design.
 - The platform is in the early stages. It runs on the JVM. There is a port to C#. All code is pushed up stream.
 - The platform uses the actor model and all messages are sent in a type-safe way.
 - Vlingo supports clustering and uses a bully algorithm to achieve consensus.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2MwEWxb

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2MwEWxb

InfoQ Podcast

Show Notes

Vaughn Vernon on Developing a Domain Driven Design first Actor-Based Microservices Framework

In June of this year, Consul 1.2 was released. The release expanded Consul’s capability around service segmentation (controlling who and how services connect East and West). On this week’s podcast, Wes and Mitchell discuss Consul in detail. The two discuss Consul’s design decisions around focusing on user space networking, layer 4 routing, Go, Windows’ performance characteristics, the roadmap for eBPF on Linux, and an interesting feature that Consul implements called Network Tomography. The show wraps with Mitchell’s discussion on some of the research that Hashicorp is doing around machine learning and security with Consul.

Why listen to this podcast:

- Consul is first and foremost a centralized service registry that provides discovery. While it has a key-value store, it is Consul’s least important feature.
With the June release (1.2), Consul entered more into the space of a service mesh with the focus on service segmentation (controlling how you connect and who can connect).
- Hashicorp attempts to limit the language fragmentation in the Company and has seen a lot of success leveraging Go across their platforms. Therefore, Consul is written in Go.
- Because Consul focused on layer 4 first, it is recommended to leverage the recent integration with Envoy for achieving high degrees of observability.
- All of the network routing with Consul happens in user space at this point; however, kernel space routing with eBPF is planned for the near term. The focus, at this point, is safely cross-compiling to every platform and addressing the most possible use cases. The focuses isn’t on the high performance use cases (yet).
- For any two servers across the globe in different data centers, instantly Consul can give you 99th percentile round-trip time between with uses a feature called Network Tomography.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2S3ZiSx

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2S3ZiSx

Mitchell Hashimoto on Consul since 1.2 and its Role as a Modern Service Mesh

In this podcast, Daniel Bryant spoke to Michelle Krejci, service engineer lead at Pantheon, about the Drupal and Wordpress webops-based company’s move to a microservices architecture. Michelle is a well-known conference speaker in the space of technical leadership and continuous integration, and she shared her lessons learned over the past four years of the migration.

Why listen to this podcast:

 - The backend for the Pantheon webops platform began as a Python-based monolith with a Cassandra data store. This architecture choice initially enabled rapid feature development as the company searched for product/market fit. However, as the company found success and began scaling their engineering teams, the ability to add new functionality rapidly to the monolith became challenging.
 - Conceptual debt and technical debt greatly impact the ability to add new features to an application. Moving to microservices does not eliminate either of these forms of debt, but use of this architectural pattern can make it easier to identify and manage the debt, for example by creating well-defined APIs and boundaries between modules.
 - Technical debt -- and the associated engineering toil -- is real debt, with a dollar value, and should be tracked and made visible to everyone.
Establishing “quick wins” during the early stages of the migration towards microservices was essential. Building new business-focused services using asynchronous “fire and forget” event-driven integrations with the monolith helped greatly with this goal.
 - Using containers and Kubernetes provided the foundations for rapidly deploying, releasing, and rolling back new versions of a service. Running multiple Kubernetes namespaces also allowed engineers to clone the production namespace and environment (without data) and perform development and testing within an individually owned sandboxed namespace.
 - Using the Apollo GraphQL platform allowed schema-first development. Frontend and backend teams collaborated on creating a GraphQL schema, and then individually built their respective services using this as a contract. Using GraphQL also allowed easy mocking during development. Creating backward compatible schema allowed the deployment and release of functionality to be decoupled.

Michelle Krejci on Moving to Microservices: Visualising Technical Debt, Kubernetes, and GraphQL

In this podcast, Daniel Bryant sat down with Zhamak Dehghani, principal consultant, member of technical advisory board, and portfolio director at ThoughtWorks. Topics discussed included: the motivations for becoming a data-driven organization; the challenges of adapting legacy data platforms and ETL jobs; and how to design and build the next generation of data platforms using ideas from domain-driven design and product thinking, and modern platform principles such as self-service workflows.

Why listen to this podcast:

 - Becoming a data-driven organization remains one of the top strategic goals of many organizations. Being able to rapidly run experiments and efficiently analyse the resulting data can provide a competitive advantage.
 - There are several “architecture failure modes” within existing enterprise data platforms. They are centralized and monolithic. The composition of data pipelines are often highly-coupled, meaning that a change to the data format will require a cascade of changes throughout the pipeline. And finally, the ownership of data platforms is often siloed and hyper-specialized.
 - The next generation of enterprise data platform architecture requires a paradigm shift towards ubiquitous data with a distributed data mesh.
Instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way.
 - Domain data teams must apply product thinking to the datasets that they provide; considering their data assets as their products, and the rest of the organization's data scientists, ML and data engineers as their customers.
The key to building the data infrastructure as a platform is (a) to not include any domain specific concepts or business logic, keeping it domain agnostic, and (b) make sure the platform hides all the underlying complexity and provides the data infrastructure components in a self-service manner.


More on this: Quick scan our curated show notes on InfoQ https://bit.ly/39exTWl
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/39exTWl

Zhamak Dehghani on Data Mesh, Domain-Oriented Data, and Building Data Platforms

In this podcast, Daniel Bryant sat down with Dave Sudia, senior DevOps engineer at GoSpotCheck. Topics discussed included: the benefits of PaaS; building a platform with Kubernetes as the foundation; selecting open source components and open standards in order to facilitate the evolution of a platform; and why care should be taken to prioritize the developer experience and create self-service operation of the platform.

Why listen to this podcast: 

 - When starting a business and searching for product-market fit, creating an application using a monolithic code base deployed onto a commercial Platform as a Service (PaaS) product is a very effective way of iterating fast and minimising operational costs.
 - There may come a point where the PaaS cannot provide bespoke requirements, or it has trouble scaling, or the costs become prohibitive. At this point many teams choose to build a custom platform using cloud technologies, such as Kubernetes. 
 - Building a Kubernetes platform can be an effective solution, but appropriate effort needs to be put into designing, building, and maintaining the platform. The platform effectively becomes another product within the business that must be managed accordingly.
 - Embracing open standards provides many benefits, especially for the long term. Implementations that are consumed through well-defined interfaces and abstraction can be more readily swapped at a later point in time. It is also generally easier to integrate components that share common interfaces.
 - Attention and resources must be provided to create an effective developer experience for the platform. It is essential to prioritize self-service operations, and also to understand the core requirements of the engineers and QA specialists that will be using the platform during their daily work.
 - Establishing an effective continuous delivery pipeline can enable more repeatable and scalable testing of applications, and also allows the codification of cross-functional requirements.
 - The cloud native landscape has now evolved to a point where most of the frameworks and tooling required to build a platform have become viable for general purpose usage. However, some assembly may still be required, and engineers should be prepared for change, as the ecosystem moves fast.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3bj3BTp
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/3bj3BTp

Dave Sudia on Migrating From a PaaS to a Kubernetes-Based Platform

In this podcast, Marty Abbott and Tanya Cordrey sat down with InfoQ podcast co-host Daniel Bryant. Abbott, CEO and co-founder of AKF Partners, and Cordrey, partner at AKF Partners, discussed topics that included: their learning from working together in the early days of eBay, why and how to avoid creating software systems that are composed of deep call chains of microservices, and how to build effective product teams.

Why listen to this podcast:

 - First introduced in the book “The Art of Scalability”, the AKF Scale Cube is a model for segmenting software components, defining microservices, and scaling products. It also creates a common language for teams to discuss scale related options in designing solutions.
 - The microservice architectural pattern is best used for implementing the “breadth” of business functionality. Engineers should avoid building deep call chains of services, as this can increase the probability of failure, and can also increase the challenges of locating and diagnosing issues. Code libraries can often be used more effectively to implement “depth” within services.
 - The AKF Availability Cube is a new model to guide discussions on how to achieve high availability. This model can also be used as a mathematical tool to evaluate the theoretical “as designed” availability of existing systems.
 - Building products using cross-functional teams is an effective approach. However, care should be taken not to accidentally create unnecessarily large teams, as this can add communication and coordination friction to the delivery process.
 - Teams should make a conscious choice to adopt new technologies, and understand the benefits and tradeoffs with doing so. Managing risk, and in particular, technology lifespan risk, is an important part of the value engineers provide to the business.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2AbpZjT
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2AbpZjT

Transcript

Marty Abbott and Tanya Cordrey on Microservices, Availability, and Managing Risk

Today on the InfoQ Podcast, Wes Reisz talks with one of the thought leaders in Microservices, CI/CD, and Cloud -- Sam Newman. The podcast covers many of the topics, techniques, and patterns that Sam writes about in his latest book, Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith. Topics covered in the podcast include understanding the problem you’re trying to solve, organizational/people changes when it comes to microservice architectures, database strategies for decomposing monolithic datastores, and why we’re seeing projects reverting from microservices to monoliths. 

Why listen to this podcast:

 - Fundamentally, microservices are distributed systems. Distributed systems have baggage (complexity) that comes along with them. The best way to deal with this complexity is not to address it. Try to solve the problem in other ways before choosing to take an organization to microservices.
 - A common issue that large enterprises run into that might be a strong indicator for implementing microservices occurs when lots of developers are working on a given problem and they’re getting in each other’s way.
 - A useful structure to follow with microservices is to make sure each service is owned by exactly one team. One team can own more than one service but having clear ownership of who owns a service helps in some of the operational challenges with microservices.
 - A release train should be a stop in the journey towards continuous delivery. It’s not the destination. If you find that you can only release in a release train, you are likely building a distributed monolith.
 - There are challenges of operating microservices when the end customer has to operate and manage it. These challenges are part of why we’re seeing projects move from microservices to process monoliths.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2XvGzmF
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2XvGzmF

Sam Newman: Monolith to Microservices

In this podcast, Lin Sun, senior technical staff member and master inventor at IBM, and Neeraj Poddar, engineering lead and architect at Aspen Mesh, sat down with InfoQ co-host Daniel Bryant. Topics discussed included: the evolution of service mesh data planes and control planes, the new Istio 1.5 architectures, Istio WebAssembly extension support, and the future of service mesh technology.

Why listen to this podcast:

 - A service mesh in one implementation approach to provide service discovery, traffic management, and cross-cutting communication concerns that engineers see when they adopt (micro)service-based. 
 - The data plane of most modern service mesh implementations run out-of-process as a proxy sidecar. This has evolved from library based implementations, such as Airbnb’s SmartStack or Netflix’s OSS libraries.
 - The recent release of Istio 1.5 saw the deployment packaging of the control plane move from a microservice-based approach to that of a monolithic implementation, named “istiod”.
 - Istio now also supports data plane extensions written in WebAssembly (Wasm). These extensions can modify requests and responses and perform out-of-band actions, such as authentication and authorization.
 - Standardisations like the Service Mesh Interface (SMI) can add a lot of value, but the user requirements, common use cases, and the core abstractions of the underlying technology must be well understood.
 - Multi-cluster and mesh expansion (out-of-cluster) support is continually improving in Istio and many other service mesh implementations.

Lin Sun and Neeraj Poddar on Istio, Wasm, and the Future of Service Mesh

In this podcast, Nora Jones, Co-Founder and CEO at Jeli and co-author of O’Reilly’s “Chaos Engineering: System Resiliency in Practice”, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: chaos engineering and resilience engineering, planning and running effective chaos experiments, and learning from incidents.

Why listen to this podcast:

 - The chaos engineering and resilience engineering fields, although inextricably linked, are often incorrectly conflated. Resilience engineering is focused on “identifying and then enhancing the positive capabilities of people in organizations that allow them to adapt effectively and safely under varying circumstances.”
 - The UX of internal or engineering-focused tooling, such as chaos experimentation tooling, is extremely important. However, engineers that create these tools often overlook the value of UX, or don’t have the relevant skills in user design research to undertake this.
 - We all work in socio-technical systems. It is important to take the time to understand both aspects. Developing empathy and working alongside teams that you are trying to influence is essential. It is extremely important to continually work to build correct “mental models” of a system.
 - The before and after of running a chaos experiment is as important as running the experiment itself. However, the aspects of planning, creating effective hypotheses, and analysing and disseminating the results are often under-resourced.
 - Incident analysis can be a catalyst to help you understand more about your system. The Learning from Incidents website, alongside books such as Sidney Dekker’s The Field Guide to Understanding Human Error and Scott Snook’s Friendly Fire, can provide excellent background information to these topics.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2C4R6xL
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2C4R6xL

Nora Jones on Resilience Engineering, Mental Models, and Learning from Incidents

In this podcast, Shannon Williams, co-founder and president at Rancher Labs and Darren Shepherd, co-founder and CTO at Rancher Labs sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the adoption of hybrid cloud across organisations, the evolution of Kubernetes as a key abstraction for portability and cross-cloud security, running thousands of Kubernetes clusters at the edge, and the value of open standards.

Why listen to this podcast

 - Organisations are adopting hybrid cloud strategies. The use of containers to package and run applications across clouds has seen large adoption over the past five years. Containers and Kubernetes are everywhere: the datacenter, the edge, embedded systems, and other locations.
 - Two enterprise use cases for Kubernetes stand out: providing standardised abstractions and APIs to increase portability across vendors and cloud platforms; and providing a framework and homogenised foundation on which to build and implement (cross cloud) security solutions.
 - Open standards support interoperability and drive innovation. The CNCF is becoming the natural home for open cloud technologies. The Rancher team have donated Longhorn, their cloud-native distributed storage platform for Kubernetes that was recently announced as generally available, to the CNCF.
 - With the success of lightweight Kubernetes distributions, such as Rancher’s K3s, engineers are starting to deploy standalone Kubernetes clusters “by the thousands” to edge locations. Rancher has recently released Fleet, a new open source project that is focused on managing large collections (“fleets”) of Kubernetes clusters.
 - Many developers and end users of Kubernetes simply want a platform-as-a-service (PaaS)-like experience. The next 12 months will see the community focus on the simplification of the Kubernetes ecosystem.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/31ZjYT1
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/31ZjYT1

Full transcript

Rancher on Hybrid Cloud, Kubernetes at the Edge, and Open Standards

Charles Humble talks to Akhilesh Gupta, the technical lead for LinkedIn's real-time delivery infrastructure, and also LinkedIn messaging. They discuss the architecture behind LinkedIn’s real-time platform, its building blocks, the frameworks used and other technical details.

Listen to the podcast for more.

Curated transcript and more information on the https://bit.ly/2DujU3U
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ 
Follow us on Instagram: @infoqdotcom

Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

Akhilesh Gupta on the Architecture of LinkedIn’s Real-time Messaging Platform

In this podcast, John DesJardins, field CTO and VP solution architecture at Hazelcast, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: how in-memory data grids have evolved, use cases at the edge (IoT, ML inference), integration of stream processing APIs and techniques, and how data grids can be used within application modernization.

Listen to the podcast for more.

Curated transcript and more information on the https://bit.ly/2FtBIgp
Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ 
Follow us on Instagram: @infoqdotcom

Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

John DesJardins on In-Memory Data Grids, Stream Processing, and App Modernization

Software engineers, architects and team leads have found inspiration to drive change and innovation in their team by listening to the weekly InfoQ Podcast. They have received essential information that helped them validate their software development map. We have achieved that by interviewing some of the top CTOs, engineers and technology directors from companies like Uber, Netflix and more. Over 1,200,000 downloads in the last 3 years.