Designing, building, and operating a microservices-based system requires changes to more than just the architecture of the system. Also impacted are the developer experience, the deployments platforms, and the management of data. InfoQ sat down with experts in related domains and learned the best practices and hidden antipatterns relevant for engineering teams adopting microservices.

Five Essential Software Architecture Podcasts 2020: Moving to Microservices

In this podcast, Daniel Bryant spoke to Michelle Krejci, service engineer lead at Pantheon, about the Drupal and Wordpress webops-based company’s move to a microservices architecture. Michelle is a well-known conference speaker in the space of technical leadership and continuous integration, and she shared her lessons learned over the past four years of the migration.

Why listen to this podcast:

 - The backend for the Pantheon webops platform began as a Python-based monolith with a Cassandra data store. This architecture choice initially enabled rapid feature development as the company searched for product/market fit. However, as the company found success and began scaling their engineering teams, the ability to add new functionality rapidly to the monolith became challenging.
 - Conceptual debt and technical debt greatly impact the ability to add new features to an application. Moving to microservices does not eliminate either of these forms of debt, but use of this architectural pattern can make it easier to identify and manage the debt, for example by creating well-defined APIs and boundaries between modules.
 - Technical debt -- and the associated engineering toil -- is real debt, with a dollar value, and should be tracked and made visible to everyone.
Establishing “quick wins” during the early stages of the migration towards microservices was essential. Building new business-focused services using asynchronous “fire and forget” event-driven integrations with the monolith helped greatly with this goal.
 - Using containers and Kubernetes provided the foundations for rapidly deploying, releasing, and rolling back new versions of a service. Running multiple Kubernetes namespaces also allowed engineers to clone the production namespace and environment (without data) and perform development and testing within an individually owned sandboxed namespace.
 - Using the Apollo GraphQL platform allowed schema-first development. Frontend and backend teams collaborated on creating a GraphQL schema, and then individually built their respective services using this as a contract. Using GraphQL also allowed easy mocking during development. Creating backward compatible schema allowed the deployment and release of functionality to be decoupled.

InfoQ Podcast

Michelle Krejci on Moving to Microservices: Visualising Technical Debt, Kubernetes, and GraphQL

In this podcast, Daniel Bryant sat down with Zhamak Dehghani, principal consultant, member of technical advisory board, and portfolio director at ThoughtWorks. Topics discussed included: the motivations for becoming a data-driven organization; the challenges of adapting legacy data platforms and ETL jobs; and how to design and build the next generation of data platforms using ideas from domain-driven design and product thinking, and modern platform principles such as self-service workflows.

Why listen to this podcast:

 - Becoming a data-driven organization remains one of the top strategic goals of many organizations. Being able to rapidly run experiments and efficiently analyse the resulting data can provide a competitive advantage.
 - There are several “architecture failure modes” within existing enterprise data platforms. They are centralized and monolithic. The composition of data pipelines are often highly-coupled, meaning that a change to the data format will require a cascade of changes throughout the pipeline. And finally, the ownership of data platforms is often siloed and hyper-specialized.
 - The next generation of enterprise data platform architecture requires a paradigm shift towards ubiquitous data with a distributed data mesh.
Instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way.
 - Domain data teams must apply product thinking to the datasets that they provide; considering their data assets as their products, and the rest of the organization's data scientists, ML and data engineers as their customers.
The key to building the data infrastructure as a platform is (a) to not include any domain specific concepts or business logic, keeping it domain agnostic, and (b) make sure the platform hides all the underlying complexity and provides the data infrastructure components in a self-service manner.


More on this: Quick scan our curated show notes on InfoQ https://bit.ly/39exTWl
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/39exTWl

Show Notes

Zhamak Dehghani on Data Mesh, Domain-Oriented Data, and Building Data Platforms

In this podcast, Daniel Bryant sat down with Dave Sudia, senior DevOps engineer at GoSpotCheck. Topics discussed included: the benefits of PaaS; building a platform with Kubernetes as the foundation; selecting open source components and open standards in order to facilitate the evolution of a platform; and why care should be taken to prioritize the developer experience and create self-service operation of the platform.

Why listen to this podcast: 

 - When starting a business and searching for product-market fit, creating an application using a monolithic code base deployed onto a commercial Platform as a Service (PaaS) product is a very effective way of iterating fast and minimising operational costs.
 - There may come a point where the PaaS cannot provide bespoke requirements, or it has trouble scaling, or the costs become prohibitive. At this point many teams choose to build a custom platform using cloud technologies, such as Kubernetes. 
 - Building a Kubernetes platform can be an effective solution, but appropriate effort needs to be put into designing, building, and maintaining the platform. The platform effectively becomes another product within the business that must be managed accordingly.
 - Embracing open standards provides many benefits, especially for the long term. Implementations that are consumed through well-defined interfaces and abstraction can be more readily swapped at a later point in time. It is also generally easier to integrate components that share common interfaces.
 - Attention and resources must be provided to create an effective developer experience for the platform. It is essential to prioritize self-service operations, and also to understand the core requirements of the engineers and QA specialists that will be using the platform during their daily work.
 - Establishing an effective continuous delivery pipeline can enable more repeatable and scalable testing of applications, and also allows the codification of cross-functional requirements.
 - The cloud native landscape has now evolved to a point where most of the frameworks and tooling required to build a platform have become viable for general purpose usage. However, some assembly may still be required, and engineers should be prepared for change, as the ecosystem moves fast.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/3bj3BTp
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/3bj3BTp

Dave Sudia on Migrating From a PaaS to a Kubernetes-Based Platform

In this podcast, Marty Abbott and Tanya Cordrey sat down with InfoQ podcast co-host Daniel Bryant. Abbott, CEO and co-founder of AKF Partners, and Cordrey, partner at AKF Partners, discussed topics that included: their learning from working together in the early days of eBay, why and how to avoid creating software systems that are composed of deep call chains of microservices, and how to build effective product teams.

Why listen to this podcast:

 - First introduced in the book “The Art of Scalability”, the AKF Scale Cube is a model for segmenting software components, defining microservices, and scaling products. It also creates a common language for teams to discuss scale related options in designing solutions.
 - The microservice architectural pattern is best used for implementing the “breadth” of business functionality. Engineers should avoid building deep call chains of services, as this can increase the probability of failure, and can also increase the challenges of locating and diagnosing issues. Code libraries can often be used more effectively to implement “depth” within services.
 - The AKF Availability Cube is a new model to guide discussions on how to achieve high availability. This model can also be used as a mathematical tool to evaluate the theoretical “as designed” availability of existing systems.
 - Building products using cross-functional teams is an effective approach. However, care should be taken not to accidentally create unnecessarily large teams, as this can add communication and coordination friction to the delivery process.
 - Teams should make a conscious choice to adopt new technologies, and understand the benefits and tradeoffs with doing so. Managing risk, and in particular, technology lifespan risk, is an important part of the value engineers provide to the business.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2AbpZjT
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2AbpZjT

Transcript

Marty Abbott and Tanya Cordrey on Microservices, Availability, and Managing Risk

Today on the InfoQ Podcast, Wes Reisz talks with one of the thought leaders in Microservices, CI/CD, and Cloud -- Sam Newman. The podcast covers many of the topics, techniques, and patterns that Sam writes about in his latest book, Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith. Topics covered in the podcast include understanding the problem you’re trying to solve, organizational/people changes when it comes to microservice architectures, database strategies for decomposing monolithic datastores, and why we’re seeing projects reverting from microservices to monoliths. 

Why listen to this podcast:

 - Fundamentally, microservices are distributed systems. Distributed systems have baggage (complexity) that comes along with them. The best way to deal with this complexity is not to address it. Try to solve the problem in other ways before choosing to take an organization to microservices.
 - A common issue that large enterprises run into that might be a strong indicator for implementing microservices occurs when lots of developers are working on a given problem and they’re getting in each other’s way.
 - A useful structure to follow with microservices is to make sure each service is owned by exactly one team. One team can own more than one service but having clear ownership of who owns a service helps in some of the operational challenges with microservices.
 - A release train should be a stop in the journey towards continuous delivery. It’s not the destination. If you find that you can only release in a release train, you are likely building a distributed monolith.
 - There are challenges of operating microservices when the end customer has to operate and manage it. These challenges are part of why we’re seeing projects move from microservices to process monoliths.

More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2XvGzmF
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Check the landing page on InfoQ: https://bit.ly/2XvGzmF

Sam Newman: Monolith to Microservices

Software engineers, architects and team leads have found inspiration to drive change and innovation in their team by listening to the weekly InfoQ Podcast. They have received essential information that helped them validate their software development map. We have achieved that by interviewing some of the top CTOs, engineers and technology directors from companies like Uber, Netflix and more. Over 1,200,000 downloads in the last 3 years.