infoq

In this week's podcast QCon chair Wesley Reisz talks to Heather Fleming, who is the VP of product and program management at GILT, where she is responsible for not only the customer-facing website, but also back office things from distribution to order processing.

Why listen to this podcast:

- GILT treats every person as an individual, with a skillset that is outside their responsibilities.
- You should be able to be your authentic self wherever you are.
- Google found creating a psychologically safe work environment was key to high performing teams.
- You can kill a great team by taking away their autonomy and empowerment.
- Great engineers that want to be managers are fearful of losing their skill, and great engineers that don't want to be managers are put in those roles and are terrible managers.

Notes and links can be found on InfoQ: http://bit.ly/1U2Wgq9

1m 49s - One of the things that makes the culture at GILT so special is the "ingredients framework" that considers every person an individual, with a skillset that is outside what their title might say and what their role and responsibilities might be.
3m 29s - One of the team ingredients is called "motivator" - the ability to make sure that the team understands what they're working on, and more importantly the "why" and how it's driving business or customer value.
4m 35s - The relationship you have with your co-workers should be deeper than many of us are allowing them to be, because they are like your second family.

Quick scan our curated show notes on InfoQ. http://bit.ly/1U2Wgq9 

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. http://bit.ly/24x3IVq

Attend Heather Fleming's session at QCon New York 2016, Jun 13-17: http://bit.ly/1U2f3og

Show notes

GILT VP Heather Fleming on Unlocking the "Secret Sauce" of Great Teams

In this week's podcast Richard Seroter talks to James Shore, author of The Art of Agile Development and one of the original signatories of the Agile Manifesto. Also on the podcast are Llewellyn Falco, creator of the open source testing tool ApprovalTests and co-founder of Teaching Kids Programming, and Rebecca Wirfs-Brock, inventor of Responsibility-Driven Design, as well as the author of books including Designing Object: Oriented Software and Object Design: Roles, Responsibilities and Collaborations.

Why listen to this podcast:

 - A lot of people know how to do TDD and refactoring for the back end, but not for the font, but the basics are the same.
 - The basics of Test-Driven Development are the same for the front or back end.
 - If you don't know what's wrong, you don't know how to fix it.
 - The most common code smells, according to Llewellyn Falco: Clutter, long lines, long methods, duplication, and inconsistency.
 - How do we make, in an agile way, the architectural work visible, and not ignore it?
 - How do you have an incremental architecture and get measurements? If I say to you I'm going to go away for six months and figure it out, that's not very measurable.

Notes and links can be found on InfoQ: http://bit.ly/1Pse2r1

1m:10s - The talk 'Agile Engineering for the Web' was about how do you bring typical Agile engineering ideas like TDD and refactoring to the front-end languages.
1m:33s - A lot of people know how to do these things on the back-end, but when you get to the front end a lot of people just throw up their hands.
2m:24s - I see CSS bugs all the time, because it's very hard to refactor CSS without breaking something.

Quick scan our curated show notes on InfoQ. http://bit.ly/1Pse2r1

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. http://bit.ly/24x3IVq

James Shore, Llewellyn Falco, and Rebecca Wirfs-Brock on TDD and Architecture

As InfoQ previously reported in March 2016, Dropbox announced that they had migrated away from Amazon Web Services (AWS).

In this week's podcast Robert Bluman talks to Preslav Le. Preslav has been a software engineer at Dropbox for the past three years, contributing to various aspects of Dropbox’s infrastructure including traffic, performance and storage. He was part of the core oncall and storage oncall rotations, dealing with high emergency real world issues, from bad code pushes to complete datacenter outages.

Why listen to this podcast:

 - Dropbox migrated away from Amazon S3 to their own data centres to allow them to optimise for their specific use case.
 - They are experimenting with Shingled Magnetic Recording (SMR) drives for primary storage to increase storage density. All writes go to an SSD cache and then get pushed asynchronously to the SMR disk.
 - Their average block size is 1.6MB with a maximum block size of 4MB. Knowing this allows the team to tune their storage system.
 - Three languages are used for the backend infrastructure. Python is used mainly for business logic, Go is the primary language used for heavy infrastructure services, and in some cases, for example where more direct control over memory is needed, Rust is also used.
 - Dropbox invest very heavily in verification and automation. A verifier scans every byte on disk and checks that it matches the checksum in the index.
 - Verification is also used to check that each box has the block keys it should have.

Notes and links can be found on http://bit.ly/preslav-le
 
Dropbox’s motivation for moving off the cloud 
2:40 - Dropbox used Amazon S3 and other services where it made sense, but they stored all the metadata in their own data centres.
3:30 - Initially this was done because Amazon had poor support for persistent storage at the time. This has since improved but it didn’t make sense for dropbox to move the metadata back.
4:01 - By that time the dropbox team was ready to tackle the storage problem and built their own in-house replacement for S3, called Magic Pocket. Magic Pocket allowed Dropbox to move away from Amazon altogether.
4:30 - The move saved money, but also allowed DropBox to optimise for their specific use case and be faster.
 
More on this:
 
Quick scan our curated show notes on InfoQ. http://bit.ly/preslav-le
 
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

InfoQ Podcast

Show Notes

Preslav Le on How Dropbox Moved off AWS and What They Have Been Able to Do Since

In this week's podcast, QCon chair Wesley Reisz talks to Kolton Andrus. Andrus is the founder of Gremlin Inc. He was a Chaos Engineer at Netflix, focused on the resilience of the Edge services. He designed and built FIT: Netflix’s failure injection service. Prior, he improved the performance and reliability of the Amazon Retail website. 

Why listen to this podcast:

- Gremlin, Kolton Andrus' new start-up, is focused on providing failure testing as a service. Version 1, currently in closed beta, is focused on infrastructure failures.
- Lineage-driven Fault Injection (LDFI) allowed Netflix to dramatically reduce the number of tests they needed to run in order to explore a problem space.
- You generally want to run failure tests in production, but you can't start there. Start in developemnt and build up.
- Having failure testing at an application level, as Netflix does, so you can have request level fault injection for a specific user or a specific device. 
- Being able to trace infrastructure with something like Dapper or Zipkin offers tremendous value. At Netflix, the failure injection system is integrated into the tracing system, which meant that when they caused a failure they could see all the points in the system that it touched.

More on this:
 
Quick scan our curated show notes on InfoQ. http://bit.ly/2fT9YiM
 
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Kolton Andrus on Lessons Learnt From Failure Testing at Amazon and Netflix and New Venture Gremlin

In this week’s podcast, Thomas Betts talks with Haley Tucker, a Senior Software Engineer on the Playback Features team at Netflix. While at QCon San Francisco 2016, Tucker told some production war stories about trying to deliver content to 65 million members.

Why listen to this podcast:

 - Distributed systems fail regularly, often due to unexpected reasons
 - Data canaries can identify invalid metadata before it can enter and corrupt the production environment
 - ChAP, the Chaos Automation Platform, can test failure conditions alongside the success conditions
 - Fallbacks are an important component of system stability, but the fallbacks must be fast and light to not cause secondary failures
 - Distributed systems are fundamentally social systems, and require a blameless culture to be successful

Notes and links can be found on: http://bit.ly/2hqzQ6K

2m:05s - The Video Metadata Service aggregates several sources into a consistent API consumed by other Netflix services.

2m:43s - Several checks and validations were in place within the video metadata service, but it is impossible to predict every way consumers will be using the data.

3m:30s - The access pattern used by the playback service was different than that used in the checks, and led to unexpected results in production.

3m:58s - Now, the services consuming the data are also responsible for testing and verifying the data before it rolls out to production. The Video Metadata Service can orchestrate the testing process.

More on this:
Quick scan our curated show notes on InfoQ. http://bit.ly/2hqzQ6K

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Haley Tucker on Responding to Failures in Playback Features at Netflix

In this week’s podcast, QCon chair Wesley Reisz talks to Keith Adams, chief architect at Slack. Prior he was an engineer at Facebook where he worked on the search type live backend, and is well-known for the HipHop VM [hhvm.com]. Adams presented How Slack Works at QCon SanFrancisco 2016.

Why listen to this podcast:

 - Group messaging succeeds when it feels like a place for members to gather, rather than just a tool
 - Having opt-in group membership scales better than having to define a group on the fly, like a mailing list instead of individually adding people to a mail
 - Choosing availability over consistency is sometimes the right choice for particular use cases
 - Consistency can be recovered after the fact with custom conflict resolution tools
 - Latency is important and can be solved by having proxies or edge applications closer to the user

Notes and links can be found on: http://bit.ly/keith-adams

3m:30s Voice and video interactions are impacted by latency; the same is true of messaging clients
4m:00s The user interface can provide indications of presence, through avatars indicating availability and typing indicators
4m:15s Latency is important; sometimes the difference is between 100ms and 200ms so the message channel monitors ping timeout between server and client
4m:40s 99th percentile is less than 100ms ping time
5m:15s If the 99th percentile is more than 100ms then it may be server based, such as needing to tune the Java GC
5m:25s Network conditions of the mobile clients are highly variable&nbsp;
6m:20s Mobile clients can suffer intermittent connectivity

More on this:
Quick scan our curated show notes on InfoQ. http://bit.ly/keith-adams

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Keith Adams on the Architecture of Slack, using MySql, Edge Caching, & the backend Messaging Server

In this week’s podcast, Thomas Betts talks with Chris Richardson, a developer, architect, Java Champion and author of POJOs in Action. Before his workshop on Microservices w/ Spring Boot and Docker at QCon San Francisco 2016, Richardson took time to discuss his ideas on how to use DDD and CQRS concepts as a guide for implementing a robust microservices architecture.

Why listen to this podcast:

 - "Microservice architecture" is a better term than "microservices".   The latter suggests that a single microservice is somehow interesting
 - The concepts discussed in Domain-Driven Design are an excellent guide for how to implement a microservices architecture
 - Bounded Contexts correspond well to individual microservices
 - Event sourcing and CQRS provide patterns for how to implement loosely coupled services
 - When converting a monolith to microservices, avoid a big bang rewrite, in favor of an iterative approach

Notes and links can be found on: http://bit.ly/2hZ8TM1

11m:51s - Microservices must be loosely coupled, usually creating a model with one database per service.
12m:45s - There is a business requirement to maintain data consistency across services, and using an event driven architecture is a good way to achieve that.
13m:38s - Event sourcing is specific technique for persisting domain objects as a series of events.
14m:11s - Just as transactions don’t like to be split across microservices, queries cannot simply join across multiple data sources. CQRS provides a solution that accommodates querying via microservices and materialized views.

More on this:
Quick scan our curated show notes on InfoQ. http://bit.ly/2hZ8TM1

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Chris Richardson on Domain-Driven Microservices Design

In this week’s podcast, Robert Blumen talks to Oliver Gould at QCon San Francsico 2016. Oliver is the CTO of Buoyant where he leads open source development efforts. Prior to Buoyant he was a Staff Infrastructure Engineer at Twitter where he was technical lead on Observability, Traffic, Configuration and Co-ordination teams.

Why listen to this podcast:

 - Stratification allows applications to own their logic while libraries take care of the different mechanisms, such as service discovery and load balancing
 - Cascading failures can’t be tested or protected against, so having a fast time to recovery is important
 - Having developers own their services with on-call mechanisms improves the reliability of the service; it’s best to optimise automatic restarts so problems can be addressed during normal working hours
 - Post mortem analysis of failures are important to improve run books or checklists and to share learning between teams
 - Incremental roll out of features with feature flags or weighted routing provides agility while testing with production load, which highlights issues that aren’t seen during limited developer testing

Notes and links can be found on: http://bit.ly/2ivoz9w

4m:05s - Each domain has different failure and operating modes, and the layered approach to resiliency means that the layer handles this automatically
4m:30s - Large systems may fail in unexpected ways
4m:35s - Twitter originally had the “Fail Whale” but this has been phased out as the system has become more stable
4m:50s - As Twitter grew, it needed to move quicker, with more engineers and less whale time
5m:10s - Automation and social tools were needed to improve the situation

More on this - Quick scan our curated show notes on InfoQ: http://bit.ly/2ivoz9w

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Oliver Gould About Architecting to Avoid and Recover from Failure

In this week’s podcast Wes Reisz talks to Neha Batra, a software engineer at Pivotal Labs. Neha spoke about pair programming in her recent QCon San Francisco 2016 presentation, and has taken time to discuss techniques to get started with the practice as well as tips for implementing it on your team. Neha also touches on vulnerability based trust and how it can help effectively build a trusting team environment.
Why listen to this podcast:

 - If you successfully start with pair programming, other tenants of XP are pulled along with you
 - Ways to get creative with remote pairing to make it work
 - The daily retro
 - Overcoming hesitance with managers when trying to implement pair programming full time
 - Vulnerability based trust building

Notes and links can be found on: http://bit.ly/2i2a0sJ

How has Pair Programming Evolved Over the Years? 

6m:17s - A lot of the fundamentals are the same, but with XP we take it to the extreme to be able to do it  eight hours a day.
6m:24s - To pair for eight hours a day we adapt to a lot of other process to create a simpler way of working, giving us an easier level to default to.
6m:44s - We use phrases in the team to make sure we agree on a test, that there are no false positives, when to refactor etc. This helps us avoid accruing code debt since we don’t do code reviews.

More on this:
Quick scan our curated show notes on InfoQ. http://bit.ly/2i2a0sJ

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Neha Batra - Pivotal Labs Pair Programming

Wesley Reisz talks to Slava Oks, who has worked at Microsoft for over 20 years on flagship products, including SQL Server. He also led the kernel team who worked on the Midori operating system. More recently, he has worked on bringing SQL Server to Linux.

Why listen to this podcast:

 - Microsoft SQL Server runs on Linux through a containerised approach called Drawbridge
 - Drawbridge implements a Linux loader and a minimal set of ABI calls to allow an in-process NT user mode kernel to run
 - SQL Server runs on top of a SQL platform layer (called SQL OS) that could be ported to run on Drawbridge
 - SQL Server had supportability commands added to allow the state of the system to be measured with SQL calls
 - A number of efficiency gains were applied to both the Drawbridge components and the SQL Server code to bring performance to within 20% of the equivalent process running on Windows

Notes and links can be found on: http://bit.ly/2kVfatX

Drawbridge
ABIs
Security
LibOS
Linux
SQL OS
Supportability
Some assembly required
Addressing
Performance

More on this:
Quick scan our curated show notes on InfoQ http://bit.ly/2kVfatX

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Architecting SQL Server on Linux: Slava Oks on Drawbridge, LibOS, & Addressing Between Windows/Linux

Wesley Reisz talks to Daniel Bryant on moving from monoliths to micro-services, covering bounded contexts, when to break up micro-services, event storming, practices like observability and tracing, and more.

Why listen to this podcast:

- Migrating a monolith to micro-services is best done by breaking off a valuable but not critical part first.
 - Designing a greenfield application as micro-services requires a strong understanding of the domain.
 - When a request enters the system, it needs to be tagged with a correlation id that flows down to all fan-out service requests.
 - Observability and metrics are essential parts to include when moving micro-services to production.
 - A service mesh allows you to scale services and permit binary transports without losing observability.

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2pFYBiT

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2pFYBiT

Daniel Bryant on Microservices and Domain Driven Design

On the InfoQ Podcast this week, Wes Reisz talks with the Queen of Patterns, Linda Rising. Linda discusses her thoughts on the importance of patterns, she answers questions about what really is a pattern, and how she became involved in working with them. Throughout the podcast she discusses a variety of organizational and personal patterns and finally wraps with patterns to apply when driving change and innovation.

Why listen to this podcast:

 - You have to realise that there’s nothing you can do about other people. The only person you can affect is yourself.
 - A pattern is not a band-aid that you use once. You use it in a context where you use it in conjunctions with other patterns.
 - Take baby steps when driving change in an organisation, and seek out a pocket of receptive people to drive it.
 - Slack is an important part to have in life, so that if something comes along you can absorb it without having to stop doing something else.
 - Listen, Listen, Listen.

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2vLIsMC

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2vLIsMC

Linda Rising on the Importance of Patterns, Her Journey, & Patterns for Driving Change/Innovation

In this podcast Shane Hastie, Lead Editor for Culture & Methods, spoke to Rebecca Parsons, CTO of ThoughtWorks, about their recent report on the need for Courageous Leadership and her forthcoming book on Evolutionary Architecture.

Why listen to this podcast:
- Courageous leadership enables organisations to solve enterprise-scale problems while still allowing empowered development teams to produce software in an agile fashion
- The importance of making the link between technology advances and what they could mean for a business organisation, rather than seeing them in isolation
- If your metric is how many experiments succeed then you will not take enough risks – turn this around and celebrate the learning from failed experiments
- The need to truly put the customer at the centre our focus, understand the kind of relationship we want with customers and how we want to interact with them
- Create an environment where people are recognised and valued for their varied perspectives and recognise that good ideas can come from many different places
- Evolutionary architecture is based on the idea that things are changing so rapidly now that you can’t predict what changes will be needed in the future
- Innovations around databases, architecture and continuous delivery enable evolutionary architecture to be a real thing today

InfoQ - Podcast

ThoughtWorks' CTO Rebecca Parsons on Courageous Leadership and Evolutionary Architecture

In this podcast Shane Hastie, Lead Editor for Culture & Methods, talks to Andrea Goulet & M. Scott Ford about their journey working together as a married couple and business partners, learning to collaborate and communicate despite having vastly different communication styles and viewpoints.

Why listen to this podcast:

 - Effective communication is a competitive advantage
 - The system that you produce will only be as good as the communication structure you have in place while you build it
 - The importance of learning to speak each other’s language – the terminology of development and business is different and it is necessary to take the time and effort to learn the different language
 - The concept of “inception layers” relating to how intensively someone is concentrating on an activity and their level of openness to interruption
 - The value of writing a daily journal in a wiki to share what’s been happening and make progress and learning visible

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2zwXlnO

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2zwXlnO

Andrea Goulet & M. Scott Ford on the Marriage of Communication & Code

In this podcast Shane Hastie, Lead Editor for Culture & Methods, talks to Don Denoncourt about remaining an engineer as one ages.

Why listen to this podcast:

•	Looking back over history and programming language changes since the early 1980’s
•	The importance of lifelong learning and putting your personal time into remaining relevant and up to date with new development platforms 
•	New graduates get to work on new things because they have just learned about them and they are prepared to take the lower paid roles than “experienced” people
•	If ongoing learning is not fun, then maybe you are in the wrong profession
•	The unconscious bias against older workers in a few teams and how they miss out on great insights 
•	Some advice for oldies and youngies working together

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2ltre65

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2ltre65

Don Denoncourt on Aging in IT and Being a Lifelong Learner

In this podcast, Deam Wampler discusses fast data, streaming, microservices, and the paradox of choice when it comes to the options available today building data pipelines.

Why listen to this podcast:

* Apache Beam is fast becoming the de-facto standard API for stream processing
* Spark is great for batch processing, but Flink is tackling the low-latency streaming processing market
* Avoid running blocking REST calls from within a stream processing system - have them asynchronously launched and communicate over Kafka queues
* Visibility into telemetry of streaming processing systems is still a new field and under active development
* Running the fast data platform is easily launched on an existing or new Mesosphere DC/OS runtime

More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2BYTMbI

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2BYTMbI

infoq

Season Saw