In this podcast from the Carnegie Mellon University Software Engineering Institute, senior researchers Jerome Hugues and Joe Yankel discuss ModDevOps, an extension of DevSecOps that embraces model-based systems engineering (MBSE) practices and technology. By making the integration between DevSecOps and MBSE explicit, ModDevOps enables developers to benefit from both the speed of DevSecOps and the risk reduction of MBSE.

Model-Based Systems Engineering Meets DevSecOps

In this podcast from the Carnegie Mellon University Software Engineering Institute, visiting scientist Rick Kazman and principal researcher Suzanne Miller discuss software architecture patterns and the effect that certain architectural patterns have on quality attributes, such as availability and robustness. Kazman also provides examples of mechanisms—such as architectural tactics and patterns—and the effects they have on availability and robustness, especially in cloud-based systems.

Software Architecture Patterns for Robustness

To better understand the potential uses of large language models (LLMs) and their impact, a team of researchers at the Carnegie Mellon University Software Engineering Institute CERT Division conducted four in-depth case studies. The case studies span multiple domains and call for vastly different capabilities. In this podcast, Matthew Walsh, a senior data scientist in CERT, and Dominic Ross, Multi-Media Design Team lead, discuss their work in developing the four case studies as well as limitations and future uses of ChatGPT.

ChatGPT and the Evolution of Large Language Models

Containerization allows developers to run individual software applications in an isolated, controlled, repeatable way. With the increasing prevalence of cloud computing environments, containers are providing more and more of their underlying architecture. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Sasank Venkata Vishnubhatla and Maxwell Trdina, both engineers in the SEI CERT Division, sit down with Tim Chick, technical manager of the Applied Systems Group, to explore issues surrounding containerization, including recent vulnerabilities.&nbsp;

Securing Docker Containers: Techniques, Challenges, and Tools

As software size, complexity, and interconnectedness has grown, software modernization within the Department of Defense (DoD) has become more important than ever. In this discussion moderated by Matthew Butkovic, technical director of risk and resilience in the SEI CERT Division, SEI director Paul Nielsen outlines the SEI’s work with the DoD on software modernization, including controlling the attack surface, incorporating industry practices such as DevSecOps, and the interplay between software, cybersecurity, and AI. &nbsp;

DOD Software Modernization: SEI Impact and Innovation

Machine learning (ML) models commonly experience issues when integrated into production systems. In this podcast, researchers from the Carnegie Mellon University Software Engineering Institute and the U.S. Army AI Integration Center (AI2C) discuss Machine Learning Test and Evaluation (MLTE), a new tool that provides a process and infrastructure for ML test and evaluation. MLTE can aid organizations across the DoD in more effectively negotiating, documenting, and evaluating model and system qualities.

Improving Machine Learning Test and Evaluation with MLTE

, a principal researcher at the Carnegie Mellon University Software Engineering Institute (SEI) and lead of the SEI’s Tactical and AI-Enabled Systems Initiative, was elected the 2026 president of the IEEE Computer Society (CS), the largest community of computer scientists and engineers, with more than 370,000 members around the world. In this SEI podcast, Lewis sits down with Ipek Ozkaya, technical director of Engineering Intelligent Software Systems, to discuss her vision and plans for the IEEE CS presidency.

Grace Lewis Outlines Vision for IEEE Presidency

Insider incidents cause around 35 percent of data breaches, creating financial and security risks for organizations. In this podcast from the Carnegie Mellon University Software Engineering Institute, Austin Whisnant and Dan Costa discuss the Insider Incident Data Expression Standard (IIDES), a new schema for collecting and sharing data about insider incidents. IIDES facilitates insider incident information handling to help organizations better protect themselves against the compromise of sensitive information and mission-critical systems, which is essential to maintaining national security and defense.

Getting the Most Out of Your Insider Risk Data with IIDES

In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Gregory Touhill, director of the SEI CERT Division, sits down with Matthew Butkovic, technical director of Cyber Risk and Resilience at CERT, to discuss ways in which CERT researchers and technologists are working to deliver rapid capability to warfighters in the Department of Defense.&nbsp;

Delivering Capability to the DoD Warfighter

A strong cyber defense is vital to &nbsp;public- and private-sector activities in the United States. In 2019, in response to an executive order to strengthen America’s cybersecurity workforce, the Department of Homeland Security’s Cybersecurity and Infrastructure Security Agency (CISA) partnered with the SEI to develop and run the President’s Cup Cybersecurity Competition, a national cyber competition that identifies and rewards the best cybersecurity talent in the federal workforce. In six years, more than 8,000 people have taken part in the President’s Cup. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), Jarrett Booz, technical lead for the President’s Cup, and John DiRicco, a training specialist in the SEI’s CERT Division, sit down with Matthew Butkovic, the CERT technical director of cyber risk and resilience, to reflect on six years of hosting the cup, including challenges, lessons learned, the path forward, and publicly available resources. &nbsp;

The Best and Brightest: 6 Years of Supporting the President’s Cup Cybersecurity Competition

Warfighters in the Department of Defense (DoD) operate in high-stakes environments where security, efficiency, and speed are critical. In such environments DevSecOps has become crucial in the drive toward modernization and overall mission success. A recent study led by researchers at the Carnegie Mellon University Software Engineering Institute (SEI) examined the state of DevSecOps within the Department of Defense. In this podcast, Eileen Wrubel,&nbsp;the SEI’s Transforming Software Acquisition Policy and Practice technical director, sits down with George Lamb, director for DoD Cloud and Software Modernization in the Information Enterprise Office of the DoD CIO, which is responsible for the DoD Software Modernization Strategy and its associated implementation plan, and Bill Nichols, lead of the SEI’s Software Engineering Measurement and Analysis work. They discuss DevSecOps successes in the DoD and opportunities for scaling its impact.

Making Process Respectable Again: Advancing DevSecOps in the DoD Mission Space

A January 2025 Defense Innovation Board study on scaling nontraditional defense innovation stated, “We must act swiftly to ensure the DoD leads in global innovation and competition over AI and autonomous systems – and is a trendsetter for their responsible use in modern warfare." In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI), chief technical officer Tom Longstaff discusses the SEI’s long-standing work to help the DoD rapidly scale technology including artificial intelligence (AI) and autonomous systems. &nbsp;

The Magic in the Middle: Evolving Scaled Software Solutions for National Defense

Container images are increasingly being used as the main method for software deployment, so ensuring the reproducibility of container images is becoming a critical step in protecting the software supply chain. However, in practice builds are often not reproducible due to elements of the build environment that rely on nondeterministic factors such as timestamps and external dependencies. Lack of reproducibility can lead to lack of trust, broken builds, and possibly mask hidden malware insertion. Vessel, a recent tool from the Carnegie Mellon University Software Institute (SEI), helps developers identify the difference between two container images to help sort benign from problematic issues. In this SEI Podcast, Kevin Pitstick, a senior software engineer at the SEI and Vessel’s lead developer, and Lihan Zhan, a software engineer at the SEI working on tactical and AI-enabled systems, sit down with Grace Lewis, lead of the Tactical and AI-Enabled Systems (TAS) applied research and development team at the SEI, to discuss the Vessel tool, its development, and application in mission-critical settings.  &nbsp;     &nbsp;

Understanding Container Reproducibility Challenges: Stopping the Next Solar Winds

In this episode, the first in a series by Suzanne Miller and Mary Ann Lapham exploring the application of agile principles in the Department of Defense (DoD), the two researchers discuss the application of the first principle, "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software." Listen on Apple Podcasts.

Applying Agile in the DoD: First Principle

In the acquisition of a software-intensive system, the relationship between the software architecture and the acquisition strategy is typically not examined. Although software is increasingly important to the success of government programs, there is often little consideration given to its impact on early key program decisions. The Carnegie Mellon University Software Engineering Institute (SEI) is conducting a multi-phase research initiative aimed at answering the question: is the probability of a program's success improved through deliberately producing a program acquisition strategy and software architecture that are mutually constrained and aligned? Moreover, can we develop a method that helps government program offices produce such alignment? In this podcast, Patrick Place describes research aimed at determining how acquisition quality attributes can be expressed and used to facilitate alignment among the software architecture and acquisition strategy.&nbsp; Listen on Apple Podcasts.

Using Quality Attributes to Improve Acquisition

In this episode, the sixth in a series by Suzanne Miller and Mary Ann Lapham exploring the application of Agile principles in the Department of Defense (DoD), the two researchers discuss the application of the sixth principle,The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. &nbsp; Listen on Apple Podcasts.

Applying Agile in the DoD: Sixth Principle

In this podcast, Tim Chick and Gene Miluk discuss methodology and outputs of the Checkpoint Diagnostic, a tool that provides organizations with actionable performance related information and analysis closely linked to business value. The Checkpoint Diagnostic utilizes process models, data mapping, and quantitative analytics to provide organizations with qualitative process baselines, quantitative performance baselines, benchmark performance comparison, and a prioritized listing of improvement opportunities. Listen on Apple Podcasts.

From Process to Performance-Based Improvement

In Department of Defense programs, a system of systems (SoS) is integrated to accomplish a number of missions that involve cooperation among individual systems. Understanding the activities conducted within each system and how they interoperate to accomplish the missions of the SoS is of vital importance. A mission thread is a sequence of end-to-end activities and events, given as a series of steps, that accomplish the execution of one or more capabilities that the SoS supports. However, listing the steps and describing them do not reveal all the important concerns associated with cooperation among the systems to accomplish the mission; understanding the architectural and engineering considerations associated with each mission thread is also essential. In this podcast, Michael Gagliardi introduces the Mission Thread Workshop (MTW), a facilitated, stakeholder-centric workshop whose purpose is to elicit and refine end-to-end quality attribute, capability, and engineering considerations for SoS mission threads. Listen on Apple Podcasts.

Introduction to the Mission Thread Workshop

The Building Security In Maturity Model (BSIMM) is the result of a multi-year study of real-world software security initiatives. It is built directly from data observed in 78 software security initiatives from firms in nine market sectors. The best way to use the BSIMM is to compare and contrast your own initiative with the data about what other organizations are doing as described in the model. You can then identify goals and objectives and refer to the BSIMM to determine which additional activities make sense for you.The BSIMM data show that high maturity initiatives are well-rounded—carrying out numerous activities in all 12 of the practices described by the model. The model also describes how mature software security initiatives evolve, change, and improve over time.In this podcast, Gary McGraw, the Chief Technology Officer for Cigital, discusses the latest version of BSIMM and how to take advantage of observed practices from high-performing organizations. Listen on Apple Podcasts.

Build Security In Maturity Model (BSIMM) – Practices from Seventy Eight Organizations

A sound security metrics program is grounded in selecting data that is relevant to consumers and collecting it from repeatable processes.  Listen on Apple Podcasts.

Initiating a Security Metrics Program: Key Points to Consider

Government agencies and private industry must build effective partnerships to secure national critical infrastructures.&nbsp; Listen on Apple Podcasts.

Public-Private Partnerships: Essential for National Cyber Security

Business leaders need to understand the risks to their organizations caused by the proliferation of botnets.  Listen on Apple Podcasts.

Tackling the Growing Botnet Threat

This podcast summarizes the inaugural Measuring What Matters Workshop conducted in November 2014, and the team’s experiences in planning and executing the workshop, and identifying improvements for future offerings. The Measuring What Matters Workshop introduces the Goal-Question-Indicator-Metric (GQIM) approach that enables users to derive meaningful metrics for managing cybersecurity risks from strategic and business objectives. This approach helps ensure that organizational leaders have better information to make decisions, take action, and change behaviors.  Katie Stewart, Michelle Valdez, Lisa Young, and Julia Allen, the developers and facilitators of this workshop, are all members of CERT’s Cyber Resilience Management team. Further details about this workshop can be found in our workshop report.  Listen on Apple Podcasts.

A Workshop on Measuring What Matters

Significant insider threat vulnerabilities can be introduced (and mitigated) during all phases of the software development life cycle.  Listen on Apple Podcasts.

Insider Threat and the Software Development Life Cycle

In our studies of many large-scale software systems, we have observed that defective files seldom exist alone. They are usually architecturally connected, and their architectural structures exhibit significant design flaws that propagate bugginess among files. We call these flawed structures the architecture roots, a type of technical debt that incurs high maintenance penalties. Removing the architecture roots of bugginess requires refactoring, but the benefits of refactoring have historically been difficult for architects to quantify or justify. In this podcast, Rick Kazman and Carol Woody discuss an approach to model and analyze software architecture as a set of design rule spaces). Using data extracted from the project’s development artifacts, this approach identifies the files implicated in architecture flaws and suggest refactorings based on removing these flaws.&nbsp; &nbsp; Listen on Apple Podcasts.

Identifying the Architectural Roots of Vulnerabilities

Due to advances in hardware and software technologies, Department of Defense (DoD) systems today are highly capable and complex. However, they also face increasing scale, computation, and security challenges. Compounding these challenges, DoD systems were historically designed using stove-piped architectures that lock the government into a small number of system integrators, each devising proprietary point solutions that are expensive to develop and sustain over the lifecycle. Although these stove-piped solutions have been problematic (and unsustainable) for years, the budget cuts occurring under sequestration are motivating the DoD to reinvigorate its focus on identifying alternative means to drive down costs, create more affordable acquisition choices, and improve acquisition program performance. A promising approach to meet these goals is open systems architecture (OSA). In this podcast, Don Firesmith discusses how acquisition professionals and system integrators can apply OSA practices to effectively decompose large monolithic business and technical architectures into manageable and modular solutions that can integrate innovation more rapidly and lower total ownership costs. Listen on Apple Podcasts.

Open Systems Architectures: When &amp; Where to Be Closed

All major defense contractors in the market can tell you about their approaches to implementing the values and principles found in the Agile Manifesto. Published frameworks and methodologies are rapidly maturing, and a wave of associated terminology is part of the modern lexicon. We are seeing consultants feuding on Internet forums as well, each claiming to have the “true” answer for what Agile is and how to make it work in your organization. The challenge now is to scale Agile to work in complex settings with larger teams, larger systems, longer timelines, diverse operating environments, and multiple engineering disciplines. In this podcast, Will Hayes and Eileen Wrubel present five perspectives on scaling Agile from leading thinkers in the field, including Scott Ambler, Steve Messenger, Craig Larman, Jeff Sutherland, and Dean Leffingwell.  Listen on Apple Podcasts.

Scaling Agile Methods

The Domain Name System (DNS) is an essential component of the Internet, a virtual phone book of names and numbers, but we rarely think about it until something goes wrong. DNS also serves as the backbone for other services critical to organizations including email, external web access, file sharing and voice over IP (VoIP). There are steps, however, that network administrators can take to ensure the security and resilience of their DNS infrastructure and avoid security pitfalls. In this podcast, Mark Langston discusses best practices for designing a secure, reliable DNS infrastructure.  Listen on Apple Podcasts.

DNS Best Practices

As software developers deal with issues such as legacy modernization, agile adoption, and architecture, they need to be able to articulate the tradeoffs of design and business decisions. In this podcast, Ipek Ozkaya talks about managing technical debt as a core software engineering practice and its importance in the education of future software engineers.  Listen on Apple Podcasts.

Technical Debt as a Core Software Engineering Practice

In this podcast excerpt, Ritwik Gupta and Carson Sestili describe deep learning and how it differs from machine learning.  “As you compose more and more non-linear functions together, you can represent a much wider function space than you could with just one non-linear function. That is why deep learning is different from shallow learning. Shallow learning doesn’t compose multiple things together. Deep learning does.”

Deep Learning in Depth: Deep Learning versus Machine Learning

Classifying errors in a component-based system is challenging. Components, and the systems that rely on them, can fail in myriad, unpredictable ways. It is nonetheless a challenge that should be addressed because component-based, software-driven systems are increasingly used for safety-critical applications. In this podcast, SEI researchers Peter Feiler and Sam Procter present the Architecture Analysis and Design Language (AADL) EMV2 Error Library, which is an established taxonomy that draws on a broad range of previous work in classifying system errors.

The AADL Error Library: 4 Families of Systems Errors

Today's major defense systems rely heavily on software-enabled capabilities. However, many defense programs acquiring new systems first determine the physical items to develop, assuming the contractors for those items will provide all needed software for the capability. But software by its nature spans physical items: it provides the inter-system communication that has a direct influence on most capabilities, and thus must be architected intelligently, especially when pieces are built by different contractors. As Dr. Sarah Sheard discusses in this SEI Podcast, if this architecture step is not done properly, a software-reliant project can be set up to fail from the first architectural decision.

Why Software Architects Must Be Involved in the Earliest Systems Engineering Activities

In this podcast, Jeff Gennari and Cory Cohen discuss updates to the Pharos Binary Analysis Framework in GitHub, including a new plug-in to import OOAnalyzer analysis into the NSA's recently released Ghidra software reverse engineering tool suite.

Reverse Engineering Object-Oriented Code with Ghidra and New Pharos Tools

The culture of computers and information technology changes quickly. The Future of Cyber Podcast series explores the future of cyber and whether we can use the innovations of the past to address the problems of the future. In our latest episode, Bobbie Stempfley, director of the SEI’s CERT Division, interviews Dr. Diana Burley, executive director and chair of the Institute for Information Infrastructure Protection, or I3P, and vice provost for research at American University. Their discussion focused on educating the cybersecurity workforce in a way that closes the gap between what students are taught in school and the skills they’ll need to use in the workplace.

The Future of Cyber: Educating the Cybersecurity Workforce

Situational awareness makes it possible to get relevant information from across an organization, to integrate that information, and to disseminate it to help leaders make more informed decisions. In this SEI Podcast, Angela Horneman and Timothy Morrow, researchers in the SEI's CERT Division, discuss the importance of looking beyond the network to acquire situational awareness for cybersecurity.

Situational Awareness for Cybersecurity: Beyond the Network

In this SEI Podcast, Marisa Midler and Tim Shimeall, network defense analysts within the SEI's CERT Division, discuss the growing problem of ransomware including the rise of ransomware as a service threats. Ransom payments from Quarter 3 of 2019 were on average $42,000, and in Quarter 1 of 2020, that average increased $70,000 to $112,000. The volume of attacks also increased by 25 percent in Quarter 4 of 2019 and by another 25 percent in Quarter 1 of 2020. The sophistication of the attacks has increased alongside their severity. Midler and Shimeall discuss steps and strategies that organizations can adopt to minimize their exposure to the risks and threats associated with ransomware.

Ransomware: Evolution, Rise, and Response

The SEI is a nonprofit, public–private partnership conducting R&D in software engineering, systems engineering, cybersecurity, and many other areas of computing, working to introduce private-sector innovations into government.

Carnegie Mellon - Software Engineering Institute

Popular tracks