Trust, but verify.

—Ronald Reagan, citing a Russian proverb

Compliance

Compliance refers to a strategy and a set of activities and artifacts that allow teams to apply Lean-Agile development methods to build systems that have the highest possible quality, while simultaneously ensuring they meet any regulatory, industry, or other relevant standards.

Note: This article is supported by a white paper, ‘Achieving Regulatory and Industry Standards Compliance with the Scaled Agile Framework®(SAFe®),’ which you can download here. In addition, there is a SAFe webinar on the topic, with Q&A, which can be viewed here.

Enterprises use SAFe to build some of the world’s largest and most important systems, many of which have unacceptable social or economic costs of failure. Examples of these high-assurance systems include medical devices, automobiles, avionics, banking and financial services, and aerospace and defense. To protect public safety, these systems are often subject to extensive regulatory or customer oversight and rigorous compliance requirements. In addition, many enterprises are subject to other government regulations (examples: Sarbanes-Oxley, HIPAA, ACA, state insurance regulations) that require similar attention and audits to ensure compliance.

Historically, organizations operating under such regulations have relied on comprehensive quality management systems (QMS). Legacy QMS systems are based on traditional, phase-gated development models and struggle to keep pace with accelerating time-to-market demands and Lean-Agile practices. Of greater concern is that even when the higher Cost of Delay (CoD) is accepted, these traditional approaches often do not increase quality or eliminate risk. As Deming notes, “Inspection is too late. The quality good, or bad, is already in the product.”

Continually addressing compliance concerns is one of the nine practices of SAFe’s Enterprise Solution Delivery competency. This article extends that knowledge and offers guidance on how to apply Lean-Agile methods to build these systems faster and better, while also addressing critical compliance requirements.

Details

Traditional waterfall practices often mandate that full system specifications are defined and committed to in detail, up-front, long before all the real system behaviors can be known. Worse, the sequential nature of phase-gate development produces large batches of work, long cycles between system integration points, and late feedback. In addition, compliance activities are typically deferred until the end of the project, providing little insight into compliance progress.

This often results in missed deadlines, disappointing business or mission outcomes, lower quality, and substantial (and late) compliance challenges. In contrast, high-assurance Lean-Agile development builds in quality incrementally—early and throughout the development lifecycle.

The Role of the Quality Management System

To satisfy compliance requirements, organizations must demonstrate that their system meets its intended purpose and has no unintended consequences that might cause harm. They must also develop the objective evidence required to prove that the system conforms to those standards. To that end, organizations that build high-assurance systems define their approved practices, policies, and procedures in a QMS. These systems ensure that development activities and outcomes comply with all relevant regulations and provide the required documentation to prove it.

Unfortunately, many QMS systems are heavily influenced by traditional phase-gated waterfall methods. This seriously inhibits, and can even prevent, the adoption of newer methods, as the older methods are hard-coded into the only approved way of working. As Figure 1 illustrates, SAFe describes an incremental approach to both development and compliance.

Figure 1. A Lean-Agile quality management system improves quality and makes compliance more predictable

This means those who want the benefits of Lean-Agile development (faster time to market and higher quality, to name a few) will typically have to evolve a Lean QMS.

The remainder of this article provides guidance on Lean-Agile strategies and patterns that develop these high-assurance systems.

Build the Solution and Compliance Incrementally

Even with a set of robust specifications, Agile Teams never have all the answers when development begins. Instead, they have a set of hypotheses that must be tested through a series of short, iterative experiments, which provide validated learning to ideally advance toward the ultimate solution. Figure 2 highlights SAFe’s incremental development approach, comparing Shewhart/Deming’s Plan-Do-Check-Adjust (PDCA) learning cycles with a traditional waterfall model.

Figure 2. Rapid Plan-Do-Check-Adjust learning cycles increase quality and reduce risk

Figure 2 illustrates two important implications for compliance. First, building smaller, working parts of the solution early allows compliance activities to also begin early, removing the large bow wave of performing such actions at the end. Each increment assesses both the viability of the current solution and its progress toward compliance, providing early feedback on the system’s ultimate fitness for use. Second, specifications are created early and evolved over time in small batches, with faster feedback on decisions and the opportunity for continuous review and assessment.

Organize for Value and Compliance

Agile Release Trains (ARTs) are the primary value delivery organizations in SAFe. Each train requires all the skills necessary to build and release the Solution, including those responsible for Quality Assurance (QA), security, testing, and Verification and Validation (V&V). (Note: While some regulations require independent, objective assurance, compliance representatives can still participate continuously as members of the ART). The result is an ART designed as illustrated in Figure 3.

Figure 3. Agile Release Trains include all disciplines including compliance

Solution and Product Management ensure that the Solution Intent and backlog properly reflect compliance requirements. Teams also ensure that their work includes appropriate compliance activities.

Build In Quality and Compliance

Built-In Quality is one of SAFe’s four Core Values, a dimension of SAFe’s Team and Technical Agility competency, and a core principle of the Lean-Agile Mindset. SAFe describes the use of Built-In Quality practices, including automation to detect compliance and quality problems and, when detected, stopping the entire system to focus everyone on resolving the problem. This philosophy applies systems thinking by ‘optimizing the whole,’ ensuring fast flow across the entire Development Value Stream and making quality everyone’s job. Quality is a culture, not a job title.

To that end, compliance concerns are also built directly into the development process, and automated wherever possible, as illustrated in Figure 4.

Figure 4. Automate compliance into design-build-test automation

Not all compliance activities can be automated, however, as some regulatory requirements mandate manual activities, including activities like Failure Mode and Effects Analysis (FMEA) and audits. This work is simply planned as part of the team backlog. The goal is to conduct these activities and reviews as the solution is being built, reducing the last sign-off activity from a large, extended event to a quick and boring ‘non-event.’

This way, the program receives early feedback on the degree to which the team’s compliance activities are being met and, conversely, how those activities may be impacting team performance. Figure 5 shows the feedback cycle between team activities and the practices defined by the Lean QMS.

Figure 5. Program Increments provide a feedback loop for compliance activities and practices

Continuously Verify and Validate (V&V)

Most high-assurance systems require V&V to ensure that:

  • The system works as designed (verification)
  • It meets the needs of the user (validation)

V&V must always occur against a known set of requirements. Otherwise, there is nothing to V&V against. As Figure 6 illustrates, SAFe uses solution intent as the repository for the existing and emerging requirements and designs.

Figure 6. SAFe’s Solution Intent provides support for Verification and Validation

Traceability within the solution intent ensures that the artifacts produced each Program Increment (PI)—the actual software, hardware components, etc.—address regulatory and compliance specifications, providing end-to-end evidence that V&V requirements have been met.

The SAFe Requirements Model Supports V&V

In SAFe, all elements of the requirements have test cases (see Figure 7), which are created at the same time as the functionality.

Figure 7. SAFe’s requirements meta-model supports Verification and Validation

Each increment yields new functionality and, consequently, adds new tests. As the number of tests grows, automation is vital to prevent testing activities from becoming a bottleneck.

Make V&V and Compliance Activities Part of Regular Flow

Verification and validation activities are performed in small batches. Each work item ensures it includes all the necessary information required for compliance. Any reviews, audits, and sign-offs are included in the work item’s Definition of Done (DoD) as shown in Figure 8. Each PI, the System Demo evaluates the solution’s progress towards its operational goals and the objective evidence necessary to meet compliance.

Figure 8. Verification and Validation activities are part of flow

Release Validated Solutions on Demand

Finally, SAFe recognizes that although the product development process happens in a predictable cadence, the release process (Release on Demand) may require additional activities. These can include:

  • Validation testing of the final release candidate (examples: medical trial, flight test)
  • Review of the objective evidence required before production approval and release
  • Customer, regulatory, UAT, signoffs, document submissions

Even then, Lean-thinking organizations always strive to fully automate delivery and, where possible, build in automated final release checks as part of a SAFe Continuous Delivery Pipeline and release on demand.

Learn More

[1] Leffingwell, Dean. Agile Software Requirements. Pearson Education, 2011.

 

Last update: 10 February 2021

 

The information on this page is © 2010-2024 Scaled Agile, Inc. and is protected by US and International copyright laws. Neither images nor text can be copied from this site without the express written permission of the copyright holder. Scaled Agile Framework and SAFe are registered trademarks of Scaled Agile, Inc. Please visit Permissions FAQs and contact us for permissions.