Assume variability; preserve options.

—SAFe Lean-Agile Principle #3

Set-Based Design

Set-Based Design (SBD) is a practice that keeps requirements and design options flexible for as long as possible during the development process. Instead of choosing a single point solution upfront, SBD identifies and simultaneously explores multiple options, eliminating poorer choices over time. It enhances flexibility in the design process by committing to technical solutions only after validating assumptions, which produces better economic results.

System development can be described as a process of continuously converting uncertainty to knowledge. No matter how well a system is initially defined and designed, real customer needs and technological choices are both uncertain and evolving. Therefore, understanding how a system needs to be implemented must adapt over time.

Details

With SBD, teams make design decisions only after they have gained sufficient knowledge and data. They do so by maintaining multiple requirements and design options for a longer period in the development cycle. As the timeline advances, they use empirical data to narrow the focus on the final design option. Through this approach, they embrace Principle #3 – Assume variability; preserve options for as long as possible, providing maximum flexibility.

Conversely, point-based design commits to a set of requirements and a single design strategy early in the process using old system information to drive decisions. It often leads to incorrect assumptions and late discoveries that require substantial firefighting and rework as the deadline approaches. This may require shortcuts, quality compromises, and—worse—missed program commitments and deadlines. Using the Continuous Delivery Pipeline to provide feedback and learning along with SBD is often the optimal approach.

SBD is an important practice for economic efficiency in Lean product development, described further in references [1] and [2]. Figure 1 shows the conceptual difference between set- and point-based design approaches.

Figure 1. Comparing set-based and point-based design approaches

Increase Economic Efficiency with Set-Based Design

Employing SBD along with the teams, the System Architect/Engineering defines the subsystems and components for the solution and identifies architecture options for each. The teams then explore multiple alternative concepts for each subsystem, filtering out options that provide less economic value, are flawed in some way that cannot meet the targets, or violate laws of physics, etc. [1].

In SAFe, teams explore set-based design alternatives, applying a hypothesis-driven Minimum Viable Product (MVP) approach with a Lean Startup mindset (see Continuous Delivery Pipeline). Design alternatives include hypotheses and assumptions. MVPs may also define experiments to gain the knowledge that allows teams to validate or invalidate those hypotheses, filter alternatives, and arrive at the optimal economic decision.

Figure 2 provides an example of a significant learning Milestone (a deadline for the decision) in which designers of a future autonomous vehicle have to select the technology to support a new initiative that prevents forward collisions.

Figure 2. An example where a future autonomous vehicle has a significant learning Milestone

Figure 2 shows teams exploring alternatives for a new ‘Obstacle Detection System’ (ODS) vehicle subsystem using lidar, radar, and camera technologies. With their corresponding hypothesis statement, they create Enablers that explore cost trade-offs, support for environmental and weather conditions, impact on vehicle design and manufacturing, quality of detection, nonfunctional requirements, etc. Teams record the results in the Solution Intent and filter the design alternatives based on validated learning.

Of course, exploring multiple design options comes at a cost to develop and maintain those options, even if they are mostly model-based or paper-based. (Note: Reinertsen points out that maintaining multiple design options is one form of u-curve optimization, and sometimes the optimum number on the curve is one. [3])

However, if there’s a high degree of innovation, variability, or immovable deadlines (e.g., the crop combine must ship in January), a set-based design is often the best choice. In this case, design efficiency depends on several factors:

  • Flexibility – Preserving a broad set of design options for as long as possible
  • Cost – Minimizing the cost of multiple options through modeling, simulation, and prototyping
  • Speed – Facilitating learning through early and frequent validation of design alternatives

Recommended practices to achieve design efficiency are described next.

Increase Flexibility in Interfaces and Design

Complex systems are built out of subsystems and component elements that collaborate to produce system behavior. Traditionally, specifications are defined early based on old knowledge with minimal understanding of what is possible and less concern for design trade-offs. A set-based approach makes trade-off decisions based on validated learning. Final designs and specifications emerge from teams exploring alternatives, gaining new knowledge, and understanding the trade-offs.

The System Architect/Engineering function typically defines the connections between subsystems and provides the context for teams to negotiate their own interfaces. In Figure 2, the system has a new ODS component responsible for preventing forward collisions. But teams must determine their own interfaces between the ODS and other subsystems—such as chassis (sensor mounting), powertrain (deceleration), braking, and lighting (brake lights).

However, interfaces are not rigid specifications. Lean allows interfaces to vary at the points of subsystem intersection. At these intersection points, system engineers may specify ranges for later negotiation. If component A is allocated additional space, weight, power, etc. over component B, how would overall system value improve? This approach allows system engineers to manage the system-level allocations while teams make their own detailed decisions, creating a collaborative environment for system-level learning, negotiation, and economic decisions.

Leverage Modeling, Simulation, and Prototyping

SBD leverages modeling, simulating, and prototyping to provide the initial learning points that help eliminate some design alternatives and confirm others. As described in the MBSE article, modeling incorporates a broad spectrum of techniques specific to the industry domain and product type, including digital twins, electrical/mechanical CAD, design thinking and user-centered design. These techniques should be applied to the parts of the system where the risk is highest, significantly reducing the cost of maintaining design alternatives for a longer period of time.

Frequent Integration Points

During development periods when new designs are being explored, uncertainty abounds. Useful knowledge is scarce. The only way to resolve the uncertainty is to test design alternatives through the early and frequent integration of the system components. Integration points are driven in part by System Demos, which occur on a fixed two-week cadence, and by Solution Demos, which typically occur on the longer Program Increment (PI) cadence.

In fact, without this frequent integration, SBD practices may create a false sense of security. They may even increase risk, as it’s possible none of the design alternatives will meet the identified targets. Frequent integration supports empirical learning with new insights used to reduce options as the system evolves, as Figure 3 illustrates.

Figure 3. Frequent integration provides critical learning points that narrow design alternatives

Take a Systems View

On large systems, decisions may span many initiatives. For example, the ODS technology decision for the autonomous vehicle should consider more than the current initiative to avoid front collisions. System Architect/Engineering owns the technical vision and helps ensure that the system meets future goals. As Figure 4 illustrates, they must guide the teams’ understanding of the larger solution when making significant technology decisions.

Figure 4. Technology decisions must look beyond the current initiative

As mentioned above, a set-based design has a cost. Exploring options can even increase the overall Cost of Delay (CoD). System Architect/Engineering must balance the possibility of over-engineering the solution (e.g. wasteful future-proofing) with being unprepared for near-term capabilities that may incur larger costs and rework ahead. And like every decision in SAFe, how much exploration should be an economic decision.

Economic Trade-Offs in Set-Based Design

Different design options have different economic implications. So understanding SBD requires knowledge of the macroeconomic goals and benefits of the system. One way to look at this is to place the alternatives on a spectrum, where a certain weight can be associated with each option.

Some of the significant economic indicators may include:

  • Cost of development
  • Cost of manufacturing
  • Performance and reliability
  • Cost of support
  • Development time
  • Technical risks

These indicators help illustrate which design options provide the greatest benefits. For instance, in the earlier collision prevention example, understanding the trade-off between the accuracy of various detection technologies vs. their added cost to manufacturing can dramatically change the decision, as Figure 5 demonstrates.

Figure 5. A trade-off curve between cost and a performance requirement (error margin) provides guidance for choosing among set-based designs

Use Set-Based Design in Fixed-Schedule Programs

Surprisingly, SBD is particularly effective for programs that require a high degree of fixed schedule commitments. Even if some of the more reliable design options don’t provide the degree of innovation that the system developers would prefer, it makes sense to keep multiple design options present. Since the schedule is unmovable, those options represent risk reduction should one of the fixed designs fail.  Risk reduction in fixed-schedule programs is often an informal (and typically less effective) form of SBD.

Learn More

[1] Ward, Allen, and Durward Sobek. Lean Process and Product Development, 2nd edition. Lean Enterprise Institute, Inc., 2014.

[2] Oosterwal, Dantar P. The Lean Machine: How Harley-Davidson Drove Top-Line Growth and Profitability with Revolutionary Lean Product Development. Amacom, 2010.

[3] Reinertsen, Don. Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing, 2009.

 

Last update: 10 February 2021

 

The information on this page is © 2010-2024 Scaled Agile, Inc. and is protected by US and International copyright laws. Neither images nor text can be copied from this site without the express written permission of the copyright holder. Scaled Agile Framework and SAFe are registered trademarks of Scaled Agile, Inc. Please visit Permissions FAQs and contact us for permissions.