The Capability Maturity Model® (CMM®) Integration (CMMI®) has been applied successfully to systems development and maintenance and has helped organizations improve their project management, engineering, and related processes. In the Software Engineering Institute's (SEI) special report "Demonstrating the Impact and Benefits of CMMI: An Update and Preliminary Results [1]," the following benefits were reported:

• Boeing Australia experienced a 33 percent reduction in the average cost to fix a defect.
• General Motors experienced an 80 percent reduction in late deliveries.
• Lockheed Martin Integrated Systems and Solutions experienced a 30 percent gain in software productivity.

CMM-based process improvement has enabled these organizations to more consistently deliver products and services on time, at high quality, and for the predicted cost.

These gains are not the exception; they are the norm. System development organizations are making great strides in transferring evolutionary capability into their customers' hands. Gains achieved by Department of Defense (DoD) contractors are transferred directly to the fighting men and women of our armed forces as they become more capable and utilize technology faster than ever before. In addition to satisfied customers and a wellequipped warfighter, the return on the investment these organizations have experienced from the implementation of CMMI is substantial. For example, Northrop Grumman [1] enjoyed a 13-to- 1 return on investment.

The acquisition process plays a critical role in how the government transfers increased capabilities into operational use. Acquisition professionals must acquire complex systems and systems of systems in order to provide these enhanced capabilities. If using CMMI can help the developers of these systems, why not apply CMMI practices to help the acquirers as well?

In late 2003, a few colleagues familiar with both acquisition practices and CMMI were asked by Mark Schaeffer, principal deputy, Defense Systems, Office of the Under Secretary of Defense (OSD) for Acquisition, Technology, and Logistics (AT&L), to interpret CMMI for use in acquisition organizations. The goal was to publish a streamlined version of CMMI best practices that could easily be implemented through self-improvement and self-assessment activities to help establish effective acquisition practices within acquisition programs. The result was "CMMI-AM [Acquisition Module]" [2], a technical report published by the SEI. Acquisition professionals in government and industry can use this module to improve their processes.

The CMMI models and the CMMI modules are two different types of products. The CMMI models, which are part of the CMMI Product Suite, are the official documents that contain CMMI best practices, and can be used with a Standard CMMI Appraisal Method for Process Improvement (SCAMPI^SM) Class A appraisal to achieve a maturity level.

The CMMI modules, however, are documents that are excerpts from a CMMI model with possible trial additions and are available for piloting and use for process improvement. Modules that are deemed successful may at some time become part of a CMMI model. A module can be used to identify strengths, weaknesses, improvement opportunities, risks, and best practices during an informal gap analysis or as informative material during a benchmarking SCAMPI Class A appraisal using a CMMI model.

Although CMMI contains many best practices that can help an acquisition organization, CMMI-AM provides additional information designed to help acquisition organizations more easily apply CMMI best practices to their processes.

Systems acquisition is no easy task. If you think about how complex commercial products are, you are seeing just the tip of the iceberg. A family car is the result of a complex mix of subcomponents that are engineered into a system. Most DoD weapon and information systems are at least this complex.

Acquirers must not only understand the operational context and codify the desired capabilities or system requirements into something that can be implemented by a development team, but also they must continuously evaluate both the evolving systems and the development teams' ability to deliver the systems on time and according to requirements, including cost, fit, and function. The acquirer must also identify the risks involved in selecting one development team or set of suppliers over another, and collaborate with them proactively over the life of the program to ensure risks imposed by the acquirer's environment or the operational environment are identified and mitigated.

Unfortunately, the state of acquisition practice is not what it could be. Difficulties abound in government and industry. Increasing complexity of the systems being acquired has overtaken the experience of those acquiring them. Acquisition professionals often do not have the systems engineering or project management experience needed to meet acquisition objectives.

Many acquirers find it difficult to do the following:

• Establish robust systems engineering practices within the program office.
• Stabilize requirements well enough to adequately work with developers/suppliers.
• Estimate the time and effort required for the program to deliver a usable capability or system.
• Enforce schedule milestones and ontime delivery of acquisition products and services.
• Assess the technical risk involved in acquiring particular products from particular suppliers.
• Implement process control measures.
• Track short- and long-term costs in relation to a budget.
• Continuously identify and mitigate risks in a team environment with all relevant stakeholders.

Since the quality of systems is governed largely by the processes used to create and maintain them, improving the processes used by both the acquirer and the supplier will improve the quality of systems. Again, improving the processes of both the acquirer and the supplier is critical. When both have mature and capable processes, the probability of success is highest.

When the acquirer's processes are mature and the supplier's processes are not, the acquirer can mentor the supplier, but the outcome is not predictable. When the supplier's processes are mature, the acquirer with immature processes often encourages short cuts and interferes with the supplier's ability to meet requirements thus adversely affecting quality, cost, and schedule. Acquirers routinely ask contractors to cut systems engineering, quality assurance, and even causal analysis and continuous improvement activities because they fail to see their immediate value to the program.

Many DoD suppliers have a head start on their government customers because they are already using CMMI best practices. To improve the state of acquisition practice, effective acquisition processes must be defined, implemented, measured, and evolved. The contribution of the acquirer must also be more clearly visible as part of program success.

The government has shown its desire to improve the state of acquisition practice in Section 804 of the National Defense Authorization Act, released in December 2002 [3]. This section states, "Service/ departments shall establish programs to improve the software acquisition process."

The requirements of such a program include the following:

• A documented process for planning, requirements development and management, project management and oversight, and risk management.
• Metrics for performance measurement and continual process improvement.
• A process to ensure adherence to established process and requirements related to software acquisition.

The act also requires that the Office of System Architecture and Investment Analysis (Communications, Command, Control, and Intelligence) and the Undersecretary of Defense AT&L support government programs by the following:

• Prescribe uniform guidance for implementation across the DoD.
• Assist the services and departments by the following:
  • Ensuring that source selection criteria includes past performance and the maturity of the software products offered by potential sources.
  • Serving as a clearinghouse for best practices in software development and acquisition in both the public and private sectors.

This summer, a team of acquisition professionals who are knowledgeable about both CMMI and CMMI-AM has begun a series of pilot appraisals using the module within select DoD programs. In these pilots, participants evaluate the effectiveness of the module in helping program offices establish process improvement programs compliant with Section 804 requirements. This piloting activity is sponsored by Dave Castellano, deputy director, Systems Engineering, Defense Systems, OSD for AT&L.

By improving acquisition processes, acquirers can take on higher-risk programs because they can balance program risk with their improved ability to manage that risk (see Figure 1). The CMMI best practices provide guidance for improving an organization's processes and its ability to manage the development, acquisition, and maintenance of products and product components. The CMMI model and the CMMI-AM assemble best practices into a structure that helps organizations examine the effectiveness of their processes, establish priorities for their improvement, and implement needed improvement.

Figure 1: Notional Depiction of a Program's Ability to Balance Risk With Healthy Acquisition Practices
(Click on image above to show full-size version in pop-up window.)

CMMI best practices apply to organizations that manage project teams who develop systems (i.e., products and services), not just to the software or systems engineering disciplines within a project team. As illustrated in Figure 2, the CMMI model that includes practices from systems engineering (SE), software engineering (SW), integrated product and process development (IPPD), and supplier sourcing (SS), when used with a continuous representation, organizes the practices into four categories: process management, project management, engineering, and support. This CMMI model was chosen to be used with CMMI-AM because it contains the largest number of best practices that are relevant to the acquisition organization.

Figure 2: Structure of CMMI-SE/SW/IPPD/SS Model With a Continuous Representation
(Click on image above to show full-size version in pop-up window.)

The CMMI-AM focuses on effective acquisition activities and practices that are implemented by first-level acquisition projects such as a Systems Program Office. Acquisition practices are drawn and summarized from the following sources of best practices:

• CMMI models.
• The Software Acquisition Capability Maturity Model.
• The Federal Aviation Administration Integrated Capability Maturity Model.
• Section 804 of the National Defense Authorization Act.

The CMMI-AM is designed to be used with CMMI best practices as an acquisition lens for interpreting these practices in acquisition environments. Figure 3 illustrates the structure of the module.

Figure 3: The Structure of CMMI-AM
(Click on image above to show full-size version in pop-up window.)

If you compare Figures 2 and 3, you will see the difference between CMMI-SE/ SW/IPPD/SS and CMMI-AM. Notice that the module does not include the Process Management process areas.

In CMMI-AM Project Management process areas, Supplier Agreement Management, Integrated Supplier Management, and Quantitative Project Management are not transferred from the model. The module adds Solicitation and Contract Monitoring as a new process area.

In the Engineering process areas of CMMI-AM, Technical Solution and Product Integration are not transferred from the model.

In the Support process areas of CMMI-AM, Causal Analysis and Resolution is not transferred from the model. The module adds Transition to Operations and Support as a new process area.

To provide a flavor of CMMI-AM's content, the following includes a best practices' example from one process area within each process area category covered in CMMI-AM.

The Project Management process areas included in CMMI-AM are Project Planning, Project Monitoring and Control, Integrated Project Management, Risk Management, Integrated Teaming, and Solicitation and Contract Monitoring.

A few of the best practices included in the Solicitation and Contract Monitoring process area include the following:

• Designate a selection official.
• Establish cost and schedule estimates.
• Evaluate proposals.

The Engineering process areas included in CMMI-AM are Requirements Management, Requirements Development, Verification, and Validation.

A few of the best practices included in the Requirements Development process area include the following:

• Establish product and product-component requirements.
• Establish operational concepts and scenarios.
• Analyze requirements to achieve balance.

The Support process areas included in CMMI-AM are Configuration Management, Process and Product Quality Assurance, Measurement and Analysis, Decision Analysis and Resolution, Transition to Operations and Support, and Organizational Environment for Integration.

A few of the best practices included in the Transition to Operations and Support process area include the following:

• Establish product transition plans.
• Identify support responsibility.
• Evaluate product readiness.

The fundamental concepts of IPPD incorporated in CMMI-AM include the effective use of cross-functional or multidisciplinary teams, leadership commitment, appropriate allocation and delegation of decision making, and organizational structure that rewards team performance.

Generic practices ensure that the improvements you make to your processes are effective, repeatable, and lasting. These practices must be considered when implementing the specific practices of the process areas.

To improve acquisition practices, practitioners, projects, and organizations must move from ad hoc acquisition practices to explicit acquisition practices. Using CMMI-AM and the Initiating, Diagnosing, Establishing, Acting, and Learning (IDEAL^SM) model, a simple improvement process, organizations can do just that (see Figure 4).

Figure 4: The IDEAL Model

Using the IDEAL model and CMMI-AM, a process improvement team would follow each phase in the loop to improve its organization's acquisition practices. The IDEAL model is available at www.sei.cmu.edu/ideal/ideal.html.

The CMMI-AM has been going through piloting, and an updated module will be available for use in early Fall 2004. However, there is nothing stopping you from using CMMI-AM now.

To get started, learn as much as you can about CMMI, CMMI-AM, and your organization's acquisition practices. To learn more about CMMI models and CMMI-AM, see www.sei.cmu.edu/cmmi/models/models.html. To learn more about CMMI, see www.sei.cmu.edu/cmmi/.

Training is available to help you get started, including the Introduction to CMMI training course and CMMI-AM tutorial. There are two types of Introduction to CMMI training available: staged and continuous representations, allowing you to choose the course that is the best fit for your company. Regardless of which course you may take, your choice does not limit your ability to use either or both representations. See www.sei.cmu.edu/cmmi/training/course-decision.html for information about selecting an Introduction to CMMI course.

Introduction to CMMI training is available from the SEI or from members of the SEI Partner Network. For more information, refer to www.sei.cmu.edu/collaborating/partners/partners-tech.html#ICMMI.

The CMMI-AM tutorial is a one-day introduction to the module designed for acquisition professionals who have attended Introduction to CMMI training and are interested in applying CMMI to acquisition. If you are interested in the CMMI-AM tutorial, contact SEI Customer Relations at customer-relations@sei.cmu.edu for more information.

Ensure that your process improvement program has senior management sponsorship and middle management support. Such sponsorship and support is critical to making the program's success possible.

Determine the scope of your initial process improvement program. You can select one or more departments, divisions, programs, or projects. Or, you can select the entire organization. However, it is wise to begin with a smaller scope.

Map your organization's processes to CMMI-AM and CMMI model. It is unlikely that the best practices will map one-to-one with your organization's processes. However, by mapping the existing processes to the practices in CMMI-AM, you will identify where there are gaps. Consider using the IDEAL model to help you implement your process improvement program.

You can conduct an informal gap analysis using CMMI-AM or, if you want a maturity level or capability level rating, you can conduct a benchmarking SCAMPI Class A appraisal using CMMISE/SW/IPPD/SS Version 1.1 Continuous with CMMI-AM as additional informative material. If you choose to conduct a SCAMPI Class A appraisal, it will require an SEI-authorized SCAMPI Lead Appraiser. If you do not already have an authorized lead appraiser, there is a list of all currently authorized lead appraisers at www.sei.cmu.edu/collaborating/partners/partners-tech.html#SCAMPI. These lead appraisers also have the knowledge to conduct more informal gap analyses using CMMI-AM.

After your gap analysis or appraisal, you will know which processes enable the most useful improvement and the results will guide your process improvement efforts.

Use CMMI-AM as a place to start improving your acquisition processes. You will benefit from the previous experience of successful organizations and develop a language that is common among organizations improving their processes — organizations that include the suppliers you work with every day.

1. Goldenson, Dennis, and Diane Gibson. Demonstrating the Impact and Benefits of CMMI: An Update and Preliminary Results. CMU/SEI- 2003-SR-009. Pittsburgh, PA: Software Engineering Institute, 2003 www.sei.cmu.edu/publications/documents/03.reports/03sr009.html.
2. Bernard, Thomas, Brian Gallagher, Roger Bate, and Hal Wilson. CMMI-AM. CMU/SEI-2004-TR-001. Pittsburgh, PA: Software Engineering Institute, 2004 www.sei.cmu.edu/publications/documents/04.reports/04tr001.html.
3. U.S. Congress. "National Defense Authorization Act for Fiscal Year 2002." Calendar No. 163, 107th Congress, 1st Session, S. 1438. Washington, D.C., 2001 www.theorator.com/bills107/s1438.html.

1. CMMI Product Development Team. CMMI for Systems Engineering, Software Engineering, Integrated Product and Process Development, and Supplier Sourcing Version 1.1 Continuous Representation. CMU/SEI-2002-TR- 012. Pittsburgh, PA: Software Engineering Institute, Nov. 2000 www.sei.cmu.edu/publications/documents/02.reports/02tr011.html.

Brian P. Gallagher is the director of the Software Engineering Institute's (SEISM) Acquisition Support Program. He is responsible for building teams from across the SEI's disciplines to support the needs of the Department of Defense and other government agency acquisition programs.

Software Engineering Institute
4500 Fifth AVE
Pittsburgh, PA 15213-3890
Phone: (412) 268-7157
Fax: (412) 268-5758
E-mail: bg@sei.cmu.edu

Sandy Shrum is a senior writer/editor at the Software Engineering Institute (SEISM). Since 1998, she has been a member of the Capability Maturity Model® Integration (CMMI®) Product Team in roles such as author, reviewer, editor, and quality assurance process owner. Shrum also serves on the CMMI configuration control board and is the CMMI communications manager. She is co-author of the book "CMMI: Guidelines for Process Integration and Product Improvement." Before joining the SEI, Shrum wrote documentation for mainframe- and Unix-based products for Legent Corporation. She has more than 16 years experience as a technical writer in the software industry. Shrum has a Master of Science in professional writing from Carnegie Mellon University and a Bachelor of Science in business administration from Gannon University.

Software Engineering Institute
4500 Fifth AVE
Pittsburgh, PA 15213-3890
Phone: (412) 268-6503
Fax: (412) 268-5758
E-mail: sshrum@sei.cmu.edu