Jim Gasbarro, NDIA PMSC Chairman, called the meeting to order, and welcomed all the attendees. Pete Wynne, Vice Chair reviewed the current agenda and previous meeting minutes. Pete Wynne motioned for acceptance of the May PMSC minutes and Gay Infanti seconded the motion. The minutes were approved. Past meeting minutes are available on the NDIA PMSC website.

Clearing House – Pete Wynne, NDIA PMSC Vice Chairman

Status of the main NDIA issues with the DoD proposed EVM Policy were reviewed. Please see the presentation for the results. A summary of the open issues from NDIA concerns are:

Canvass industry regarding the 10-15 calendar day due date for EV data. Due Date September 7. Walt Berkey
Provide input on the impact of the threshold changes in terms of the number of contracts affected. Due date September 7th Walt Berkey
Provide suggestions for tailoring the CPR based on current C/SSR experience, to include the contents of format 5 and best practices for variance analysis. Due Date September 7 Pete Wynne
Provide suggestions for tailoring the IMS [on contracts below $50m]. Due date September 7 Pete Wynne
Still discussing the issue of EVMS validation, to include how the validation process works, acceptance of validations, other ways to obtain validation, and remedies available if a system is determined to be non-compliant with the Industry EVMS Standard.
DCMA is trying to time EVMIG update with implementation of DoD revised EVMS policy later this year.

Of note in discussion, David Treacy mentioned that DOE will require EVMS on FFP type contracts greater than $50. This is in contrast to DoD proposed optional application on FFP, and OMB FAR proposed changes to require EVMS on all major acquisitions.

The business case presenting the impact of the proposed DoD EVM policy changes was presented. Based on the proposed new thresholds, EVM will be required on all cost or incentive contracts valued at $50M or greater; however, it will no longer be required on contracts below $20M. All contracts requiring EVM will require an IBR and ongoing surveillance. Contracts below $20M that are designated as "major acquisitions" by DoD or OMB may be subject to EVM requirements in accordance with the FAR and OMB direction.

Walt Berkey, LMCO, summarized the Joint Government/Industry Working Group activities in three discussion topics that are of importance in developing the DOD EVM policy revisions. [NDIA Note: These were three separate presentations given Tuesday and Wednesday but consolidated into one place.]]

1) Application of EVMS:

Draft EVMS policy is perceived by industry to require an increased EVM workload on contracts requiring the application of EVMS. The Non recurring threshold will drop from 73M to $50M and the production threshold will drop from $315M to $50M which will increase the number of contracts required to implement a full 32 guideline compliance EVMS. The floor of 6.3M is being raised to $20M, which will decrease the number of low-end contracts currently implementing a C/SSR using a management system agreed to in the contract negotiations. The issues discussed were the Business Case assumption that contracts eliminated on the low-end equal the work content of the contracts added on the high end. Consensus was that this was not true and an equivalency factor will need to be developed for the business case. Another aspect discussed was what it means to require EVM below $50 M and above $20 M. Does this mean a 32 Guideline capable or compliant EVMS? If not, how will tailoring be accomplished? This question was not resolved during the discussion. Consensus of the government participants appeared to be that applying EVM means an EVMS fully compliant with the 32 ANSI/EIA 748 guidelines. Tailoring is to be addressed at a near term government meeting. There are differences between OMB FAR proposed changes “major acquisitions” from those proposed in the draft DoD EVMS policy. OMB will require EVM on all major acquisitions regardless of contract type. DoD draft policy is basing the application on the PM's assessment of risk. These areas of conflict will continue to be addressed as more information is provided.

2) Network Schedule and schedule analysis integrated with the CPR:

This area has confusion related to when required based on EVM being required on contract. The discussion focused this problem into three areas. On complex and high-risk contracts, a network schedule provides value to the program manager. On pure production contracts which are being managed using an MRP system and risk is significantly lower it does not make sense to layer a network schedule on top of the operating MRP system. Discussion also addressed the middle ground (i.e. FPI contract for LRIP or EMD contracts) where both development and production exist concurrently. In that case, a need for both a network schedule and MRP schedule could be needed to manage the contract.The discussion concluded that more information on tailoring and application expectation is needed to resolve this topic.

Recommendations:

Utilize network schedule and critical path on the high risk
development contracts
Implement networking of tasks traceable to the IMP events and
accomplishment criteria where IMP is utilized
To reflect the Statement of Work in manageable activities that
provide visibility to both the government and the contractor where the IMP is not utilized
Use the MRP/MRP II line of balance schedules where implemented for Production contracts.

3) Shorter deliver time for the CPR:

Multiple scenarios were discussed regarding the timeliness of CPR delivery addressing the OSD desire for delivery in 10 working days. Typical variations that impact the delivery of a CPR include accounting system closing dates and analysis scope (e.g. CPR formats and schedule analysis). Some information conveyed that early delivery of data will increases adjustments as the analysis reveals the need.

Three models were discussed showing the delivery time lines in work days and calendar days based on a prime contract and two subcontract data tiers. This showed delivery dates between 26-33 days assuming linear integration of data. Joe Houser recommended adding the relationship of increased reliance

on the use of estimated actuals to improve the delivery of the data. Several recommendations were made during this discussion. First is to not separate the data from the analysis. Second is to limit the data flow down to the first tier subcontractors. Third was to flow the data from all subcontract tiers and perform all analysis at the prime level. Conclusion was that tailoring guidance needs developed regarding CPR data reporting requirements.

Jim Gasbarro briefed his concerns as the departing PMSC Chair. He presented a wake-up call regarding the state of EVMS within the industry. OMB and DoD significant changes in policy will require EVMS by many new agencies and contractors. Significant questions:

Where do contractors go for expertise? Where do agencies go for EVMS expertise?

Business Case Evaluations have gotten the attention agencies regarding EVMS. Are we going to relearn or take the lessons learned over the last 30 years of EVMS application? He suggested as focus for the future: 1) Too many guides exist or will be created if we don’t have a common approach; 2) A validated contractor for one agency should be validated for others as well; 3) Need rules on EVM on FFP; 4) Pre-Award IBRs need explored; 5) Need common how-to guides; 6) Need Government Executive Agent for EVMS; 7) Need to tackle the issue of 3^rd party validations; 8) Need better integration of PMSC and PMI-CPM efforts.

In discussions, Dave Muzio, OMB, offered to write letter to Gen. (Ret>)Farrell, NDIA President, requesting that the PMSC serve as a focal point for EVMS for all government Departments and agencies. He would also write a letter to the Chief Acquisition Officers Council and the Chief Information Officers Council recommending that each department and agency attend the PMSC meetings to learn about EVMS and participate in the discussions and recommend that civilian agency contractors join NDIA

and participate in PMSC. Others brought up the CMMI model of levels as a model for EVMS maturity required by a customer. The concern was raised that EVMS is more of how we use the data verses a maturity model. The group consensus was that the “How-To” level of documents discussions needs put on hold at least until the January 2005 session as too much effort is required to handle the existing changes. See discussion on Wednesday for the action PMSC decided to capture on Jim’s challenges.

Pete Wynne briefed the status of the NDIA PMSC Guideline Intent document. Revision 9A (draft) is available on the NDIA PMSC website. It was developed by a joint industry and government working group. It is copyright and permission given within the document to copy as long as credit is given as specified. In discussion several government agencies discussed they were already using the intent document for validation and training, and it had wide acceptance. With the broadening of application to other agencies pending, the consensus was that the intent document needs updated with a table of contents and finally issued in non-draft format.

Walt Berkey, Lockheed Martin, briefed the status of the IBR Guide. GEIA Engineering Bulletin: HB-748-1 was published in May 2004 which is the PMSC IBR Guide with a cover page. The handbook now is available from GEIA for a charge. He continued with the status on the UK and US Reciprocity Agreement. APM and NDIA PMSC signed a reciprocity agreement effective July 1, 2004 recognizing equivalence of the EVMS guidelines in the ANSI/EIA 748 and the APM Earned Value Management (EVM) Guide. This is the first international agreement based on the ANSI/EIA 748 Guidelines.

Gay Infanti, Northrop Grumman, briefed the status of the Risk Management Team. They conducted a survey from November through June 2004 and received 121 responses. They have been unable to follow-up with most of the respondents who offered to share best practices - either the contact information is out of date or the individuals have failed to respond. Many best practices have been collected - some have been shared with the committee through briefings presented at committee meetings. Sharing best practices more broadly, with the program management community at large, will require permission of the process owners/authors due to potential proprietary or intellectual property issues.

A few members of the Risk Management Team will facilitate a workshop at the fall IPM conference and have submitted an abstract that includes a wrap up of the survey results and presentation a white paper for discussion. This white paper is currently in process and it addresses next steps toward integrated program management. The paper will contain recommendations to NDIA PMSC concerning what needs to be done to further foster the integration of risk management with earned value management and will include some suggested implementation steps. The team plans to be complete the publication of this paper. They recommend a follow-on team to address their recommendations including 1)the development of language to specifically address risk management within guidelines and handbooks, as well as how to integrate the two processes, and 2) work with DAU to further incorporate program management process integration into its training curriculum.

Joan Ugljesa, AIM Consultant, briefed the status of the XML Working Group. The group has been chartered by the committee since August 2003. In January 2004 it was decided by motion to take the formal standards path. They have made contacts with several standards groups to determine the best approach and to evaluate development requirements. The best path forward is to work with the Open Applications Group (OAGi). This is a non-profit industry consortium that uses a broad set of schemas that are widely used nationally and internationally. They are endorsed by and affiliated with other industry associations. Once work begins development can be completed in 5 to 8 months. The only requirement is that three members of OAGi are needed for new development work; Boeing is currently a member, Lockheed Martin is in the process of renewing their membership. AIM would need to be a member of OAGi to lead the development work, however, the membership fee is expensive. She requested NDIA sponsorship and the motion was tabled on the basis that PMSC has no funding sources. Without sponsorship or finding an existing OAGi member to take the lead, the team is on hold until membership issues with OAGi can be resolved.

Richard Zell, DCMA, briefed Lessons Learned from the ISO Quality world. The ISO process separates the standards from the compliance aspect. The American Society for Quality (ASB) origination owns the ISO standards in the US. They formed a for profit Registrar Accreditation Board (RAB) to qualify voluntary auditors. The RAB defined what pass/fail criteria was for reviews. They then certify auditors for third party compliance reviews. The Registrar (Auditor) then conducts compliance reviews and gives certificates of compliance. They systems are then re—reviewed every three years. In discussions it was brought out that the EVMS validation process must be recognized by customers, and is a program management process. The obvious application would be to third party validations, but additional work need to be examined about how would train and certify reviewers, and what would the validation mean in the eyes of the customer?

Jim Gasberro discussed new business topics. It was decided that the Guideline Intent document needs to be released by the end of the year with a table of content and linkage to validation and surveillance. It was decided that the committee needs to explore a formal relationship with PMI-CPM.

Jim Gasberro then adjourned the working group

Minutes of the NDIA Program Management Systems Committee (PMSC)
Battelle Memorial Institute, Columbus OH August 24-25

Jim Gasbarro, NDIA PMSC Chairman, called the Wednesday meeting to order, and welcomed all the attendees.

Steve Kelly, Battelle, gave an introduction into the history and business of Battelle. They started as a 501(C) trust from the will of Gordon Battelle in 1925. The core of there business is translating scientific discovery to innovative applications. They have embraced EVMS as a Program Management tool and mandated its use on all contracts.

Pete Wynne, NDIA PMSC Vice Chairman briefed the PMSC actions taken from yesterdays August 23^rd, 2004 meeting.

Pete Wynne will lead the team finalizing the ANSI Guideline Intent document. The guide will have references to validation and surveillance frameworks.Bob Loop is the other member of the team.
Walt Berkey will lead a team developing the Integrated Program Management Document Framework. This is intended to provide a roadmap between the IBR, Intent, and Surveillance guides to assist new practitioners in finding out what document to go to get guidance.Other members of this team are Richard Zell, Dick Treacy.
Buddy Everage will lead a team discussing the System Acceptance Process. This issue is critical with the expansion of validation to civilian contractors.Scope includes how to monitor and own the process.Other members include Dave Treacy, Richard Zell, Jim Gasberro, Jim Gordon, Floyd Reece, Joe Hauser.

The consensus of the membership was that the first two actions must be completed by yearend to be available for Agencies to begin to use.

In discussion Bob Loop briefed that he spoke with Ruth Franklin, the NDIA Director for Procurement. She confirmed that the “D” in NDIA does not limit our involvement to defense. Civilian contractors can join NDIA and federal government agencies can attend and participate equally with the DoD attendees. Ruth also noted that Government Agencies can also join NDIA as limited members. The committee gave Dave Muzio the approval to pursue designation of NDIA as the focal point for OMB EVMS policy. It was also noted that the committee would need to work with Dave on the exact draft wording before finalization.

The discussion was tabled regarding an EV Executive Agent. The suggestion was made to use the community of practice website through DAU for EVMS questions. Pete Wynne and Bob Loop took the action to work with Dave Bachman regarding this possibility.

POCs were discussed for the IBR, Intent, and Surveillance Guide. There was a suggestion to add the membership of the team that created the document as the point of contact. There is an issue with the surveillance guide; none of the membership has been attending NDIA PMSC meetings. No decision as of yet

Dave Muzio requested NDIA PMSC provide a paragraph to understand how to get documents and the points-of-contact. Action was given to Pete Wynne and Bob Loop to provide this paragraph.

The interface between PMI-CPM and NDIA PMSC was discussed. Pete Wynne as Chair has action to work with Neil Albert on how to effectively start this coordination.

Jim Gordon, NRO discussed a request that DCAA participate in the NDIA PMSC meetings. He had a meeting with them recently were they presented a plan for EVMS surveillance. Action: Pete Wynne to work with Ruth Franklin to write a letter to Bill Reed and request DCAA involvement.

Walt Berkey’s Wednesday briefing was summarized in Tuesday’s minutes.

Dave Muzio, OMB, briefed the status of the proposed FAR EVMS Clause Update. The proposed EVMS solicitation provision and contract clause would be required for all projects defined as major acquisitions, regardless of dollar value, including both fixed price and cost-type contracts.

- IBR's should be performed before contract award, whenever possible; however, contracting officers may determine in writing that the IBR cannot be performed before award.

   - In Part 7, Acquisition Planning, there is proposed new instructions that say, where the size and complexity of the project would make the cost of preparing for the IBR unreasonable within the
     normal proposal preparation costs, the project should request sufficient funds in the business case requesting funding for the planning phase of acquisition to pay for up to two offerors to undergo
     an IBR before award.

The above FAR proposed changes to the FAR are necessary to implement OMB's Circular A-11, Part 7, Planning, Budgeting, Acquisition, and Management of Capital Assets, and the Supplemental, Capital Programming Guide, requirements that agencies use EVMS on all major acquisitions. OMB reviews the use of EVMS on all business cases requesting funding submitted as part of the budget process, and use of EVMS on all other major acquisitions as part of the Budget and Performance Integration reviews. OMB reviews agencies policy, use of contract language and contractor validation as proof that EVMS is being used properly.

Neil Albert, MCR, briefed the formation of a team to update Military handbook 881 on the Work Breakdown Structure. The team will have 5 tasks:

1. Update for DoD 5000 changes.

2. Identify how the WBS can support each acquisition phase.

3. Reflect changes to new and improved approaches to product development.

4. Show how WBS is developed and maintained throughout the life cycle to meet program management goals.

5. Show how the WBS is the common link which will unify the planning, scheduling, estimating, budgeting, configuration management, and performance reporting disciplines.

The update is planned for year-end.

Debbie Tomsic, OUSD(AT&L)ARA, presented an OSD EVM activities update. Debbie briefed the status of the project to update the WBS Handbook and the status of the proposed new DoD EVM policy. The policy change recommendations and CPR and IMS DIDs have been revised to accommodate many of the key issues raised by industry during the comment period. Ongoing actions include:

Finalizing the policy change recommendations/rationale and DIDs for formal coordination within DoD.

Finalizing the business case (will incorporate input from industry).
Finalizing the tailoring guidance for CPR and IMS reporting (will incorporate input from industry).
Developing guidelines for applying EVM on FFP contracts.
Updating the EVM Implementation Guide to include tailoring guidance and to focus on how to contract for EVM.

The effective date of the new policy is contingent upon approval of the changes and all pertinent documents being updated and published. The new policy will apply to contracts awarded after the effective date and will not be retroactive. Interim policy will be issued if timing dictates, and the DoD 5000.2 will be revised during the next update cycle.

The back-up charts in Debbie's presentation summarize the DoD responses to the main issues raised by industry via the NDIA comments on the proposed policy changes.

MDA Update: Dave Milton discussed (no charts) status. The major issue is IBR excuses. A recent GAO audit required justification if the IBR was conducted greater than 6 months after award. They also had concerns about software metrics used to track progress including EVMS. As an agency they are also pursuing development of a common WBS.

AF Update: Major John Garrett discussed (no charts) Personnel changes happening (did not enumerate). They are focusing on implementation of SMART tool (System Management Acquisition Requirements Tool) which has EVMS metrics within it. And he mentioned the Air Force is significantly concerned with retirement issues.

DOE Update: David Treacy briefed the agency's existing thresholds are nearly identical to the proposed DoD thresholds. They have found steep learning curves in EVMS implementation with some suppliers such as Universities. Culture and existing management processes are the greatest obstacles. DOE is developing an EVM Application Guide. He is very concerned with the resources available to perform validations. He is supportive of both alternative methods of demonstrating EVMS compliance, including a formal third party validation process and perhaps self-assessment with government oversight. However, considerable development work remains before the government community (where the validation requirement originates) is prepared to accept alternative methods.

Navy Update: Kathy Llewellyn. Contractor personnel have raise concerns about Department of Navy practices when conducting audits and reviews. It was requested that these concerns should be forwarded to her (Kathryn.Llewellyn@navy.mil or by telephone at 703-693-8825). She emphasized that if anyone has any concerns with any issues with Navy personnel involved with EVM, please call her and the issue will be addressed promptly. DASN Management & Budget has requested audits on DoN implementation of EVM at three hardware systems commands; specifically, Naval Air Systems Command; Naval Sea Systems Command; and the Space and Naval Warfare Systems Command. The Audits are performed by Naval Audit Services, with assistance by the AIR 4.2 EVM staff. The NAVAIR audits have been completed; NAVSEA audits are near completion; and the SPAWAR audits will commence in FY 2005. Findings – EVM is not being used correctly and there is significant disparity in implementation within the Department of the Navy. Additionally, all audit and inspection findings are now required to be reported via the Echelon 1 organization to the next highest level of the Service as material weaknesses.

NRO Update: Lt.Col. Jim Gordon discussed (no charts) that an Earned Value Mangement Council has been chartered for the Intelligence Community (IC). The purpose is the exchange of information and best practices for implementation, surveillance, and analysis of EVM. Individually, the members of the IC primarily rely on third party validations by DCMA normally execute surveillance agreements instead of system validations or certification . Jim also briefed that he has been hired by the comptroller’s organization and a new EVMS director will be hired. His final action is to provide a briefing of the “State of EVMS in the NRO” to executive management. He agreed to share assessments of individual company EVM Systems and implementation with the appropriate EVM focal points.

Pete Wynne discussed that the next meeting is scheduled in January 2005 at West Palm Beach, hosted by Pratt and Whitney. The August 2005 meeting is planned for California, hosted by CS Solutions.

Jim Gasberro called for any additional items. There were none. He then closed by thanking our hosts – Battelle – for all of their gracious efforts. The August 2004 meeting was then adjourned.

YOUR SOURCE FOR EDUCATION, ACCESS, AND ADVOCACY

Download a summary of Divisions here

Find your Division today.

NDIA Division Excellence Award

NDIA's Education offerings exist to build a more capable, qualified, and
efficient Defense Industrial Base to support our national security through
doing business with the Department of Defense.

August 2004 meeting

NDIA Program Management Systems Committee Meeting

Batelle Memorial Institute, Columbus OH

August 24-25, 2004

Presentations, August 24-25, 2004 (in PDF format)

Integrated Program Management (IPMD)