Selecting the Right Consultant: Scores and Consensus

Three Key Questions:

1.  How detailed should evaluation criteria be in a Request for Proposals (RFP) or Request for Qualifications (RFQ)?  
2. Should the evaluation criteria include the weighting or maximum number of points for each criterion?  
3. What if the mathematical result of your selection committee doesn't result in selecting the consultant that most of the committee believes would be best for the project?

Lean Toward Transparency:  In the interests of not only transparency and openness in the selection process, but of communicating to consultants what experiences, personnel, and approach are most important to a public agency, I think it is best to always include detailed evaluation criteria with the maximum number of points possible for each criterion.

Concerns With Detailed Criteria and Weighting:  Some public agencies believe that including detailed criteria and weighting may not result in the best selection if the scores don't match the general assessment of the selection committee.  

Consensus Meeting:  To deal with this concern, I think it is a good practice for members of the evaluation committee to individually rate proposals or qualifications, and then come together as a committee to discuss their scores.  The purpose of such a meeting is to attempt to develop consensus, not meaning that each evaluator will have the same scores or that there will only be one score for the entire committee.  Instead, a consensus meeting allows members to discuss different scores and perspectives.  What one person saw might not have been apparent to another evaluator.  

Modifying Scores Based on Other Perspectives:  During the consensus meeting, evaluators should be free to modify their scores based on opinions and assessments from other members of the committee.  Of course, evaluators should always be free to not modify their scores based on the discussion.  The process of scoring proposals and qualifications, even if based on criteria, can be a somewhat subjective process, and the consensus discussions of the committee can help to balance out scoring.  Scoring only in isolation, devoid from discussions, may not result in the best selection for an agency.  That is the value of the consensus meeting of the evaluation committee.

Document With Comments:  Members of the selection evaluation committee should not merely assign numerical points for each criterion, but should make comments on an evaluation form noting some of the strengths and weaknesses of each proposal or statement of qualifications.  Not only can such comments help guide the consensus meeting of the committee, but they can be valuable justifications in the event of a protest, and can help capture some of the reasons for a selection in the event the public agency provides a debriefing to unsuccessful consultants.  Finally, some federal funding sources require a written document of the strengths and weaknesses of each proposer to be part of the procurement file, and the comments on each evaluation form can serve as a good source of information to help develop this document.

Mike Purdy's Public Contracting Blog 

© 2011 by Michael E. Purdy Associates, LLC 

http://PublicContracting.blogspot.com