Evaluation Criteria for Public Participation Methods

Jason Diceman — Wed, 19 Sep 2007 13:57:38 -0600

In search of a solid frame of reference for comparing various international citizen assembly processes I am investigating, I discovered an excellent paper called “Public Participation Methods: A Framework for Evaluation” by Gene Rowe and Lynn J. Frewer, two Ph.D.s working at the Institute of Food Research in Norwich, United Kingdom. Below I copied and pasted together all nine key criteria defined in the paper and their related suggestions. You can use these criteria and suggestions to help plan and evaluate any stakeholder engagement processes you are involved with, such as members surveys, focus groups, public hearings and citizen juries. I think it is very useful.

The original paper was published in 2000 in the Science, Technology, & Human Values journal and has since been the most read article on the journal's web site. You can download a free copy of the PDF from socialsciences.wur.nl or see the attached here. The full paper is 27 pages long with 75 bibliographic entries. It is definitely a key paper in the emerging canon of research on public participation mothods.

A Summary of the Criteria from “Public Participation Methods: A Framework for Evaluation” by Gene Rowe and Lynn J. Frewer

Acceptance Criteria

Representativeness: The public participants should comprise a broadly representative sample of the population of the affected public.
Suggestions: select a random stratified sample of the affected population; involve the use of questionnaires to determine the spread of attitudes with regard to a certain issue, using this as a basis for the proportionate selection of members.
Independence: The participation process should be conducted in an independent, unbiased way.
Suggestions: steering committee or management team incorporates members from diverse bodies or neutral organizations, such as university academics; disclosure from participants of any relationship to the sponsoring body ; rhe use of a respected facilitator.
Early [public] involvement: The public should be involved as early as possible in the process as soon as value judgements become salient.
Suggestions: Public debate should thus be allowed on underlying assumptions and agenda setting and not just on narrow, predefined problems
Influence: The output of the procedure should have a genuine impact on policy.
Suggestions: ensure that there is a clear acceptance beforehand as to how the output will be used and how it might direct policy; use of the media to inform the general public about the specific ways in which the output has influenced policy.
Transparency: The process should be transparent so that the public can see what is going on and how decisions are being made.
Suggestions: releasing information on aspects of the procedure, varying from the manner of the selection of the public participants to the way in which a decision is reached to the minutes of meeting; if any information needs to be withheld from the public, for reasons of sensitivity or security, ... admit the nature of what is being withheld and why.

Process Criteria

Resource accessibility: Public participants should have access to the appropriate resources to enable them to successfully fulfil their brief.
Suggestions: information resources (summaries of the pertinent facts); human resources (e.g., access to scientists, witnesses, decision analysts); material resources (e.g., overhead projectors/whiteboards); and time resources (participants should have sufficient time to make decisions).
Task definition: The nature and scope of the participation task should be clearly defined.
Suggestions: [clearly define at the outset] the scope of a participation exercise, its expected output, and the mechanisms of the procedure.
Structured decision making: The participation exercise should use/provide appropriate mechanisms for structuring and displaying the decision-making process.
Suggestions: A variety of decision-aiding tools might be incorporated into a participation procedure, such as decision analysis, decision trees, multiattribute utility theory, and the Delphi technique; structure the decision process in groups; important to structure the decision process in [small] groups; ian ndependent decision analyst could be usefully involved; use of an [experienced] group facilitator to employ rules for effective group decision making.
Cost-effectiveness: The procedure should in some sense be cost-effective.
Suggestions: take account of the potential costs of the alternative methods, in both time and money, and to consider the extent to which they fulfill the other criteria.

If I find other evaluation frameworks I will post them to this blog.