Policy Analysis Criteria

PPA 670 POLICY ANALYSIS

ESTABLISHING ANALYSIS CRITERIA

What Are Criteria
Reliability and Validity
Economic Criteria
Equity Criteria
Technical Criteria
Political Criteria
Administrative Criteria

WHAT ARE CRITERIA

Every time a policy problem is identified, some statement of goals is adopted. The goals are what the adopted policy alternative should accomplish. Goals are broad, formal, long-term problem-solving achievements that are desired. An example might be to make sure that all rivers are safe, clean, and usable.

Goals are translated into objectives. Objectives are more concrete statements about desired end states, with time tables, target populations, and resource limits. An objective might be to make the Colorado River safe for swimming and fishing.

Criteria are the measurable dimensions of objectives. Criteria are used to compare how close different proposed policy alternatives will come to meeting the goals of solving the problem. Criteria set the rules to follow in analyzing and comparing different proposed policy alternatives (solutions).


Sample criteria for improving river water quality might be:: effectiveness--how much of an improvement in water quality will this alternative produce?; cost--how much will it cost to improve the quality of the river using this alternative?; technical--do we have the equipment and know-how to use this alternative?; political--is this alternative politically acceptable?

Measures are the actual measurements that will be taken of each proposed policy alternative. For example, measures such as the following might be employed:

effectiveness--how many milligrams of pollutants per liter of water will this alternative clean up?

cost--how many dollars will be required to implement this alternative?

technical--is the necessary equipment for this alternative available and are people trained to use it?

political--what percentage of the voting-age population will favor this alternative in a statewide poll?

One difficulty in specifying criteria and measures is that many problem statements have vague, fuzzy, or even conflicting goals. This is often necessary in order to get consensus on taking some action about the problem. But this complicates the selection of criteria.

If dirty rivers are a problem, and the goal is to have clean rivers, what is the most important considerations in choosing between different ways of cleaning up the rivers? Is it cost? Is it effectiveness? Is it equity?

What do we mean by "clean"? It is impossible to get rivers 100% clean. Do we use Federal, State, or local standards on admissible levels of toxicity? How will we measure the level of cleanliness that different policy alternatives are likely to produce?

RELIABILITY AND VALIDITY

The criteria and their measures must be unambiguous. They should be relatively straightforward and simple to measure. Their application should produce uniform results, no matter who does the measuring of different alternatives. And repeated measurements of the same alternative should produce the same results, again, no matter who does the measuring.

Criteria and measures should be appropriate to the unit of analysis. That is, if the goal of a proposed policy alternative is to change the investment strategies of cities, the unit of measurement is cities, not individuals. Be sure to specify whether the unit of measurement is households or families, census tracts or neighborhoods, school children or school districts, etc.

ECONOMIC CRITERIA

Most policy analysis involves at least one economic criterion. These include impacts on the economy, on expected public sector revenues, on government spending, etc.

The most common economic criteria are costs. These may include:: borrowing costs--the costs of borrowing funds; decreases in net worth--decreases in assets and/or liabilities; direct costs--directly attributable to the policy alternative; indirect costs--additional impacts not included in the goals; intangible costs--costs that cannot be counted or quantified; monetarizable costs--can be expressed in dollars; one-time fixed costs--new capital expenditures, equipment, training, etc.; operations and maintenance costs--ongoing costs of the alternative; opportunity costs--other things that could have been done with the same resources instead; tangible costs--can be counted and quantified

Costs need to be counted. One cannot assume that the money was going to be spent anyway. Costs should be identified as completely as possible, eliminating unpleasant surprises down the road.

Another type of cost criterion that is often employed is marginal cost. That is, if some good or service is already being produced, how much more will it cost to produce one additional unit of output?

The types of costs that are considered in marginal analysis are:: fixed costs--these do not vary in the short run, no matter how many units are produced; variable costs--these vary directly with the volume of output of goods or services; average costs--the total of units of output divided by the total costs of output; marginal costs--the costs of producing one additional unit of output; sunk costs--these are costs that can be ignored as they have already been spent in the past

Another type of economic criterion is benefits. Benefits are the opposite of costs. Benefits are ways in which the policy actors will be better off. Benefits can be measured in many of the same ways as costs, including:

direct benefits--directly attributable to the policy alternative

increases in net worth--increases in assets and/or liabilities

indirect benefits--additional benefits not included in the goals

interest earned--interest that will accrue or be paid

intangible benefits--benefits that cannot be counted or quantified

monetarizable benefits--can be expressed in dollars

one-time benefits--one-time reduction in the problem

ongoing benefits--continuing decreases in the problem

tangible benefits--can be counted and quantified

Benefits are often more difficult to quantify than costs. One alternative is to use "shadow prices," or the value of the benefits in a perfectly competitive market, for example, free recreation facilities, wilderness areas, parks, etc.

EQUITY CRITERIA

Efficiency and effectiveness are technical and economic questions, but equity is a public question. Equity asks about the social allocation of burdens and benefits. Equity asks the questions of "who pays?" and "who benefits?"

A proposed policy alternative may impact equity if it will change the distribution of burdens and benefits in society. There is no universally approved optimal or right answer for how benefits and burdens should be distributed in society. That is a continuing area of contention, and essentially a political decision.

However, there are guidelines for equity, such non-discrimination, and the same treatment for those equally situated and different treatment for those unequally situated.

Horizontal equity asks whether burdens and benefits are being shifted among groups in society which are relatively equal.

Vertical equity asks whether burdens and benefits are being shifted among groups in society which are relatively unequal.

Inter-generational equity asks whether burdens or benefits are being shifted from one time period to another, whether younger generations will have to pay more and receive less than older ones, or vice versa.

Groups are often identified on the basis of:: residence; income; citizenship; race or ethnicity; sex; age; family status; home ownership; educational status; veteran status; criminal record; substance abuse; health

Problems in assessing equity include:: how should the population be sub-divided?; how should groups be defined?; should historical criteria, the status quo, or desired states be used?; what is a burden?; what is a benefit?; what is a degree of need?; what is an ability to pay?

TECHNICAL CRITERIA

Effectiveness is often used as a criterion by which to judge policy proposals. Effectiveness is the extent to which the proposed policy will attain the goals set forth in the problem statement. For example, if the goal is to decrease the current teenage driver accident rate, how much will each policy alternative decrease the rate below current levels?

Another technical criterion is technical feasibility. This asks whether the technology exists or is readily available to implement a proposed alternative. For example, one proposed policy alternative may be to install in all cars a breath analyzing device that would not let a car start if the driver has been drinking. However, this technology is not widely or cheaply available.

Other technical criteria may question whether the measurement of criteria can be conducted at the desired level of reliability and validity. For example, are there tests that can adequately measure whether students in bilingual education programs have the same level of literacy as students in non-bilingual education programs?

POLITICAL CRITERIA

Many times the client for the policy analysis will hold a political office. In that case, the policy analyst must often include political criteria in the assessment of proposed policy alternatives.

Political viability asks whether or to what extent a proposed policy alternative will be acceptable to relevant powerful groups, decision makers, legislators, administrators, citizens, neighborhoods, unions, or others.

Other ways of assessing political viability include:: acceptability--is the proposed alternative acceptable to policy makers, policy targets, the general public, voters, etc.?; appropriateness--is the proposed alternative appropriate to the values of the community, society, the legislature, etc.?; legal--is the proposed alternative legal under current law, or will statutes have to be amended or enacted?; responsive--will the proposed alternative meet the real or perceived needs of the target group, the public, etc.?

ADMINISTRATIVE CRITERIA

Many public policies are implemented by public agencies. Therefore, administrative operability or administrative ease are often used as criteria for judging proposed public policies.

Questions that may be addressed include:: authority--does the agency have the authority to implement the proposed policy?; commitment--does the proposed policy have the commitment of top managers, field staff, and support staff?; capacity--does the agency have the resources to implement the proposed policy, in terms of staff, skills, money, training, expertise, etc.?; support--are the facilities, equipment, and other support available for the proposed policy?