How do you design metrics?
What is a metric and why do you need one?
An incentive mechanism such as a competitive tender allows landholders to bid for providing a service. Depending on the objective of the tender, different bids may propose different activities to achieve different types of environmental outcomes.
To be able to evaluate and compare the different bids, you need a way of scoring the environmental benefits of each bid. A metric is a way of determining that score. Only then can you take into account the bid prices and determine which bids offer the best value for money.
Using a well-designed metric helps ensure that bids are assessed in an efficient, transparent, and equitable manner.
How difficult is it to design a metric?
Designing a metric can be a complex activity requiring specialist skills that a regional NRM body may not have in-house.
A number of factors influence the design of a metric—these include the desired outcomes and their priorities (what improvements do you want to see?), the current condition of the environmental attribute (how will you measure improvements?), and the biophysical science available (is the data available for you to measure?).
Nine key elements of a metric
Before constructing your metric, examine the relevance of the following key elements and decide if you need to incorporate them into your metric:
- —you need to be able to measure both; consider, for example, hectares versus species.
- Spatial relations— do some spatial combinations of management changes yield greater outcomes?
- Relative change—what benchmark are you measuring improvements against? Is the baseline a duty of care? Is the duty of care requirement already defined?
- Location—are there upstream/downstream impacts? Consider distance to benefit.
- Timing—how long will it take to achieve the outcome? Is short-term change preferable over long-term change? Do you rate permanent change more highly?
- Implementation risk—what is the likelihood of the landholder failing to change management practices?
- Outcome uncertainty—what is the probability that the desired outcome will not result?
- Irreversibility/thresholds—what happens if nothing is done? extinctions? thresholds passed?
- Adverse impacts—will the solution create problems?
How to construct a metric
The two most important elements of your metric are quality/quantity as compared to your baseline, and spatial relations. Other elements are often used to weight the quality/quantity element.
If the outcome is difficult to measure directly, consider using a surrogate outcome that is easier to measure. And if you can't measure the outcomes, consider measuring the outputs.
Due to the complex nature of metrics, check first if a metric already exists that can be tailored to meet your needs.
Download a printable version of this fact sheet (PDF, 58 kB)*
Useful resources
The Designer Carrots website metric essentials section provides detailed information to help program designers to understand and initiate metric development.
Related topics
*requires Adobe Reader
Last updated 22 March 2011