Revaluation Methodology for Measuring Value

From P2P Foundation
Jump to navigation Jump to search

* Policy Note: Revaluation: a participative approach to measuring and making change. A CECAN Evaluation and Policy Practice Note for policy analysts and evaluators, Note No. 7, Summer 2017

URL = https://www.cecan.ac.uk/sites/default/files/2018-01/07%20-%20Revaluation%20%28online%29.pdf


Characteristics

" What are the main elements of Revaluation?

Revaluation is both a process for measuring change in complex systems, and for making change in those systems:

• Shortcuts the process by which the attributes of an intervention are found to be effective, in order that they can then be replicated elsewhere. Instead it evaluates in real time, helping those working in a system to understand their effectiveness as their action or intervention unfolds, in order to increase their impact.

• Can be used for both formative (developmental) and summative (impact) evaluations: wherever the question “what is the value of this activity?” is asked.

• Is experienced by those working in a system as a simple participative process involving three phases of activity:

    • telling their stories about the intervention or system,
    • translating those stories into items of data expressing pieces of value,
    • then negotiating with other participants to reach a settled account of the overall value.

• Is a process that is responsive to context, and the resources of the system (including available time), and is thus different in each setting.

• Is intended to function as an extension of what participants are already doing, rather than an additional burden which feels apart from their work, and undertaken for someone else.

• Calculate involves presenting quantitative outputs and outcomes; and manipulating numbers (summing, or converting using proxy data and multipliers) to arrive at a single figure, usually in £s. The dominant metric in orthodox evaluation.

• Calibrate involves qualitative judgements about the relative merits (or cost/benefits) of different actions and outcomes. Based on how actors decide where to direct their efforts (and how much effort to make), both as individual decision making and socialised in groups.

• Capacitate involves measuring the characteristics and capacity of a movement or network, plus the potential of that network to increase its capacity in future, and thus the value it can generate (ie its emergent qualities). Included here are relational data, exploring the connections in a network including in space (eg maps) and time (eg calendars)."

What key terms are used in Revaluation?

"The central device of Revaluation is the 6 box grid for revealing value. This is introduced to actors at the start of the measurement process, and is used as a dashboard to summarise the kinds of value revealed by the end of the study.

The main premise of Revaluation is that to develop a full account of the value in a system, we have to measure in three dimensions (the 3 Cs), each of which includes visible and invisible value, as shown below: evaluation is an innovative process for measuring the full value of activity in complex systems. It is a new approach, developed in the context of NHS Change Day 2015, a grassroots social movement for improving patient experience. The organisers of Change Day felt it was impossible to evaluate – based in part on having tried to do so in previous years – and framed the research task as one of “reimagining evaluation”." (https://www.cecan.ac.uk/sites/default/files/2018-01/07%20-%20Revaluation%20%28online%29.pdf)

Typology

"In each of these dimensions, Revaluation explores both visible and invisible value:

• Visible value is based on known data, that which is observed in a system and already collected (at least in some parts of the system). As such, visible value tends to relate to changes past. It can also be considered as direct value: intended impacts and outcomes, in line with the objectives of an intervention.

• Invisible value is based on knowable data, which could be gathered and generated if actors or commissioners were interested in doing so. A simple example is Return on Investment data, in which wider outcomes are monetised (eg using proxy multipliers) to make their £ value visible. By extension, invisible value is also that which is hidden by actors (in systems where their work is not supported by commissioners) or which is indirect: not in line with (or unforeseen by) the intended outcomes of the intervention as commissioned." (https://www.cecan.ac.uk/sites/default/files/2018-01/07%20-%20Revaluation%20%28online%29.pdf)

Examples

"Revaluation has ...been applied in diverse settings including work on the natural environment, encouraging physical activity, and family nursing. It is continuously being refined through practice. Revaluation is centrally concerned with revealing the value of an activity or intervention in a complex system. Rather than asking “what works”, its first question is “what is going on?” For this and other reasons, Revaluation has been described both as “a paradigm shift in evaluation” and “not evaluation”." (https://www.cecan.ac.uk/sites/default/files/2018-01/07%20-%20Revaluation%20%28online%29.pdf)