Monitoring and Evaluation: Differences and Synergies between the Two
February 3, 2021Understanding Logical Framework and Theory of Change
February 18, 2021What are SMART Indicators in Monitoring and Evaluation?
To understand the ‘Indicator’ we need to start with, what is a variable? A variable is any characteristic, number, or quantity that can be measured and ‘varies’ between different respondents/units of measurements. Age, sex, income and expenses, place of birth, date of survey, vehicle type are examples of variables.
An indicator is a ‘variable’ that measures ‘things’ closely related to a specific construct but not precisely – it only ‘indicates’ just like your index finger would! For example, age is an indicator of aging but the precise aging of the body of two people 45 years of age will be different. Total coliform bacteria in drinking water indicates a ‘potential’ health hazard, but not all bacteria in the water are pathogenic (many bacteria are good for health like those in curd/yogurt). Assets owned by a household is an indicator of its wealth. Scores in an examination is an (arguable) indicator of knowledge or skills gained in school.
How are indicators constructed and used?
In monitoring and evaluation, indicators are typically used to measure changes throughout the program’s life in various activities, outputs, or outcomes. These indicators often require us to combine one or more variables from our dataset. For example, consider an outcome – coverage of toilets in a village. To measure this outcome using a survey, we can just summarize a single variable or question asking ‘whether the surveyed household had a toilet or not’. However, if the outcome was something more complex like ‘learning outcome among children’, then we will probably need to administer a knowledge test of several questions and somehow combine all those variables to obtain a kind of test-score or an index of learning. If the outcome of interest is ‘income’, then perhaps we need to collate data on many variables that ask income earned and money spent to earn that income from each member of a household.
Indicators are usually summarized as percentages in the case of binary or Yes/No type of variables or as means in the case of variables that are continuous numbers. For example, a variable ‘whether a household has a toilet’ can get coded 0 (for no) and 1 (for yes). The average of these 0s and 1s can be, say, 0.38 or 38 percent of households who answered yes. In case of learning outcomes, we can come up with a cut-off of say 80% and then create a new variable – ‘whether the child scored 80% or higher in the test’ and then summarize it to get ‘% of children who passed the threshold for expected learning level’. Often organizations need to measure coverage for especially activities and outputs such as number of people trained, number of blood donation camps, number of people who called to get advice, etc. These are simply numbers.
Monitoring and evaluation studies evaluate a ‘change’ in the indicator (often towards a pre-specified target). For example, we may need to know ‘the change in coverage of household toilets’ between 2014 and 2020. Or, ‘change in the proportion of children with adequate learning levels between treatment and control groups’. Note, many organizations specify the indicator as “change in % of toilets built between 2014 and 2020”, but it is not very accurate (but can be convenient in communication). The indicator is ‘% of households with toilet’ and the change in this indicator is what you want to measure/report as your impact.
So, What are SMART indicators?
You may have heard that the indicators we define and use in M&E should be SMART. But, the acronym SMART has been used in so many contexts and with different definitions that each letter in S M A R T can have different and sometimes overlapping interpretations as summarized below. When defining and constructing your indicators, you should ideally test on these criteria to decide if they are smart-enough for you. Note, a SMART indicator is not the one that is most accurate and precise, but the one that meets your learning or reporting needs the best within the constraints of time, resources, and money!
I hope you found this blog valuable. Share it with your colleagues if you did.
Still have M&E questions that Google can’t answer? Schedule a free micro-consulting meeting with us and we’ll try our best to help you!
