Research Impact Methods for evaluating impacts of direct giving and cash transfers

The challenge

In Australia, children born under the poverty line are one of the most vulnerable groups in society – reliant on government support to improve their well-being and ensure they have access to a healthy upbringing. Within this legislative space, direct cash transfer programs have vast potential to improve the well-being of vulnerable populations: however; there is a current lack of confident knowledge on optimal approaches. This research explores the available methods of quantitatively evaluating the impact of social programs; in particular, those that involve direct giving and cash transfers to vulnerable families with young children. It considers the relative strengths and weaknesses of various options and offers suggestions about which option(s) to pursue in the Australian context at this time.

Solution

Supported by the Paul Ramsay Foundation, the research team aim to provide an overview of the evaluation methods that may be relevant to understand the impact of cash transfer programs. The research and associated report seek to explain the use of impact evaluations and how to prepare for them; outlining the strengths and weaknesses of different experimental and quasi-experimental methods, and identifying potential Australian data-sets. The paper includes a discussion of two existing Australian direct cash transfer schemes: Baby bonus and the Coronavirus Supplement. It considers whether these can be evaluated as ‘natural experiments’ with broader implications.

Outcome and impact

The findings of this project carry vast potential to generate direct outcomes for both policy and society. While the evaluated improvement of direct cash transfer programs will meet the immediate needs of disadvantaged families – the evaluation methods outlined in the report carry broad application across government and industry.

In relation to the evaluation of current direct cash transfer programs available to families with young children, the paper concludes that conducting a Randomised Controlled Trial is likely to be the best option to identify the best implementation choices. The attached report outlines these findings, as well as recommendations for further studies. 

Transcript

Everything we do at the Economics Department at UTS is underpinned by our mission statement, which is to contribute to social welfare in all of our activities. By social welfare we mean the well being of all people in society, and so that is a part of what we do. 

This work that we've been doing for the Paul Ramsay Foundation is a report on the impact evaluation method options to evaluate the impact of direct cash giving to families with vulnerable children. The best way to answer is using randomized controlled trials. So, we've seen disciplines such as medicine in particular, but also culture using randomized controlled trials to test various interventions for roughly a hundred years. These methods are seen as the gold standard for causal inference, but they're not used particularly widely for trialing social programs and, in particular, in Australia they haven't been used very broadly at all. 

I've been studying questions on intergenerational mobility, most recently about what we call absolute intergenerational mobility, which is about what proportion of people have a higher income than their own parents. This is a kind of summary indicator of social economic progress. When we talk about intergenerational economic mobility, we're talking about broadly speaking the relationship between people's incomes or their other kind of economic outcomes and their parents. The stronger the correlation, the less equality of opportunity we have because in that situation people are kind of born into their own outcomes whereas if the relationship is weaker than people have greater opportunities to achieve highly in their own right – independently of what their parents have done. 

In recent times the amount of economic growth we've had in Australia has declined and is projected to stay relatively low, whereas the level of inequality over recent decades has been increasing. Australia has historically had quite a high level of absolute income mobility, intergenerational mobility, but that's under threat because of the changes in the economy that it's seeing now. The Treasurer has expressed interest in this directly and wants us to do some more work to sort of flesh out these projections for the future.

Client testimonial

Evaluation of innovative programs is essential if we are ever going to make meaningful change. Like so many organisations working for purpose, PRF looks to evaluations and evidence to inform our work. We were pleased to partner with UTS Business School as they delivered rigorous research and evidence on the cash transfers programs.

George Argyrous
Head, Measurement, Evaluation & Learning, Paul Ramsay Foundation