Impact Evaluation Methods
UPCOMING DATE: TBC
DURATION: 3 Days; 9am-5pm
COST: $2,500.00 per person (Discounts available)
LOCATION: UTS Business School, Ultimo NSW
Government departments, private firms and NGOs constantly make decisions which affect people’s lives in many ways. Governments and NGOs design and deliver various programs across many realms including education, health and justice. Financial organisations assess eligibility for loans and help customers to navigate financial decisions. A fundamental tasks for all such organisations is to determine what impact their programs and policies have on the people they serve. This task is known as ‘Program Impact Evaluation’.
Given the crucial importance of such evaluations, as well as the steadily increasing industry demand for professionals with the skills to conduct such evaluations, our expert academics working within the Economics Discipline Group of the UTS Business School have developed our very first ‘Impact Evaluation Methods’ short course.
The course covers the most important contemporary econometric evaluation techniques, without focusing on technical details. It emphasises hands-on application rather than theory, and covers essential topics and skills, such as randomised experiments, regression, instrumental variables, regression discontinuity designs, and differences-in-differences methods, as well as related methods.
Enrol to build your capability in conducting more effective assessment and comparison of impacts achieved through the programs and policies of your organisation, ensuring that funding and efforts are allocated to where they will achieve the greatest impact.
Interested In Hearing More? Get in Touch With Us.
If you have any questions about this course, you can get in touch with the UTS Business School Engagement Hub team on (+61) 2 9514 3504, or email us at business.practice@uts.edu.au.
Key details
| DURATION | 3 Days; 9am - 5pm |
| LOCATION | UTS City Campus, Ultimo NSW |
| COURSE COST |
$2,500.00
|
| WHERE TO ENROL? | Visit the UTS Open enrolment page. |
Course outline
1. INTRODUCTION TO IMPACT EVALUATION METHODS
This module covers:
- The importance of program impact evaluation
- The scourge or selection bias
- The magic of randomised controlled trials
- Applications
- Methods which uncover causal relationships versus methods that estimate associations
- Case studies and brief historical context
- A quick overview of quasi-experimental methods
2. INTRODUCTION TO STATA
Stata is the leading econometric software package. It is the package of choice for most
impact evaluation practitioners. This module covers:
- Introduction to Stata environment
- Data manipulation
- Data visualisation
- Descriptive statistics
- Menu-driven commands
- ‘Do files’
3. INTRODUCTION TO REGRESSION ANALYSIS
Whilst not strictly necessary, regression is a fundamental tool for most impact evaluations. This module introduces participants to the intuition and anatomy of
regression. Participants will learn to run basic regressions in Stata, to interpret the output, and to visualise the results.
4. RANDOMISED CONTROLLED TRIALS (RCTs OR EXPERIMENTS)
This module covers:
- Lab experiments vs field experiments;
- Critical design and execution considerations
- Randomisation
- Statistical power
- Clustered designs
- Interpretation of results
5. PRACTICAL CONSIDERATIONS WITH RCTs
This module covers:
- Threats to the validity of RCTs
- What can go wrong (and how to fix it)
- Ethical considerations
- Considerations for running imperfect experiments given constraints
- Interpreting and communicating results
6. INSTRUMENTAL VARIABLE (IV) REGRESSION
IV Regression has a long and rich history and has many applications. It is a key technique for causal inference in its own right. It is also a key tool for implementing all of the quasi-experimental techniques.
Whilst IV can be confusing, this module will show that its essence is straightforward. Participants will learn how IV can also be used to analyse the results of RCTs.
The module will also introduce participants to the ‘local average treatment effect’ and ‘external validity’.
7. REGRESSION DISCONTINUITY DESIGN (RDD)
Experiments are not always feasible or practical. But sometimes quasi-experimental techniques can be almost as powerful. RDD is a quasi-experimental technique that has emerged in recent years as the ‘Silver Standard’ of causal inference - the closest cousin of RCTs. A single graph is usually enough for a good RDD to be convincing.
Participants will learn about the intuition and power of RDD, and will implement RDD in the Stata environment, both graphically and through regression analysis.
8. DIFFERENCES-IN-DIFFERENCES (DiD) & RELATED TECHNIQUES
DiD is the most commonly used quasi-experimental technique. Like other quasi-experimental techniques, DiD seeks to mimic RCTs. In DiD, treated groups and comparison groups are followed over time.
Participants will implement and interpret DiD regressions in the Stata environment. Participants will also be introduced to Panel Data, as well as the synthetic control method.
Who will benefit?
- Existing practitioners working in roles that require them to conduct such evaluations or testing variation in programs;
- Decision-makers that are required to engage with results of evaluation work;
Course facilitator
Professor Peter Siminski | Professor, Economics Discipline Group
One of Australia’s leading applied microeconomists, Peter has led the Impact Evaluation Methods course design and will contribute to its delivery. He has previously used all of the techniques covered in the course in his own research and has been teaching the techniques to undergraduate and postgraduate students since 2009. Much of his research has applied modern impact evaluation techniques to study the causal effects of Australian government policies and programs on people’s lives. Peter has published in leading journals such as the American Economic Review, AEJ: Applied Economics and The Review of Economics and Statistics. He is Associate Editor of the Economic Record and is on the Editorial Board of the Economics of Education Review.
