Performance-Based Research Fund (PBRF), University of Otago, New Zealand

The University of Otago is clearly New Zealand's top-ranked university for research once all four of the average-quality measures in the 2012 Performance-Based Research Fund (PBRF) Quality Evaluation are taken into account.

The PBRF is the New Zealand Government's formal tertiary institution research performance assessment.

The Tertiary Education Commission's interim report on the 2012 PBRF Quality Evaluation contains a rich and detailed view of the research performance of Tertiary Education Organisations (TEOs), research strengths in different subject areas, and the performance of individual Nominated Academic Units (NAUs) within TEOs.

A full copy of the report, and an accompanying audit report, can be found at http://www.tec.govt.nz/Funding/Fund-finder/Performance-Based-Research-Fund-PBRF-/quality-evaluation/. As well as the Quality Evaluation results, the report also analyses Research Degree Completions (RDC) and External Research Income (ERI), the other two components of the PBRF funding formula.

^ Top of page

Results Summary

Key results for the University of Otago from this report are:

Otago has 189 A-rated staff (representing 179.6 fulltime employees or FTE) and 546 B-rated staff (504.1FTE), second only to the University of Auckland. A-rated researchers are judged as possessing high international standing in their fields and B-rated as enjoying high national standing. A summary of where the University stands in terms of the FTE of A/B/C/R staff (and comparisons with 2006/2003) is:

	A(FTE)	B(FTE)	C(FTE)	CNE(FTE)	TOTAL(FTE)
2012	179.56	504.12	290.95	193.56	1168.19
2006	137.85	439.37	261.94	150.86	990.02
2003	91.11	344.06	409.85		845.02

Otago's staff profile shows very good balance and commitment to developing new researchers, with 275 of the University's 'new and emerging' (NE) staff (245 FTE) receiving funded grades. Of these 221 (193.6 FTE) received 'C(NE)' grades, 53 (49.4 FTE) received 'B' grades, and 2 received 'A' grades.

^ Top of page

Average Quality Scores

Four new average quality score (AQS) metrics have been introduced for 2012, two relative to staff (AQS(N) and AQS(S)) and two relative to students (AQS(E) and AQS(P)).

The new average quality score (AQS(N)) provides an indication of the quality of research undertaken by staff at each tertiary education organisation whose evidence portfolios (EPs) were submitted for assessment.
The AQS(E) provides an indication of the extent to which teaching and learning at degree-level and above is underpinned by research at each TEO.
The AQS(P) indicates the extent that postgraduate degree-level teaching and learning is underpinned by the research undertaken at each TEO.
Finally, the AQS(S) indicates the extent to which staff whose EPs were assigned a funded Quality Category are representative of all academic teaching and research staff at each TEO.

A summary of Otago's results in the four different AQS measures (and comparisons with 2003/2006) is:

	AQS(N)	AQS(E)	AQS(P)	AQS(S)
2012	4.96	1.55	8.14	18.47
2006	4.89	1.43	7.98	17.26
2003	4.49	1.19	6.93	14.62

All of these measures have shown increases in 2012 compared to 2006 and 2003. The student-focused measure reveals that for every 100 postgraduate students at Otago there are more than 20 A- or B-rated staff, giving a ratio of better than 5:1.

Otago's rankings in the four different AQS measures, and comparisons with 2003/2006, are as follows:

	AQS(N)	AQS(E)	AQS(P)	AQS(S)
2012	3^rd	2^nd	1^st	4^th
2006	2^nd	1^st	1^st	2^nd
2003	3^rd	1^st	1^st	4^th

^ Top of page

Scorecards versus League Tables

The University has a strong, balanced scorecard to report, and summary table of the positions of all New Zealand universities for all four average-quality scores is presented below. As the TEC report states, the University of Otago is the only TEO to be ranked in the top four in all four AQS measures.

The 2006 PBRF evaluation, which used one average-quality measure, found that Otago was the top-ranked university for research quality¹. When the full scorecard is considered, Otago retains this distinction.

2012	AQS(N)	AQS(E)	AQS(P)	AQS(S)	SUM
Otago	3^rd	2^nd	1^st	4^th	10
Victoria	1^st	6^th	3^rd	1^st	11
Auckland	2^nd	3^rd	5^th	3^rd	13
Canterbury	4^th	4^th	7^th	2^nd	17
Lincoln	7^th	1^st	2^nd	7^th	17
Waikato	5^th	7^th	4^th	5^th	21
Massey	6^th	4^th	6^th	6^th	22
AUT	8^th	8^th	7^th	8^th	31

^ Top of page

PBRF: the facts

PBRF was introduced in 2003 to encourage and reward research excellence in New Zealand's tertiary education sector.

It is managed by the Tertiary Education Commission (TEC) and it has progressively replaced a student numbers-based model for supporting research.

Any Tertiary Education Organisation (TEO) which is approved to teach degree programmes is eligible to take part in the PBRF.

PBRF provides criteria for evaluating research performance. The most significant of these is the quality evaluation (60 per cent weighting).

The quality evaluation process comprises a peer review assessment of evidence portfolios submitted by tertiary institutions to TEC. The portfolios describe eligible staff members' research outputs and their contributions to the research environment as well as peer esteem.

This quality evaluation enables TEC to calculate an aggregate quality score for each TEO. This gives stakeholders reliable information about the quality and level of research in the tertiary education sector.

Altogether, 31 TEOs participated in the 2006 PBRF quality evaluation. Of those, the University of Otago received the highest quality score.