March 7, 2022
7 March 2022
Ms Judi Zielke
Interim CEO
Australian Research Council
Dear Ms Zielke
The Group of Eight (Go8) welcomes the opportunity to respond to the Australian Research Council’s public consultation on its Excellence in Research for Australia (ERA) 2023 Submission Guidance Documentation.
The Go8 represents Australia’s eight consistently leading research-intensive universities, with seven of its members ranked in the world’s top 100 universities. It undertakes 70 per cent of Australia’s university research and spends some $6.5 billion on research each year. In ERA 2018, more than 99 per cent of Go8 research was rated above world standard.
The Go8 has consistently advocated that the Government must maintain its commitment to the public accountability of universities for research excellence and ensure it is rewarded. ERA has been the mechanism for this quality assurance for the sector and the Government, and to be effective must be appropriately rigorous and administratively manageable. The 2020-21 ERA EI Review not only gave effect to the Go8’s 2018 recommendation that the initiative be evaluated, but also took up Go8 recommendations regarding how the administrative burden of ERA could be reduced while its transparency is increased.
The Go8 notes and supports the request in the Acting Minister of Education Stuart Robert’s letter of expectation to the ARC that the ARC fast-track development of more efficient and robust assessments of the quality of research. We note your response to the Minister regarding the ARC’s efforts in this respect and look forward to being involved in any discussions regarding a new scale for ERA quality, including a revised ERA ‘world standard’ benchmark, and benchmarks for citation analysis and peer review assessment.
Noting the above as context, the Go8 provides the following recommendations supported by detailed discussion.
Recommendations:
- That the ARC consider and mitigate to the extent possible the impact of constrained timeframes on institutions capacity to submit to ERA 2023, given delayed guidelines and the need for institutions to incorporate the 2020 ANZSRC in systems
- That the ARC expedite the rollout of annualized submission processes as soon as possible as a means of facilitating less administrative burden in ‘bulk’ on universities
- That the ARC implement its proposed response in ERA 2023 to the impact of the pandemic in relation to the waived requirement for the 30% peer review sample to be a representative sample of research output types
- That findings of the work of the Indigenous Research Code working group be published as soon as possible given the need for institutions to incorporate any relevant recoding in their systems
- That the ARC provide context for the need for a Statutory Declaration as a new requirement, and consider removing it if on balance it appears to add no major value
Discussion
General
The Go8 is largely supportive of the expected trajectory of ERA as reflecting the ARC’s acceptance of all 22 recommendations of the 2020-21 ERA EI Review. Go8 institutions are eager for the implementation of these to be accelerated, including those that will impinge on post-2023 rounds, such as the move to an annual data collection, as a means of reducing the load at three yearly intervals on institutions.
With respect to ERA 2023, the Go8 would urge that the sooner new or revised specifications can be released, the more time is provided for institutions to prepare – noting that there are indications that institutions are already behind their usual preparation schedules. This includes the provision as soon as possible of the specifications related to the new Indigenous Field of Research. Other elements whose finalisation has been delayed but should also be released as soon as possible to give sufficient time for universities to prepare include:
- ERA benchmarks, rating scale and world standard definition
- By-line implementation
- Draft ERA 2023 Journal List
Likely impact on institutions
A conservative estimate of the implementation costs of ERA 2023, based on the figure calculated for a smaller Go8 university, is that this would be $1.25 million at the lower range. The costs of systems, including new systems or updates on systems due to introduced changes – such as the new ANZSRC, vendor payments, ongoing and specialty staff brought on to assist, all add to the costs of complying with ERA 2023.
Nevertheless, the Go8 considers ERA – as noted earlier – to be an essential benchmarking exercise in the absence of any other exercise or system; particularly as a means to reinforce the level of performance for key funders who may otherwise be sceptical of global ranking systems.
Workability of the changes in ERA 2023
The Go8 notes that the proposed changes will be largely implementable when finalised with some specific recommended changes, such as by-lining by affiliates, being welcomed to streamline collection processes.
Proposed response in ERA 2023 to the Pandemic
The proposed waiving of the requirement for the 30% ERA peer review sample to be a representative sample of research output types and the voluntary provision universities may have to explain the impact of the pandemic on a discipline in their submission are both welcomed by the Go8.
It is unclear nevertheless how the strict peer review sampling requirement waiver is pandemic related, or how the ARC proposes to factor in consideration (e.g. of ratings) of the impact of the pandemic on a discipline.
Incorporating the new ANZSRC
Go8 institutions had begun initiating changes to their systems as early as when the new ANZSRC was released. Feedback to date is that mapping of the new ANZSRC FoR codes has proved challenging in relation to some FoRs such as law, psychology and engineering, entailing a significant number of changes in systems and in coding and preparing those systems. Go8 universities are also awaiting and dependent on the findings and deliberations of the ARC Indigenous Studies Excellence and Engagement Group (ISEE) in considering how to incorporate the new FoR 45 for Indigenous Studies into their respective submissions.
Changes to address false and misleading data
Go8 institutions continue to support any measures intended to ensure fair and transparent reporting processes.
Institutions should be given the right of reply to any recalculations of citation profiles resulting from removal of work of individuals or research focuses.
Statutory Declaration
The Go8 urges reconsideration of the additional requirement to have a Statutory Declaration from the Director of HR in the absence of an institutional byline given it can be an additional and onerous administrative requirement, with added administrative costs.
Sensitivity flag
More detailed advice from the ARC on definitions for sensitivity flags and related data is needed. The sensitivity flag should only apply to those outputs selected as the subset to be provided in full for peer review. Although the sensitivity flag is appropriate, it will be challenging to collect and submit data relating to ERA systematically and comprehensively. It may be necessary to change university systems to accommodate these requirements.
Other feedback
Some Go8 feedback has indicated that the suggested 50% allocation to the Indigenous codes with FoR 45 is too high and not sensitive to the potential complexities that might be present with any particular output. For example, an interdisciplinary publication with Indigenous content or methods that would normally code to two other FoRs would be forced to be split 50-25-25, and potentially disadvantage the underlying disciplinary FoRs). It is too early to determine if 50% is an appropriate standard allocation without knowing the outcome of the current consultation by the ARC Indigenous Studies Excellence and Engagement Group (ISEE) on how to code to FoR 45. Rather than making this a rule now, the 50% allocation rule should be deferred until the ISEE recommendations and coding instructions are finalised.
Thank you again for the opportunity to contribute to this consultation.
Yours sincerely
VICKI THOMSON
CHIEF EXECUTIVE