REF 2021 & ECRs: the current situation

This is based on a talk that I gave at an event on “Publishing and Research Strategies for ECR Classicists” at the Institute of Classical Studies on 20th September 2017. If you want to cut to the chase, the slides are here, full notes of my talk below.

It is a slightly tricky time to be writing this post: there have been a host of announcements about changes to the REF policy in recent months, weeks, and even days, but much remains undecided, making it difficult to offer advice of the kind that was possible for REF 2014. That said, ECRs (and especially those currently on the job market) are very much living through this uncertainty, and can’t simply wait it out until next summer to make decisions that impact upon careers.

With that in mind, this post is intended as a reflection on the current state of affairs. While I can’t offer anything too concrete, I have tried to clarify, or at least clearly set out, the main areas of uncertainty relating to ECRs, and to give pointers on where to find information and what to look out for as further details are released. I’ve also offered some preliminary advice for ECRs based on what I think can be inferred thus far.

All of this comes with the (big) disclaimer that these are my own opinions only, some areas are still open to interpretation, this is by no means definitive, and this may well yet change: and n.b. that none of the information that has been released is final policy – that will come next year.

  1. REF2021: an overview

The Research Excellence Framework is the system for allocating research funding in UK HEI; it is based on assessing 3 factors with :

  • outputs (60%);
  • impact (25% – an increase from 20% in 2014);
  • and environment (15%)

The focus of what follows is on research, although impact may be of relevance to ECRs (see paragraphs 19-26 of the Initial Decisions document for the latest on impact).

The previous (and first) round of the exercise was run in 2014 and raised some areas of difficulty, including a substantial impact on ECRs as my earlier work documents. A period of review and consultation has therefore been in progress, starting with the Stern Review in 2016, a Consultation exercise on the findings of the Stern Review held in March 2017 (for which the summary of responses was published last week), and some Consultation events. The Initial decisions on the REF were published at the start of September 2017, along with a Circular Letter. There have also been a series of webinars over the summer, most recently this one in July on Developing staff and outputs, not to mention several blog posts by David Sweeney on various topics. (yes, I’m exhausted just writing all of that).

The timetable indicates that draft guidance will be published next summer, with final guidance and criteria in winter 2018-19 (see Initial decisions, paragraphs 49-51). Based on the approach thus far, I think we can expect that until then there will be this ongoing slow-release of pieces of information as they become available which is helpful, but also means that there will be a lot of discussion and changes to keep track of.

2. What we know so far

The broad parameters of research assessment will stay the same: “outputs” (broadly, publications) will be submitted to the REF by HEIs and sent to subject panels who assess by peer review; quantitative data will be used in some disciplines where appropriate (see Initial decisions, para. 6). Peer reviewers will rank submissions on a 1-4* basis:

ref
Quality profile from REF2014

The timeframe for publications that can be submitted will be 1st January 2014 – 31st December 2020.

A key change from the last round will be the new Open Access policy, which states that from April 2016 journal articles submitted to the REF must be available in an Open Access form, i.e. an institutional or subject repository (n.b. “early deposit”, i.e. within 3 months of acceptance, is crucial; if you aren’t already, then get in the habit of submitting to your institution’s repository as soon as your work is published. If you have any doubts about embargo periods/ potential conflict with your journal contract, get in touch with your institution’s OA team who will be very used to dealing with this and know what to look for). Note that OA does not apply to monographs for REF 2021, but it is expected that it will for the next cycle (post-2021).

Another change that has been announced is the move to better accommodate interdisciplinary research: each sub-panel will have at least one dedicated panel member to oversee and assess interdisciplinary research, there will be an “interdisciplinary identifier” for outputs, and support of interdisciplinary research will be considered in the environment template. (for more on interdisciplinarity, see Initial decisions, para. 15).

There are also moves for improvements in equality and diversity considerations, particularly in terms of improving the representativeness of panel appointments (i.e. those assessing work) and there have been announcements thus far as to how this will be approached (see Initial decisions, para. 44, and p. 19)

So far, I think there are some encouraging moves in what has been announced. However, much of what is still to be decided impacts substantially on ECRs and I’ve outlined the three key areas below.

3. Staff submissions

This concerns who “counts” as a researcher and is submitted to the REF. In REF 2014, eligible staff were defined using core eligibility criteria but institutions did not need to submit all of those meeting these requirements. This ran into a number of issues, which REF 2021 aims to address through the proposal of 100% return: i.e.

“all staff who have a significant responsibility to undertake research”

will be submitted; alternatively, HEIs can decide not to return all staff but must draw up a code of practice as to how the decision is undertaken.

The proposal of “significant responsibility” works to core eligibility requirements which are essentially the same as REF 2014:

  • academic employment function of ‘research only’ or ‘teaching &  research’
  • are independent researchers [i.e. not research assistants unless ‘demonstrably’ independent]
  • minimum employment of 0.2 full time equivalent
  • have a substantive connection to the submitting institution
  • RAs on projects won’t be eligible, and further guidelines being produced around the definition of “independent researcher” (see Circular Letter, Annex A, para. 3-8)

While the principle of all-staff submission has been broadly accepted, uncertainty remains as to how eligible staff will be identified – there were initial suggestions of using HESA contract data which have been ruled out, and it now looks as though it will be up to HEIs to determine who has “significant responsibility” and meets the above criteria. While there is a lot of discussion around this, and some of it confusing in the terminology, I don’t think ECRs need to focus too much on the finer details at this point in the cycle – this is fairly high-level policy, and while it will impact upon you depending on what role you are in towards the cut-off point, at this point I would work to the assumption that if you are going for academic teaching & research (T&R) posts, then you will be expected to be REFable.

Which takes us onto the next uncertainty.

4. Number of outputs 

A key change from the last REF is the announcement of “decoupling”. The outline premise of this is that staff and outputs are less directly associated with one another so that the exercise is about institutions, not individual performance.

This means that instead of having a portfolio of outputs attached to each individual (4, with reductions for ECR status and other staff circumstances), the number of submissions required will be an average across the whole department, with a minimum and maximum number of submissions per individual. The proposed no. of submissions is:

  • Minimum of 1 per staff;
  • Average and maximum tbc; so far, a suggested multiplier of 2 outputs per staff, and a maximum of 6.

(for more on this, see the Developing policy on staff and outputs webinar).

Decisions are ongoing around measures to promote equality and diversity in staff and output selection, including drawing up codes of practice on applying criteria for identifying staff, HEIs’ approach to output selection, and reductions for individual staff circumstances (see Webinar slide 9/ roughly 20 mins into the presentation) – from what I can see, it isn’t clear if an ECR reduction is being discussed (see here for what ECR reduction meant in REF 2014) although the minimum/average system makes this a little less crucial than previously.

In principle, decoupling seems to be a good thing and potentially reduces some of the pressures and competitiveness that ECRs experienced in the last REF. However, as things stand there is still uncertainty as to what the number of submissions will be required – although I think we could fairly safely assume that for an ECR, 1-2 will be the expectation. Perhaps more uncertain is how this will play out on the job market, both in terms of individual hiring decisions, and the impact on hiring cycles more broadly of the kind of impact that we saw last time.

5. Portability

The proposal of non-portability of outputs has been the key concern for ECRs, and is still by far the most uncertain. This proposal put forward that

“outputs should be eligible for submission only by the institution where the outputs were demonstrably generated”

That is, outputs would essentially “belong” to the institution where the work was carried out and not move with you when you moved institution. The aim was to prevent the hiring poaches of “REF stars” that occurred towards the end of last cycle.

However, clearly for ECRs this has huge consequences given the level of job mobility in the current market where fixed-term contracts are on the rise (not to mention that, given the number of teaching-only contracts, a huge amount of research is undertaken in unpaid time). This is reflected in the latest documentation, which states that

“we also recognise the significant concerns raised about this proposal in consultation responses, including the unintended consequences for staff mobility (particularly for early-career researchers) and publication behaviour, and about burden, practical implementation and retrospective application.”

(see Circular letter, paras. 16-17)

The proposal now being put forward in the current round of consultation (see Annex A, paras. 9-11) is that for this cycle, transitional arrangements are put in place, with 2 options being proposed:

  1. The simplified model, whereby outputs would be eligible for return by the institution where research carried out as well as by the newly employing institution.
  2. The hybrid approach, with a deadline (to be determined), after which a limited number of outputs would transfer with staff, with eligibility otherwise linked to the originating institution.

 

refnp

(Source: Developing policy on staff and outputs webinar: 19 July 2017, slide 14)

There has been recognition of the cost/burden of implementing the hybrid model, and the simplified model will be much more straightforward for ECRs, but at this point we don’t know which way this will go – I’d like to assume option A, but who knows. It does look like there has been enough discussion of the negative effects of non-portability (not just on ECRs – staff mobility is necessary at all levels, for many reasons), that hopefully outputs will be portable at least up to a certain date, but this will be the key area for ECRs to follow in coming months.

6. What can ECRs do?

Based on this assessment, my take on this would be to assume:

  • that if you are aiming for an academic job you will be in a REFable position;
  • that a minimum of 1 and average of 2 submissions will be required;
  • that portability will potentially apply to current outputs but you may need to stay informed about further changes and especially the cut-off date if the hybrid model is adopted.

Again, this is only what I can infer from the current documentation and this could all change quite quickly, but I think these assumptions are fairly safe in that they will allow you to work to a few pointers I’ve suggested below, while leaving you open to any subsequent changes.

7. Publication strategy

So to finish, some suggestions of what to do if you’re soon to be or currently on the job market or in a fixed-term position:

Focus on quality not quantity: fewer, high-quality publications is better than lots of lower-ranking pieces. This has always been true, and it will probably be more so in the new system where a lower number of outputs per staff are being submitted: institutions will want to make each one count. I have written more on quality in this post on publication strategy (the REF info is clearly out of date, but the pointers on quality apply). The quality not quantity rule is also time-efficient if you are stretched for publication time as an ECR; make everything you can publish count for as much as possible on your CV.

Timing and portability: at this point, I would probably not hold back on publishing articles that you have ready. If you are on the job market, you need publications on your CV (we spoke a bit yesterday about how you are playing 2 games as an ECR: the hiring game, and the REF game, which are entangled but also slightly different from one another).

However, looking ahead to the future (and I’ll update on this in due course): if a deadline for non-portability comes into play then you may want to start thinking strategically later in the cycle; if you are on the job market late in the cycle and have something ready for submission then it may be worth holding off on hitting the submit button until you are newly employed in a REFable post. I would consider having an unpublished “safety piece” that travels with you on the job market until you are in post (the exact piece might change of course, but the principle is to have one piece held back).

8. Stay informed

The final guidance is expected in winter 2018-19. In the meantime, keep up to date with any changes: HE news outlets, blogs, and the HEFCE website will have all the latest, and I’ll be doing my best to keep up to date with the changes and writing on ECR impacts.

It is hugely time consuming (and stressful) trying to keep up with all of this if you’re an ECR on the job market, so to those in more senior positions, please help with dissemination, either to the ECRs within your direct reach and/or at an institutional level. A lack of and mis-information around the REF was cited as a huge contributor to the pressure of the REF 2014 in my study of its impact on ECRs, so please help with this if you can. Which brings me onto my final point.

9. The REF and ECR mental wellbeing

My final piece of advice is around the mental health impact of the REF on ECRs, particularly during this period of uncertainty in which the very thing that for years you have been told your employability depends upon is now up in the air (and it may well feel as though the pack of cards deciding your future has been thrown up in the air, and we’re waiting to see where they land).

My 2015 report showed an overwhelming detrimental impact of the REF upon ECR mental health and wellbeing: respondents both in secure early-career positions and on the job market noted that the REF exacerbated feelings of anxiety, insecurity and precarity, and some experienced substantial mental health impacts of this. ECRs also said that they felt isolated and unable to admit to these impacts in the highly competitive workplace environment that REF created.

I don’t want to place undue worry on ECRs at this point: there is still much to be determined, and once decisions are made, this REF could well play out better for ECRs. But I do think it’s important to acknowledge that many ECRs are feeling incredibly anxious and even more precarious as a result of the current uncertainty, and that if you are, you’re not fretting over nothing or worrying unduly before the final policy is announced; if anything, this period of uncertainty before the policy is decided is, I think, going to be the hardest time for ECRs on the job market. The job market has been pretty horrendous in terms of mental health impacts over the last few years (and I have another post coming up with more thoughts on this), and I can only imagine that the last few months have been even more stressful. So if you’re feeling like this you’re by no means alone; and again, if you’re in a position to do so, then please offer any support, advice, and reassurance that you can to ECRs.

By way of conclusion, a round up of the works cited in this post:

 

 

Advertisements

3 thoughts on “REF 2021 & ECRs: the current situation”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s