Tag Archives: REF

REF 2021: update for ECRs

This post provides an update to my previous blog post on “REF 2021 and ECRs: the current situation” in which I outlined the key decisions and remaining areas of uncertainty (mostly the latter) surrounding the next Research Excellence Framework.

Today, quite a few of those uncertainties have been clarified by the publication of “Decisions on Staff and Outputs“. I will soon update my earlier post to reflect this latest document (and offer a few more interpretations of the ECR implications), but in the meantime here is a brief summary of the key points of relevance for ECRs that have been announced today:

Staff eligibility: this latest document confirms that the Stern recommendation that “all staff with significant responsibility for research” are returned will be implemented. 6.a of the document outlines the core criteria (which broadly follows that of REF 2014), but there is more detailed guidance this time on what constitutes “significant responsibility” in paras. 11-13: “those for whom explicit time and resources are made available to engage actively in independent research, and that is an expectation of their job role.”

100% of staff defined in this way are expected to be returned. Meanwhile, teaching-only contracts with no research element won’t count as returnable under these guidelines; neither will RAs employed on project work (see Independent Research, paras. 14-15) nor those without a “substantive connection” to the submitting HEI. This seems to accord with what was suggested previously (see section 3 here); although I’d add that, while the intention here is to offer a more rigorous approach as to who is submitted, there does still seem to be a large amount of flexibility as to how “significant responsibility” will be determined.

Decoupling: the proposed decoupling of staff from submissions (see my previous explanation of decoupling in section 4 here) is going ahead, as described in paras. 25-29, with the numbers of outputs now confirmed: a minimum of one per researcher; a maximum of 5; an average of 2.5 per FTE, across the submitting unit. That means that as an ECR you will need at least one output, and more than that would be beneficial as the submitting unit will be looking for 2-3 per person.

The big change here from REF 2014 (although it is one which we’ve been expecting) is that there is no “ECR discount” that would be deductable per person; instead, the average and min./max. figures account for this across the whole submitting unit (as previously the expectation was 4 per person, so the average is lower). There are guidelines to account for exceptional individual circumstances (paras. 30-32), thus addressing earlier concerns that circumstances such as substantial periods of parental leave/ illness etc would not be acknowledged; deductions for individual circumstances will be applied to the total number of outputs required of the submitting unit as a whole. ECRs without exceptional circumstances, however, need to work to the minimum/average figures.

Portability: (background context in section 5 here). After much talk of the non-portability of research we have clarification as follows (paras. 33-36):

We will implement a transitional approach to the non-portability of outputs in REF 2021, whereby outputs may be submitted by both the institution employing the staff member on the census date and the originating institution where the staff member was previously employed as Category A eligible when the output was demonstrably generated.

*Added note: see para. 34 for the definition of “demonstrably generated”:  “for REF 2021 ‘demonstrably generated’ will be determined by the date when the output was first made publicly available.”

This means that if you publish while at institution A, and you move to institution B, your output can count at both institutions. This has been a big area of concern for many ECRs and I’m relieved to (finally!) see a clear decision on the issue, and one which recognises and prevents the potentially disastrous consequences for ECRs that non-portability may have had.

Open Access: the latest guidance (paras. 37-40) seems to align with what has been suggested for a long time now about Open Access requirements for REF 2021: “The policy will require outputs to be deposited as soon after the point of acceptance as possible, and no later than three months after this date (as given in the acceptance letter or email from the publication to the author) from 1 April 2018.” There are some further exceptions outlined in the next paragraph (39). As I’ve mentioned before, if you’re at all confused about OA requirements then I would suggest that you familiarise yourself with your institution’s OA support and get in touch with the relevant team if you have any queries and concerns about the process.

A couple of final points: the census date is now confirmed as 31st July 2020; and more detailed guidelines on the above are expected mid-2018.

***

That summarises most of what I’ve read so far; I’ll potentially add to / clarify these points in coming days and if helpful, offer further guidance on ECR implications and what to do next; but for the moment, it looks like the outline framework is fairly clearly in place and ECRs can now start planning accordingly.

Advertisements

REF 2021 & ECRs: the current situation

This is based on a talk that I gave at an event on “Publishing and Research Strategies for ECR Classicists” at the Institute of Classical Studies on 20th September 2017. If you want to cut to the chase, the slides are here, full notes of my talk below.

It is a slightly tricky time to be writing this post: there have been a host of announcements about changes to the REF policy in recent months, weeks, and even days, but much remains undecided, making it difficult to offer advice of the kind that was possible for REF 2014. That said, ECRs (and especially those currently on the job market) are very much living through this uncertainty, and can’t simply wait it out until next summer to make decisions that impact upon careers.

With that in mind, this post is intended as a reflection on the current state of affairs. While I can’t offer anything too concrete, I have tried to clarify, or at least clearly set out, the main areas of uncertainty relating to ECRs, and to give pointers on where to find information and what to look out for as further details are released. I’ve also offered some preliminary advice for ECRs based on what I think can be inferred thus far.

All of this comes with the (big) disclaimer that these are my own opinions only, some areas are still open to interpretation, this is by no means definitive, and this may well yet change: and n.b. that none of the information that has been released is final policy – that will come next year.

  1. REF2021: an overview

The Research Excellence Framework is the system for allocating research funding in UK HEI; it is based on assessing 3 factors with :

  • outputs (60%);
  • impact (25% – an increase from 20% in 2014);
  • and environment (15%)

The focus of what follows is on research, although impact may be of relevance to ECRs (see paragraphs 19-26 of the Initial Decisions document for the latest on impact).

The previous (and first) round of the exercise was run in 2014 and raised some areas of difficulty, including a substantial impact on ECRs as my earlier work documents. A period of review and consultation has therefore been in progress, starting with the Stern Review in 2016, a Consultation exercise on the findings of the Stern Review held in March 2017 (for which the summary of responses was published last week), and some Consultation events. The Initial decisions on the REF were published at the start of September 2017, along with a Circular Letter. There have also been a series of webinars over the summer, most recently this one in July on Developing staff and outputs, not to mention several blog posts by David Sweeney on various topics. (yes, I’m exhausted just writing all of that).

The timetable indicates that draft guidance will be published next summer, with final guidance and criteria in winter 2018-19 (see Initial decisions, paragraphs 49-51). Based on the approach thus far, I think we can expect that until then there will be this ongoing slow-release of pieces of information as they become available which is helpful, but also means that there will be a lot of discussion and changes to keep track of.

2. What we know so far

The broad parameters of research assessment will stay the same: “outputs” (broadly, publications) will be submitted to the REF by HEIs and sent to subject panels who assess by peer review; quantitative data will be used in some disciplines where appropriate (see Initial decisions, para. 6). Peer reviewers will rank submissions on a 1-4* basis:

ref
Quality profile from REF2014

The timeframe for publications that can be submitted will be 1st January 2014 – 31st December 2020.

A key change from the last round will be the new Open Access policy, which states that from April 2016 journal articles submitted to the REF must be available in an Open Access form, i.e. an institutional or subject repository (n.b. “early deposit”, i.e. within 3 months of acceptance, is crucial; if you aren’t already, then get in the habit of submitting to your institution’s repository as soon as your work is published. If you have any doubts about embargo periods/ potential conflict with your journal contract, get in touch with your institution’s OA team who will be very used to dealing with this and know what to look for). Note that OA does not apply to monographs for REF 2021, but it is expected that it will for the next cycle (post-2021).

Another change that has been announced is the move to better accommodate interdisciplinary research: each sub-panel will have at least one dedicated panel member to oversee and assess interdisciplinary research, there will be an “interdisciplinary identifier” for outputs, and support of interdisciplinary research will be considered in the environment template. (for more on interdisciplinarity, see Initial decisions, para. 15).

There are also moves for improvements in equality and diversity considerations, particularly in terms of improving the representativeness of panel appointments (i.e. those assessing work) and there have been announcements thus far as to how this will be approached (see Initial decisions, para. 44, and p. 19)

So far, I think there are some encouraging moves in what has been announced. However, much of what is still to be decided impacts substantially on ECRs and I’ve outlined the three key areas below.

3. Staff submissions

This concerns who “counts” as a researcher and is submitted to the REF. In REF 2014, eligible staff were defined using core eligibility criteria but institutions did not need to submit all of those meeting these requirements. This ran into a number of issues, which REF 2021 aims to address through the proposal of 100% return: i.e.

“all staff who have a significant responsibility to undertake research”

will be submitted; alternatively, HEIs can decide not to return all staff but must draw up a code of practice as to how the decision is undertaken.

The proposal of “significant responsibility” works to core eligibility requirements which are essentially the same as REF 2014:

  • academic employment function of ‘research only’ or ‘teaching &  research’
  • are independent researchers [i.e. not research assistants unless ‘demonstrably’ independent]
  • minimum employment of 0.2 full time equivalent
  • have a substantive connection to the submitting institution
  • RAs on projects won’t be eligible, and further guidelines being produced around the definition of “independent researcher” (see Circular Letter, Annex A, para. 3-8)

While the principle of all-staff submission has been broadly accepted, uncertainty remains as to how eligible staff will be identified – there were initial suggestions of using HESA contract data which have been ruled out, and it now looks as though it will be up to HEIs to determine who has “significant responsibility” and meets the above criteria. While there is a lot of discussion around this, and some of it confusing in the terminology, I don’t think ECRs need to focus too much on the finer details at this point in the cycle – this is fairly high-level policy, and while it will impact upon you depending on what role you are in towards the cut-off point, at this point I would work to the assumption that if you are going for academic teaching & research (T&R) posts, then you will be expected to be REFable.

Which takes us onto the next uncertainty.

4. Number of outputs 

A key change from the last REF is the announcement of “decoupling”. The outline premise of this is that staff and outputs are less directly associated with one another so that the exercise is about institutions, not individual performance.

This means that instead of having a portfolio of outputs attached to each individual (4, with reductions for ECR status and other staff circumstances), the number of submissions required will be an average across the whole department, with a minimum and maximum number of submissions per individual. The proposed no. of submissions is:

  • Minimum of 1 per staff;
  • Average and maximum tbc; so far, a suggested multiplier of 2 outputs per staff, and a maximum of 6.

(for more on this, see the Developing policy on staff and outputs webinar).

Decisions are ongoing around measures to promote equality and diversity in staff and output selection, including drawing up codes of practice on applying criteria for identifying staff, HEIs’ approach to output selection, and reductions for individual staff circumstances (see Webinar slide 9/ roughly 20 mins into the presentation) – from what I can see, it isn’t clear if an ECR reduction is being discussed (see here for what ECR reduction meant in REF 2014) although the minimum/average system makes this a little less crucial than previously.

In principle, decoupling seems to be a good thing and potentially reduces some of the pressures and competitiveness that ECRs experienced in the last REF. However, as things stand there is still uncertainty as to what the number of submissions will be required – although I think we could fairly safely assume that for an ECR, 1-2 will be the expectation. Perhaps more uncertain is how this will play out on the job market, both in terms of individual hiring decisions, and the impact on hiring cycles more broadly of the kind of impact that we saw last time.

5. Portability

The proposal of non-portability of outputs has been the key concern for ECRs, and is still by far the most uncertain. This proposal put forward that

“outputs should be eligible for submission only by the institution where the outputs were demonstrably generated”

That is, outputs would essentially “belong” to the institution where the work was carried out and not move with you when you moved institution. The aim was to prevent the hiring poaches of “REF stars” that occurred towards the end of last cycle.

However, clearly for ECRs this has huge consequences given the level of job mobility in the current market where fixed-term contracts are on the rise (not to mention that, given the number of teaching-only contracts, a huge amount of research is undertaken in unpaid time). This is reflected in the latest documentation, which states that

“we also recognise the significant concerns raised about this proposal in consultation responses, including the unintended consequences for staff mobility (particularly for early-career researchers) and publication behaviour, and about burden, practical implementation and retrospective application.”

(see Circular letter, paras. 16-17)

The proposal now being put forward in the current round of consultation (see Annex A, paras. 9-11) is that for this cycle, transitional arrangements are put in place, with 2 options being proposed:

  1. The simplified model, whereby outputs would be eligible for return by the institution where research carried out as well as by the newly employing institution.
  2. The hybrid approach, with a deadline (to be determined), after which a limited number of outputs would transfer with staff, with eligibility otherwise linked to the originating institution.

 

refnp

(Source: Developing policy on staff and outputs webinar: 19 July 2017, slide 14)

There has been recognition of the cost/burden of implementing the hybrid model, and the simplified model will be much more straightforward for ECRs, but at this point we don’t know which way this will go – I’d like to assume option A, but who knows. It does look like there has been enough discussion of the negative effects of non-portability (not just on ECRs – staff mobility is necessary at all levels, for many reasons), that hopefully outputs will be portable at least up to a certain date, but this will be the key area for ECRs to follow in coming months.

6. What can ECRs do?

Based on this assessment, my take on this would be to assume:

  • that if you are aiming for an academic job you will be in a REFable position;
  • that a minimum of 1 and average of 2 submissions will be required;
  • that portability will potentially apply to current outputs but you may need to stay informed about further changes and especially the cut-off date if the hybrid model is adopted.

Again, this is only what I can infer from the current documentation and this could all change quite quickly, but I think these assumptions are fairly safe in that they will allow you to work to a few pointers I’ve suggested below, while leaving you open to any subsequent changes.

7. Publication strategy

So to finish, some suggestions of what to do if you’re soon to be or currently on the job market or in a fixed-term position:

Focus on quality not quantity: fewer, high-quality publications is better than lots of lower-ranking pieces. This has always been true, and it will probably be more so in the new system where a lower number of outputs per staff are being submitted: institutions will want to make each one count. I have written more on quality in this post on publication strategy (the REF info is clearly out of date, but the pointers on quality apply). The quality not quantity rule is also time-efficient if you are stretched for publication time as an ECR; make everything you can publish count for as much as possible on your CV.

Timing and portability: at this point, I would probably not hold back on publishing articles that you have ready. If you are on the job market, you need publications on your CV (we spoke a bit yesterday about how you are playing 2 games as an ECR: the hiring game, and the REF game, which are entangled but also slightly different from one another).

However, looking ahead to the future (and I’ll update on this in due course): if a deadline for non-portability comes into play then you may want to start thinking strategically later in the cycle; if you are on the job market late in the cycle and have something ready for submission then it may be worth holding off on hitting the submit button until you are newly employed in a REFable post. I would consider having an unpublished “safety piece” that travels with you on the job market until you are in post (the exact piece might change of course, but the principle is to have one piece held back).

8. Stay informed

The final guidance is expected in winter 2018-19. In the meantime, keep up to date with any changes: HE news outlets, blogs, and the HEFCE website will have all the latest, and I’ll be doing my best to keep up to date with the changes and writing on ECR impacts.

It is hugely time consuming (and stressful) trying to keep up with all of this if you’re an ECR on the job market, so to those in more senior positions, please help with dissemination, either to the ECRs within your direct reach and/or at an institutional level. A lack of and mis-information around the REF was cited as a huge contributor to the pressure of the REF 2014 in my study of its impact on ECRs, so please help with this if you can. Which brings me onto my final point.

9. The REF and ECR mental wellbeing

My final piece of advice is around the mental health impact of the REF on ECRs, particularly during this period of uncertainty in which the very thing that for years you have been told your employability depends upon is now up in the air (and it may well feel as though the pack of cards deciding your future has been thrown up in the air, and we’re waiting to see where they land).

My 2015 report showed an overwhelming detrimental impact of the REF upon ECR mental health and wellbeing: respondents both in secure early-career positions and on the job market noted that the REF exacerbated feelings of anxiety, insecurity and precarity, and some experienced substantial mental health impacts of this. ECRs also said that they felt isolated and unable to admit to these impacts in the highly competitive workplace environment that REF created.

I don’t want to place undue worry on ECRs at this point: there is still much to be determined, and once decisions are made, this REF could well play out better for ECRs. But I do think it’s important to acknowledge that many ECRs are feeling incredibly anxious and even more precarious as a result of the current uncertainty, and that if you are, you’re not fretting over nothing or worrying unduly before the final policy is announced; if anything, this period of uncertainty before the policy is decided is, I think, going to be the hardest time for ECRs on the job market. The job market has been pretty horrendous in terms of mental health impacts over the last few years (and I have another post coming up with more thoughts on this), and I can only imagine that the last few months have been even more stressful. So if you’re feeling like this you’re by no means alone; and again, if you’re in a position to do so, then please offer any support, advice, and reassurance that you can to ECRs.

By way of conclusion, a round up of the works cited in this post:

 

 

A Culture of Publish or Perish? The Impact of the REF on ECRs

This week I attended the Westminster Higher Education Forum on “Next Steps for the Research Excellence Framework“, having been invited to speak on “the Impact of the REF on Early Career Researchers”. I felt very fortunate to have the opportunity to put forward the perspectives of ECRs in this setting, and especially encouraged by the highly positive response to my paper – if my talk was, overall, a sobering reflection on the status of ECRs in the current HE landscape, then it was heartening to hear that many people were interested in taking these issues seriously.

I include below a brief outline of my 8-minute talk, with a few added notes resulting from the event. There is now much more work to be done on analysing the nuances of the survey data presented below, and I aim in due course to write a more detailed piece on this.

A Culture of Publish or Perish? The Impact of the REF on ECRs

In this talk I aim to highlight some of the ways in which the REF has impacted upon early career researchers, using this as a spring-broad to think about how the next REF might better accommodate this career group.

In my role at the Institute of Advanced Study at the University of Warwick I work closely with a community of early career researchers and have experienced first-hand the many impacts that this REF has had on my peer group; but I wanted to ensure that this talk reflected a broader range of experiences across UK HE, and therefore in preparation I distributed an online survey asking ECRs about their experiences and opinions on the REF 2014.

Survey overview

– 193 responses collected between December 2014 and March 2015

– responses gathered via social media and email from across the UK

– 81.3 % had completed PhDs within the last 8 years

– 41.5 % were REF returned

– 18.7% were currently PhD students

– 10.9% had left academia since completing a PhD

5 main points emerged as most significant from among the responses:

  1. Provision of information about the REF to ECRs

I asked ECRs how well informed they felt about the REF, e.g. through training, university advice, or other sources.

Over 50% felt that they were well/moderately well informed, but 40% of respondents felt that they had little or no information. Those who felt ill-informed about the REF included both those ECRs who were returned  – and didn’t know why or to what effect – and others who were not submitted, because they were didn’t have enough outputs or were in a sessional or non-REFable position. Although information about the REF wasn’t deemed relevant to this second group, lack of understanding about the REF is nonetheless problematic for the career mobility of these groups.

Of the 25% who were very well informed, about a third of those noted excellent information provided at institutional level through training events, departmental discussions, and mentoring or other support. However, many were informed only through independent research: reading educational supplements such as THE, Union publications, and using social media to gather information.

There was also much noted about the lack of information specifically targeted at ECRs: some noted confusion, even at institutional level, as to how ECR was defined, or how the career-stage discount applied. This information was in the REF guidelines, but not especially accessible to ECRs: better institutional training around the particularities of ECR status is needed and, judging from many of the survey responses, this would significantly ease some of the uncertainty that this group face. [ note: see my REF for ECRs guide here]

As I mentioned in the Q&A, a clear issue that emerged to me throughout the surveys was the need for better communication, at every level: in the REF guidelines, at the level of university management, at departmental level, and between mentors/supervisors and ECRs themselves.

  1. Publication strategy

Publication strategy was the primary area ECRs felt the REF impact, and the main point to emerge from respondents was that, like other academics, ECRs are well aware of the need for strategic targeting of publication: focusing on those areas of research that have most strategic importance, aiming for 4* publication outlets, and holding back or rushing out publications to fit the REF cycle. ECRs have learnt to play the game.

However, some respondents noted strong concern that the value and quality of their research was becoming restricted within a narrow criteria that doesn’t allow for innovative pursuits, or that they were under pressure from supervisors or mentors to neglect areas of research that might otherwise be beneficial but that wouldn’t produce strong REF outputs. The issue of research restrictions seemed particularly troubling in the case of interdisciplinarity, towards which ECRs have otherwise been driven in recent years [although the subsequent  talk by Tim Hall aimed to dispell the ‘myth’ that the REF doesn’t accommodate interdisciplinarity].

  1. Job applications

The intense focus on publication had a huge impact on ECR job applications: among respondents there was an overwhelming sense of an increasingly competitive job market in which hiring committees focused on solely on REFable publications, creating the pressure to therefore be REF-able straight out of PhD. As a consequence, the REF dangerously intersects with the increasing trend towards casualization: un-REFable ECRs become stuck in a limbo of casualised contracts; but these precarious, short-term and typically teaching-heavy workloads further preclude having the time and resources to work towards 4* outputs that would get them better jobs

The casualization of ECRs is further exacerbated by the hiring cycles that the REF creates: long dry spells followed by a hiring spree. This can work in favour of those who finish PhDs at the right time in the REF cycle; but as several respondents made clear, even those hired for excellence are still at risk of the precarity of casual contracts, hired only for duration of the REF cycle and then let go when it’s over.

The connections between the REF and the casualization of HE seem, at least to ECRs, intricately bound up and this is perhaps the most troubling change it has driven in recent years.

  1. Cultural pressures

I also asked in the survey an open-ended question about any other impacts that the ECRs wanted to express, and this is where many noted the wider cultural shift in academia. ECRs overwhelmingly felt that the REF created a huge amount of pressure and anxiety which impacted particularly on those at the bottom rung of the career ladder. At departmental/ collegiate level many noted a culture of aggression and bullying, as well as the creation of a two-tier hierarchy between teaching and research which is used to inhibit career mobility of those in teaching positions. Comments about the effects on individuals’ mental health were prevalent: words such as ‘insecurity’, ‘pressure’, and ‘anxiety’ occurred numerous times throughout the survey responses. There was also a clear sense of a high level of disillusionment and dissatisfaction at the profession, and cynicism focused around the REF. In some respects, “the REF” has become a byword for a wider culture shift in academia – a shift driven by processes that extend beyond the assessment exercise itself – but it is nonetheless a focal point around which ECRs see very real, material impacts. If that is so, then perhaps with some work, the REF also has the potential to drive more positive changes in coming years.

One final point emerged about the potential positive of the REF:

  1. Public engagement and impact

68% of respondents felt that the REF had changed their attitude towards impact, and were thinking more about public engagement from an early stage of their research. While there have been problems raised with the measurement of impact, it is perhaps encouraging to see a trend shift in this direction coming from those starting out on their careers, and it is especially encouraging looking ahead to the REF2020: if the weighting of impact does (as expected) become more significant then ECRs will be well-placed to address this remit as they progress into the next stages their careers.

***

The original survey that I undertook in preparation for this talk is now closed, but my work on this topic is on-going and I would welcome further comments; please get in touch via email if you would be interested in participating in follow-up work.

Upcoming talk: Westminster Higher Education Forum on “Next Steps for the REF”

I have been invited to speak on 23rd April at the Westminster Higher Education Forum event on “Next Steps for the REF” about the impact of the REF on early career researchers. I’m really pleased to have the opportunity to talk about this subject as it has been such a dominant issue for the many ECRs I know and have worked with over the last few years. In preparation for the talk I have been running a survey on early career researchers’ experiences of the REF – with just over a month until the talk I will be closing this soon to collate responses, but the survey will remain open for a few more days so please do take a moment to complete it if you can. I hope to make available a summary of my findings in a later post.