Writing promotion and tenure evaluations for life scientists : thoughts on structure and content

Writing promotion and tenure evaluations for life scientists: thoughts on structure and content. Experts in a discipline are frequently called upon to provide external evaluations of promotion and tenure (P/T) candidates at other universities and research agencies. Nonetheless, there is scant published information on the techniques and strategies used to produce a thorough and insightful P/T evaluation. External P/T evaluations must be matched to the candidate’s specific promotion criteria, and to their faculty appointment, both of which may vary substantially among institutions. A P/T evaluation letter should be based on independent and objective standards. Evaluation of the research dossier is typically easiest because of the many quantitative tools available (e.g., citation frequency, impact factors and h–index). Assessment of teaching performance should be based on comparisons of quantitative evaluation data compared to departmental means and standard deviations, although these metrics are not error–free. Written comments by students are also informative, as is evidence of successful mentoring of graduate students. Documentation of the successful use of innovative science pedagogy (e.g., flipped classrooms and active learning) also provides helpful evidence for evaluation of a teaching dossier. Finally, outreach faculty may be evaluated on the basis of: 1) publications, and 2) quantitative evaluations of workshops or other presentations.


Introduction
The roles and responsibilities of professors in the 21 st century have increased tremendously in the last 50 years.Although several extant guides discuss how to be a productive faculty member, in general, these publications are directed towards new faculty (Boice, 2000;McGuckin and Ladhani, 2013;Wilson, 2013;Whitehead, 2016) rather than senior faculty.Besides the more obvious job responsibilities such as teaching and research, professional service is an obligation of all researchers and professors.Typically, this service falls disproportionately on senior faculty, and, not surprisingly, women and minority faculty (Guarino and Borden, 2017).One aspect of professional service that has received little attention in the published literature, especially in the life sciences, involves the strategies and techniques used to serve as an external evaluator for a candidate up for promotion/ tenure (henceforth P/T) at another university (see Goldman (2017) for a discussion of this topic with respect to law).Such evaluations represent one of the most important areas of professional service performed by faculty, and have consequences far beyond the acceptance or rejection of a manuscript or grant proposal.P/T letters play a critical role in determining job stability and current and future salary for faculty in the life sciences.In this essay I will explore the techniques and considerations that I have used in 39 external faculty/researcher evaluations for P/T.These evaluations include: 1) associate/full professors with teaching and research appointments, 2) associate professors with teaching or outreach appointments, and 3) research chairs and government scientist promotions, at institutions ranging from teaching universities to R1 research universities.Geographically, these institutions encompass locations in both the United States and Europe.My essay is based on generalities derived from these experiences, but I cannot overemphasize that any P/T evaluation must be based on the standards specific to the institution making the review request, and these may vary substantially among institutions.For example, institutional evaluation criteria may range from whether or not the candidate would be promoted and tenured at the evaluator's university, to simple reviews of the candidate's strengths and weaknesses.Because this essay is based solely on my own experience as a professor in the United States, it contains an element of subjectivity; hence, its greatest value will be as a platform for discussion among colleagues rather than as a canonical guide to the P/T evaluation process.

So you have received an evaluation request
A variety of factors should be considered before you agree to write a P/T evaluation.First, some faculty are concerned that writing a negative P/T letter will incur future repercussions.I know from personal experience that writing negative letters is not an enjoyable aspect of professional service, but it is a necessary one.Assuming you are familiar with the productivity norms of your field, one way to obtain an idea of the qualifications of a candidate is to request a summary of their accomplishments or a copy of their vita prior to agreeing to the evaluation.With this information in hand, you will be able to make a more informed decision regarding whether or not you are willing to undertake the review.Of course if everyone only writes positive P/T evaluations, then ultimately the quality of our field and the professoriate in general, will decline.Second, you should establish whether or not your evaluation letter will be kept confidential.Some public universities are bound by sunshine laws which permit the candidate to view their outside evaluation letters in an unredacted form, and this should be considered in your decision-making process, especially if your field is a small one.Even when confidentiality is supposed to exist, you cannot assume that everyone privy to the decision will keep all information confidential, including the identity of external evaluators.This likely is more of a consideration for associate professors writing evaluation letters than for full professors.Finally, given the time necessary to read relevant materials and write a pithy but thorough letter, it is important to assess whether your schedule is sufficiently clear to provide a timely evaluation.
Once you have agreed to serve as an outside evaluator, you should receive the candidate's P/T dossier from an administrator, but check to ensure it includes the promotion/tenure standards for the academic unit as well as the proportion of the candidate's position composed of research, teaching, and outreach/service (commonly abbreviated FTE [full time equivalent] or EFT).I have occasionally written evaluations for candidates working in units that had no specific requirements or guidance for P/T other than possession of a national or international reputation.How this is determined typically is left up to the evaluator and requires substantial thought; especially given that opinions on the matter may differ.

The letter and what should be considered
Unless your subdiscipline is small, and the host department composed of faculty in the same specialty, the professors reading your evaluation will likely be unfamiliar with you or your work.Consequently, I begin my letters with a short introduction of my academic record and background.Having a professional web site that includes a vita and publication list greatly aids this process, because it allows you to dispense with much descriptive text via a link to the site.I then describe my relationship to the candidate's field of research: 1) are we in the same specialty (animal community and population dynamics) or am I writing from a more general (ecologist) perspective?2) are there portions of the candidate's research/teaching that are outside of my expertise and ability to judge? 3) finally, I describe my relationship with the candidate.Do I have a relationship with the candidate, either professional or personal, and how do I distinguish between these?Of course, if you truly have a personal relationship with the candidate then you should not be evaluating them for P/T unless your field is so small that everyone socializes with everyone else.Regardless of my relationship with the candidate, I describe whether or not we have met and under what circumstances (e.g.professional meeting, workshop, grant review panel, seminar reception).For example, here is a comment from an evaluation letter where I previously had a professional interaction with the candidate, 'Although I have met XXX and XXX has recently helped me with references to a chapter I have written, we do not have a personal relationship, and I believe I can evaluate their work objectively.'I consider myself to have a personal relationship with a candidate if we regularly socialize outside of our professional milieu, but do not consider a relationship personal if our socializing only occurs within a professional context (e.g., a large dinner at a meeting) and is occasional.Of course, opinions will vary on what does and does not constitute a personal relationship; consequently, I think it is important to disclose any relationship between a candidate and evaluator.I also include a statement regarding my ability to complete the evaluation in an unbiased manner.
I begin my second paragraph with a statement that 'we all have too much to do and too little time to do it in' and then follow with my conclusion regarding the candidate's suitability for P/T, based on the criteria of the home unit.Here is an example, although for confidentiality I have modified the specific details, pronouns and promotion criteria: 'I fully support Dr. XXX promotion to Associate Professor with tenure at Anonymous State University.They would have no trouble being promoted to that rank in my home unit or, based on previous promotions, in Units Z, Y or Z here at University of Georgia.Their research work on the dynamics of planetary bodies and satellites is top notch and I believe will stand the test of time.In addition, their teaching and grant records are strong and indicate that their work is highly innovative.They clearly have exceeded your stated criteria for promotion to Associate Professor at Anonymous University.'I try to keep this paragraph short but thorough, and comment on all relevant criteria.I make this my second paragraph to ensure that members of the evaluation committee/faculty will be able to reach my conclusion quickly, rather than wading through what may be a dense multi-page letter.Frankly, I have witnessed more than one discussion in a P/T meeting where it was clear that faculty had not thoroughly read the outside evaluations, although structuring the letter in this manner also runs the risk of faculty not reading the entire evaluation.
I begin the next few paragraphs by restating the P/T criteria of the candidate's home institution and describe my evaluation of their performance based on their specific appointment (e.g., 50 % research or 20 % research; 80 % teaching or 30 % teaching).I typically begin with something like 'Your guidelines for promotion and tenure state an individual must have achieved at least satisfactory performance ratings in all applicable areas (based upon appointment) of teaching, research/creative endeavors, and service, and an excellent rating in at least one.'If the candi-date has taken an unusual amount of time to apply for promotion, or there are gaps in their productivity, it may be worthwhile to enquire whether there are special circumstances that may warrant consideration such as illness or family issues, although some would argue this is inappropriate and all evaluations should be based on productivity alone.Let your professional conscience be the judge.
The different portions of the P/T evaluation vary in their difficulty.In life sciences, the easiest portion to write is the research evaluation, because multiple quantitative tools are available for evaluation of a candidate's research and professional reputation.Examples of possible metrics are: 1) number of citations for a candidate's papers, which also yields the number of papers that have been highly cited (although I know of no objective definition for a highly cited paper in ecology, organismal biology or resource management, based on my experience I would consider a highly cited paper to have 50 or more citations although clearly date of publication must be considered too), 2) the number of papers in regional, national or international journals, 3) journal impact factors, although some disagree about their merit, and 4) research impact indices such as the h-index.Other possible criteria are: 1) invitations to speak at universities outside of the candidate's region, 2) regional or national awards, 3) service on agency, state or regional (e.g., Department of Natural Resources, Department of Transportation) or national (NSF, NIH or USDA) research or grant review panels, and 4) service on editorial boards for regional, national or international journals.If the criterion is an international reputation then one may use the same categories elevated to an international level, although it is likely that only a few faculty will have received international awards.Sources such as Web of Science and Google Scholar are useful tools for collecting these data and I prefer Google Scholar because it casts the widest net for citations.In addition, it is worthwhile to examine citations under both the candidate's full name and first initials, given that European journals frequently list authorship under the latter rather than the former.I have found that even Google Scholar repeatedly misses some of my highly cited papers if I do not also search under my first two initials.I also am quite wary of using online h-index calculators which are easy to use but rarely accurate, and if an h-index is needed I calculate it by hand using papers obtained via Google Scholar.This is much more time consuming but remember your evaluation will partially determine whether the candidate will keep or lose their job.Regardless of the availability of metrics to describe research productivity and quality, good P/T letters are always most effective when the reviewer can demonstrate a detailed and personal knowledge of the candidate's work.Consequently, I always try to read five to ten of the candidate's papers focusing on more recent work, in order to discuss the specifics of their research.A thorough evaluation will describe specifics of the candidate's research designs and results, as well as the implications of their findings for the field as a whole.
Although it is easy to play the 'numbers game' (i.e., numbers of papers, numbers of citations and h-indices), an objective evaluation of a candidate's research requires thoughtful consideration and insight.Clearly, neither citations nor counts of publications alone provide a sufficient basis for an objective evaluation, especially in these days of 'pay to publish'.Although predatory journals with unethical publication policies and editorial board appointments exist, it should not be too difficult to detect publications in such journals via the many possible metrics discussed above (e.g., impact factors, number of citations).However, even restricting your evaluation to legitimate journals, one must acknowledge that publication rates vary greatly among fields and, of necessity, this must be considered in your evaluation.It always is wise, although not necessarily easy, to determine normative publication rates for fields outside your specialty.
What may be even more problematical is ascertaining the candidate's actual contribution to their multi-authored publications, especially if not described in the narrative.There are many criteria used to determine authorship and author order in the life sciences, which we have recently reviewed (Grossman and DeVries, unpublished).Nonetheless, in my own work, I abide by the convention of earning either first authorship or last authorship, because these connote who did most of the work (first) and who provided the intellectual impetus for the work and provided funding (last).However, consider the case where a candidate has met the minimum requirement for research productivity but has no senior authorships.If their papers mainly represent the work of a graduate student senior author and a faculty member junior author, I would not be too concerned, given this is the typical pattern of authorship in my field for graduate student publications.But what if the candidate works in a field where large collaborative projects are the norm, has no senior authorships, and no papers with fewer than 10 coauthors; a case I have always found problematic?Here is an example of how I handle this situation 'The sole shortcoming that I find with XXX's research record is that most of XXX's papers have many authors and XXX typically is not the first.Consequently, it is difficult to judge XXX's specific contribution to the work.Nonetheless, I have given XXX the benefit of the doubt and assumed that XXX's contributions were significant.'Clearly each case may warrant a different approach and there certainly are no easy answers regarding evaluation of a candidate's publication record when many or most of the publications include many coauthors and the candidate generally is not the senior author.I will confess that for promotions to full professor or research chairs, I feel that demonstration of continued substantive scholarly performance, as evidenced by senior authorships on research papers, is required.
Although a P/T letter should not be overly personal, the use of a personal 'touch' always makes it easier to read and also imparts authenticity to the author's words.Here are some statements from my own previous letters 'XXX's list of funded proposals reads like a veritable Who's Who of funding sources and I wish I had as much success, especially recently!'Or 'Nonetheless, if you deem this letter inadequate, please contact me immediately.The last thing I would want to do is compromise the candidacy of such an excellent faculty member.'In conclusion, however, your letter still must retain the required objectivity of an outside evaluator.
Typically, the research evaluation, at least in the United States, will require assessment of the candidate's history of obtaining competitive grants and contracts.Certainly in this day and age, proficiency at obtaining grants is a requirement for promotion in the United States, at least at most research oriented universities.The number and fiscal magnitude of grants will vary by department and institution; hence, there is no point in suggesting specific numbers and amounts.It is reasonable to assume that a candidate's grant support should be sufficient to support graduate students or full-time technical help.Nonetheless, being proficient at obtaining grants does not necessarily demonstrate adequate scholarship for promotion.It is not uncommon to review candidates that have exceptional records at obtaining grants and contracts, but weak publication records.Conversely, you may encounter cases where the publication record is excellent but the grant record is mediocre.Again, the standards of the home institution must be followed when evaluating these cases.Nonetheless, my personal opinion is that someone who publishes in top notch journals without substantial grant support is more deserving of P/T than someone with a strong grant record but a poor to mediocre publication record.
As I review previous P/T evaluation letters, I find that I have not generally been asked to evaluate a candidate's proficiency in teaching, although summaries of teaching evaluations as well as teaching awards frequently have been included in promotion dossiers.Perhaps this is because I have only worked at a research (R1) institution.Quantitative teaching evaluations have their own set of biases including gender bias (Boring et al., 2016), although it is easy to compare them to departmental means and standard deviations.Student comments on evaluations also provide information, as do unsolicited letters of commendation from students.All of these metrics need to be considered carefully in terms of their usefulness for a tenure evaluation.
There are a variety of additional criteria that are useful when evaluating a candidate's teaching performance.For example, teaching narratives may contain information regarding the successful use of both standard and innovative tools for science pedagogy such as active or authentic learning, flipped classrooms, and new applications of educational technology such as music or karaoke videos (Grossman and Watson, 2015;Grossman and Simon, 2018).If the dossier contains little information on the candidate's teaching performance it will be useful to discuss any instances where you observed the candidate's presentation skills such as during presentations at scientific meetings, or lectures, or from presentations at grant review panels, or simply acknowledge that the information in the dossier is insufficient for an evaluation.
A final aspect of instructional responsibilities warrants mention, and that is the supervision and mentoring of graduate students.It is reasonable to expect that a candidate for promotion to associate professor at a research university should have an active graduate program, and should have graduated multiple students prior to being awarded promotion and tenure.For promotions to full professor at an R1 university, I am reluctant to write a positive letter for a candidate that has not graduated both multiple PhDs and Master's students.Again numbers vary too much from field to field to identify specific minima, but evidence for an active graduate program should be readily identifiable.
On occasion you may be asked to evaluate faculty whose main appointment is service/outreach.For these evaluations, the main criteria I use are: 1) evidence of outreach publications, and 2) whether workshops, panels or presentations are evaluated via some quantitative instrument such as a Likert scale questionnaire.Again, the evaluation must be based on the institutional requirements for promotion.Publication may or may not be a requirement for these faculty but those who are publishing in refereed journals have clearly shown excellent performance.Outreach faculty who only publish technical bulletins may still meet P/T guidelines, and many extension service publications fall in some middle ground where they are sent out for external review but generally do not have a real chance of being rejected.Assessment of workshop productivity is often difficult because of a lack of quantitative assessment.One would hope that if outreach faculty are required to go through a P/T process that quantitative assessment data would be provided to an evaluator.
External evaluation of P/T candidates is an important component of professional service for faculty, and helps to maintain a high quality professoriate.In this essay I have shared my own strategies, judgements and experiences in order to provide guidance to faculty unfamiliar with the P/T process and stimulate collegial discussion of these issues.I hope these suggestions are of use to professors young and old.