Disclaimer: I am no expert in admissions. This series of blog posts is a hastily organized braindump resulting from abundant introspection, and discussions I’ve had with prospective grad students. Much of this is my view of the process and holds (if at all) only for North American PhD programs in ML/CV/Robotics.

This article is for you if you want to know what the constituents of an informative letter of recommendation (LOR) are.

Letters of recommendation are an extremely important component of your application package. In many cases admission probabilities are boosted by high-impact letters and dwindled by poorly drafted letters. Nearly all of you would’ve approached referees for letters, but many of you might not have read a letter.

I will enlist what aspects of a letter make it more impactful, hoping it will enable you to think deeply about your reference letter requests. I figure this post will also help first-time letter writers for North Americal grad school applicants. Often, first-time applicants from a new school/city (particularly in underrepresented groups) are disadvantaged by their referee’s lack of expercience writing strong letters, or due to the absence of concretely defined expectations. I hope to level the playing field a bit, with this post.

Note: This post is also a critique on referees that ask students to draft letters for themselves (I know firsthand a lot of professors/managers in India do this). This is an abhorrent practice and often ends up hurting the student’s chances of securing an admit. Nearly all universities realize that most students from certain parts of the world tend to write their own letters and discount these letters. On the off chance that you’re reading this post and are a referee that asks students to write their own letters, please don’t! You’ll do a great deal of good for a student’s career by spending an hour or so carefully drafting a letter.

Assumptions: This article assumes that your referee knows you very well and has the time to compose a well-thought out letter to support your case.

Length

There’s a misconception among several referees and students that letters of recommendation must be limited to a page or less. This is not true. On average, most letters I’ve read are 1.5-2 pages in length (only rarely longer).

Levels of support

Good letters will not wait until the end to deliver the punchline. They will begin (and conclude) by making a concrete recommendation. For North American letters, there are four broad levels of support that letters offer. Here’s examples of each:

  • “I recommend XYZ for the [Masters PhD] program at your university”.
  • “I strongly recommend XYZ for the [Masters PhD] program at your university”.
  • “I offer my strongest recommendation possible for XYZ to be admitted to the [Masters PhD] program at your university”.
  • “I would not recommend XYZ for the [Masters PhD] program at your university”.

Letters of the last type are few; often referees will not offer to write a letter in such a case.

Calibration and Context

An LOR is an endorsement for you by your referee. Now, the admissions committee may or may not know you or your referee. Consequently, there is a large variance in which letters of reference are perceived. Good letters aim to minimize this variance by calibration: speaking about your achivements, abilities, and potential in a manner that’ll enable apparent comparison to other applicants. Good letters will provide an accurate context in which to evaluate your letter.

Letter writer profile

The first step to calibrating letters is to contextualize the profile of the letter writer. This is usually done by providing a very short bio of the referee, stating the referee’s experience in their current role, and—importantly—the number of students at various levels supervised by them over their career (e.g. X Postdocs, X PhDs, X Masters students, and X undergrad students/interns). It is also important to note the range of positions these alumni went on to take (faculty positions? research scientist at top-orgs? gradschool at top-K places?). It is also important to explicitly state who among these people (and at what stages of their careers) you are being compared with.

Association with the letter writer

Letter writers need to explicitly state their association with you (advisor/course instructor/manager/colleague/etc.). Letters only have an impact when the referee knows you long enough to speak about your abilities (for research programs in ML, I suspect it would take students a bare minimum of 3-6 months to cultivate this association). Letters from a professor you only took a semester-long class with are therefore not a great choice.

Objective evaluation matters

Most letters objectively compare you to a cohort of students chosen by the letter writer. They often place you in an appropriate quantile within the cohort (e.g. top 2%, bottom 50%, top candidate, etc.). Some letters go to the extent of comparing the student to a recent graduate from the target program and make a strong case for admission. Such objective scores are hard to assign, but form a crucial component of the letter and are looked at in detail. In addition to an objective evaluation in the letter, nearly all schools require the referees to fill out idiosyncratic questionnaires that provide additional analysis of the student within the peer group.

Comparisons with other students in the current application cycle

Some letter writers take their job truly seriously. They compare the student with everyone else they are writing a letter for during the admissions cycle and also provide a preferred ranking scheme. This adds an additional layer of credibility to a letter. This also makes it hard for students to forge letters; as all applicants will now need to have consistent scores and ranks.

Well-motivated description of research

For the most part, a letter will pick one or more of your projects to talk about in detail. It’s important to clearly motivate these research projects so that people from outside your primary area may better appreciate your work. It is somewhat uncommon, but not inconceivable, to have a figure to better illustrate the core idea/results of your project.

Personal qualities and traits

Another important part of the letter is a description of your personal qualities and traits. Are you an extremely fun person to work with? Do you often organize lab events / socials and other activities? What is your preferred style of work? A good letter would explicitly highlight all these and more. If you haven’t been in contact with your referee for a while, you would want to send in a gentle reminder of all activities you carried out, so it doesn’t slip away.

Quotes from closer mentors

Many advisors are hands-off, meaning your are often closely supervised by a senior student from the group. Good letters will include direct feedback from your closer mentors and provide a better picture of your abilities.

Explaining blips and blemishes

If you had a poor grade in an important course or haven’t had a publication thus far, you can have your letter writers vouch for you and explain away such blips and blemishes. You can often ask your letter writers to talk about ongoing or unpublished work you’re carrying out and why it’s exciting.

I hope this clears some air of confusion about reference letters. I’d be very curious to hear any views/takes on LORs. Please drop me an email line or a Twitter DM if this is you.

Here are a few other articles in this series.

  1. How to pick your grad school
  2. Writing a statement of purpose