Week 10: How a Reviewer Turns a Reading Into a Score

Last week, I talked about how reviewers actually read — not how they are supposed to read, but how they read at 11 PM on a Tuesday with six more applications in the stack.

This week, I want to talk about what happens next. The reviewer has finished reading. The application is closed. And now they are sitting in front of a blank critique template, and they have to do something that is far harder than most applicants realize: they have to translate an experience into a judgment.

This is not unique to grant review. Every time your work is evaluated by another human being — a hiring manager scanning a resume, an editor reading a manuscript, a client reviewing a proposal, a boss assessing your performance — the same translation happens. Someone finishes engaging with your work, and then they sit down and convert what they experienced into structured language that carries consequences for you.

And in every one of those contexts, the same invisible problem applies: what they write is not a perfect record of what they read. It is a record of what they remember. And what they remember is not the same as what you wrote.

Understanding that gap — between the reading and the record — is one of the most consequential things you can learn about how your work gets evaluated. In any field. By anyone.

I happen to know this gap intimately from the reviewer's side of the NIH grant process. That is the specific version I am going to describe today. But the deeper truth underneath it applies far beyond study section.

The template is not neutral

The first thing a reviewer sees when they open the critique template is structure. Under the Simplified Review Framework, the template organizes their evaluation into three factors. Factor 1: Importance of the Research — encompassing both Significance and Innovation, evaluated together with a single criterion score. Factor 2: Rigor and Feasibility — your Approach, also a single criterion score. Factor 3: Expertise and Resources — your team and environment, assessed for sufficiency but not scored numerically.

This structure seems straightforward. It is not.

The template asks a reviewer to do something that does not come naturally to most evaluators in any domain: disaggregate a holistic experience into discrete scored categories. Your reviewer did not read your application thinking "now I am evaluating importance" and then "now I am evaluating rigor." They read it as a continuous experience — one where significance bled into approach, where the strength of the preliminary data shaped their confidence in the timeline, where a well-constructed team made a challenging design feel feasible and an unclear team description made a straightforward design feel uncertain.

Now they have to pull that experience apart. And the way they pull it apart is shaped by something most people never think about — whether they are filling out a grant critique template, writing a performance review, or scoring a job candidate after an interview: what was easiest to remember.

The availability problem

There is a concept in cognitive psychology called the availability heuristic. It means that people assess the importance or frequency of something based on how easily it comes to mind. In everyday life, this is why people overestimate the risk of plane crashes and underestimate the risk of heart disease — dramatic events are more cognitively available than statistical ones.

In any evaluation context, the availability heuristic means that the parts of your work an evaluator can most easily recall when they sit down to write are the parts that will dominate the evaluation. Not necessarily the most important parts. The most memorable ones.

In grant review, this plays out with striking specificity. A vivid example in your preliminary data — a graph that told a clear story, a finding that surprised the reviewer — will be cognitively available when they write about Importance. A confusing paragraph in your analytic plan — one they had to reread, one that slowed them down — will be available when they write about Rigor and Feasibility. Not because it was the most serious issue in your approach, but because the friction of reading it created a stronger memory trace than the five paragraphs around it that were perfectly clear.

This is why the information architecture I wrote about last week matters beyond the reading experience itself. The structure of your work does not just determine whether an evaluator can find your arguments. It determines which arguments are cognitively available when the evaluator sits down to render judgment.

Think about that in any context where your work is assessed. The section of a proposal that was clearest will be the section that shows up in the feedback. The part of a presentation that created friction — a confusing slide, a rushed transition — will be the part the audience remembers when they discuss it afterward. Not because it was the most important thing you said. Because it was the most available thing in their memory when they went to form an opinion.

I have experienced this myself as a reviewer. I have finished writing a critique, felt satisfied that it was fair and thorough, and then gone back to the application to check something — and found a strength I had not mentioned. Not because I disagreed with it. Because by the time I sat down to write, it had been displaced by something more cognitively vivid. The strength was on page nine, stated in a single sentence embedded in a methods paragraph. The weakness I wrote about at length was on page four, immediately visible under a subheading that named the very thing I was concerned about.

The applicant who buries a strength and headlines a weakness has made a structural choice that follows the evaluator all the way to the scorecard. This is true in grant review. It is true everywhere.

What evaluators write first

Most reviewers do not write their critique in the order the template presents it. They start with whatever they have the most to say about.

For many reviewers, that means they start with concerns. Not because they are looking for reasons to criticize, but because concerns generate more cognitive energy than affirmation. A strength you agree with produces a simple sentence: "The significance of the proposed research is well established." It takes thirty seconds to write. A concern you need to articulate — one where you are trying to be fair, trying to distinguish between a genuine limitation and a preference, trying to express what bothered you without overstating it — takes time. It requires precision. It requires going back to the source to make sure you are representing the argument accurately before you explain why it did not persuade you.

This asymmetry is not unique to grant review. Think about the last time you wrote a performance evaluation, a book review, a peer critique, a reference letter. The positive things you had to say probably came quickly and in broad strokes. The concerns — if you had them — took longer to articulate, required more careful wording, and occupied more space on the page.

The consequence: concerns often receive more detailed, more carefully worded treatment than strengths. And the person reading the evaluation later interprets that asymmetry as a signal. They conclude the evaluator cared more about the weaknesses. They read the length of the concern as a proxy for its weight.

Sometimes that is true. But sometimes the evaluator scored the work highly and simply spent more words on the parts that required more careful articulation. A short, declarative strength — "the investigative team is exceptionally well suited to conduct this research" — may reflect genuine conviction. A long, carefully hedged concern — "while the analytic approach is generally sound, the justification for the clustering adjustment could be strengthened" — may reflect something minor the reviewer wanted to document without overstating.

The length of a critique point is not a reliable proxy for its weight. The score is.

The ceiling most applicants do not know about

Now I want to tell you something about scoring that reshapes how every strategic decision in your application should be weighted.

After the reviewer has written their narrative critique, they assign preliminary scores. Factor 1 — Importance of the Research — gets a score from 1 to 9. Factor 2 — Rigor and Feasibility — gets a score from 1 to 9. Then they assign an overall impact score, also 1 to 9, which reflects their global assessment of the application's likelihood of making an important contribution to the field.

Here is what most applicants do not understand, and what fundamentally changes how you should allocate your writing energy: the Factor 1 score sets the ceiling.

A flawless approach — rigorous design, airtight methods, impeccable analytic plan — cannot rescue an application where the reviewer was not convinced the question matters enough. If a reviewer scores your Importance a 4, your overall impact score cannot be better than a 4, no matter how strong everything else is. The approach serves the question. If the question does not compel, the approach has nothing to elevate.

This is the opposite of how most applicants spend their time. Most applicants pour their revision energy into the Approach section — tightening the methods, strengthening the analytic plan, adding sensitivity analyses. And those things absolutely matter. But they matter within the ceiling that Importance has already set.

The applications that score in the top percentile are not the ones with the most bulletproof methods. They are the ones where the reviewer finished reading the Significance and Innovation sections and thought: this has to be funded. Once a reviewer reaches that conviction, the Approach section becomes a question of "can this team pull it off" rather than "should anyone pull it off." The bar for rigor does not lower, but the reviewer's orientation toward the application shifts. They are reading the approach looking for reasons to believe rather than reasons to doubt.

That shift — from skeptical evaluation to invested assessment — is the single most important thing that can happen during a reviewer's reading of your application. And it is determined by Factor 1.

I want you to think about what that means for where you spend your revision time. If you are resubmitting, and your summary statement shows strong marks on Approach but a middling assessment of Importance, the instinct is to fine-tune the methods. The strategic move is to rebuild the case for why this work matters.

And this principle extends beyond grants. In any evaluative context, the person assessing your work makes a judgment about whether the thing you are proposing is worth doing before they make a judgment about whether you can do it well. If you have not won the first question, excellence on the second question operates under a ceiling you may never see in the feedback — because most evaluators will not write "your methods were strong but I was never convinced this mattered." They will simply score accordingly and articulate their concerns in language that sounds methodological when the root was motivational.

The critique as a script

There is one more thing about the critique-writing process that I did not fully appreciate until I had reviewed for several cycles, and it connects everything I have described today to what happens in the study section room.

When a reviewer writes a critique, they are writing for two audiences simultaneously. The first audience is the applicant — the person who will eventually read this and use it to decide whether and how to revise. The second audience is the room — the other scientists on the study section who will hear this critique presented during discussion.

These two audiences need different things. The applicant needs specificity. The room needs narrative — a coherent, efficient summary that conveys the essential character of the application in a few minutes.

When a reviewer is tired, when they are writing their fifth critique in a cycle, when the template is open and the clock is moving — the audience that wins is usually the room. Because they know the discussion is coming. They know they have to stand up and present this. And so they write, often unconsciously, toward presentability — toward the version of their assessment that will be most coherent when spoken aloud.

This has a consequence: critiques sometimes trade specificity for narrative flow. A concern that was nuanced and conditional in the reviewer's mind may appear as a cleaner, more definitive statement — because clean statements present well in a room. A strength may be described in broad terms rather than with the specific evidence that supported it — because broad characterizations are easier to speak.

When you read your summary statement, you are reading a document that was written, at least in part, to be performed. The reviewer was not just recording their assessment. They were preparing a script.

This is true of most evaluative writing, if you think about it. The manager writing your performance review knows it may be discussed in a calibration meeting. The editor writing a rejection letter knows the language may need to be defensible. The interviewer writing up their assessment of a candidate knows a hiring committee will weigh it. Evaluative documents are almost never purely private. They are drafted with awareness of a future conversation, and that awareness shapes what ends up on the page.

What this means for how you write — anything

If I could give every person whose work is evaluated by busy humans one insight from the critique-writing side of the process, it would be this: you are not just creating something that will be experienced. You are creating something that will be translated — first into a judgment, and then, usually, into a conversation about that judgment.

The parts of your work that are easiest to recall will dominate the judgment. The importance of what you are doing sets the ceiling for how favorably everything else is evaluated. And the language the evaluator uses to describe your work — in a critique, in a meeting, in a hallway conversation — will closely track the language you gave them, if you gave them language worth borrowing.

In grant writing, this means your Specific Aims page is doing more work than any other single document. It sets the importance frame. It provides the language a reviewer will carry into the template and, later, into the room. A compelling Aims page does not just introduce your application. It writes the first draft of your critique — the version you want the reviewer to deliver.

Every structural choice you make either supports or undermines that chain of translation — from page to memory, from memory to critique, from critique to the room. Every layer you make easier is a layer where your argument arrives intact.

What is one part of your work — a grant, a proposal, a pitch — where you know the substance is strong but you suspect the evaluator would struggle to describe it from memory?

Next week: when the summary statement arrives — the sentence most applicants skip past, why your Resume and Summary of Discussion tells you more than the individual critiques, and the most common misread that derails resubmissions.

~Lisa Carter-Bawa, PhD, MPH, APRN, ANP-C, FAAN, FSBM

More insights at lisacarterbawa.com/insights

Lost In Translation (LIT) grantsmanship curriculum is available as an institutional cohort for cancer centers, training programs, and research-intensive departments.

The LIT Founders Cohort launched in August — the first cohort outside of an institutional partnership.

NEW: Four seats remain for the next cohort — LIT February 2027 Deadline Cohort — and begins October 19 and is taking applications here. Five live Monday sessions, noon–1:30 PM ET, built around the Five Translation Gaps and timed so your Specific Aims page is reviewed, revised, and ready for the February 5 NIH deadline. The cohort includes full access to the LIT course (14 modules + 13 workbooks) plus a post-cohort Translation Feedback Brief — a one-on-one review of your revised Aims page and/or full grant returned in time for your submission.

10 seats. $997 includes the full LIT course and cohort experience. Already own the course? The cohort experience is $500. Apply here: https://forms.gle/5AWXyPwufN7LKSxr6

If October isn't the right time for you — or if you're exploring whether this course is the right fit — I've created a brief interest form where you can tell me a little about where you are in your grant journey and what you're looking for. There's no commitment involved; it simply helps me plan future cohorts and formats around what investigators actually need. I'll follow up personally as new opportunities come together.

Contact lisa@soultosoulleadership.com with any questions or visit lisacarterbawa.com/losttranslation to learn more.