An important part of grant writing is understanding the grant review process. If you can put yourself in the position of the reviewer, and ask yourself .... will the reviewer easily comprehend the rationale and significance of the experiments I am proposing? .... then you will be well on the way to preparing a proposal that will be reviewed favorably. Later in this course you will get some experience in reviewing grant proposals prepared by your fellow students in this course, and will participate in a "mock" grant review panel, somewhat analogous to the USDA-NRICGP panel meetings where grant proposals are ranked. In this lecture I would like to outline how the review process actually occurs.
The Program Director for each Program appoints, each year, a scientist who is knowledgeable about the Program field and who will act as Panel Manager to oversee the grant reviews and panel discussions/evaluations for all grants submitted to that program in that year. The Panel Manager for each program is changed every year. The Panel Manager, together with the Program Director (and associated staff) is responsible for putting together a team of 6 to 16 (depending on the program and number of proposals received) individuals who are experts in fields relevant to the program, who will serve as Panel Members. The Panel Members are not always changed each year; in order to provide some continuity from year-to-year, certain Panel Members are asked to serve on the Panel for a second term. If a program receives 120 proposals, the Panel would be comprised of 12 Panel Members. Each Panel Member would typically be expected to review 20 of these proposals. Of these proposals, each Panel member would be expected to serve as Primary Reviewer for 10 of these proposals, and Secondary Reviewer for the additional 10 proposals. Each Panel Member would also receive an additional 10 proposals to read [written reviews not required]. Thus, for each proposal received, one Panel Member would be the Primary Reviewer, one would be the Secondary Reviewer, and one would be a Reader.
For each proposal, whether Secondary or Primary, a written review is prepared in advance of the Panel Meeting (which generally occurs in Washington D.C. over a period of 3 - 5 days [depending upon the number of proposals to be reviewed]). Standard Research Grant proposals are
reviewed by several criteria in the written reviews:
- the scientific merit of the proposal:
1. novelty, uniqueness, and originality
2. conceptual adequacy of hypothesis or research question
3. clarity and delineation of objectives
4. adequacy of description of the undertaking
5. suitability and feasibility of methodology
6. demonstration of feasibility through preliminary data
7. probability of success of project
- qualifications of proposed project personnel and adequacy of facilities:
1. training and awareness of previous and alternative approaches, performance record and/or potential for future accomplishments
2. time allotted for systematic attainment of objectives
3. Institutional experience and competence in subject area
4. adequacy of available or obtainable support personnel, facilities and instrumentation
- relevance of the project to long-range improvements in and sustainability of U.S. agriculture
1. documentation that the research is directed towards a current or likely future problem in U.S. agriculture
2. development of basic research ideas towards practical application
Each reviewer is asked to rate each proposal overall as either:
- very good
Each Panel Member receives (about 2 months in advance of the Panel Meeting) not only the 20 proposals he/she is designated to review, but also a copy of the Project Summary page from each of the other 100 proposals. The Project Summary is the most important page of your proposal in the review process, because it is the only page that is seen by the entire Panel!
Each Panel Member must declare a Conflict of Interest with respect to any of the proposals that will eventually be discussed by the Panel if he/or she has worked in any way with any of the PIs on any of the grant applications over the past 5 years, or if any proposal to be considered by the Panel is from the same Institution as the Panel Member. By declaring a Conflict of Interest, the Panel Member is excluded from all discussions about the specific proposal, and must exit the room when such discussions occur in the Panel Meeting.
In addition to the Primary and Secondary Reviews conducted by the Panel Members, the Panel Manager must contact a host of other "experts in the field" to serve as ad hoc reviewers for individual proposals. Often as many as 4 to 6 ad hoc reviewers are contacted for each proposal. However, ad hoc reviewers do not always respond in a timely fashion, and it is not uncommon to find that only 2 or 3 ad hoc reviews are available at the Panel Meeting for any given proposal. Nevertheless, these ad hoc reviews serve an important part in the review process, since the written statements prepared by the ad hoc reviewers must be presented to the entire panel by the designated Primary Reviewer, who must reconcile his/her own review of the proposal with the independent ad hoc reviews. The job of the Primary Reviewer of each proposal at the Panel Meeting is to present a brief overview (to the rest of the Panel) of the objectives of the proposal, its strengths and its weaknesses in his/her own opinion, followed by a summary of the ad hoc reviewer's opinions. The Secondary Reviewer then presents his/her opinion of the proposal (strengths and weaknesses) and the proposal is then opened to discussion by the Reader and other Panel Members. It is extremely important to realize that the other 10 Panel Members may have only seen the Project Summary page of the proposal up to this discussion point. The Panel discussions are intended to arrive at a concensus about where a proposal should be ranked. The following categories are generally used to rank proposals by the Panel:
- High priority for funding
- Medium priority for funding
- Low priority for funding
- Some scientific merit
- Do not fund
Within each category, proposals are also ranked against other member proposals; e.g. within the Medium Priority funding category is proposal X superior to proposal Y, A, G and D etc.
Each Reader is responsible for taking notes on the Panel Discussion pertaining to each proposal, and for preparing a Panel Summary Statement concerning the overall Panel impressions of the proposal, the concerns of the Panel and the other reviewers, and the overall recommendation. This Panel Summary is prepared during the Panel Meeting by each Reader, is proof read by the Primary and Secondary Reviewers and is then read aloud to the entire Panel by the Reader. The Panel then provides comments and
suggestions about improving the wording of each Panel Summary Statement, as required.
Generally, discussions of any one proposal will be limited to about 10 to 15 minutes each. Although this may seem a small amount of time, consider that for a discussion of 120 proposals approximately 3 days would be allowed for the Panel Meeting. Assuming 6 proposals are discussed per hour, then you have 20 working hours to get through 120 proposals! This is 2.5 working days (assuming an 8 h-day). Many of the discussions (particularly where ad hoc, Primary or Secondary reviewers express markedly different opinions about a proposal) greatly exceed 15 minutes. These extra discussions go into "over-time", often leading to >12 h working days for the Panel. The Panel must reserve at least 0.5 days at the end of the Panel Meeting to consider a final
ranking of all of the proposals, especially the "Outstanding", "High Priority" and "Medium Priority" categories.
This final ranking of the proposals becomes the Panel Recommendation which in turn becomes the Program Recommendation to NRICGP which is then transferred to the Awards Branch of USDA-NRICGP for awarding the grants [typically approximately 22 - 27% are funded]. Reviews (including the Panel Summary Statement (with the names of the reviewers omitted)) are then returned to each applicant.
Postdoctoral fellowship proposals are also evaluated on the quality of the training environment, including:
- Documentation that arrangements have been made with an established investigator to serve as mentor.
- Documentation that arrangements have been made for the necessary facilities, space, and materials to conduct the proposed research.
- Potential for the postdoctoral fellow to initiate an independent research program.
In evaluating proposals for the support of research conferences, the following factors are considered:
- Relevance of the proposed conference to agriculture in the U.S. and the appropriateness of the conference in fostering scientific exchange.
- Qualifications of organizing committee and appropriateness of invited speakers to the topic areas being covered.
- Uniqueness and timeliness of the conference.
- Appropriateness of budget request.
The following evaluation factors are used in reviewing applications for Research Career Enhancement Awards, Equipment Grants, and Seed Grants:
- The merit of the proposed activities or research equipment as a means of enhancing the research capabilities and competitiveness of the applicant and/or institution.
- The applicant's previous research experience and background.
- The appropriateness of the proposed activities or research equipment for the goals proposed.
- Relevance of the project to long-range improvements in and sustainability of U.S. agriculture.