Novelty First Policy-Based Intelligent Review Framework (IRF) for the Evaluation of Research Proposals

Abstract

Despite its many limitations, peer review is the most preferred research assessment scheme for research proposal assessment at the individual level. Although scientometric assessment offers effective assessment frameworks, certain limitations, including the proven and potential misuse of scientometric indicators, hinder its wide adoption. Informed peer review is viewed as an effective way of harnessing the advantages of peer review and scientometric/quantitative assessment wherein one may complement the limitations of the other. Informed peer review frameworks are still prone to many inherent challenges in scientometric assessment and peer review. The importance of intelligent review frameworks that can be more advanced and effective than informed review frameworks lies there. With the advent of AI and generative AI (GenAI), a plethora of opportunities are available to convert informed peer review frameworks to intelligent review frameworks but not without challenges and concerns. In this work, we discuss the possible opportunities for effective AI intervention in an existing informed peer review framework to transform it into an intelligent review framework. Although the selected existing informed peer review framework emphasized the ‘novelty first’ policy, it did not provide any means or guidelines to execute it. The proposed conceptual ‘intelligent review framework’ addresses this very well by exploring the effective use of AI/ML techniques for the process and is envisioned to have the flexibility to adapt to future technological developments in AI, GenAI, etc. Possible challenges and a roadmap for possible evolution with anticipated technological changes, etc., are also discussed.

1. Introduction

As externally funded (mostly publicly funded) research projects are vital in determining the growth of science and technology, an intelligent and evolvable evaluation system framework for the assessment of such proposals is inevitable for STI governance. Given that AI is intended to take over many traditional systems with possibilities of human replacement or coexistence with humans, experts approach it with extra caution for some applications. STI governance, especially research evaluation, is one such application wherein even traditional scientometrics that may offer not only evidence but also intelligence are still heavily weighed against judgmental approaches such as peer review. With respect to research evaluation for STI governance, both scientometric/quantitative evaluation and peer review can benefit from AI’s transformative potential perhaps later than sooner because of many challenges, including issues such as transparency issues, due to the black box nature of some AI algorithms noted by Farber (2024). Gen AI models are far from claiming human-like intelligence/decision-making ability (Raman et al., 2024), and some experts even advocate a blanket ban of those for peer review. However, blanket bans on any technological change cannot stand the test of time, and the prudent approach is to diligently explore transformative potential and cautiously and systematically integrate AI models/approaches for systems such as peer review that are facing heavy challenges because of human limitations in tackling the massive rise in research activities. With respect to quantitative/scientometric evaluation, to be truly AI-powered, a systemic transformation of the field in terms of objectives and evaluation methods/systems is needed (Milojević, 2025), wherein AI-powered scholarly discovery platforms can be decisive (Rahman et al., 2026).

The evaluation/assessment of research project proposals for fellowships/grants/awards at the individual level is highly challenging, especially in the case of thrust areas or national priority areas. Evaluations and decision-making that are purely based on peer review (which is widely adopted for ex ante evaluations at the individual level) can be tricky. Unlike the emphasis on quality to be ensured by peer review in research publishing driven by corporate competition for reputation, the government’s immunity to such market forces, power against accountability and systemic lack of transparency sometimes lead to elite capture and cronyism (Mom et al., 2018), wherein ‘bureaucracy-elite academicians lobby’ exhibit favoritism for misappropriating/misallocating grants/resources via intentional bias in peer review. The study also revealed that women are more affected than men are in this regard. Another crucial pitfall is the bias against novelty or fear/intolerance of risk (Fedorov et al., 2010; Edwards et al., 2011; Petsko, 2011; Wang et al., 2018; Azoulay & Greenblatt, 2025; Veugelers et al., 2025). All these factors combined with the increased bureaucratization and administrative burden related to publicly funded projects orchestrated not only for genuine requirements such as crisis response but also for bureaucratic-over control and social and political side-payments (Bozeman & Jung, 2017) hinder novelty in research.

On the other hand, while its usage in nations that adopted performance-based funding systems (PRFS) for ex post evaluation (Hicks, 2012) is more sensible than peer review (Abramo et al., 2019; Abramo, 2024), we also have before us examples of misuse of scientometric/bibliometric indicators for review of proposals for funding. Sometimes using indicators can be more effective and easier for executing corrupt practices than peer review can be. The impact factor metric (Garfield, 1963) and h-index (Hirsch, 2005) might be the most misused indicators for ex ante evaluation of research proposals wherein applicants are individuals despite the caution provided during their introduction itself. For example, Madhan et al. (2018) criticized Indian funding agencies, including the DST of suffering from h-index syndrome, and attributed them not only to the misuse of IF and the h-index but also to the institutionalization of such misuse. Thus, indicators that are free from bias (favoring or giving undue advantage to some) and are not complex/sophisticated to interpret are vital for ensuring the success of scientometric assessment for the review of candidates for funding.

Informed peer review frameworks (IPRFs) that use both scientometric/quantitative assessment and peer review are sometimes regarded as better options (van den Besselaar & Sandström, 2020). However, IPRFs that take into account both bibliometric/quantitative scores and peer-review-based assessment scores to calculate final scores are also prone to the pitfalls of either the assessment procedures or both. Earlier, Abramo et al. (2019) opined that as the scale of assessment increases, scientometrics might be more useful than peer review (indicating the importance of scientometric evaluation and informed peer review over peer review). Based on the analysis of production functions of peer review and scientometric assessment for large-scale assessment that considers labor (in this context, the number of peers or scientometricians at work) and materials (resources that support analysis) costs, Abramo (2024) found that the increase in volume of proposals overburdens peers and reemphasized the ‘benefit of scale’ of scientometric assessment in long-term. In OECD countries (including those having PRFSs), and in developing countries like India that mostly rely on ex ante assessment, a rise in number of private universities and overall doctoral enrollment is visible (Sarrico, 2022; Jayaram, 2023). Nations can no longer stick to peer review assessment alone as the overburden and burnouts of peers might worsen the effects of limitations of peer review. Thus, scientometric assessment cannot be kept away forever in ex ante evaluations. Although informed peer review can be costlier than both scientometric and peer-review-based assessment, as a rise in the volume of proposals is anticipated, the ‘benefit of scale’ pointed out by Abramo (2024) for scientometric assessment in the PRFS is applicable in the case of ex ante informed peer review too, and in the long run, it may turn out as cost effective. Therefore, an informed peer review framework that ensures responsible use of indicators and is free from various previously discussed limitations of peer review should be an integral component of the national STI governance. This is the major motivation behind this work.

Recently, an informed peer review framework (IPRF) that attempted to minimize the drawbacks of both was introduced by Lathabai et al. (2023). The IPRF recommended the service of two teams, (i) scientometric/quantitative analysis teams and (ii) peer review teams, that work independently of each other. The responsibilities and functions of both the teams and the features and computations involved in IPRF are discussed in Section 2.3. Although the scientometric team computes a composite score viz. CS from citation-dependent indicators, it is almost immune to biases/limitations induced by citations, as its computation does not include raw citations or its simple variations, as in the case of IF and the h-index. Once proposals are ranked on the basis of the CS, those must be passed to the peer review team in a ‘least rank first’ fashion to review the merits of the proposal. Peer reviewers assess the proposals on the basis of the criteria set by them with or without the direction of funders, and CS has no role/influence other than deciding the order. To aid the peer review team and reduce the possibility of ineffective evaluation or biased/corrupt evaluation, a ‘novelty first policy’ was recommended. This means that while determining the quality/merit of a proposal, novelty should be given predominant consideration. The rationale is that novelty or new ideas, new theories or methods to solve problems (Bornmann et al., 2019b) are the key drivers of scientific/technological progress, and the earlier perspective of novelty as an unmeasured (Leahey et al., 2023) output of recombination (Kaplan & Vakili, 2015) is no longer valid after the introduction of methods to measure novelty, which we will eventually discuss. Thus, the diligent construct of the IPRF, which has an independent scientometric framework and a peer review framework, is supposed to have as much objectivity and balance as possible. However, the IPRF has several limitations, such as (i) limited/restricted analytical and operational space for the scientometric team and (ii) no guidance on how to assess novelty effectively despite the stress on novelty. With the advent of AI/ML, there are possibilities for addressing these drawbacks without worrying too much about cost. Thus, a framework that harnesses scientometrics and peer review in a better fashion than IPRF, other existing informed peer review models, scientometric evaluation models and peer review models that incorporate AI/ML methods effectively in both scientometric and peer review modules is needed. This gap is addressed in this work.

Therefore, we propose a conceptual framework that is envisioned to be comprehensive, robust, flexible and evolvable and retains the advantages of IPRF (Lathabai et al., 2023). This framework considers the fact that the wind of change in the form of AI is already blowing in scientometrics, which is also evident from works on hybrid human–AI tools for scientometrics (Correia et al., 2023) and many others. The proposed framework bestows more responsibilities/functions to the scientometric team so that peer reviewers in this framework can enjoy much support from the scientometric team in contrast to the IPRF. To ensure ‘novelty first policy’, in addition to the proposals ranked according to the novelty-incorporated composite score (CS′), the list of novel papers (and possibly their short summaries of novel contributions) in the relevant literature will be handed over to the peer review team. This may significantly aid peer reviewers in determining the novelty of the current proposals. The novelty of a proposal can be determined by examining whether it has any theoretical novelty, methodical novelty or applicational novelty over the papers that are determined as novel/disruptive/paradigm shift pivot papers in the fields/themes relevant to the call. Several other features envisioned for the framework that may be crucial for its evolvability are discussed in detail in Section 3. We prefer to call this framework the ‘Intelligent Review Framework’ or ‘IRF’ because the reasons are already obvious from the above discussion.

2. Developments Relevant to the IRF

Although we have yet to discuss the possible structure, function and evolution of the proposed design of the IRF, we clarify here that although the assessment is supposed to be carried out at the individual level, the following assessments are necessary for comprehensive and effective quantitative evaluation of the individual: (i) assessment of the body of literature related to field(s) or thrust areas being funded to determine key developments (for comparative determination of the relevance/novelty of submitted proposals) and the key contributors (to identify the experts for peer review) and (ii) assessment of the expertise of individuals/candidates who submitted proposals as in the case of IPRF. Prominent scientometric methods, techniques or frameworks useful for these are discussed next.

2.1. Scientometrics for the Assessment of Novelty, Disruptiveness and Paradigm Shifts in Research Fields

Scientometrics offers a myriad of frameworks, tools and methods for assessing research fields by mining scientific/patent networks via network analysis frameworks and tools (Shibata et al., 2007; Liu & Lu, 2012; Lathabai et al., 2018) for disciplinary as well as interdisciplinary analysis (Leydesdorff & Rafols, 2011; Wagner et al., 2011); Topic modeling (Mohammadi & Karami, 2022) and approaches that combine network analysis and text mining (Klavans & Boyack, 2017; Kim et al., 2022; Yan et al., 2024) are also gaining momentum. The indicators that could pinpoint or handpick works with substantial novelty/disruption/paradigm shifting potential are crucial for the proposed design of the IRF.

• Novelty, disruptiveness and paradigm shift indicators

As extensions of the discussion of the importance of knowledge recombination and technical refinement (Grant, 1996) and disruptive innovation theory by Christensen (1997), several indicators for novelty/disruptiveness and paradigm shift detection/prediction were introduced. Azoulay (2019) noted that if a paper receives more citations in such a way that the citing works acknowledges/cites few of its intellectual forebears (papers cited by the paper of interest), then that work must be of disruptive potential. Therefore, many indicators are introduced on the basis of the cited references and citations received by papers or patents, represented as citation networks and networks derived from them, such as cocitation/bibliographic coupling networks and journal cocitation networks (reflecting knowledge origin, flow and recombination). Works that attempt to determine novelty via text or semantic similarity to previous works are also available. More such effective attempts can be expected in the AI era. In this work, we discuss 13 indicators (please also refer to Supplementary Table S1 for details) related to scientific and technological novelty, disruptiveness and paradigm shift detection/prediction.

Freeman’s betweenness centrality: Chen (2006) demonstrated the use of Freeman’s betweenness centrality (Freeman, 1977), and the rationale is that papers with high betweenness in cocitation networks are found in paths between two clusters/research fronts that lead to paradigm shifts.

Atypicality indicator: Uzzi et al. (2013) introduced a novel indicator with the rationale that atypical or unusual combinations of cited references (that lead to unusual journal pairings) can be suggestive of novelty. Atypicality score, z >0, indicates conventionality, and z < 0 indicates atypicality.

Novelty score U: Lee et al. (2015) developed the novelty score U, an improved version of the computationally intensive indicator by Uzzi et al. (2013), wherein atypicality/novelty is expressed as opposed to commonness. U is computed as the qth percentile (typically q = 10) of commonness at the paper level (obtained from commonness at the journal level).

Flow vergence index (FV index): Prabhakaran et al. (2015) defined an indicator/model as papers/patents that received (i) more citations (than the number of citations made) or (ii) a good number of citations from quality papers (which are, in turn, reasonably cited), or both are more likely to directly/indirectly transfer knowledge to more knowledge pathways in the citation network.

Flow Vergence (FV) gradient or$∆FV$: Lathabai et al. (2015) developed the FV gradient indicator to pinpoint pivots of paradigm shift from papers that have high FV potential, because in a cited–citing pair/link where j cites i, if the citing paper exhibits more FV potential (${FV}_j>{FV}_i, { ∆FV}_{ji}={FV}_i-{FV}_j <0)$, then j can be a crucial paper associated with a paradigm shift, because usually i will have more FV potential than j does. The FV gradient can predict paradigm shifts too (Prabhakaran et al., 2018).

Patent-based novelty indicators: Verhoeven et al. (2016) devised three indicators, as patents can be novel if they possess novelty in recombination (NR) and novelty in technological origin (NTO) and scientific origin (NSO) of knowledge. Patents are assigned a binary value if they cause a new IPC combination (NR = 1), cause an IPC combination in the IPCs associated with them (NTO = 1) and cause a new IPC-WOS category combination (NSO = 1), rather than considering the frequency/magnitude. These three scores are then used to determine patents with different inherent novelties.

W score: It was introduced by Wang et al. (2017), according to which papers that led to journal combinations that are not only new/nonexistent but also reused (reoccur due to further publications) after introduction are novel.

CD index: The consolidated disruptive index (CD index) (Funk & Owen-Smith, 2017) treats a patent k as substantially disruptive if, among the patents that cite k, a smaller share cites the predecessors of k. A method to compute the magnitude of future use of k was also provided by them.

Disruptiveness indicator (DI): L. Wu et al. (2019) introduced a disruptiveness indicator, DI, that considers the citation preferences of works that cite a work and do not cite it. In other words, m is a work with high disruption potential if (i) the number of works that cite m but do not cite those cited by m > the number of works that cite m also cite those cited by m and (ii) there are fewer works that do not cite m but cite works that are not cited by m.

Modified DI index or ${DI}_m^{nok}$: Recognizing that consideration of works that do not cite m works sometimes forces the DI indicator to behave in a fashion counterintuitive to its purpose, Q. Wu and Yan (2019) introduced ${DI}_m^{nok}$ (validated by Bornmann et al., 2019b), a corrected DI index, the formula of which is without the number of works that do not cite m but cite works that are not cited by m.

Depth and Breadth indicators: Bu et al. (2021) introduced two indicators related to disruptiveness, namely, depth and breadth. If articles that cite paper i also cite other articles that cited i are more numerous (i.e., if papers citing i are highly bibliographically coupled through i), then i can be disruptive, as it creates more depth. The breadth indicator is computed as (1-depth), which indicates that it possibly caused knowledge flow to more works (but those that need not be well cited as of now but can give rise to multiple pathways).

The FV index (Prabhakaran et al., 2015) can also reflect both depth (as it incorporates eigenvector centrality) and breadth (as it incorporates the indegree/number of citations received).

Semantic novelty (SN) indicator: Instead of similarity via cited references, Shibayama et al. (2021) considered semantic similarity between cited references where semantic similarity is computed via the cosine similarity of ‘word embedding vectors’ of a cited reference pair and the qth percentile of the vector for all pairs associated with a work. In addition, semantic novelty (SN) is the opposite of semantic similarity; papers whose references have less semantic similarity with each other are treated as more novel.

Integrated novelty-impact indicator: Yan and Fan (2024) devised an indicator NI that reflects both novelty and impact, wherein the NI is computed as the weighted sum of impact and novelty. Impact is computed on the basis of citation sentiment, and novelty is computed on the basis of semantic novelty (via outlier detection in word embedding analysis). Papers with high impact according to citation sentiment and time heterogeneity and high semantic novelty can be highly novel.

Thus, most of the indicators are effective and valid (validated using seemingly effective methods) but computationally intensive. For instance, the effectiveness of the FV index was demonstrated by Prabhakaran (2018) using dynamic empirical validation in multiple fields, and its applicability in even volatile fields was validated by Lathabai et al. (2024). Patent-based novelty indicators were validated on R&D prize award (by R&D magazine) data and the EPO patent database by Verhoeven et al. (2016). The predictive power of the FV gradient was validated using static and dynamic analyses by Lathabai et al. (2018). A convergent validity test by Bornmann et al. (2019a) revealed that U is better than W is. The effectiveness of disruptiveness indicator is validated using five different approaches, including one that involves Nobel Prize Winners’ articles. The modified disruptiveness indicator is validated using a convergent validity test with the F1000 prime by Bornmann et al. (2019b). The semantic novelty indicator is validated against self-reported novelty scores using a survey, and the integrated novelty-impact indicator is empirically validated in two fields in the respective introductory papers themselves. Thus, methods in the pre-AI era, especially network-based novelty/disruptiveness/paradigm shift indicators designed with profound methodical rigor, can be promising assets for IRF. However, a trend toward the adoption of advanced text analysis and NLP techniques for the determination of semantic novelty is obvious. This might signal a gold rush toward the profound and proactive adoption of LLMs or the GenAI, and the combination of pre-AI network-based indicators for novelty/disruptiveness/paradigm shift indicators in combination with AI-powered approaches can work well in the scientometric evaluation part of the IRF.

2.2. Peer Review and AI

• Peer review in the pre-AI era

Major factors that favor peer review as the preferred option in ex ante evaluation are (i) delays in the recognition/detection of novel works by scientometric/quantitative methods (Wang et al., 2017) and (ii) intimidatingly narrow tolerance of risk by funders, including public funders (Fedorov et al., 2010; Edwards et al., 2011; Petsko, 2011), associated with highly novel or offbeat research that deviates from mainstream research and the fear of going wrong (Boudreau et al., 2016; Lane et al., 2022). These often lead to suboptimal selections and nurturing of incremental research (Park et al., 2015; D. Li, 2017). Although several studies are on peer review for science funding, studies that consider the role of risk in the assessment of research proposals that lead to suboptimal selections are scarce. Through a linguistic analysis of review reports, van den Besselaar et al. (2018) revealed that peer reviews are more oriented toward the analysis of weak points in proposals than toward the assessment of novelty/innovativeness/high risk–high disruption potential. Linton (2016) suggested the combination of the Nobel Prize (in economics), winning the Black–Scholes model (Black & Scholes, 1973) and peer review, wherein the Black–Scholes model (adopted to the research assessment setting) provides a means to assess the relative attractiveness of projects (instead of financial value). The subjective expected utility (SEU) approach was recommended by Franzoni and Stephan (2023), in which peer reviewers provide values and probabilities for primary and secondary outcomes (along with verbal comments explaining the values) and compute a composite score as the sum of the product of value and probabilities for outcomes.

• AI developments and possible effects on peer review

The prevailing concern of the misuse of AI in scientific writing (which is also applicable to research proposal development) since the introduction of Scigen (an AI writing tool capable of creating nonsensical papers that intensified manifold introduction) in 2005 (Bohannon, 2015) intensified manifold because of the introduction of ChatGPT/GPT-3.5 (Stokel-Walker, 2022). AI tool usage in academic writing (also applicable for research proposal writing for funding) possibly enhances six areas (Khalifa & Albadawy, 2024), such as idea generation (L. Li et al., 2024), content structuring and literature synthesis, data management, editing and ethical compliance (Zairul et al., 2025). In the short term, advanced versions of some of the above-discussed AI-based research proposal generators, aiding tools or currently trending GenAIs, may become successful/accepted, making the traditional approach to peer review (especially those focused on weak points of the proposals) completely useless. AI detectors as a first-line of defence can be useful, but those are prone to serious false positives (than false negatives) (Dalalah & Dalalah, 2023). This urges proactive action to aid reviewers (both journals and proposals) with better tools and via the formulation of standards, procedures and guidelines for authors/candidates as well as reviewers.

Owing to the manifold increase in proposal submissions, the need to automate every process from initial proposal submission to the final decision will be felt sooner, making AI intervention inevitable. Munagandla et al. (2024) proposed AI-driven research proposal management systems that could considerably reduce administrative overheads, bottlenecks and delays arising from routine tasks such as document validation, reviewer assignment and tracking deadlines. More directly useful tools for reviewers are AI-based content analysis tools or summarizers, especially when a large volume of proposals need to be analyzed. LLM-based text summarization and content analysis (Malinen, 2024) and LLM-based functionalities such as Query Master and PDF summarizers (Agarwal & Behera, 2024) can be helpful but insufficient from a semantic point of view (Carius & Teixeira, 2024), revealing their insufficiency from a semantic point of view. Early versions of AI products integrated into Dimensions, Web of Science and Science Direct for content analysis/document summarization or field summarization, targeting both reviewers and researchers (Dimensions, 2024; Hanumanth, 2025; Elsevier Ltd., 2025), are also being developed. As these tools are trained on scientific publication data, they are supposed to perform better than other tools trained on internet sources (which are prone to fake or misinformation).

Thus, most of the developments in AI usage for peer review, especially for ex ante review of research proposals, are still in the nascent stage, and the current policies, measures and regulations are not generally in favor of the use of these tools for automatic content analysis, the generation of review reports, etc. In this crucial juncture, we propose the design of ‘Intelligent Review Framework’ or IRF. As already mentioned, before discussing it, its precursor, the IPRF introduced by Lathabai et al. (2023), needs to be discussed.

2.3. Performance Assessment at the Individual Level and a Revisit to IPRF Based on Relevance-Based Expertise Scores

The use of raw indicators such as IF, h and h-type indicators such as the h(2)-index (Kosmulski, 2006), g-index (Egghe, 2006), f and t indices (Tol, 2009), $h_m$(Schreiber, 2008), $h_{\alpha }$ (Hirsch, 2019), etc., has drawn criticism as individual assessment indicators. Expertise indicators inspired by h and g, such as x and x(g) (Lathabai et al., 2021), but reflecting core competency areas and potential core competency areas of institutions, can be used for individual-level assessment, as used in IPRF by Lathabai et al. (2023). The key recommended features of the IPRF are listed below:

• Two independent teams—(i) the first team for scientometric/quantitative analysis, which uses a scientometric framework for informing the second team about the overall performance of candidates concerning the fields/themes considered for the award in the form of ranks—and (ii) the second team (peer review committee) qualitatively evaluate the merit of each proposal, wherein the order of consideration of proposals is based on the ranks provided by the first team during scientometric assessment (last rank to first rank). Notably, while the first team assigns ranks on the basis of the performance of the candidates, scores/ranks are not taken into account for initial screening or shortlisting. The order is assigned to make shortlisting easier. The details of the decision framework used by the peer review committee are discussed in the third section.
• Scientometric framework (of the first team): Given the use of indicators such as the h-index or its variants, the IF is not suitable for the assessment of individuals for funding and related applications; indicators that are not too complex and sophisticated but are free from the limitations of both are necessary. Recently, Lathabai et al. (2021) introduced expertise indices such as x and x(g) (inspired by h and g indices) that help determine the core competent areas (top x thematic areas that have thematic strength ≥x, where thematic strength can be reflected by indicators such as citations, altmetric score, etc.) and potential core competent areas (x(g)-x areas below the x areas, where x(g) areas managed to gather average citations ≥x(g)) of an actor (such as institutions and individuals) and demonstrated the same results in the case of institutions. Although that framework uses keywords or important terms that are suitable for representing thematic areas (NLP techniques used as applicable), the relevance of publications that are to be mapped to thematic areas for the computation of thematic strengths was not considered. Relevance-incorporated expertise indices (wherein relevance scores come with concepts associated with publications available with the Dimensions database) were introduced to overcome this limitation and to reduce (if not eliminate) the need for NLP. Instead of the thematic strengths computed using the injection method, relevance-incorporated thematic strengths were proposed to be computed using the double injection method (Lathabai & Prabhakaran, 2023) for the computation of relevance-incorporated expertise indices. The relevance-incorporated expertise indices were defined as follows:

$x^{\left[R\right]}$-index: A scientist is supposed to have an $x^{\left[R\right]}$-index value of $x^{\left[R\right]}$ if he/she has published papers in at least $x^{\left[R\right]}$ thematic areas with relevance-incorporated thematic strengths exceeding $x^{\left[R\right]}$. These $x^{\left[R\right]}$ areas that form the $x^{\left[R\right]}$-core can be treated as the core competency areas of the scientist.

${x\left(g\right)}^{\left[R\right]}$-index: A scientist is supposed to have an ${x\left(g\right)}^{\left[R\right]}$-index value of ${x\left(g\right)}^{\left[R\right]}$ if he/she has published papers in at least ${x\left(g\right)}^{\left[R\right]}$ thematic areas such that the average relevance-incorporated thematic strength from these areas exceeds ${x\left(g\right)}^{\left[R\right]}$.

Once the relevance-incorporated expertise indices are computed after the publication profile of each candidate who submitted proposals is analyzed, the core competency and potential core competency areas can be determined as the top $x^{\left[R\right]}$ areas and the next ${x\left(g\right)}^{\left[R\right]}-x^{\left[R\right]}$ areas. This helps determine the composite score CS using the following formula:

$$CS=0.35 \alpha +0.15 \beta +0.35 \gamma + 0.15 \delta$$

(1)

where $\alpha$ and $\beta$ represent the level of agreement between the t = |T| thematic areas (the list of areas in which T is preferably decided and communicated to the scientometric team by the funding agency) considered for funding (possibly thrust/national priority areas) and that of the thematic areas that fall within the lists of core competency (represented by $T\_x^{\left[R\right]}$) and potential core competency (represented by $T\_{x\left(g\right)}^{\left[R\right]}/T\_x^{\left[R\right]})$ of the researcher, respectively. These can be computed as follows:

$$\alpha = {\displaystyle \frac{|T \cap T\_x^{\left[R\right]}| }{\left|T\right|}}= {\displaystyle \frac{|T \cap T\_x^{\left[R\right]} | }{t}}$$

(2)

$$\beta = {\displaystyle \frac{|T \cap (T\_{x\left(g\right)}^{\left[R\right]}/T\_x^{\left[R\right]}\left)\right| }{\left|T\right|}}= {\displaystyle \frac{|T \cap (T\_{x\left(g\right)}^{\left[R\right]}/T\_x^{\left[R\right]}\left)\right| }{t}}$$

(3)

$\gamma$ and $\delta$ are the normalized strengths of a candidate in their core competency areas and potential core competency areas, respectively. For a candidate with $\left|T \cap T_{x^{\left[R\right]}}\right|=$ n and |$T \cap (T\_{x\left(g\right)}^{\left[R\right]}/T\_x^{\left[R\right]})|$ = m, then

$$\gamma = {\displaystyle \frac{{\sum }_{i=1}^n{S}_i}{{\sum }_{k=1}^t{max.TS}_k}}$$

(4)

$$\delta = {\displaystyle \frac{{\sum }_{j=1}^m{V}_j}{{\sum }_{k=1}^t{max.TS}_k}}$$

(5)

$S_i$ and $V_i$ represent the relevance-incorporated strengths of the candidate concerning arbitrary core competent areas $i \in T \cap T\_x^{\left[R\right]}$ and arbitrary potential core competent areas $j\in T \cap (T\_{x\left(g\right)}^{\left[R\right]}/T\_x^{\left[R\right]})|$, respectively. ${max.TS}_k$ is highest/maximum strength possessed by a candidate (applied for funding) in area $k \in T$.

Specifically, $x^{\left[R\right]}$ (number of core competency areas) and ${x\left(g\right)}^{\left[R\right]}$ (number of core and potential core competency areas taken together) do not inherit the limitations of h and g from which those are inspired because they represent the number of areas of expertise and because of the relevance-based computation. Additionally, these are used to compute $\alpha$ and $\beta$ (Equations (2) and (3)); hence, these are not directly used in CS computation. $\alpha$ and $\beta$ take values between [0, 1], as these are obtained by normalization of the number of core competency areas and the number of potential core competency areas that are relevant for the call upon $t$ = |T|, which is the total number of areas relevant to the call. This normalization further eliminates almost all the possibilities of citation-related bias. A candidate can never have both $\alpha$ and $\beta$ equal to 1. Similarly, in the case of $\gamma$ and $\delta$, relevance incorporation for the computation of $S_i$ and $V_i$ and normalization of their sums, as shown in Equations (4) and (5), significantly reduces the effect of citation-induced bias. As $\gamma$ and $\delta$ (both lies between [0, 1]), for an applicant to have $\gamma$ = 1, he/she should be the one (among all the applicants) with the highest thematic strengths in all his/her core competent areas relevant to the call. Similarly, for the applicant, $\delta$ = 1 if he/she has the highest thematic strengths in all his/her potential core competency areas relevant to the call. A candidate can never have both $\gamma$ and $\delta$ equal to 1.

After the CS is computed, the ranking of the applicants (applications) is performed accordingly, and the applications are passed on to the peer review team in the determined order.

3.

Peer review procedure: The peer review team analyzes the proposals (in the specified order). The size of the short list (i.e., the number of candidates to be shortlisted and called for final presentation and interview) is determined in the following way. If there are total N awards (predetermined), and if there are M groups (according to age/career length or stage), the total number of shortlists will be y N, and shortlists from a group will be y N/M, where y can be 2, 3, etc., depending on N, and the time selection can board them to listen to presentation, seek clarifications, have discussions and/or interview the candidates. From each group, proposals are shortlisted on the basis of evaluation according to parameters decided by the panel/board. Among the parameters used to evaluate (or assign scores), novelty was recommended as of primordial importance (and to be assigned more weightage rather than treating all the parameters equally). Apart from the novelty of the first policy, the key highlight of the IPRF is the diligent effort to eliminate the influence of scientometric evaluation on the shortlisting decision because it addresses two possible scenarios that may arise during the peer review process. These are as follows:

Scenario 1: If all the proposals from the top (y N/M) candidates (according to CS scores) are more meritorious (merit of proposals decided by qualitative parameters) than the other proposals are, then the top (y N/M) proposals can be shortlisted in the first pass itself.

Scenario 2: If one or more of the top (y N/M) proposals are not up to the mark and some of the other proposals having lower CS scores appear to be better in terms of merit than they are, a second pass is required to decide the best among those, and if the proposals from lower CS-scored candidates are confirmed to be of better merit, those can be shortlisted. It can be suggested that such candidates themselves receive training and guidance in that respect and it can be considered to be provided if needed, which, in a way, contributes to capacity/expertise building exercise too.

Now, we are in a position to discuss the Intelligent Review Framework (IRF), which is a conceptual framework that also includes a computational framework. Specifically, the definitions and equations discussed in Section 3 are novel and are part of the computational framework of the IRF.

3. Intelligent Review Framework (IRF)

We propose the IRF on the basis of a comprehensive and diligent review of related literature (see Section 2) and different kinds of exposures, experiences and expertise of the authors. The prolonged exposure of two authors at both the receiving end as well as at various evaluation committees, the work experience of one author in a government establishment for STI policy development, our experience in the field of scientometrics and contributions that include various evaluative indicators, frameworks, etc., and our experience in scientometric analysis of AI/ML/LLMs, especially their reliability, ethical considerations, etc., and our empirical validation of the suitability of AI/ML/LLMs for decision-making in managerial/governance/SDG-related tasks were helpful in designing the IRF, a conceptual framework that also embeds a computational framework. The IRF has to be implemented by a permanent institutional structure that is part of the STI governance of nations. Nations are recommended to have a permanent institution viz. National Scientometric Task Force or NSTF (consisting of scientometricians, prominent members of the scientific community, HR professionals and developers proficient in AI/ML). Prominent members of the scientific community will have advisory roles unless they are selected as peer reviewers upon scientometric evaluation against a call. The tenure of the peer review panel ends when all the awardees submit the final project report. If the prominent member of the scientific community agrees to be a peer reviewer, he or she will not be a part of the advisory board until the candidates are awarded. After the declaration of awards, he/she will also be excluded from key discussions/meetings/reviews during the progress of the awarded projects in the capacity of advisory members. However, he/she has to attend those in the capacity as peer reviewers whenever possible. In the case of multiple calls from the same discipline with sufficient thematic overlap, an ad hoc scientometric team will be directed to prepare a wider list of possible experts to be invited for the panels to reduce the number of members engaged in multiple panels as much as possible to reduce the possibility of elite capture in NSTF and IRF. Of course, all these invoke new challenges in some disciplines where some of the experts might be in early career stages/not hold permanent positions, which is one of the legal requirements to be appointed as a panel member in many countries. Both teams are under the oversight of the NSTF; if there is any confusion regarding scientometric evaluation results that leads to the recommendation of peer reviewers or any general issues (common in any workplace) or issues specific to a particular call in the IRF, or if there is a need for more support and facilities, the NSTF (senior scientometricians in consultation with advisory board members concerning the field, HR officials) has to intervene and resolve such issues, and future training programs can be updated by flagging such issues, along with strategies to avoid/overcome them.

The implementation of the IRF requires two independently working teams, as in the case of the IPRF. The first team, i.e., the scientometric team, has a greater operational and analytical role and has more responsibility for supporting the peer review team. NSTF is supposed to (i) carry out scientometric research and development activities related to indicator assessment, field assessment, individual assessment, etc., with special emphasis on applications of AI/ML techniques for all these (including the development and management of relevant AI-powered information systems, databases and knowledge bases); (ii) watch, track and predict the dynamics of all disciplines and subfields, including current and potential national priority areas, to provide intelligence (higher gravity than evidence-based information) for STI policy making and governance; (iii) recruit/train scientometricians; (iv) track, determine and review ethical considerations related to AI/ML usage in general and specifically in research practices and STI governance along the technological advancement and evolution of AI/ML; and (v) form/depute ad hoc teams for aiding funding agencies with IRF activities. Thus, the function (v) of the NSTF is more crucial for the scientometric component of the IRF. In regard to the peer review part of the IRF, unlike the case of the IPRF and other existing systems, peer reviewers are selected from a recommended list prepared by the scientometric team on the basis of an evaluation of expertise using state-of-the-art methods to eliminate the effects of elite capture and cronyism. Additionally, peer reviewers are blind to candidates’ details (making peer review a double-blind process) to eliminate any chances of favoritism. These measures also eliminate the possibility of gender discrimination. NSTF might provide the service of discourse analysts for aiding peer reviewers in the qualitative analysis of proposals on request.

The operation of the IRF in relation to a call for a proposal spans four phases: (1) Ideation of calls for proposals to receive proposals, (2) determination of experts and formulation of the panel for peer review, (3) computation of composite scores and (4) peer review and shortlisting for interviews.

3.1. Phase-1

When funders ideate the funding program and the call for a proposal, they can consult the NSTF, who will constitute an ad hoc team (to be headed by senior NSTF officials) to conduct research on the literature and provide feedback on which traditional and interdisciplinary fields are to be considered (Figure 1). Specifically, the NSTF is supposed to continuously monitor, track and predict development in all disciplines and subfields, including national priority areas (see item (ii) of functions of the NSTF). For this purpose, they might consider vital general properties (Coccia, 2018) and the sociolinguistic and discipline genre properties behind the evolution of fields (Stichweh, 2001). In the initial phase of the IRF roadmap, the use of AI/ML/LLM is not recommended. However, in later phases, they may consider the use of computational sociolinguistic and discipline genre analysis with the aid of proven AI/ML/LLM tools. Specifically, according to function (i) of the NSTF, its R&D activities also cover the sociolinguistic and discipline genre aspects. The scientometric team also determines the thematic areas associated with the fields using state-of-the-art methods and tools. Currently, topic modeling techniques such as LDA and transformer-based techniques such as BERTopic and SciBERT can be used, or any suitable LLMs can possibly be used. Once the fields and thematic areas are considered, the aspects to be considered for research proposal evaluation (at phase 4) according to the fields determined as relevant for the call and level of emphasis on each aspect (other than novelty) demanded by such fields are communicated to the funders/regulators. If the funders/regulators are satisfied, they may initiate the call. By using their dedicated project/proposal management systems, if any, funders can receive applications. Once the call deadline ends, funders have to transfer the applicant information and proposals to the NSTF, and they can form a master database (MDB) to store the information organized conveniently with the help of the development team to work further on that information in later stages.

3.2. Phase-2

The scientometric team constituted by the NSTF performs detailed field analysis and thematic area analysis to determine novel, disruptive or paradigm-shift-related works and patents (using relevant networks such as paper citation networks, patent citation networks, and other derived networks to compute the indicators discussed in Section 2.1) and performs content analysis of the same manually or via AI-based tools (Figure 2). Until effective tools are developed by the development team for content analysis, those already developed by Clarivate Analytics (Hanumanth, 2025) or so can be used. The team should prioritize the development of a single tool/package to determine all the novelty scores given in Section 2.1, because multiple tools/packages are currently required to compute them. Unless and until the most effective novelty, disruptiveness or paradigm shift indicators are in place, the following procedure can be used.

Procedure for classification of works into different classes of novelty, disruptiveness or pivotal roles in a paradigm shift using an AI-powered scientometric approach

The rationale of this procedure is that papers/patents that are picked up by multiple novelty, disruptiveness and paradigm shift indicators can be treated as more important in the fields/themes and can be marked as high novelty/disruptive/paradigm shift papers/patents. Thus, depending on whether more indicators are highlighted or not, different grades/classes can be present for papers/patents in a field, which can be assigned by a clustering exercise. The steps of the procedure are given below.

• Computation of scores: For all works/patents in a field or thematic area, compute scores for novelty, disruptiveness or FV gradient scores.
• Unidirectionality of indication should be ensured: By unidirectionality, uniformity in indication characteristics should be present. For instance, if a high value of some scores indicates high novelty, disruptiveness and a paradigm shift to a pivotal role, low values of others can indicate that. All scores should be transformed to ensure a unidirectional indication (preferably a high score indicating more novelty, disruptiveness and pivotal role in a paradigm shift). Additionally, some indicators, such as in Verhoeven et al. (2016), do not provide a score but attempt to classify patents. A scientometric team should develop an indicator that works according to the principle but provides a quantitative score and adapts it to suit the assessment of papers. For instance, an expression such as the one below can be considered for papers.

$$NS=a NR\prime +b NSO\prime +c NTO\prime$$

(6)

$NR\prime =$ Number of new/first occurrences of Journal or WoS classification.

$NSO\prime =$ Number of new journal combinations or WoS classifications via papers associated with such journals or WoS classifications.

$NTO\prime =$ Number of connections between a WOS category/Journal and IPC that did not exist before.

The following values can be taken for a, b and c: a = 0.335, b = 0.335 and c = 0.33.

Similarly, the FV gradient or ${∆FV}_{ij}$ (Lathabai et al., 2015) is a score assigned for links in paper citation networks or patent citation networks, not vertices (that represent papers/patents). Therefore, the following transformation can be considered. First, all the links have to be ranked after being sorted in descending order. As links associated with (i) detected pivots have negative scores and (ii) predicted pivots have positive scores close to zero, the ranks of such links will be very high and high, respectively. The ranks of the links can be provided as scores to the sources (citing papers). Then, the scores of the detected pivots (sources of negative FV gradient links) will be extremely high, the scores of the predicted pivots (sources of links with an FV gradient close to zero) will be high, and the remaining papers will have lower scores than both of these categories. Thus, the FV gradient score or FVG of the citing paper (or source of a link) can be determined as follows:

$${FVG}_i={r(∆FV}_{ij})$$

(7)

where ${∆FV}_{ij}$ is the FV gradient weight of link $l_{ij}$ (from i to j).

3.

Normalization (optional): Nevertheless, problems can arise from some indicators (in Section 2.1) taking scores between 0 and 1 while others are not. Normalization options for others can be considered if needed.

For instance, ${FVG}_i$ can be easily normalized to be contained in ]0,1] by using

$${FVG\left(norm\right)}_i={\displaystyle \frac{{r(∆FV}_{ij})}{max\left\{R\right\}}}$$

where R is the set of ranks of the FV gradient values of all the links.

4.

Dimensionality reduction: To determine the most influencing indicators or avoid the less relevant ones, techniques such as principal component analysis or others can be used.

5.

Clustering of papers/patents according to these scores: Appropriate unsupervised clustering techniques such as k-means or more advanced ML algorithms can be used to cluster papers according to the scores of indicators selected after step 4.

6.

Cluster examination: Clusters having papers/patents scoring high for all indicators, clusters having papers scoring high for most of the indicators, clusters having papers scoring high for only some of the indicators, etc., can be determined. Papers that are found to score high for multiple novelty/disruptiveness/paradigm shift indicators (found most important clusters) can be selected for detailed content analysis.

7.

Content analysis: Content analysis of all the papers/patents in most important clusters can be carried out both manually and using AI-aided tools. Thematic analysis using BERTopic, SciBERT, etc., can also be performed. For other clusters, only recent works/patents need to be analyzed rigorously. The specific contributions of all the selected papers and how those advanced the applicable field/fields papers need to listed out (in an appropriate format) by the ad hoc scientometric team. Upon verification, if satisfactory, these results can be stored in another ad hoc database, ADB1. This information needs to be accessed by peer reviewers in phase-4 as this can be crucial to them because it can serve as a kind of benchmark of state-of-the-art against which novelty in proposals needs to be evaluated.

Specifically, in steps 1, 4, 5, 6 and 7 of the 7 step-procedure, AI/ML methods and techniques are applicable. For these steps, robustness can be ensured by consistency checks, i.e., by rerunning algorithms on different parameter settings as applicable and ensuring results are consistent or do not vary significantly. For instance, robustness of the clustering algorithms can be ensured by rerunning at different parameter settings and analyzing consistencies in terms of number of clusters, cluster memberships, inter cluster distances and other clustering quality indicators.

Specifically, the implementation challenges associated with the 7-step procedure may vary across STEMM fields and can be more challenging in non-STEM fields. Fields in which citation practices that determine various network structures (such as citation and cocitation) are more robust can be less challenging. In such fields, the procedures, especially cluster examination and content analysis, should be performed more cautiously, and in case of doubts, senior scientometricians in NSTF can be approached.

Second, the important authors or contributors within the fields or thematic areas can also be determined. As a rule of thumb, we recommend the use of CS scores introduced by Lathabai et al. (2023) for IPRF to be used instead of the most productive, most cited or highest h-index authors from the field or themes. Other preferable options are the use of the x-index (Lathabai et al., 2021) alone or the ${\Psi }_c$-index (Lathabai & Prabhakaran, 2023). The experts thus identified will be communicated to the funders. Funders can contact the experts on the list and invite them. Details of those who accept or express willingness to serve as peer reviewers in the panel can be shared with an ad hoc team, and the information will be stored in the ad hoc database ADB2.

The NSTF will update the National Expert Network database (if there is any), which is not an ad hoc database such as the databases created thus far but is to be created and managed by the NSTF as part of the ongoing STI data governance activities of STI governance.

3.3. Phase-3

In this stage, the scientometric team has to extract the applicant’s details, including their publication information, from the master database MDB (created during phase 1). Once the details are extracted, the publication profile of the actor must be created (Figure 3). This can be accomplished through an API search in the Dimensions database, which provides the concepts associated with publications and their relevance scores and the scores for impact attributes such as citations or Altmetric attention scores. Once the profiles are created, the relevance-incorporated thematic strengths of the applicant can be computed according to the double injection procedure for IPRF given by Lathabai et al. (2023). This can aid in the determination of the $x^{\left[R\right]}$ and ${x\left(g\right)}^{\left[R\right]}$ indices, which, in turn, can be used to compute CS scores. Similarly, a suitable patent database such as USPTO or the ones in which candidates have obtained patents should be considered to extract inventive profiles of the applicants, and some intelligent topic modeling schemes can be employed to determine terms and their relevance for patents, and the same procedure given by Lathabai et al. (2023) can be used to determine composite scores or CS scores related to patents. Notably, both patents and papers need to be considered if the call is open for both industry and academia alike and if applications are also received from scientists and inventors almost alike. To execute the novelty-first policy, the IRF additionally examines and takes into account whether the applicants already contributed any novel, disruptive or pivotal papers on paradigm shifts and such patents. A search in ADB1 (created during phase 2) is required for this.

The number of novel, disruptive and paradigm shift pivotal role patents and papers can be used to compute the improved composite score CS, which also emphasizes ‘novelty’, unlike the original CS used in IPRF. CS’ is expressed as follows:

$$CS\prime =0.25 CS+ 0.25 CS\left(pat\right)+{0.25 \eta }_P+ 0.25 {\eta }_{Pat}$$

(8)

$${\eta }_P={\displaystyle \frac{{NDPS}_P}{{TNDPS}_P}}$$

(9)

$${\eta }_{Pat}={\displaystyle \frac{{NDPS}_{Pat}}{{TNDPS}_{Pat}}}$$

(10)

where ${NDPS}_P$ and ${NDPS}_{Pat}$ are the numbers of novel/disruptive/paradigm-shift pivotal papers and patents, respectively, whereas ${TNDPS}_P$ and ${TNDPS}_{Pat}$ are the total numbers of novel/disruptive/paradigm-shift pivotal papers and patents, respectively, found in the analyzed fields or thematic areas considered for the funding program. Notably, in the IRF, as the weight of the CS (used in the IPRF) is 0.25, the weight of the citation-dependent terms α, β, γ and δ are reduced to their one-fourth values, suggesting the elimination of any residual effect of citation bias. Specifically, in Equation (8), the weight given to novelty terms is 0.5 (0.25 each for their proportionate contribution to the total number of novel papers and patents in fields/themes related to the call). This emphasis on novelty in scientometric evaluation is the first crucial step in ensuring ‘novelty first policy’. Applicants’ CS scores, along with the scores that helped their computation, are stored in another ad hoc database, ADB3.

3.4. Phase-4

In the previous stages, the proposals were not analyzed. In this stage, the proposals saved in the MDB have to be extracted by the development team. This will then undergo thorough technical checks, including checks for AI-generated content (Figure 4). This is extremely important until and unless a robust AI-powered proposal management system that includes research integrity systems or FoSci systems that can ensure the screening of proposals according to research integrity guidelines stipulated by the emerging discipline of forensic scientometrics (McIntosh & Vitale, 2024) is used during the submission time itself. If a scientometric team finds any technical as well as ethical or research integrity issues, including AI-generated content, they can contact the NSTF for verification and confirmation, and if they also find those suspicious, the scientometric team can advise funders to either seek clarifications from applicants (if issues are seemingly resolvable in a fairly quick manner and proposals can be resubmitted quickly) or reject those proposals on technical grounds (if issues are too great to be resolved). Now, the proposals cleared by the AI detector and technical issues checker and those resubmitted proposals with resolved issues will be considered eligible proposals. Eligible proposals are then sent to both the peer review panel and the AI peer review simulator.

Evaluation by an AI peer review simulator: The AI peer review simulator is not merely an AI peer review report generator. The simulator is supposed to be equivalent to human subject experts or peer reviewers in terms of knowledge. If not now, these simulators will soon be a reality. Currently, explorations of simulating Delphi studies using AI tools that are supposed to have human expert equivalent subject knowledge, and hence, can serve as Delphi panels, are ongoing. However, the perfect version of such tools will take many more years to be realized. Until such simulators are available, the best AI peer-review report generators can be used. The simulator/report generator is also asked to provide scores against the same evaluation criteria prescribed to the human peer review panel and a detailed report in the format given to its human counterpart. Details of the format and suggestive criteria given to peer reviewers are discussed in the following subsection. One part (the incorrect statements in AI-generated summaries) will be absent in the format given to the AI simulator. All the eligible proposals and their summaries, reports and scores generated by the AI peer review simulator/report generator will be saved in a knowledge base KB by the NSTF. Proposals and their summaries (from the KB), field analysis summaries and novel contribution details in the field (from ADB1) will be passed on to the peer review panel.

Evaluation by Peer Reviewers: Now, the peer review panel has to follow the procedure given below:

• Quickly review the AI-generated summaries. Note the contributions and novel elements highlighted in those reports.
• Thoroughly read the proposals. Analyze the proposals according to the instructions given to them (by the funders, scientific experts, technical experts and government/regulatory bodies in consultation with the NSTF).
• Assign scores against various prescribed aspects/heads (also agreed upon by funders and regulators in consultation with the NSTF). If it is a call related to a specific discipline, common basic aspects such as feasibility (technical as well as financial) methodical rigor and novelty should be evaluated at the level of emphasis demanded by the discipline. Ethical considerations, a major aspect that exhibits stark variation across disciplines, should be evaluated as stipulated by relevant international as well as regional discipline-specific ethical guidelines and procedures, and if there is a dedicated board/body/committee, adjudication can be sought if needed. Apart from this, any unique aspect relevant to that discipline alone is applicable and should also be evaluated. In the case of thrust/priority areas, all the above general aspects and aspects that become relevant only during transdisciplinary research assessment, along with unique aspects relevant only to a particular thrust/priority area, should be evaluated. If existing ethical frameworks/committees are not competent to assess the special ethical considerations demanded by the thrust/priority area, ad hoc committees (that can later be merged into the national research ethics and integrity framework) need to be formed, and their performance should be ensured in a fast-track mode. In particular, note that scores are given for different criteria and that peer reviewers need not compute an aggregate score.
• While scores are assigned against novelty criteria, the field analysis summary and novel contribution details provided to them from ad hoc database ADB1 needs to be checked. It should be examined whether the proposals have theoretical, methodological and applicational novelty compared with the existing novel/disruptive/paradigm shift pivotal papers or patents and/or might cause disruptions or paradigm shifts if the projects are executed successfully. This is the decisive step in the peer review process to ensure ‘novelty first policy’.
• Prepare a detailed report in a prescribed format that comprises (i) a summary to be written by peer reviewers, (ii) the list/details of any incorrect statements made by AI-generated reports and why those are incorrect and (iii) remarks and recommendations about the proposal.

The peer reviewers’ reports, scores and recommendations will also be stored in the KB. This will enable comparison of the (i) AI proposal summaries by human reviewers vs. the AI peer review simulator and (ii) scores given by human reviewers vs. the AI peer review simulator, which may lead to eventual improvement of the AI peer review simulator with the help of AI-powered knowledge management techniques and, hence, is vital for the roadmap of IRF. On the basis of the evaluation score similarity and textual report similarity, we envision the evolutionary roadmap of the IRF (see Section 4).

Now, both the CS′ scores (AI-powered scientometric evaluation) and the scores given by peer review (AI-powered peer review) against various criteria are available in ADB2 and KB, respectively, and can be used for shortlisting decisions. As both evaluations can be as efficient as possible (although they might not be absolutely perfect), the final score for proposals that might lead to shortlisting should be computed in the following way:

$$FS=\theta C{S}^{\prime }\left(rsc\right)+\varnothing peer review score$$

(11)

where $\theta +\varnothing$ = 1. To ensure peer review is given more importance, it is recommended to keep $\theta$ well below 0.5 (50%). Furthermore, $CS\prime \left(rsc\right)$ denotes rescaled CS′ scores. The rescaling is to ensure scale computability with peer review score. This is because, CS′ (computed by Equation (8)) lies in ]0,1], and if peer review scores are assigned in ]0,100], CS′ needs to be scaled up. Thus, Equation (11) becomes

$$FS=\theta (100\times C{S}^{\prime })+\varnothing peer review score$$

(12)

Peer review score encompasses scores for novelty and scores for technical and financial feasibility (indicating the risks involved in the proposal) and other aspects. Aspects apart from novelty can be taken together and denoted as aggregate score, then Equation (11) becomes

$$FS=\theta C{S}^{\prime }\left(rsc\right)+\varnothing (\mu novelty score+\pi aggregate score)$$

(13)

where $\mu + \pi$ = 1.

Let $\theta =0.4$ (and $\varnothing$ = 0.6) to ensure 60% weightage to peer review. Now, to ensure novelty first policy, let $\mu$ = 0.6, which makes $\pi =0.4$, then Equation (13) becomes

$$FS=0.4 C{S}^{\prime }\left(rsc\right)+0.36 novelty score+0.24 aggregate score$$

(14)

In Equation (14), for the term $0.4C{S}^{\prime }\left(rsc\right)$, 50% is for novelty (see Equation (8)). This makes overall 20% allotment for novelty determined by scientometric/quantitative evaluation. As 36% of the overall score is dedicated to the novelty score (qualitatively given by peer reviewers), the overall representation of the novelty aspect in FS is 56%, ensuring the ‘novelty first policy’. This, of course, is suggestive and can be regarded as a rule of thumb. As novelty is in trade off with risk (due to technical and economic feasibility), weightage to novelty (and hence, aggregate scores) needs to be finalized in terms of how much risk tolerance is affordable for the particular call on discussion among funders, regulators and scientific and technical experts in the country in consultation with the NSTF. A minimalistic demonstration (please see Section 3.5) on synthetic data might make this point clear.

Once FS is computed for all the applications, as in IPRF, applications have to be classified into M groups according to effective age or stage of career. If the funders are willing to award N proposals and the top level of NSTF and funders and peer reviewers are willing to hear presentations from and interview the top y N applicants to make a final decision on the award (where y can be 2, 3, etc.), then from each group, the top (y N/M) applicants will be selected according to their FS score.

3.5. Demonstration of the Computational Framework Using Synthetic Data of 7 Applicants of Same Effective Age/Career Length

Let a call for proposals be associated with five subfields/focus themes denoted by FT1, FT2, … and FT5. By the novelty computation process at phase-3, let the number of novel papers determined for five subfields/focus themes be 100, 125, 95, 65 and 150, respectively, making the total number of novel papers to be considered for the call = 535.

Now let us consider 7 applicants whose essential details required for scientometric part (extracted and computed by scientometric team) are given in

Table 1. Details (essential for computation of CS′) of 7 applicants.

	TF1	TF2	TF3	TF4	TF5
A	(C, 10, 500, 3)	---	(C, 15, 750, 4)	---	(P, 5, 100,1)
B	(C, 15, 600, 2)	(P, 10, 275, 2)	---	(C, 25, 950, 3)	---
C	(P, 9, 110, 1)	---	(C, 30, 2000, 6)	(P, 10, 100,1)	---
D	---	(C, 25, 1350, 4)	(P, 7, 300, 2)	(P, 12, 210, 2)	----
E	(P, 5, 90, 0)	(C, 35, 1950, 5)	---	---	(C, 15, 400, 1)
F	---	(P, 8, 350, 0)	---	----	(C, 40, 2750, 7)
G	----	----	(P, 18, 350, 3)	(C, 25, 650, 0)	(P, 7, 400, 2)

in quadruple format. Let us assume that no applicant has obtained a patent in relevant subfields/themes and evaluation is focused on publication data only. The items from left to right indicate (i) whether the subfield/theme is one of the core competency subfield/theme (indicated by letter C) or one of the potential core competency subfield/theme (denoted by P) of an applicant, (ii) number of publications by the applicant in that particular subfield/theme, (iii) citations received by the publications by the applicant in that particular subfield/theme and (iv) number of novel/disruptive/paradigm shift papers by the applicant in that particular subfield/theme.

Using Equations (1)–(5), CS for all 7 applicants can be computed. This is given in

Table 2. CS computation of 7 applicants.

	α	β	$\mathit{\gamma }$	$\mathit{\delta }$	CS
A	2/5 = 0.4	1/5 = 0.2	(500 + 750)/8200 = 0.152	100/8200 = 0.012	0.225
B	2/5 = 0.4	1/5 = 0.2	(600 + 950)/8200 = 0.189	275/8200 = 0.034	0.241
C	1/5 = 0.2	2/5 = 0.4	2000/8200 = 0.244	(110 + 100)/8200 = 0.026	0.219
D	1/5 = 0.2	2/5 = 0.4	1350/8200 = 0.165	(300 + 210)/8200 = 0.062	0.197
E	2/5 = 0.4	1/5 = 0.2	(1950 + 400)/8200 = 0.287	90/8200 = 0.011	0.272
F	1/5 = 0.2	1/5 = 0.2	2750/8200 = 0.335	350/8200 = 0.043	0.224
G	1/5 = 0.2	2/5 = 0.4	650/8200 = 0.079	(350 + 400)/8200 = 0.091	0.171

. Specifically note that t = 5 (as there are 5 subfields/themes) is used for computation of α and β. For, γ and δ, ${\sum }_{k=1}^t {max.TS}_k$ needs to be computed. From Table 1, ${max.TS}_{TF1}$ = 600, ${max.TS}_{TF2}$ = 1950, ${max.TS}_{TF3}$ = 2000, ${max.TS}_{TF4}$ = 900 and ${max.TS}_{TF5}$ = 2750, making ${\sum }_{k=1}^t {max.TS}_k$ = 8200.

Now, using the information of total number of novel papers by applicants (by summing up the number of novel papers by applicants in all applicable subfields/themes given in Table 1) and total number of novel papers considered for the call, CS′ can be computed as given in

Table 3. CS′ and FS computation of 7 applicants (with 56% overall weightage to novelty).

	Total Number of Novel/Disruptive/ Paradigm Shift Papers	${\mathit{\eta }}_{\mathit{P}}$	CS′	CS′ (rsc)	Novelty Score	Aggregate Score	FS
A	8	8/535 = 0.015	0.0600	6	80	100	55.201362
B	7	7/835 = 0.013	0.0636	6.36	75	100	53.5427314
C	8	8/535 = 0.015	0.0585	5.85	85	100	56.9416059
D	8	8/535 = 0.015	0.0530	5.3	78	100	54.1990449
E	6	6/535 = 0.011	0.0708	7.08	70	100	52.0316617
F	6	6/535 = 0.011	0.0592	5.92	82	100	55.888646
G	5	5/535 = 0.009	0.0452	4.52	65	100	49.2080921

.

In stage-4, after examining proposals and comparing them against 535 novel papers from the 5 applicable subfields/themes, peer reviewers assign novelty scores. If novelty and aggregate scores for the submitted proposals by applicants are awarded as given in Table 3, FS scores can be computed. The case of FS score computation using overall novelty score weightage of 56% (weightage of novelty score on peer review 36%) is also given in Table 3, if no risk is identified with any of the proposals (aggregate scores = 100).

Most novel proposals are found to have high FS scores, and if three out of 7 can only be shortlisted, C, F and A (in this particular order) will make it into the shortlist (please see bold font in Table 3).

Now let us examine various scenarios for a range of overall novelty weightage 0.25–0.75, wherein the proposals by applicants who received top scores for novelties (in peer review) are associated with varied levels of risks. No-risk scenario is characterized by aggregate score of 100, low-risk scenario is characterized by 90 < aggregate score < 100, moderate-risk scenario is characterized by 75 < aggregate score ≤ 90, and high-risk scenario is characterized by aggregate score < 75. From all scenarios (except no-risk scenario), one specific representative aggregate score is chosen for computing FS scores.

No-risk scenario: FS scores for applicants with novelty weightage range 0.25–0.75 in no-risk scenario (aggregate score = 100) are shown in Figure 5. This indicates that if there are no issues/risks related to technical and economic feasibilities of the proposals, most novel proposals will always make it onto the shortlist. However, distinction from top three and others become clearer when overall novelty weightage (${\displaystyle \frac{\theta }{2}}+\varnothing \mu )$ > 0.35.

Low-risk scenario: In low-risk scenario, let C, F and A (applicants with top novel proposals) be awarded aggregate score = 91. Then, FS scores for applicants with novelty weightage range 0.25–0.75 are shown in Figure 6. This indicates that effect of novelty weightage (emphasis) starts only when overall weightage (${\displaystyle \frac{\theta }{2}}+\varnothing \mu$) > crosses 0.44 (as C becomes eligible for shortlisting). At 0.6 (60% overall weightage), F also becomes eligible for shortlisting, increasing complexity of decision-making by strengthening the need to carefully address novelty–risk tradeoff. Thus, the range 0.44–0.6 is chosen as ‘emphasis zone’, the ranges prior to and beyond the ‘emphasis zone’ are marked as ‘under emphasis zone’ and ‘over emphasis zone’, respectively.

Moderate-risk scenario: In moderate-risk scenario, let C, F and A (applicants with top novel proposals) be awarded aggregate score = 76. Then, FS scores for applicants with novelty weightage range 0.25–0.75 are shown in Figure 7. This indicates that effect of novelty weightage (emphasis) starts only when overall weightage (${\displaystyle \frac{\theta }{2}}+\varnothing \mu$) > crosses 0.58 (as C becomes eligible for shortlisting). At 0.67 (67% overall weightage), F also becomes eligible for shortlisting, increasing complexity of decision-making by strengthening the need to carefully address novelty–risk tradeoff. Specifically note that when proposals are becoming riskier, novelty emphasis has to increase by increasing the risk tolerance level. Furthermore, the emphasis zone (0.58–0.67) is narrow compared to the low-risk scenario, indicating the importance of fixing the overall weightage for novelty increases when proposals’ risks grow.

High-risk scenario: In high-risk scenario, let C, F and A (applicants with top novel proposals) be awarded aggregate score = 50. Then, FS scores for applicants with novelty weightage range 0.25–0.75 are shown in Figure 8. This indicates that effect of novelty weightage (emphasis) starts only when overall weightage (${\displaystyle \frac{\theta }{2}}+\varnothing \mu )$ > crosses 0.67 (as C becomes eligible for shortlisting). At 0.75 (75% overall weightage), F also becomes eligible for shortlisting, increasing complexity of decision-making by strengthening the need to carefully address novelty–risk tradeoff. Specifically note that ‘emphasis zone’ shifted further to 0.58–0.75 and is narrowest in high-risk scenario.

The above analysis used synthetic data for 7 applicants and one set of novelty scores (given in Table 3). How the aggregate score variation among top novel applicants in different scenarios might have impacted FS is not covered. A series of simulation studies and sensitivity analyses on many large synthetic and real-world datasets (of awarded grants in custody of funders) are required for a holistic understanding prior to implementation. Despite these limitations, the current analysis clearly indicates that it is not possible to fix the same weightage to all fields as different levels of risks are associated in different fields and demand different levels of risk tolerance. Even for same field, two different calls involving different set of subfields/themes require careful evaluation of risks, and affordable risk tolerance should be fixed by extensive and inclusive stakeholder discussion before determining the overall weightage of novelty score to ensure the right level of emphasis on novelty. Transdisciplinary fields/national priority areas cannot grow with low risk tolerance and at least moderate risk tolerance (in some cases high risk tolerance is required). The choice of 56% weightage might be useful to ensure that novelty is emphasized with low and moderate risk tolerance.

4. Discussion of the Evolutionary Roadmap of the IRF

The current study introduced a conceptual framework, namely, the IRF, for the evaluation of research proposals at the individual level. Its novelty lies in the following facts: (i) it integrates a means to realize ‘novelty first policy’ in the ex ante evaluation of research proposals, and (ii) it introduces an effective integration of AI-based peer review (in peer review) and AI-powered scientometric assessment. Owing to its sophistication and scale, it cannot be demonstrated and validated in a pre-implementation manner. As the IRF relies on the ‘benefit of scale’ and requires the establishment of the NSTF or a similar functional body, which is a matter of national STI governance and policy, we can recommend only national STI policymakers and other concerned ministerial/governance offices to carefully analyze our proposed conceptual framework and critically evaluate the ‘benefit of scale’ against general, field-specific and country-specific challenges.

Upon cost/challenge–benefit analysis, if the establishment of NSTF (or the alteration of existing similar systems to meet requirements) and IRF is found to be feasible, keeping up with the pace of AI development and filtering out the best ones according to parameters such as performance, safety and reliability is a major challenge. This is vital because while the capabilities of AI/ML/LLMs are improving at a greater pace, reliability improvement is not keeping up, and in some applications, reliability decline (Allen & Peterson, 2026; Kübler et al., 2026) is also witnessed. As per the METR (Model Evaluation and Threat Research), the number of tasks that AI agents/models can perform (at a 50% success rate) doubles every 7 months (Kwa et al., 2025), and this result is beginning to become popular in some circles as Moore’s law for AI. However, for the functions that are to be carried out by the NSTF, including its IRF-related functions, a slower growth rate and a much slower adoption rate are anticipated, as a 50% success rate is not affordable in matters related to national policy. Apart from this, data security and data governance in the AI era (Sharma & Sharma, 2025) in general and, specifically, STI governance applications and ethical considerations of AI/ML/LLMs and the field-dependent and country-dependent variations in those and the evolution of these factors are also decisive in determining the participatory evolution of AI in IRF, especially for AI-powered peer review of proposals.

The NSTF’s scientometric staff (not the ad hoc scientometric team in IRF) should continuously track the developments in AI/LLMs related not only to performance but also reliability over time. By doing so, NSTF will be able to prescribe the most reliable and performing model among the available options to the ad hoc team. Additionally, in due course, if some efficient tools to detect hallucinations are developed, those should be adopted. Manual checking of the reports generated by the AI-peer review simulator for hallucinations and misinterpretations/out of context interpretations by the ad hoc scientometric team who operates the reports generated by AI-peer review simulators is mandatory before the summary of the proposals (not the scores and evaluation reports) generated by the AI-peer review simulator is handed over to the peer reviewers. Incorporation of AI summarization modules managed by or associated with scholarly databases like WoS, Scopus and Dimensions for retrieval-augmented generation (Lewis et al., 2020) can also improve reliability by reducing hallucinations. Using prompt engineering techniques like chain of thoughts, negative constraint prompting that prohibits the model from making up information, etc., can also be used to address hallucination issues for reliability improvement. Once the summaries are handed over to peer reviewers, along with the review of the proposal, they need to highlight any incorrect statements/claims in the summary generated by AI-peer review simulator, which is a second layer reliability improvement process. Owing to the importance of IRF in STI governance, as recommended by Banerjee et al. (2025), permanent human oversight should be there to ensure reliability, as some researchers argue that inherent hallucinations in LLMs will not disappear completely.

In the case of accuracy of peer review (i.e., the AI-powered peer review simulator), it can be tracked by the NSTF by examining numeric as well as text similarity/agreement scores computed at phase 4 (stored in KB2). Until both scores cross 0.5, only human evaluation can be considered valid. After 0.5 crossing (50% success rate), the AI evaluation can be considered to complement the human evaluation with strict human supervision. In that case, human evaluation should be given more weight than AI evaluation for the computation of FS, as given in Equation (15).

$$\begin{gathered} FS=0.4 C{S}^{\prime }+0.24 novelty score \left(H\right)+0.12 novelty score \left(AI\right)+ \\ 0.16 aggregate of other scores \left(H\right)+0.8 aggregate of other scores \left(AI\right) \end{gathered}$$

(15)

In Equation (15), among the peer review scores, scores from human evaluation are given more weight (66.66%), and the weight of AI peer review is approximately 33.33% to ensure caution against the potential limitations of AI peer review. However, the overall emphasis on novelty-first policy is ensured in Equation (12) because the overall weight of novelty-related terms is maintained at 56%.

Suitable thresholds can be set (e.g., 0.95 for numerical agreement and 0.85 for text agreement), beyond which human supervision can be minimized, and AI scores can also be considered with equal weight for shortlisting. Then, the FS can possibly be computed with equal weight to that of the human and AI reviews (i.e., 50% each for human and AI peer review), resulting in FS computation in the following manner.

$$\begin{gathered} FS=0.4 C{S}^{\prime }+0.18 novelty score \left(H\right)+0.18 novelty score \left(AI\right)+ \\ 0.12 aggregate of other scores \left(H\right)+0.12 aggregate of other scores \left(AI\right) \end{gathered}$$

(16)

In Equation (16) too, the overall weight of novelty remains 56%, ensuring the emphasis on ‘novelty first policy’.

Evolution of AI Participation in the IRF’s Peer Review Section: Two Scenarios

Let us consider two scenarios in which the reliability of AI/ML/LLMs has improved significantly over time and in which AI-powered peer review systems have evolved such that the agreement between AI-generated peer review and human-generated peer review has improved continuously over the years. If a scenario such as the one shown in Figure 9 (Scenario 1) is encountered, up to 2035, the evaluation can be completely human (wherein AI involvement is restricted to providing a proposal summary and aiding novelty identification). After that, AI peer review may be used to complement human evaluation under careful human supervision. In the scenario given in Figure 9, AI involvement in FS score computation is a question even in 2041. Therefore, FS will be computed as in Equation (14) up to 2035 and by using Equation (15) after that. In the case of scenario 2 (Figure 10), human evaluation (without the involvement of AI in score assignment) is the only option until 2031. After that, AI peer review can be used to complement human evaluation under careful human supervision until 2039. After 2039, AI peer review can be given more or less the same importance as that of human review, and the requirements of human supervision can be restricted to cases where wide deviations are encountered. Thus, in scenario 2, Equation (11) can be used up to 2031, Equation (12) can be used from 2031 to 2039, and subsequently, Equation (13) can be used. The possible evolutionary configuration of the IRF emphasizing HR:AI ratios at different phases that leads to an evolutionary roadmap of the IRF in scenario 2 (optimistic in terms of AI-powered peer review) is depicted in Figure 11.

Apart from the IRF-related functions of the NSTF, its continuous research endeavors may aid in the phase-1 activities of the IRF in the immediate or short-term future, such as (i) the assessment of LLMs and AI tools, as well as the development of sciento-LLMs or LLMs, and (ii) the development of AI-driven research proposal management systems such as the one proposed by Munagandla et al. (2024) to perform some of the mundane administrative tasks (not every task suggested by them) to aid funders. Assuming that AI developments in scientometric applications and peer review occur at a slower rate than that given by Moore’s law of AI, the HR:AI ratio in the IRF during configurational upgrades might also change in a slower and more careful way.

Although we have performed a comprehensive analysis of the existing state-of-the-art in scientometrics for conceptualizing IRF, many uncertainties are related to this context, especially in terms of AI usage. Additionally, the gravity of tracking challenges induced by the uncontrolled and disorganized structure and growth of scientometrics should be addressed while Sciento-LLMs, AI-powered novelty assessment frameworks (AINAFs) and AI-powered peer review systems should be upgraded. The uncertainty related to solving those issues and challenges may be crucial for the roadmap. If some of the challenges are resolved sooner, NSTF/IRF should also be prepared for that. If nations are going to have NSTF and IRF as suggested, the benefits or threats of AI, no matter how soon they materialize, can be addressed and become advantageous via diligent human intervention. The most crucial step for NSTF and IRF lies in training, recruiting and keeping ergonomic support to Scientometricians throughout the transition toward Industry 5.0.

5. Conclusions

This study proposes a conceptual Intelligent Review Framework (IRF) as a policy-based, AI-enabled alternative to conventional proposal assessment systems that rely excessively either on peer review or on simplistic scientometric indicators. Its principal contribution lies in being, to the best of our knowledge, the first framework to operationalize a ‘novelty first policy’ in research proposal evaluation through an integrated architecture that combines AI-powered scientometric assessment, structured peer review and a phased governance design. The framework advances beyond the earlier informed peer review model by assigning a broader role to the scientometric team: identifying priority themes, detecting novel and disruptive works in the relevant literature, recommending experts, strengthening proposal screening and supporting the development of AI-based review tools. A major finding is that novelty can be treated not as a vague aspiration but as an assessable dimension through the combined use of novelty, disruptiveness and paradigm-shift indicators; thematic relevance scores; and improved composite measures such as CS′ and FS. The framework also contributes a governance mechanism in the form of a national scientometric task force, which can support long-term institutionalization, standardization and technological upgrading.

However, even if a nation has rich scientometric talents and institutions that can be upgraded to the NSTF, we recommend that national policymakers and responsible ministries/departments in government carefully weigh challenges, including discipline-specific challenges and country-specific challenges, against the ‘benefit of scale’ offered by the IRF and consider them before their implementation. Additionally, the massive scale (establishment and sustenance of the NSTF and IRF or suitable revamping current institutions) demands the need for a heavy-budgeted pre-implementation commissioned study. As part of that study, expert survey (ensuring inclusivity and appropriate representation level of various stakeholders) should be there concerning the IRF and the computational framework embedded in it. Apart from that, a series of simulation studies and sensitivity analyses on many large synthetic and real-world datasets (of awarded grants in custody of funders) is vital to develop a holistic understanding on how to determine the affordable risk tolerance level in different fields/subfields/themes and thereby decide the level of emphasis on novelty for a call. Furthermore, as the framework is evolvable, the validation is not a one-time exercise and should be carried out at each upgradation point specified in the evolutionary roadmap.

IRF envisions ensured fairness in evaluation via many features. Firstly, the peer review selection via scientometric evaluation and peer reviewers’ lack of access to applicants’ details significantly reduces the possibility of favoritism by elite-capture and cronyism and other biases, including gender bias. Second and foremost is the novelty emphasis in both scientometric and peer review assessments, which directly targets the bias against novelty and fear/intolerance of risk. This might hinder the intentional or unintentional favoring of less novel proposals by rejecting novel proposals by simply stating it is either risky or ambitious. Satisfactory justification on why it is risky and why the risk is beyond the prior determined tolerance standards of the applicable fields needs to be clarified and communicated to the applicant in the event of exclusion from the short list. How crucial is affordable risk tolerance level determination in fixing novelty weightage is demonstrated by a modest sensitivity analysis. Thirdly, to ensure responsible, ethical and fair usage of AI in peer review, any attempt by human peer reviewers to use AI/LLM at their own end is prohibited. The proposal submissions (without applicants’ information), AI-generated summaries of the proposals, details of contributions of major novel publications related to the call, etc., will be available to peer reviewers in read-only format, that too via the system specifically designed by NSTF. Access to these documents will be managed via temporary credentials.

Moreover, the study shows that the transition from informed review to intelligent review must remain cautious, staged and policy sensitive. AI can substantially strengthen large-scale proposal processing, reviewer support, thematic intelligence, technical screening and evidence-based shortlisting; however, human judgment remains indispensable, especially in the near term, because proposal evaluation involves contextual reasoning, ethical scrutiny and risk-sensitive interpretation. Another salient feature is that the IRF is explicitly evolvable: it incorporates a roadmap in which the human–AI balance can be recalibrated as AI systems mature and as agreement between AI-assisted and human review improves. Thus, the IRF should not be seen merely as an automation framework but as a governance-oriented review architecture designed to improve fairness and reduce misuse and bias rooted in elite capture and cronyism, which include gender bias, strengthen novelty recognition and prepare research funding systems for the next phase of AI-integrated scientific assessment.