Journal of Knowledge-Research Studies

Analysis of Research Excellence Assessment Frameworks and Providing Policy Requirements for Iran

Document Type : Original Article

Author

Assistant Professor, Policy evaluation and Monitoring of Science, Technology, and Innovation Department, National Research Institute for Science Policy (NRISP), Tehran, Iran

Abstract

Purpose: Many countries have developed and implemented research excellence frameworks  tailored to their specific social, cultural, and academic context. This study aims to investigate and compare national research excellence frameworks based on their objectives, indicators, levels of implementation, and evaluation processes.
Methodology: This applied and library-based research was conducted using a comparative approach and the Beri model (1969). The study employed a descriptive-comparative method to analyze the structure, implementation, and evaluation mechanisms of selected frameworks.
Findings Seven major research excellence frameworks were identified and examined, including Research Excellence Framework (REF), Standard Evaluation Protocol (SEP), Excellence in Research for Australia (ERA), Committee for Evaluation of Italian Research (CIVR), Canada First Research Excellence Fund (CFREF), Excellence Initiative (EI), Research Assessment Exercise (RAE). The comparative analysis revealed both similarities and differences among these frameworks in terms of objectives, evaluation indicators, levels of comparison (national, international, or both), assessment approaches (quantitative, qualitative, or mixed), scoring methods (quantitative or qualitative), and implementation processes.
Conclusion: evaluating the quality and societal impact of research is essential for determining the role and accountability of academic institutions . Therefore, the development and use  of comprehensive, context-sensitive indicators and metrics are necessary to assess research excellence effectively, taking into account each country’s unique requirements and policy priorities.
Value: This comparative study provides an opportunity to analyze and compare national research excellence frameworks in terms of their objectives, indicators, scoring methods, approach, levels of implementation, and execution processes.

Keywords

References
Andras, P. (2011). metrics, quality, and management implications. Research Evaluation, 20(2), 90-106. https://doi.org/10.3152/095820211X12941371876265
Arocena, R., Göransson, B., & Sutz, J. (2019). Towards making research evaluation more compatible with developmental goals. Science and Public Policy, 46(2), 210-218.‏ https://doi.org/10.1093/scipol/scy051
Aubert Bonn, N., & Bouter, L. (2023). Research assessments should recognize responsible research practices. Narrative review of a lively debate and promising developments. Handbook of Bioethical Decisions. Volume II: Scientific Integrity and Institutional Ethics, 441-472. http://dx.doi.org/10.1007/978-3-031-29455-6_27
Azadi Ahmadabadi, G., &Ghanadi Nejad, F. (1404). Prioritizing Research: Current Needs and Future Directions. Iranian Institute of Information Science and Technology. https://irandoc.ac.ir/book/6181  [In Persian]
Bornmann, L., & Marx, W. (2023). The measurement of research excellence: Challenges, developments, and future directions. Informetrics, 17(1), 101345. https://doi.org/10.1016/j.joi.2023.101345
Bucchi, M., & Papponetti, V. (2007). Research evaluation as a policy design tool: Mapping approaches across a set of case studies. The Fondazione Eni Enrico Mattei Note di Lavoro Series Index. http://dx.doi.org/10.2139/ssrn.999929
Davidson, P. M., Homer, C. S. E., Duffield, C., & Daly, J. (2011). A moment in history and a time for celebration: The performance of nursing and midwifery in Excellence in Research for Australia. Collegian, 18(2), 43–44. https://doi.org/10.1016/j.colegn.2011.02.001
Eslami, Z. , Hakimzadeh, R. & Saboury, A. A. (2018). Comparative Study of Explanation the Framework for Assessment he Quality of Research in Iran's Higher Education System with the Countries of Britain, Australia, the Netherlands, Italy and Hong Kong. Journal of Research in Educational Systems, 12(42), 59-76. http://doi.org/10.22034/jiera.2018.84465 [In Persian]
Eslami, Z. , Hakimzadeh, R. & Saboury, A. A. (2018). Comparative Study of Explanation the Framework for Assessment he Quality of Research in Iran's Higher Education System with the Countries of Britain, Australia, the Netherlands, Italy and Hong Kong. Journal of Research in Educational Systems, 12(42), 59-76. http://doi.org/10.22034/jiera.2018.84465 [In Persian]
Evans, E. D., Gomez, C. J., & McFarland, D. A. (2016). Measuring paradigmaticness of disciplines using text. Sociological Science, 3, 757–778. http://dx.doi.org/10.15195/v3.a32
Evans, G., Rob, M., &  Hamish, S. (2021). Excellence in Research. https://www.mbie.govt.nz/dmsdocument/16598-excellence-in-research
Fallon, D. (2008). Germany's" Excellence Initiative". International Higher Education, (52). http://dx.doi.org/10.6017/ihe.2008.52.8036
Gaunand, A., Hocde, A., Lemarié, S., Matt, M., & De Turckheim, E. (2015). How does public agricultural research impact society? A characterization of various patterns. Research Policy, 44(4), 849-861. https://doi.org/10.1016/j.respol.2015.01.009
Geuna, A., & Piolatto, M. (2016). Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while). Research Policy, 45(1), 260–271. http://doi.org/10.1016/j.respol.2015.09.004
Geyer, A. (2014). The German Excellence Initiative. In: Promoting Research Excellence New Approaches to Funding(PP.87-110). OECD. https://www.upla.cl/noticias/wp-content/uploads/2015/01/StateofHigherEducation2014.pdf#page=88
Goodarzvand Chegini, M., and Yousefi, S. (2024,   February  18). Analysis of the Onion Research Model and its Application in Social Science Research. Tehran Institute of Humanities and Cultural Studies. https://civilica.com/doc/2024893 [In Persian]
Government of Canada, I. C. (2022). Canada First Research Excellence Fun: Application Process. Canada First Research Excellence Fund. http://www.cfref-apogee.gc.ca/programprogramme/apply-demande-eng.aspx
Haslam, N., & Koval, P. (2010). Possible research area bias in the Excellence in Research for Australia (ERA) draft journal rankings. Australian Journal of Psychology, 62(2), 112–114. https://doi.org/10.1080/00049530903334489
Jamali, H. (2011). Research Evaluation: Approaches, Techniques and Challenges. Rahyaft, 21(49),39 -52. https://rahyaft.nrisp.ac.ir/article_13512.html?lang=en [In Persian]
Jones, C. R., & Walmsley, A. (2022). A change would do you good: advances in research impact in sustainable tourism and some ‘home truths’ for the sector. Journal of Sustainable Tourism, 30(9), 2073-2088. https://doi.org/10.1080/09669582.2021.2000995
Jonkers, K. & Zacharewicz, T. (2017). Research performance-based funding syste A comparative assessment. Luxembourg. Publications Office of the European Union. http://doi.org/10.2791/70120, JRC101043.
Musgrave, S., & Hajek, J. (2011). Excellence in research for Australia and sustainable data. In Sustainable Data From Digital Research 2011: Humanities Perspectives on Digital Scholarship (pp. 193-205). The Custom Book Centre. https://ses.library.usyd.edu.au/handle/2123/7939‏
Parks, S., Ioppolo, B., Stepanek, M., & Gunashekar, S. (2018). Guidance for standardising quantitative indicators of impact within REF case studies. Santa Monica, and Cambridge: RAND Europe.‏ https://www.rand.org/pubs/research_reports/RR2463.html
Peruginelli, G., & Pölönen, J. (2023). The legal foundation of responsible research assessment: An overview on European Union and Italy. Research Evaluation, 32(4), 670-682. https://doi.org/10.1093/reseval/rvad035
Pinar, M., & Horne, T. J. (2022). Assessing research excellence: evaluating the research excellence framework. Research Evaluation, 31(2), 173-187. http://dx.doi.org/10.1093/reseval/rvab042
Radi, S. (2025). Major Challenges in Knowledge Transfer and Research Excellence in Higher Education Institutions. In Navigating Quality Assurance and Accreditation in Global Higher Education (pp. 435-456). IGI Global Scientific Publishing. https://www.igi-global.com/chapter/major-challenges-in-knowledge-transfer-and-research-excellence-in-higher-education-institutions/365745
Reed, M. S., Ferré, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M., & Holden, J. (2021). Evaluating impact from research: A methodological framework. Research Policy, 50(4), 104147. https://doi.org/10.1016/j.respol.2020.104147
REF (2016). Building on Success and Learning from Experience: An independent review of the Research Excellence Framework. London: Department for Business, Energy and Industrial Strategy.
Royal Netherlands Academy of Arts and Sciences, Association of Universities in the Netherlands. (2020). Standard Evaluation Protocol 2021–2027. Protocol for research assessment in the Netherlands. Netherlands: KNAW. https://storage.knaw.nl/2022-06/SEP_2021-2027.pdf
Salajegheh, M. & Jahanshahi Javaran, R. (2021). Comparison of Researches Assessment Criteria in America, UK, Italy, Canada, Australia, Finland, Hong Kong, Denmark and Iran. Popularization of Science, 11(2), 48-77. http://doi.org/10.22034/popsci.2021.260018.1068 [In Persian]
Salimi, Q. & Hosseini, N. (2017). An Explanation of Research Excellence Framework (REF): a Step towards Providing a Model for Research Quality Assessment System in Nation’s Higher Education. Rahyaft, 27(65), 83-98. https://dor.isc.ac/dor/20.1001.1.10272690.1396.27.65.6.4 [In Persian]
Saunders, N. K., Lewis, P., & Thornhill, A. (2023). Research Methods for Business Students, 9th Edn. Pearson Education. https://www.researchgate.net/publication/240218229_Research_Methods_for_Business_Students
Tan, C. (2024). A Foucauldian analysis of research Assessment in a postcolonial context: the example of Hong Kong. Journal of Education Policy, 39(4), 660-678. https://doi.org/10.1080/02680939.2023.2269382
Thiem, A., & Dusa, A. (2012). Qualitative comparative analysis with R: A user’s guide (Vol. 5). Springer Science & Business Media. https://link.springer.com/book/10.1007/978-1-4614-4584-5
Tijssen, R. (2020). Re-valuing research excellence: From excellentism to responsible assessment. Transforming Research Excellence. http://dx.doi.org/10.5281/zenodo.3607319
Tijssen, R., & Kraemer-Mbula, E. (2018). Research excellence in Africa: policies, perceptions, and performance. Science and Public Policy, 45(3), 392-403. https://doi.org/10.1093/scipol/scx074
Trevisol, J. V., & Brasil, A. (2023). Evaluation policies for research and graduate education in the Netherlands: lessons on self-evaluation and institutional autonomy. Revista Brasileira de Educação, 28, e280107. https://doi.org/10.1590/S1413-24782023280108
Williams, K., & Grant, J. (2018). A comparative review of how the policy and procedures to assess research impact evolved in Australia and the UK. Research Evaluation, 27(2), 93-105. http://doi.org/10.1093/reseval/rvx042
Wiśniewska, M. Z., & Grudowski, P. (2024). The culture of excellence and its dimensions in higher education. The TQM Journal, 36(2), 593-615. https://doi.org/10.1108/tqm-11-2022-0325
Zacharewicz, T., Lepori, B., Reale, E., & Jonkers, K. (2019). Performance-based research funding in EU Member States—a comparative assessment. Science and public policy, 46(1), 105-115. https://doi.org/10.1093/scipol/scy041
Journal of Knowledge-Research Studies
Volume 4, Issue 2 - Serial Number 12
September 2025
Pages 111-131
Files
Share
How to cite
Statistics