From PhD Research to International Audience: My Learning Journey on Educational Assessment and Rasch Measurement Theory
From PhD Research to International Audience: My Learning Journey on Educational Assessment and Rasch Measurement Theory
By Muhammad Affan Ramadhana
I was driving when the email notification from PROMS 2025 committee arrived. I peeked through my smartwatch but could only see the subject headline: Congratulations – Distinguished Student Scholarship PROMS2025.
Having arrived home, I tried to read the email word-by-word, in case I misread it. Alhamdulillah. My plan to present my research at the conference is saved. I was awarded travel grant to attend and present at the conference. This award is given by Pacific Rim Objective Measurement Society (PROMS) and Singapore University of Social Sciences (SUSS) as the committee of PROMS conference this year. I did not expect much when I submitted the application. I just thought that it is better to fail while trying than to regret never trying.
“The award recognizes outstanding students across Southeast Asia who have demonstrated academic excellence, research potential, and a commitment to advancing the field of measurement and assessment.” (PROMS 2025 website).
My Research Background
I was trained academically in English language education field. I obtained my Bachelor of Education degree in 2011 and my Master of Education in 2014, both from Universitas Negeri Makassar (UNM), South Sulawesi province, Indonesia. Since 2022, I was admitted to the PhD Program at the Faculty of Education, Universitas Islam Internasional Indonesia (UIII). I did my undergraduate thesis on English test analysis using classical test theory. Now, my doctoral research focuses on educational assessment and evaluation, with specific interest in language testing and assessment, rater-mediated assessment, and Rasch measurement theory.
It was in the Educational Assessment class, when I was first introduced to Rasch measurement model by Dr. Bambang Sumintono. As one of prominent figures in Rasch measurement in Indonesia, he clearly explained several traditions in measurement model and present a solid argument on why using raw score in measurement is problematic. Later in the following weeks, I was introduced to multi-rater assessment – a concept that I later used in my dissertation research.
My dissertation specifically explores assessment behavior of Indonesian university lecturers when grading English essays. This research focus emerged from my experience listening to stories of how my colleagues admit inconsistencies when assessing students’ writing. In rater-mediated assessment, inconsistencies and biases can be a threat to validity, reliability, and fairness of the assessment, which can impact the students’ future opportunities.
My dissertation research utilizes Many-Facet Rasch Measurement (MFRM) analysis, which is a modern psychometric model built on the idea that observed ratings are influenced by multiple facets, where each contributes to the final measurement. These facets include examinee ability, rater severity, and assessment rubric criteria. MFRM allows estimation of the logit scale for each facet and adjustment for inconsistencies. This model is important for identifying systematic biases or interactions that may be introduced by indirect factors.
From PROMS 2024 to PROMS 2025: My First Steps in the International Arena
Participating in PROMS 2024 provided a very valuable first-time experience for me. PROMS, as a scholarly society that promotes objective measurement and Rasch measurement, provides a forum for researchers from various backgrounds to share their works. In the 2024 conference, I directly witness the presence of leading experts in the field of Rasch measurement, such as Prof. Trevor G Bond (Founder of PROMS), Prof. George Engelhard (President of PROMS), and Prof. Yan Zi (Vice-President of PROMS), among others. These names I previously knew from books and journal articles only.
I was just started to deeply learn about this field when I participated in PROMS 2024 in Kuala Lumpur, Malaysia last year. I actually did not understand most of the theory when I presented my preliminary study. It was just my strategic action to gain insights and measure the worthiness of my plan and preparation for dissertation research. But I did my best. The feedback that I received, both from Rasch experts and from language assessment experts, resulted in substantial refinements to the analytical framework and methodology of my dissertation research. This experience also strengthened my determination to continue exploring this field.
Upon browsing the PROMS 2025 website, I noticed that the committee of this year’s conference opened applications for Distinguished Student Scholarship. I have just finished analyzing data from the first phase of my dissertation research when I saw the opportunity. I immediately prepared my extended abstract, curriculum vitae, and most importantly, recommendation letter from dissertation supervisor. He strongly supported me and believed that participating in PROMS 2025 will provide valuable feedback at the crucial stage of my dissertation research.
In PROMS 2025, I will present the result of first phase of my dissertation research. In the study, I analyzed assessment behavior of 36 university lecturers in Indonesia. They were tasked to assess English essays written by undergraduate students from various Asian countries. The assessment was conducted using standard analytical rubric with five categories (content, organization, vocabulary, language use, and mechanics).
The analysis indicates the presence of at least three groups of raters based on their level of severity/leniency with significant differences across raters. The statistical fit analysis also identified raters who exhibited inconsistent scoring behavior (overfit and misfit). The most interesting finding of this study was the identification of biases between rater background and scoring criteria. The results show different assessment behaviors of lecturers based on their academic qualifications, field of study, professional rank, and gender. This study is expected to offer practical insights for assessment committees to minimize potential biases, especially when designing multi-rater English writing assessment programs
My Future Plans
Receiving this scholarship will provide me with the opportunity to present some of my dissertation findings at the PROMS 2025 conference. The main event will be held on July 22-23, 2025, located in Singapore University of Social Sciences (SUSS) campus. The Distinguished Student Scholarship presentation session is scheduled for the afternoon of Day One. I believe the scholarship will provide me with a platform to gain insights and interact with leading experts and colleagues from across the Pacific region and beyond. I believed the forum will help me expand my professional network into the field of objective measurement.
I am very grateful for this opportunity and the recognition given through the PROMS 2025 Distinguished Student Scholarship. I believe research focused on improving educational practices, no matter how small, can have international relevance. I hope to make the most of this opportunity to learn from experts, share my research findings, and continue to contribute to the development of the field of educational measurement and language assessment, particularly within Rasch measurement theory.
Islamic Feminists’ Approaches
Nurmila, N. (2025). Islamic Feminists’ Approaches. In: Bano M, ed. The Cambridge Companion to Women and Islam. Cambridge Companions to Religion. Cambridge University Press; pp. 214-233. https://www.cambridge.org/core/books/abs/cambridge-companion-to-women-and-islam/islamic-feminists-approaches/7D0AA2AE8DDAC391F4BB7B067A724C00
Summary
Nina Nurmila provides a review of key scholarly figures in the field of Islamic feminism and their methodological approaches. She emphasizes how, in Indonesia, their influence has permeated the wider Muslim community more effectively than in other contexts, thanks to local scholars disseminating their ideas through vernacular Islamic scholarship.
Grit, Innovation, and Adaptation: Key Themes in FoE’s 3rd Graduate Forum for Education Future
Grit, Innovation, and Adaptation: Key Themes in FoE’s 3rd Graduate Forum for Education Future
March 24, 2025
Contributor: Supriyono | Editor: Dadi Darmadi | Photo: Virda Lalitya Umam
The Faculty of Education (FoE) at Universitas Islam Internasional Indonesia (UIII) has successfully hosted its 3rd Graduate Forum & Colloquium on March 18-19, 2025, at the UIII Campus in Depok, West Java. Themed "Education for the Future: Adaptation, Innovation, and Grit in a Globalized World," the event convenes graduate students to explore the dynamic evolution of education.
In her opening address, Assoc. Prof. Charyna Ayu Rizkyanti underscored the forum's relevance, stating, "talking about the future—who can truly predict it? We’re witnessing the world changing very fast. However, if there’s one thing that we know for sure, education will always be at the heart of how we navigate this uncertain future,” she said.
She emphasized the indispensable role of adaptation, innovation, and grit in modern education, explaining that adaptation ensures survival, innovation drives progress, and grit fortifies resilience in the face of adversity.
Meanwhile, Prof. Nina Nurmila, Dean of the Faculty of Education at UIII, highlighted the forum's function in equipping the students for academic dissemination. She remarked, "This serves as a vital rehearsal for you—to present your research and articulate your ideas to a broader academic audience." Prof. Nurmila also emphasized the significance of timely study completion and scholarly productivity, urging students to remain diligent in their academic publications and maintain the tradition of graduating together onetime.
The forum featured two distinguished keynote speakers. On the first day, Prof. Corina D. Riantoputra, a psychology lecturer from the University of Indonesia, explored the primary theme, "Education for the Future: Adaptation, Innovation, and Grit in a Globalized World." On the second day, Aria Nakissa, Ph.D., from UIII's Faculty of Islamic Studies, presented "Study Completion and Productive Publication," aiming to inspire students and faculty members to achieve timely graduation and maintain a robust academic publication record.
Organized by the Education Student Association (ESTUDIA), the forum also marked the inauguration of ESTUDIA’s new leadership cabinet, with Mukhamad Imron Rosadi, an MA in Education student from batch 4, assuming the role of president.
The 3rd Graduate Forum & Colloquium serves as a dynamic platform for participants to engage in intellectual discourse, exchange insights, and challenge prevailing perspectives, embodying the ethos of shaping the future of education in an era of rapid transformation.
Self-handicapping scale: evaluation of psychometric properties among Malaysian and Indonesian university students using Rasch rating scale model analysis
Sumintono, B., Law, M.Y. and Sitasari, N.W. (2025). Self-handicapping scale: evaluation of psychometric properties among Malaysian and Indonesian university students using Rasch rating scale model analysis. Journal of Applied Research in Higher Education. https://doi.org/10.1108/JARHE-08-2024-0429
Abstract
Purpose – This study aimed to evaluate the psychometric properties of SHS in Malaysian and Indonesian university students’ populations using the Rasch Rating Scale Model.
Design/methodology/approach – Specifically, the persons’ and items’ reliability and separation, rating scale’s functionality, unidimensionality, item targeting, item quality and item bias were evaluated using Winstep 4.8.1.0 on a sample of 318 Malaysian and 470 Indonesian university students.
Findings – Both samples show good unidimensional measures. In terms of certain psychometric attributes, the Indonesian and Malaysian samples have relatively similar qualities. The adoption of SHS in measuring the selfhandicapping tendency indicates the scale works well for both Indonesian and Malaysian samples. Practical implications – The findings allow researchers in Malaysia and Indonesia to confidently use the SHS to measure self-handicapping behaviours among university students. This will then enable the design and implementation of a comprehensive intervention programme aimed at reducing self-handicapping and improving the psychological well-being of these future change agents in both countries.
Originality/value – This instrument was first tested in the United States, but its psychometric properties have yet to be evaluated in Malaysia and Indonesia.
Keywords Self-handicapping, Self-handicapping scale, Psychometric properties, Rasch rating scale model, University students
Paper type Research paper