From a Course Assignment to Publication: My Experience with PISA 2022 Data

By Dwi Oktariani

When I entered the Graduate Seminar on Large-Scale Assessment course in the second semester of my doctoral study at the Faculty of Education, Universitas Islam Internasional Indonesia (UIII), I never imagined that the final assignment from the course would lead me to a publication in a reputable international journal. The course, taught by Bambang Sumintono, Ph.D., was a continuation of Advanced Educational Assessment, which had previously started to introduce me to the world of assessment and education measurement.

My academic background is in performing arts education, especially dance. Before starting my doctoral study, most of my research and practice were closely connected to arts and communities. I teach traditional dance at Universitas Tanjungpura, work as an arts practitioner, do choreography and performances, and conduct research on indigenous communities in West Kalimantan. The research world that I knew was very close to people, movement, tradition, and cultural practices. Data involving thousands of respondents, statistical analysis, SPSS, and especially large-scale assessment felt like a world very far from me.

My more serious encounter with quantitative approaches actually began at UIII. In the assessment course, I started learning about the Rasch Model and how to use Winsteps software. I began to understand how the quality of an item is examined and how assessment can produce accountable information. This experience slowly broadened the way I viewed educational research and showed me that there were still many areas outside the field I had been working in that I needed to learn.

A more challenging journey began when PISA was introduced in the Graduate Seminar on Large-Scale Assessment course. For the midterm assignment, I critically examined large-scale assessment practices and began questioning the relatively limited position of the arts in that discussion. I then carried this interest into my final assignment. I wanted to know whether a dataset as large as PISA could actually “speak” about the arts. From this simple question, I started developing a study on gender differences in affective engagement, creativity belief through the arts, and Indonesian students’ participation in school-based arts activities using PISA 2022 data.

The process was much more difficult than I expected. I had never used SPSS before, let alone worked with large-scale secondary data. From the thousands of data available through the OECD, I chose the Indonesian data because of my own curiosity: what does PISA actually look like when we focus on Indonesian students? I started looking for the questions given to students, understanding the code for each item, identifying relevant variables, and selecting data that could be used in the study. At that stage, the more I searched, the more I felt that I did not know enough. I even visited the libraries at UPI and UNY to look for previous studies that connected PISA with arts education. However, I did not find any study that was exactly the same as what I wanted to do. I also studied the work of senior students who had used PISA data to understand how such data could be developed into an article for a Scopus-indexed journal.

The more I read, the broader the problem became, and the more difficult it was for me to keep the study focused. At the same time, the final assignment deadline was getting closer. I was tired, but stopping was not an option. In the end, I returned to something I understood best: the arts. I went through the PISA 2022 questions again and finally selected seven items related to affective engagement, creativity belief, and participation in visual arts, music, and theatre.

I also did not go through this learning process alone. My classmates became an important part of the journey. Munaya helped me a lot in learning how to do data cleaning, something that was completely new to me at the time. Mbak Nofi and Pak Umam kept encouraging me when I felt overwhelmed by the secondary data. In class, the lecturer also continued to show us different examples of academic articles that we could study as references when developing our final assignments. From this experience, I realized that learning at the doctoral level also grows through asking questions, trying, making mistakes, discussing, and helping one another.

Eventually, the final assignment was completed. The results showed that female students had higher affective engagement with the arts, while male students reported a higher frequency of participation in several school-based arts activities. For me, these findings were not only statistical results. I started to see how large-scale data could offer another perspective on issues in arts education that I had previously understood mainly through practice, observation, and field experience.

After the final grade was released, I did something that was actually quite spontaneous. I tried submitting the paper to Asian Education and Development Studies (AEDS), Emerald, a Scopus Q2 journal. I did not even make major changes. The manuscript I submitted was basically the final assignment from the course. It was also my first experience creating an Emerald account and trying the submission process to an international journal by myself.

When I received the news that the manuscript had passed the initial stage and entered the review process, I was actually confused about how I should react. I sent the review decision to the lecturer and asked what it meant because at that time I still did not fully understand how publication in a Scopus journal worked. He congratulated me and encouraged me to begin “navigating” the reviewer revisions. That phrase turned out to be quite an accurate description of what happened over almost the following year.

The review process became another learning space for me. At one stage, I was very happy when one reviewer gave a minor revision. However, in the next round, I was faced again with a major revision. The manuscript eventually went through several rounds of revision before it was accepted. Even near the end, issues related to consistency of terms, punctuation, number formatting, and proofreading still brought me back to another revision.

Part of this process took place during a very personal period in my life because I was going through childbirth. My concentration was certainly not always at its best. Strangely, however, I do not remember the revision process as an unpleasant experience. I actually enjoyed it. Every reviewer comment showed me a part of my own writing that I had not noticed before. A manuscript that I initially considered only a course assignment slowly became a much more mature piece of work.

The experience of being a single author also left its own impression on me. I handled the entire process myself, from submission, reading and responding to reviewer comments, reanalyzing the data when necessary, revising the manuscript, to communicating with the editorial team. There was freedom in it, but also full responsibility. That was exactly what made the experience enjoyable. I could directly see how my ability developed from one version of the manuscript to the next.

In the end, the article, “Gender Differences in Affective Engagement, Creativity Belief Through the Arts, and Participation in School-Based Arts Activities: Evidence from PISA 2022 Indonesia,” was accepted and published in Asian Education and Development Studies. I still love traditional dance, indigenous communities, and research that allows me to meet people directly and engage with their cultural practices. Learning about PISA and quantitative research simply gave me another way to look at arts education. I now understand that issues in the arts can be read through community stories and teachers’ experiences, but they can also be explored through thousands of student responses in an international dataset.

For this reason, I am very grateful to the lecturer, who, through the Graduate Seminar on Large-Scale Assessment course, opened a door to a research area that I had never imagined I would enter. Thank you for the publication examples, experiences, and encouragement that were always shared with us in class. I am also grateful to my classmates who became part of this learning process, especially Munaya, Mbak Nofi, and Pak Umam. This experience made me understand that a course assignment at the doctoral level does not have to end when a grade has been given. For me, perhaps this is the most valuable meaning of the journey: learning does not always make us feel more comfortable. Sometimes, learning takes us into unfamiliar territory, makes us feel many times that we do not know enough, and then slowly expands the boundaries of what we think we are capable of doing as researchers.