How Trustworthy Are Meta-Heuristic Algorithms in Developing Short Forms of Psychological Scales? A Comparison of Algorithms and Expert Judgments
SAGE OPEN, cilt.16, sa.4, 2026 (SSCI, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 16 Sayı: 4
- Basım Tarihi: 2026
- Doi Numarası: 10.1177/21582440261495260
- Dergi Adı: SAGE OPEN
- Derginin Tarandığı İndeksler: Social Sciences Citation Index (SSCI), Scopus, EBSCO Education Source, Education Abstracts, ERIC (Education Resources Information Center), Directory of Open Access Journals, Academic Search Ultimate (EBSCO), Social Science Premium Collection (ProQuest), Education Source Ultimate (EBSCO)
- Hatay Mustafa Kemal Üniversitesi Adresli: Evet
Özet
The study aimed to examine the psychometric consistency of meta-heuristic optimization algorithms (Ant Colony Optimization, Bee Colony Optimization, Simulated Annealing, and Particle Swarm Optimization) in the scale-shortening process by comparing their performance with short forms developed based on expert judgment. A descriptive comparative research design was employed. Data were collected from 1,052 participants using the 21-item Beck Depression Inventory. In addition, item-selection judgments were obtained from 27 experts specializing in the field of depression. Using each meta-heuristic optimization algorithm, 10-item short forms were generated. A separate 10-item short form was also constructed based on the aggregated inclusion and exclusion decisions of the experts. The resulting short forms were evaluated in terms of Cronbach's alpha coefficients, confirmatory factor analysis (CFA) indices, and R2 values. The findings indicated that all algorithms yielded acceptable levels of model fit and reliability. The expert-based short form demonstrated the highest internal consistency and AVE, whereas some algorithm-generated forms showed slightly more favorable model-fit indices. The average agreement between algorithmic item selection and expert judgments was approximately 50%, suggesting partial divergence between statistical optimization and expert evaluation. Overall, the findings suggest that although algorithm-based approaches can produce psychometrically sound short forms, they may not adequately capture the theoretical content and conceptual breadth of the construct on their own.