<![CDATA[National accreditation tiering of scholarly journals is a labour-intensive administrative process, and indexing platforms increasingly harvest full article metadata in the hope that publication content will support automated screening. We test whether that content is worth harvesting. Using the operational database of a regional journal indexing platform covering 305 journals and 26,210 harvested articles in Central Java, Indonesia, we construct a three-level ordinal accreditation target and predict it with a Frank-Hall ordinal random forest over 36 registry metadata variables, after an explicit audit that removes columns acting as label proxies. The registry model reaches a quadratic weighted kappa (QWK) of 0.549 under 5 × 5 repeated stratified cross-validation, with pooled out-of-fold QWK 0.572 (95% CI 0.456-0.679) and accuracy 0.678 against a majority baseline of 0.384. We then add journal-level representations built from 4,020 article abstracts using three sentence encoders, three pooling schemes, four PCA sizes and two fusion strategies. Abstract content is informative on its own (QWK 0.283) but no configuration met an improvement criterion fixed before the results were inspected (∆QWK ≥ 0.05 and p < 0.05); the largest observed gain was +0.023 (p = 0.17). A power analysis places the minimum detectable effect at 0.176 QWK, so small effects cannot be excluded. SHAP and permutation importance agree that article processing charge disclosure, aim-and-scope length and journal age dominate. For platform designers, the practical implication is that content harvesting is not required for this screening function.]]>
Copyrights © 2026
