Purpose – This study aims to determine empirically the appropriate number of items that meet all item selection criteria, from classical test theory to item response theory.Methodology – This study used the Mathematical Mindset Scale (MMS) as an instrument for a non-test. It was conducted on 259 Mathematics Education students in Indonesia to determine the appropriate number of items that meet all item selection criteria, from classical test theory to item response theory. This quantitative research has been analyzed, including sample adequacy tests, construct validity and reliability, the goodness of fit, model suitability to polytomous data, assumption tests (unidimensional, invariance, local independence), characteristic curve graphs, information function graphs, and standard error graphs.Findings – After several trials, 11 items passed the classical tests and item response theory criteria. It is also known from this study that non-test instruments can be less than 20 as long as they meet the analysis criteria at each stage.Contribution – This research contributes to helping increase insight and knowledge for researchers in the fields of education and psychology, especially those interested in non-test measurement tools and IRT and classical analysis.