PhET Simulations Performance to Measure Complex Problem-Solving Skills in Physics Subject Matter: Evidence of Validity and Reliability

This study aims to analyze the performance of the Physics-Complex Problem-Solving (Ph-CPS) skills test using Physics Education Technology (PhET) simulations to measure Complex Problem-Solving (CPS) skills using physics subject matter as the problem context. This study is instrumentation research that produces a CPS skill measurement instrument that will be used in physics classes. The results indicated that the Ph-CPS test meets the criteria for complex problems and minimal complex system criteria. Construct validity and reliability analysis using the Rasch model showed that only Ph- CPS 1, Ph-CPS 2, and Ph-CPS 4 items were valid and reliable. The results of the concurrent validity analysis indicated that Ph- CPS items 1, 2, and 4 align with the PISA 2012 CPS test. Ph-CPS item 3 is considered too easy because it has similarities with Angry Birds games, so it is classified as a “familiar problem”. The results of the concurrent validity analysis show that Ph-CPS items 1, Ph-CPS 2, and Ph-CPS 4 have concurrent validity against the 2012 PISA test. Regression analysis between physics concept mastery and Ph-CPS skills showed no significant influence of physics concepts on Ph-CPS skills. It indicates that the Ph-CPS test measures domain-general skills.