How Differences among Data Collectors are Reflected in the Reliability and Validity of Data Collected by LikertType Scales?

Abstract

The purpose of this study is to investigate association of data collectors’ differences with the differences in reliability and validity of scores regarding affective variables (motivation toward science learning and science attitude) that are measured by Likert-type scales. Four researchers trained in data collection and seven science teachers who did not undergo any training, gathered data from 391 ninth-grade students. The data collection instruments were the “Motivation toward Science Learning Scale” and “Science Attitude Scale.” Data collection applications were conducted in four stages, two of which were accomplished four weeks apart by the researchers. The remaining two stages were accomplished four weeks apart by the teachers. A principal component analysis, confirmatory factor analysis, Cronbach’s alpha reliability analysis, Pearson correlation test for convergent validity, and t-test for the differences between the mean scores of each data collection stage were used for the data analysis. The results showed that motivation toward science learning and attitude toward science were high but the factor structures and reliability values, which were obtained by different data collectors, were different for the two scales. As another result, the convergent validity between the scores on the scales was shown to be sufficient for the measurements. However, the results of difference tests on the mean scores of the applications showed that there was a statistically significant difference between the mean scores of the two motivation scale applications by the teachers.