This study aims to analyze the student and school level variables that affect students’ self-efficacy levels in mathematics in China-Shanghai, Turkey, and Greece based on PISA 2012 results. In line with this purpose, the hierarchical linear regression model (HLM) was employed. The interschool variability is estimated at approximately 17% in China-Shanghai, approximately 22% in Turkey, and approximately 23% in Greece. This study showed a positive association between variables of self-confidence, teacher support, and attitude toward school, all of which are among Level 1 variables, and mathematics self-efficacy in all three countries. A negative association was observed to exist between the variables socio-cultural index and educational opportunities at home and mathematics self-efficacy in all three countries. While pre-school education in China- Shanghai and Turkey were negatively associated with students’ mathematics self-efficacy levels, the same variable was positively associated with students’ mathematics self-efficacy in Greece. While the variable mathematical anxiety was negatively associated with students’ mathematics self-efficacy in China-Shanghai and Greece, it was positively associated with students’ mathematics self-efficacy in Turkey. The variable interest in mathematics, in turn, was negatively associated with mathematics self-efficacy solely in China- Shanghai. Regarding the association between mathematics self-efficacy levels and the school level variables, a near-zero positive association was found between class size, deemed significant for Turkey, and self-efficacy levels. The association between teacher to student ratio in school and self-efficacy levels was found to be negative in all three countries. The variable teacher’s morale, however, was positively associated with self-efficacy level in China-Shanghai and Turkey.