Abstract

Numerous meta-analyses addressing the effect of technology on student achievement differ by focus, scope, content, sample, and methodological quality, making the interpretation of the overall effect challenging. To overcome this problem, this dissertation implemented a systematic quantitative synthesis procedure (second-order meta-analysis) to answer the question: does technology use enhance student achievement in formal face-to-face classroom settings as compared to traditional (no/low technology) settings, while taking methodological quality into consideration. Literature searches and review processes resulted in 37 relevant meta-analyses involving 1253 different primary-studies (approximately 130,300 participants). After examining the lists of primary studies, 25 meta-analyses incorporating 1055 primary studies (approximately 109,700 participants) were found to have greatest coverage of the overall set of primary-studies while minimizing the problem of overlap in primary literature. Analyses revealed a variety of weaknesses in the implementation of the meta-analytic procedures. To synthesize the 25 effect-sizes from the unique meta-analyses, two standard error approaches were used, one based on sample sizes in the primary studies, and one based on number of studies included in individual meta-analyses. The weighted mean effect-sizes from the two approaches, 0.315 and 0.333 respectively, were significantly different from zero. Results from the first approach revealed a high level of heterogeneity while those from the second one were homogeneous. Moderator analysis for results from the first approach revealed that higher methodological quality meta-analyses and higher inclusivity regarding the covered literature and incorporated research designs in a meta-analysis were associated with lower average effect-sizes. To validate these findings, 574 individual effect-sizes (60,853 participants) were extracted from 13 meta-analyses that provided sufficient information. The weighted mean effect-size of 0.304 was significantly different from zero and highly heterogeneous thus supporting the findings of the second-order meta-analysis with both approaches. The results consistently represent a medium strength effect-size, favouring the utilization of technology. Guidelines for conducting a second-order meta-analysis with advantages and disadvantages of the used approaches are presented and discussed with suggestions for applicability in different settings. Implications for technology use are offered and recommendations for future meta-analyses are suggested, including the need for greater systematicity, rigour and transparency in implementation and reporting.