Meta-Analysis Matthew Burns University of Missouri Overview Resources
Meta-Analysis Matthew Burns University of Missouri Overview Resources https:// www.meta-analysis.com/downloads/Meta%20Analysis%20Fixed%20vs%20Random%20effec
ts.pdf https://www.meta-analysis.com/pages/formulas.php School Psychologists and Research
Keith (2008) three research activities Conducting Research Consuming Research Synthesizing Research
Most Famous Articles in our Field? Types of Errors in Narratively Synthesizing Research Selectivity Erroneous detailing Nonrecognition of faulty author conclusions
Suppression of contrary findings The larger the literature, the less useful reviews Center et al. (1995) study of Reading Recovery Reading Recovery Pretest
Posttest Clay Book Level Test Word Reading Test Neale Analysis of Reading Passage Reading Test Diagnostic Spelling Test Phonemic Awareness Test
Cloze Test Word Attack Skills Test .59 2.55 4.64 2.32
7.26 18.27 .68 Control
Pretest Posttest .47 .83 9.96
30.86 12.59 4.82 55.55 22.86 51.59 24.23
63.50 1.83 7.43 .17
2.57 2.77 12.17 1.40 5.73 22.67
39.27 23.17 41.33 17.47 34.47 SIGNIFICANCE = SIZE OF SAMPLE X
SIZE OF EFFECT A standardized estimate of the difference between experimental and control groups X1 X2
d= (s1 + s2)/2 2r d=
1-r2 Hedges g Hedges g Upwardly biased for small sample sizes Multiply g by (1-3/[N-9])
N total sample size Effect Size Websites http://www.campbellcollaboration.org/resour ces/effect_size_input.php http://www.cognitiveflexibility.org/effectsize/ http://www.lyonsmorris.com/ma1/
Interpreting ES Cohen (1988) .20 small .50 medium .80 large
Somewhat arbitrarily selected More (was) to be gained than lost by supplying a common conventional frame of reference (p. 25). PostTest d = -0.18
Pre to Post Test d = 0.76 Synthesizing Meta-Analysis
A methodology for systematically examining a body of research (Glass, 1976). Examine the effect of variables on the phenomenon of interest Exhaustive search Established and explicit inclusion criteria Report an empirically derived effect size
Integrate research to create stronger generalizations Meta-Analysis of Reading Recovery Reading Recovery All students
k d 16 .63
Discontinued 5 1.11
Not Discontinued 4 -.04 Interventions for first graders
All 26 .63 Other than RR *
(2000) 10 .64
Elbaum, Vaughn, Hughes, & Moody Interventions for Children with LD Reading comprehension Direct instruction Psycholinguistic training
.39 .15 .12 .08 Contributions to Learning Hattie 2009 The student d = .40
The school d = .23 The teacher d = .49 The curriculum d = .45
Meta-Analysis Hattie 2009: Curriculum Vocabulary Instruction d = .67 Phonics Instruction d = .60 Comprehension Instruction d = .58 Whole Language d = .06 Perceptual-Motor d = .08
Teacher Roles: Hattie 2009 Activator Facilitator Feedback d = .72
Meta-cognition d = .67 Direct Instruction d = .59 Mastery Learning d = .57 Form Assess. d = .46 Simulation/game d = .32
Inquiry-based d = .31 Class size d = .21 Problem-based d = .15 Inductive teach d = .06
Total Total d = .60 d = .17
Why Conduct Meta-Analytic Research Conflicting results Clarity in a large literature Quantify relationships between variables across studies to build theory
Conflicting Results: Problem Solving Teams Mean Effect Sizes Clarity: Disability Simulations Theory
Need for Meta-Analyses PROCESS Beginning Identify your research questions broadly stated Identify key words that used to describe
relevant concepts in literature Develop inclusion criteria (For what are you looking?) Search Terms Comprehensive
Limit false positives, BUT almost eliminate false negatives Include synonyms Include vocabulary Based on literature!!!! IOA
Searching the Literature
Identify source for electronic search Conduct search Eliminate based on abstract Eliminate based on article Search reference lists Contact authors of found articles
Learner control AND reading AND math* AND literacy AND science AND social studies AND history AND language AND English language arts AND writing AND English language learner AND computer assisted learning AND classroom AND computers AND educational technology AND school
Codes Four general codes Substantive features of what is being studied Research methods Characteristics of the researcher and research context How the study is reported
Terms must be based on literature Clearly described in paper Coding manual pilot and IOA Type of Meta-Analysis Fixed Effects Random Effects
https:// www.meta-analysis.com/downloads/Meta-analysis%20Fixed-effect%2 0vs%20Random-effects%20models.pdf Fixed Effects Meta-Analysis Very similar studies same population etc. One true effect sizes
Use when all studies are identical and identified Five drug company studies Six studies on Reading Recovery Random Effects Model Articles are similar but vary somehow There is a distribution of effect sizes
Combined effect Fixed effects = the one common effect Random effects = mean of the population of true effects Effects weighting (more on that later) Random effects not as influenced by extremely small or large studies (part of population)
Variance for Random Effects v= n1 + n2 + g n1*n2
2*(n1 + n2) 2 Weighting Random Effects Q measure of total variance and homogeneity SCD Effect Size
Cohens d is not appropriate for SCD No-assumptions effect size (Busk & Serlin, 1992) Autocorrelation Percentage non-overlapping data (PND, Scruggs, Mastropieri, & Casto, 1987) No confidence intervals
Affected by outlying data Auto correlation r = .54 Auto correlation r = .57 Auto correlation r = .48
Total Auto correlation r = .57 Auto correlation r = .72 Non-Overlap of All Pairs (NAP) Parker & Vannest (2009) Based on area under the curve (AUC)
Compares overlap between each data point in A with each data point in B NAP = number of comparison pairs with no overlap divided by the total number of pairs Can be converted to other effect sizes How Large?
ROC with large effect based on visual analysis of multiple-baseline data = .96 Small effect = .90 Petersen-Brown, Karich, & Symons, 2012 Phi
Remember PND, PAND, NAP unknown distribution cannot convert to CI Phi commonly used effect size Cohen 1989 Converting to Phi NAP uses formula by Ruscio (2008)
AUC to phi Interpret Phi as ES
.10 = small .30 = medium .50 = large Derived from large-group designs Convert d to correlation with .20, .50, and .80
Threats to Validity Nonindependent effects Exclude repeats Average
Gaps in the literature Apples and oranges File drawer problem Fail-Safe N (Orwin, 1983) Nfs =
N0 (d0 dc) dc N0 = Number of studies d0 = Mean of observed effects
Assigning servers to service should be independent of network topology. Configure server with whatever IP address the service expects. VM keeps the same IP address even after migration. Uniform high capacity: Maximum rate of server to server traffic flow should...
13-1 Genetic Engineering Selective Breeding Humans use the variations in plants, animals, and other organisms to pass down the desirable traits Examples: Dogs, livestock (cows, chickens), plants we eat Hybridization Crossing dissimilar individuals to bring together the best of both...
Briefly commander of Georgia's Continental battalion (replaced by Lachlan McIntosh) Button Gwinnett's Plantation on St. Catherine's Island. Lyman Hall. GA representative, who signed the Declaration of Independence. 12th governor of GA.
Levels of learning goals Next Activity: Work on your learning goals with your table group Testing achievement of learning goals: Slide 17 The Montillation of Traxoline when assessment goes astray Slide 19 Slide 20 Slide 21 Ideas for implementation: Slide...
For example, empty bottles, old utensils, cardboard tubes, bubble wrap, fabric and yarn, eyeglass cases, old tools, cookie tins, shoes, gloves, and so forth. Collecting materials can also make for a great scouting project or volunteer activity.
Pas de convention à ce jour pour travailler pour un service mutualisé. Collaboration à définir sur des missions transversales (nécessité de plusieurs conventions) En théorie, iln'est pas possible de solliciter l'ACFI sur des projets transversaux pour le service commun.
Shore up weak hive, make another nuc, help a friend, etc. Let them make honey (watch for plugging) Eat it, Store it, use for overwintering, use to boost weak stores, etc. Give them room/ something to do. Be prepared to...
Ready to download the document? Go ahead and hit continue!