Research – scholarly or scientific investigation or inquiry (The American Heritage Dictionary of the English Language)

Educational Research – the search for facts to be used in the solution of problems

Technology – the application of science; the body of knowledge available to a civilization that is of use in fashioning implements, practicing manual arts and skills, and extracting or collecting materials (The American Heritage Dictionary of the English Language)

Educational Technology – the combination of instructional, learning, developmental, managerial, and other technologies as applied to the solution of educational problems (Gentry, C.S. (1987). Educational media and technology yearbook 1987. Littleton, CO: Libraries Unlimited.)

Today’s content derived from: Research Methods in Education: An Introduction - Sixth Edition; William Wiersma ISBN 0-205-15654-1

Nature of Educational Research

• Empirical
• Systematic Process
• Validity
• Internal
• External
• Reliability
• Variety
• Basic
• Applied
• Qualitative
• Quantitative

Nature of Educational Research

• Empirical – result of research must take some sort of information form so that knowledge can be generated

• Systematic Process
• Identify the Problem
• Reviewing Information about the Problem
• Collecting Data
• Analyzing Data
• Drawing Conclusions

• Validity – research should be based on fact or evidence that it is capable of being justified
• Internal – the extent to which results can be accurately interpreted
• Random Samples of Sufficient Size (~25 to 30)
• Consistent Treatment – only difference overall between two groups is the treatment
• Sample – subjects actually studied
• External – the extent to which results can be generalized to populations
• Population – target group of subjects

• Reliability – research must be conducted consistently and must be repeatable; necessary for validity

• Variety
• Basic Research – the extension of knowledge
• Applied Research – solution of a problem
• Qualitative – describes phenomena in words instead of numbers or measurement
• Quantitative – describes phenomena in numbers or measurement

Quantitative Research Design

• Purpose:
• To Provide Answers to Research Questions
• To Control Variance
• Variance – not all individuals are the same or have the same score or measurement
• Controlling Variance – being able to explain or account for variance caused by variables being studied
• Randomization – spreads an effect of a variable evenly across the groups of the study:

• Building in Factors as Independent Variables – allows researchers to determine effects of those factors:

• Holding Factors Constant – potential variable is reduced to a constant; reduces any effect the factor may have on the dependent variable (e.g., all students have same IQ)
• Statistical Control – adjusting the dependent variable scores to remove the effect of the control variable (e.g., a relationship is likely to exist between IQ scores and test performance)
• Characteristics of Good Research Design
• Freedom from Bias – the data and the statistics computed from them do not vary in any systematic way, but only as would be expected on the basis of random fluctuations (i.e., variance not controlled)
• Freedom from Confounding – the effects of two or more independent variables cannot be separated (e.g., three teachers each using a different method)
• Control of Extraneous Variables – extraneous variables (which are not being examined) may have an effect on the dependent variable and should be identified and minimized or eliminated
• Statistical Precision for Testing Hypotheses – appropriate data must be obtained with enough precision so hypotheses can be tested with confidence (i.e., differences in samples are real and are not due to chance)

• Characteristics:
• One or more variables is manipulated by the researcher to determine the effects of that manipulation
• Participants randomly assigned to treatments
• Question: What is the effect of the manipulation of the experimental variable?
• Design – the structure by which variables are positioned or arranged in the experiment
• Experimental Variable – independent variable being deliberately manipulated
• Dependent Variable – item upon which effects of the experimental variable manipulation are measured
• Criteria for a Well-Designed Experiment
• Adequate Experimental Control – enough restraints on the conditions of the experiment that the researcher can interpret the results (i.e., determine if experimental variable has an effect)
• Lack of Artificiality – necessary if results are to be generalized
• Basis for Comparison – to determine whether or not there is an experimental effect
• Adequate Information from the Data – with enough precision to make decisions about the hypotheses
• Uncontaminated Data – to reflect the experimental effects
• No Confounding of Relevant Variables – other variables may have an effect on the dependent variable
• Representativeness – to generalize the experimental results
• Parsimony – with all other characteristics being equal, a simpler design is preferred to a more complex one
• Posttest-Only Control Group Design
• Contains as many groups as there are experimental treatments, plus a control (or comparison) group
• Subjects are measured only after the experimental treatments have been applied
• Pretest-Posttest Control Group Design
• Contains as many groups as there are experimental treatments, plus a control (or comparison) group
• Subjects are measured both before and after the experimental treatments have been applied
• Factorial Design – involves two or more independent variables (factors). Measures effects of each independent variable as well as interaction effects between the variables
• Repeated Measure Design – the same subject is measured more than once on the dependent variable
• Provides information about possible delayed effects
• Provides information about the duration of an effect
• Time Series Design – involves repeated measurement with an experimental treatment inserted between two of the measurements; uses control group

• Characteristics:
• One or more variables is manipulated by the researcher to determine the effects of that manipulation
• Participants "naturally" assigned to treatments (e.g., by classes)
• Question: What is the effect of the manipulation of the experimental variable?
• Problems of Validity
• Limitations should be clearly identified
• Equivalence of the groups should be discussed
• Possible representativeness and generalizability should be argued on a logical basis
• Posttest-Only, Nonequivalent Control Group Design
• Generally weak in Validity
• Pretest-Posttest, Nonequivalent Control Group Design
• Pretest aids in checking the similarity of the group
• Time Series Design – involves repeated measurement of one or more intact groups with an experimental treatment inserted between two of the measurements
• Single Subject Designs – repeated measurements changing only one variable at a time

• Characteristics:
• Variables are studied as they exist, without manipulation
• Random samples of population preferred
• Question: What are the characteristics of the variables? What are the relationships between the variables?
• Methodology of Survey Research
• Define the Research Problem and Begin Survey Design
• Develop a Sampling Plan (if not sampling entire population)
• Develop Survey Instrument – consider how data will be collected, tabulated, summarized, and analyzed
• Include Demographic Information
• Categorization Scheme for Open-Ended Items
• Train Observers or Testers - reliability
• Pilot Study
• Conduct the Survey
• Follow-up – non-response may bias the results
• Cross-Sectional Designs – data collected at one point in time
• Longitudinal Designs – data collected over time and at specific points in time
• Trend Study – a general population is studied (i.e., sampled) over time
• Cohort Study – a specific population is studied over time
• Panel – the initial random sample (of a general or specific population) is used throughout the data-collection times

• Characteristics:
• A holistic description of present phenomena is developed
• Question: What is the nature of the phenomena?
• Rarely applies to Instructional Technology Research

• Characteristics:
• A description of past events is developed
• Question: What was or what happened?
• Rarely applies to Instructional Technology Research