In this paper, I will prepare a mock interview on the merits and weaknesses of quasi-experimental designs. The designs in question include: (a) single group post-test only designs, (b) single group pretest/posttest design, (c) single-group time-series design, (d) nonequivalent control group posttest-only design, (e) nonequivalent control group pretest/posttest design and (f) multiple-group time-series design. I will look into explaining what is, and when it is appropriate to use a detailed design, the merits, and limitations of a specific design, the statistical analysis used in each specific design. I will also include a research question for each of the specific designs.
Single Group Post-test Only Designs
Interviewer: What are the sole group post-test only plans and when can we use them?
Researcher: I would say that sole group post-test only plans are experiment designs that involve manipulation of a variable and subjects are randomly assigned to experimental groups. In one group post-test only design, participants receive treatment and are observed for effects afterward. It best applies where there is no comparison group or when you want to evaluate between-subject effects. Take, for instance; a firm wants to know if its employees benefit from a recent training conference. Following the implementation of the training (treatment), the ‘group’ of employees are given short surveys (post-tests) to assess their satisfaction with the training program.
Delegate your assignment to our experts and they will do the rest.
Interviewer: What are some of the merits and limitations of this design?
Researcher: Well, one-group posttests are preferred in cases where carry-over effects are likely. They also work well with large group samples because there is a likelihood that a random assignment will result in equality between groups. With posttest-only, it is possible to measure characteristics of the sample that might relate to the outcome of interest. This is by far the weakest type of quasi-experimental designs. It cannot account for the rule of selection bias, as the researcher is unable to collect the baseline information about the equivalence of the group. Also, without a control group, there is no way of knowing the attitudes of participants had they not been subjected to the treatment. The results of posttests-only are quite difficult to interpret, as there is no clear definition of the relationship between independent and dependent variables before research.
Interviewer: Which statistical analysis method does this design implement?
Researcher: Analysis of this quasi-design entails selecting one group or unit based on a similar characteristic, then subjecting them to a treatment program, and finally recording the effects of the program on the participants. A case study on focus groups will be appropriate to use here as it examines decisions.
Research Question: what are the effects of smoking on health?
Single Group Pretest/Posttest Design
Interviewer: Now let us have a look at this next one. Could you explain what unique group pretest/posttest design is, and when to use it?
Researcher: Certainly, this quasi-experimental model studies one group or unit repeatedly over time. Quasi-experiments are similar to randomized controlled trials in many respects, but there are many challenges in designing and conducting a quasi-experiment when internal validity threats are introduced from the absence of randomization (Matthew, 2018) Researchers use this when they want to find out the influence of introducing a treatment to a unit having observed its prior characteristics. Here, a single group of participants is pre-tested, given an intervention measure and the post-tested. It is best used where the number of subjects is small since it conducts pretests and the posts tests that take time.
Interviewer: Can you share some of the merits and limitations of this quasi-design?
Researcher: Very much so, this design is one of the most frequently used experiments simply because it establishes a relationship between the variables under study. It establishes the differences of within-subjects outcomes. Pretests scores are useful to the researcher as they aid in verifying the success of randomization, therefore, help in categorizing units into a group. The downside to this design is that it can be time-consuming because it conducts tests before treatment and then observes effects after. The other demerit is that repeated testing sometimes introduces the problem of differential carryover effects.
Interviewer: What statistical analysis tactics does it employ?
Researcher: Well, the analysis of this design largely depends on your sample size. Larger samples can use longitudinal approach to analyze data. A paired sample t-test can be the appropriate statistical test to use with this. A paired t-test compares the means from the same group at different times. Rietveld in the paired t-test and beyond (2017), states that “tests for paired sample designs require an evaluation of their assumptions.” This emphasizes the significance of prior studies.
Research Question: What is the effectiveness of an anti-drug education program on school students’ attitudes toward illegal drugs?
Single-group Time-series Design
Interviewer: Onwards and what is a single-group time series quasi-experimental design and where or when is it best applicable?
Researcher: It is a type of quasi-experimental design where a series of periodic measurements are done on a single group or unit in intervals over time. It is applicable in instances of studying education or behaviors of subjects over a long time. It involves the creation of a comparison group is most often used when it is not possible to randomize individuals or groups to treatment and control groups (White & Sabarwal, 2014).
Interviewer: In your opinion, what are the merits and limitations of this design?
Researcher: This method is useful in showing trends over time. It examines the dependent variable over an extended period, both before and after manipulation of the independent variable. It works with an appropriate control group. The shortcomings of this design are that interpretations problems can arise due to the complexity of results. There is also no control for history and instrument reactivity may play a part in undermining effects.
Interviewer: What statistical analysis methods are practical to use?
Researcher: Given the lengthy nature of the research, I would suggest longitudinal and case studies as the best options to use in interpreting information.
Research Question: What is the weekly productivity of schoolteachers in September?
Nonequivalent Control Group Post-test-Only Design
Interviewer: What does this ‘long’ quasi-experiment design entail and when is it ideal to use?
Researcher: It is quite a mouthful, here; participants in one experiment group are exposed to treatment, and another nonequivalent group also undergoes the treatment, and a comparison of the results is done. The nonequivalent comparison group has similar characteristics to the treatment group, but there will be no random allocation of participants in this group. It is used in instances where a researcher wants to test phenomena between two groups with similar characteristics but in different settings.
Interviewer: Understood, then what are the advantages and demerits of this design?
Researcher: Of course, it provides compressed data due to the presence of a control group. It also eliminates most of the confounding variables that arise in research. The limits include possible bias in judgment, as there is no randomization. Occasions are more likely to produce the main effect due to repeated testing, maturation and practice and cumulative carryovers (Campbell & Stanley, 2015). It is also subject to non-random subject loss.
Interviewer: What are the systemic analysis styles to work with on this design?
Researcher: Cross-sectional and sequential methods are appropriate to use in this analysis since they focus on two groups with similar characteristics but at the same time.”
Research Question: What are the science scores of students from stream A using the new syllabus and B using the old curriculum, taught by Professor Makina?
Nonequivalent Control Group Pretest/Posttest Design
Interviewer: Can you tell me what this design entails, and when I can use it in research?
Researcher: I certainly can. Here, an experimental group is given a pretest, receives treatment, and then a posttest. At the same time, a nonequivalent control group gets a pretest, does not receive treatment, and then receive a posttest. This design applies to instances where a researcher tests the effects of treatment on the experiment group.
Interviewer: Can you expound on the advantages and limitations of this design?
Researcher: Yes, I would say that this method is very effective as evidenced by its use in therapy. It gives insight to variations in characteristics. The limitation of this kind of research is the susceptibility to selection differences, and does not account for historical error.
Interviewer: What are the statistical analyses of this design?
Researcher: This kind of study employs the use of a cross-sectional form of data measurement. It works on units with similar characteristics even when one undergoes different interventions.
Research Question: What are the differences in performance of students in streams A and B, taught by Professor Makina?
Multiple-group Time-series Design
Interviewer: Finally yet importantly, what does this design encompass and what can you tell me about when or where I can apply it?
Researcher: This is a type of quasi-experimental design where a series of periodic measurements are taken on multiple groups to assess the effects of within-subject comparisons from before and after treatments. It applies in areas where the researcher might be interested in finding the effects of treatments between two subjects or groups.
Interviewer: What can you tell me about the benefits of this design?
Researcher: Of course! Changes in subjects’ responses during the experimental phase combined with the reversal of the effects in the treatment observation phase provide strong evidence for a causal link between the independent and dependent variables. They involve many observations made before, and after treatment, they give additional qualities relating to the topic of interest.
Interviewer: What statistical analysis methods are applicable here?
Respondent: Since the researcher has control over the treatment process, this method applies basic, interrupted, and control time series design to compare the patterns of change over time from before to after treatment.
Research Question: who is responsible for student performance between teachers, parents and students’ initiative?
References
Campbell, D. T., & Stanley, J. C. (2015). Experimental and quasi-experimental designs for research Ravenio Books
Maciejewski, M. L. (2018). Quasi-experimental design. Biostatistics & Epidemiology , 1-10.
Rietveld, T. & Hout, R. (2017) The t-test and beyond: Recommendations for testing the central tendencies of two independent samples in research on speech, language and hearing pathology Journal of Communication Disorders, Volume 68 September 2017, Pages 44-57
White, H., & S. Sabarwal (2014). Quasi-experimental Design and Methods, Methodological Briefs: Impact Evaluation 8, UNICEF Office of Research, Florence.
