The author of this research project, Allen Rubin, was selected both to design the exercise and conduct the outcome evaluation. During this evaluation, she obtained findings that were largely unusable. In addition, the associated project director further advanced claims that Allen Rubin quietly abused the design that she had taken part in conceiving. It was noted for example, that there were instances of political self-interest that marred the exercise. This is the general verdict passed on an exercise that was otherwise well-intended.
Among the problems that were faced was the evaluation design. The initial intention was to take a random approach towards conducting the experiment. This was, however, not practicable due to issues with administration and ethics. Hence, they adopted an overflow approach. The overflow families were going to receive regular evaluation services by the field officers. The problem with this overflow approach, however, is that this design can only be used where there are sufficient numbers of valid family units. This is why this exercise was only carried out in a semi-overflow manner, since it was not assured that the available number of case families would suffice. The overflow approach was combined with a simple time-series design, which utilized data from families that had a history with substance abuse starting from two years prior.
Delegate your assignment to our experts and they will do the rest.
Another issue is the fact that there was a miscommunication during the planning and design of the research instruments. The designer had said that the data to be collected had been stored in a database, which was not the case. The researchers, therefore, had to manually obtain the data from written case files, which were very voluminous and scattered, making the exercise very painstaking and time-consuming. Additionally, the number of study cases had been brought down unexpectedly, reducing the sample size by 75 percent than initially planned. Obviously, this reduced the statistical validity of the whole exercise.
Half a year into the implementation of the quasi-overflow design, a research assistant discovered that some of their study subjects were being sent to a mental health center to receive a somewhat similar treatment to what they were administering. This compromised the design status of the research exercise since the researchers required to make comparisons that were uninfluenced. This resulted in the omission of a few research parameters from the design instruments.
Conclusively, to avoid issues similar to those that were encountered during this study, it is highly recommended that every participating party be involved in the design and evaluation of the research exercise from the very beginning. This allows for all parties get a practical feel of the study so that they identify with every facet of the exercise from the beginning.
References
Rubin, A. (1997). The family preservation evaluation from hell: Implications for program evaluation fidelity. Children and youth services review , 19 (1-2), 77-99.
