Additional Assignment Information

The basic requirement in the assignment is to show that you can evaluate and critique a scientific paper. The key point to remember, here, is that science has to do with the accumulation of evidence. This evidence might be gathered using qualitative methods, quantitative methods, or both. The quality of the science in any paper is therefore dependent upon the status of the evidence presented. Many factors impact upon the status (reliability, validity and usefulness) of the evidence, including:

The sample upon which any conclusions and recommendations are drawn. Key issues here include:

·         Is it representative of the population for which inferences are being made?

·         Is it representative of the group from which it is being drawn?

·         How has the sample been drawn?

·         Are there any observable – or potential – biases in the sample?

·         Is the sample size sufficient for the particular research questions asked and the conclusions drawn?

·         How are participants accessed?

·         Does the method by which the sample is accessed carry any possibility of obtaining biased results? If so, how and why? How could any such biases be avoided?

·         If there are limitations in the sample, then do the authors show an awareness of them and what they might mean for the interpretation of the results?

The precise means by which data are generated once participants are accessed. Here attention might be paid to:

The reliability of any measures/instruments used:

·         Is any reliability evidence offered on the current sample? Is it sufficient for you to assess the quality of any instruments?

·         Is any reliability evidence offered on the basis of previous samples (studies)? Again, is it sufficient for you to assess the quality of any instruments used?

·         Could further evidence for reliability be presented? If so, what is missing?

The validity of any instruments/measures used or data presented:

·         Is any validity evidence offered either for the current sample or on the basis of previous studies?

·         What type of validity is demonstrated?

·         Is the validity of any instruments or results sufficiently demonstrated?

·         What are the potential threats to the validity of instruments/results?

·         How could the validity of the findings be improved?

The general research design – is this appropriate to the research questions investigated in the paper? For example:

·         What type of research design is used?

·         Is it appropriate for the research (study) goals and objectives?

·         Does the way the study is conducted answer the research questions asked?

·         What are the strengths and limitations of the design?

·         How could it be improved, supplemented or built upon?

The statistical analysis. You are not expected, here, to understand the intricacies of every possible statistical test that might be used. You might, however, comment on the general strategy used in analysis, e.g. are differences or associations being investigated and how clearly/thoroughly are results presented?

If qualitative methods are employed - as part if not the entire research strategy – then issues to be considered include:

·         Are these methods appropriate for the stated research (study) goals?

·         Is the interpretation of qualitative data offered defensible? If so, what has been done to demonstrate this (e.g. inter-coder agreement, transparency of themes and so on)?

·         How are participants’ responses analyzed?

·         What material (if any) is used to stimulate participants’ responses (e.g. use of media clips)? How has this material been identified? Is sufficient information provided on this? Against what criteria is the material chosen? Is the material valid for the purpose it is used?

·         How are any identified themes and categories in the data derived at?

·         To what extent does the analysis and presentation of the data reflect such things as sensitivity to context, commitment and rigour, coherence and transparency?

Does the study throw up any ethical issues? If so, are they sufficiently addressed? Is sufficient information presented upon which to make a judgement?

Frequently asked questions:

1.     Do I have to read around the topic of the paper and critique it on the grounds of theory/content/concepts etc.?

2.     Do I have to critique the style of writing?

Some reading around the topic might be an advantage but your primary task is to critique the method adopted, not to write a literature review of the area. Reading around the subject might show you how others have tackled the same research question but who is to say that other authors have done a better job of it? The focus of the assignment is on research methods. Don’t lose sight of that.

Neither is your task to act as an ‘editor’ correcting what you believe to be points of grammar and sentence construction etc. These things are open to dispute at the best of times. Again, remember that you are critiquing the paper on its research method. Does it do what it sets out to do? Does it explain the procedure sufficiently? Could you replicate the study on the basis of the information presented? Could you improve upon it? How? These are the sorts of issues to focus upon.