Evaluating literature about research
If the literature that you are evaluating describes or reports research then it is important not to accept all that is claimed at face value. You should make an evaluation of the standard of the research and its claims before relying on it. The following questions may help you to do this (adapted in part from Rumsey 2004 and Thompson Learning).
What search question am I asking of this text?
Whenever assessing literature, you should always do so in light of your search question.
- Why have I chosen this text?
- What is my research question?
- What will critically reading this text contribute to my research project?
What type of literature is this?
It is important to consider the type of literature you are assessing. You may choose to be more critical of a peer reviewed paper than of a newspaper article.
- Is this an account of practise?
- Is this literature that reports research?
- Is this theoretical literature?
- Is this a policy document?
What is the research question (if applicable)?
The research question should summarise what the source is about.
- What research question is posed by the author(s)?
- What hypothesis do they offer?
- Does the literature answer the research question and accept or reject the hypothesis?
Was the design of the study effective (if applicable)?
If the study design is inappropriate or short sighted, then the conclusions based upon it are unsound.
- Were the experiments used appropriate to answer the research question and test the hypothesis?
- Were variables sufficiently controlled?
- Was appropriate statistical analysis used?
What is being claimed?
Assessing whether the authors have the right to make the claims that they have is one of the most important evaluations you can make.
- Are the claims made by the author(s) consistent with each other?
- Are the claims made by the author(s) consistent with the wider research field?
- With how much certainty do the authors make their claims?
- Is there sufficient evidence to support the author(s) claims?
- How widely do the author(s) claim that their work can be applied?
- What is the main argument of the text? What are the implications of the results?
Are the research claims credible?
The facts of research often depend on someone's interpretation. If research claims are made, are they credible?
- Who conducted the research?
- Who sponsored or paid for it?
- What was the research looking for?
- Did the researcher(s) seem to have a point to prove?
- What was the research not looking at?
- Where was it looking?
- How old is the research?
- What were the research methods used?
- How significant is the research?
- Is it frequently referred to by other authors?
Is there a comprehensive bibliography?
The bibliography is a key source of further information and additionally indicates how well researched the author(s) are on a given topic.
- Have the authors used recent publications?
- Have the authors relied on a wide variety of literature?
- Can you use these citations to further your own research?
- Can you identify key researchers or research groups in the field?
- Can you identify other journals that publish similar/related papers?
Usability of the source?
Ensure the source is accessible before you decide to use it.
- Is it easily accessible? Is it accessible from off campus locations?
- Is it a format that you can open on your own PC?
- If you have read a paper abstract, can you access the full text?
- Is it clear who authored the information for referencing and evaluation reasons?