Chapter 25 – Choosing a significance test for your data
This chapter takes the readers through the criteria for choosing an appropriate test of significance for the data they have obtained and the effect they are investigating.
Exercises
Exercise 25.1
Identifying simple two-condition designs
Some two-condition research designs are outlined below. Your job is to read the information (all of it!) and decide which test it is most appropriate to use. You should read the criteria for selecting tests contained in the first part of Chapter 23 before attempting the exercise. The tests that are possible are listed in the table below. Select parametric tests unless there is information contrary to their use.
Related t
Unrelated t
Single sample t
Mann-Whitney U
Wilcoxon T
Pearson correlation
Spearman correlation
Chi-square
Sign test
- Children are classified as high or low social media users and researchers investigate whether their scores on a reading test differ significantly.
Show answer
Unrelated t
2. The same children as in (1) are recorded as extroverts or introverts with the enquiry being: are extroverts more likely than introverts to be high social media users?
Show answer
chi-square
3. Students are tested for self-esteem before and after the exam period to see whether there is a significant change in self-esteem.
Show answer
Related t
4. A researcher believes that stress has an effect on physical health and so measures people’s stress levels with a questionnaire and records the number of times they have visited the doctor with minor ailments over the past two years. She believes perceived stress levels will predict number of visits.
Show answer
Pearson correlation
5. A class of school pupils is asked to solve a set of simple maths problems, each working on their own. They are then asked to solve similar problems, but this time they are told there will be a prize for the best performance. The dependent variable is the difference between their two performances and it is found that these scores are very different from a normal distribution in terms of kurtosis.
Show answer
Wilcoxon
6. The same researcher as in (4) tests the hypothesis that stress affects self-esteem and expects higher stress levels to be related to lower self-esteem scores and vice versa. In this project she finds the self-esteem scores are heavily skewed and cannot remove this with a transformation.
Show answer
Spearman’s correlation
7. A sample of people is found who have just completed their second degree. A researcher is interested in whether their second degree category is better than their first degree category. Since degree grades are categorical the only record is whether the second degree was better or worse than the first.
Show answer
Sign test
8. Participants are divided into two groups. One group is asked to doodle (by filling in letters) while listening to a guest list of names invited to a party. A control group does the same task without doodling. It is predicted that the doodle group will perform better when asked to recall as many names as possible. The variances of the two groups are very different and there are quite different numbers of participants in each group.
Show answer
Mann-Whitney U
Exercise 25.2
Identifying ANOVA designs
From the following brief descriptions of research designs try to identify the ANOVA type with factors and levels. For instance, an answer might be one-way unrelated or 2 x 3 x 2 mixed.
One-way
2 x 3
3 x 3
2 x 3 x 2
3 x 4
Unrelated
Repeated measures
Mixed
1. Participants are asked to rate a fictitious person having been told they are either pro-hanging, anti-hanging or neutral.
Show answer
One-way unrelated
2. Participants are presented with both positive and negative traits for later recall and are induced into either a depressed, neutral or elated mood. Does induced mood affect numbers of either positive or negative traits recalled?
Show answer
2 (traits) x 3 (mood) mixed
3. All participants are asked to name colours of colour patches, non-colour words and colour words.
Show answer
One-way repeated measures
4. Male and female clients experience either psychoanalysis, behaviour modification or humanistic therapy and effects are assessed. Do males and females respond differently to the types of therapy?
Show answer
2 (sex) X 3 (therapy) unrelated.
5. Participants are given either coffee, alcohol or a placebo and are all asked to perform a visual monitoring task under conditions of loud, moderate, intermittent and no noise.
Show answer
3 (stimulant) x 4 (noise) mixed.
6. Older or younger participants are asked to use one of three different memorising methods.
Show answer
2 (age) x 3(memory method) unrelated.
7. Extroverts and introverts are asked to perform an energetic and later a dull task after being given a stimulant. On a subsequent occasion they are given a tranquiliser and repeat the two tasks. Later still they are given a placebo and asked to repeat the tasks.
Show answer
2 (personality) x 2 (task) x 3 (drug) mixed
8. Participants perform tasks in front of an audience and when alone. They are first asked to sort cards into three piles, then four piles and finally five piles.
Show answer
2 (audience) x 3 (sort task) repeated measures.
Weblinks
Choosing a significance test for your data
This great site for social science statistics provides calculators for all tests up to One-way ANOVA (select the calculators top tab), several other useful pages and a ‘quiz’ tab where you can select the number of questions you’d like to answer:
Social Science Statistics (socscistatistics.com)
This is part of David Lane’s hyperstat site listed in Chapter 13:. Everything statistical:
Rice Virtual Lab in Statistics (RVLS) (onlinestatbook.com)
All about choosing an appropriate statistical test for your data:
Choosing the Right Statistical Test | Types & Examples (scribbr.com)
“Statistics at Square One”. Statistical concepts explained. Part of the British Medical Journal pages but open to all at present:
http://bmj.com/collections/statsbk/index.shtml
A reminder about other useful sites linked in earlier chapters- The analysis Factor – Chapter 21
Research Methods Knowledge Base – linked in Chapter 1
This is the BBC’s Bitesize Maths site with simple explanations of maths you’ve probably forgotten from school but need now!
www.bbc.co.uk/education/subjects/z6vg9j6
The link to G*Power (again):
www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/download-and-register