Scales are controversial: A weighty discussion
Daniel Weisberg Dr. Sarah R. Beck
The rating scale in literature
• Amsel and Smalley (2000)
– Two dolls choose one of two gift boxes and keep whatever is inside – How happy is Susan with the gift she received? Not at all happy, a little happy, pretty happy, or very happy? (The Initial q.)
– After seeing what could have been won… – How happy is Susan with the gift she received? Not at all happy, a little happy, pretty happy, or very happy? (The Alternative q.)
The rating scale in literature
• Amsel et al., (unpublished manuscript) – exp. 4
– Imagine Billy the Bird got a present. How would he feel? Happy, sad, or not happy or sad? (Initial q.)
– Billy opens the present and he found it was better than what he expected it to be. Would Billy be happier, the same as before, or less happy than before? (Alternative q.)
The rating scale in literature
• Amsel et al., (unpublished manuscript) – exp. 3
– Participants dealt two face-down cards vs. experimenter’s one face-up card. Asked to turn over one card – which tied the experimenters. – How do you feel about the card you turned over? Happy, sad, or not happy or sad? (Initial q.)
– Participants see what they could have won… – How do you feel now about the card you turned over? Happy, sad, or not happy or sad? (Alternative q.)
Three scales
– Identical initial and alternative question – Flip over faces (happier vs. less happy) – Addingthe ‘now’ Adding the ‘now’
– Implemented this scale into four experiments, but added two extra faces – Ranging from 1 (very unhappy) to 5 (very happy)
Experiment 1
– Differences in children’s understanding of their own and others’ regret and relief (submitted, Developmental Psychology) – A games task Alternative box, alternative question (‘now’), – Regret trials (win 2/3,Initial box, initial question, initial score alternative could have won 8) score – Relief trials (win 2/3, could have won 0)
Experiment 1
– Alternative score – initial score – = range of between -4 and 4 for each participant – A negative result significantly different from 0, demonstration of understanding of regret – A positive result significantly different from 0, demonstration of understanding of relief – From 5-6-years-old, children were able to experience regret t(20) = -3.700, p = .001 – From 7-8-years-old, children were able to experience relief t(18) = 4.925, p < .001. – (Age 5: t(20) = .899, p = .379)
Experiment 3
– One condition replicated the games task of experiment 1 – Regret Age 5-6: t(13) = -3.044, p < .009 Age 6-7: t(12) = -5.189, p < .001 – Relief Age 5-6: t(13) = -.960, p = .355 Age 6-7: t(12) = 1.244, p = .237 Adults t(7) = 5.383, p = .001
A new scale?
– May be some participants who use extremes of the scale – Regret – initial score of 1, alternative score cannot be lower – Relief – initial score of 5, alternative score cannot be higher – Three-point scale for initial question – Five-point scale for alternative question
– Pilot on 20 children (age 5-6: n = 10, age 6-7: n = 10) in experiment 4
Experiment 4
– One condition replicated games task in experiment 1 & 3 with a different methodology – Regret and relief trials
Relief replication trial
Experiment 4
– 5-6- and 6-7-year-olds were combined – Regret replication trial: t(19) = -1.116, p = .278 – Relief replication trials: – t(19) = .309, p = .761
5
4
3
2
1
– Inconsistent pattern of results
Experiment 4b
– Regret replication trials Initial win trials
• Win 2/3, could have won3 lost 8 • 5-6: t(31) = -4.574,p = .126 1.574, p < .001 • 6-7: t(38) = -7.149,p < .001 4.312, p < .001
– Relief lose trials trials Initial replication
• Win 2/3, could have won 0 Lose 2/3, could have lost 8 • 5-6: t(31) = -.360, p = .721 2.745, p = .010 • 6-7: t(38) = 1.404, p = .168 5.531, < .001
However…
– Relief trials e.g., Initial win trials (win 2/3, lost 3) and initial lose trials (lose 2/3, lost 8) Initial score of 5 – alternative score cannot be higher – Regret trials e.g., win 2/3, won 8 – Initial score of 1 – alternative score cannot be lower – Take out affected participants and re-run analyses…
Analyses re-run
.355
A new scale… take 2?
Experiment 1/3
Initial Win+ trials (W2/3, W8) Regret Initial Win- trials (W2/3, L3) Relief Initial Lose+ trials (L2/3, W3) Regret Initial Lose- trials (L2/3, L8) Relief
“How do you feel about choosing Experiment your box now?”4b “Happier, sadder or the same as before?” (counterbalanced) Experiment 4b
From bad to worse…
– Relief trials: ‘happier’ is correct. ‘The same’ or ‘sadder’ is wrong – Regret trials: ‘sadder’ is correct. ‘The same’ or ‘happier’ is wrong
Sadder Happier Sadder Happier Regret (Sadder) Sadder Happier Relief (Happier) Happier Sadder Happier Sadder Happier Sadder
Sadder Happier Sadder Happier Regret (Sadder) Sadder Happier Relief (Happier)
Happier Sadder Happier Sadder Happier Sadder
Summary
Initial: How do you feel about choosing your box? Alt: How do you feel about choosing your box now?
Initial: How do you feel about choosing your box? Alt: How do you feel about choosing your box now?
Happier, sadder or the same as before?
Initial: How do you feel about choosing your box? Alt: How do you feel about choosing your box now? Happier, sadder or the same as before?
Alternatives?
Initial: How do you feel about choosing your box? Alt: How do you feel about choosing your box now? Happier or sadder than the face you chose?
Alternatives?
Initial: How do you feel about choosing your box? Alt: How do you feel about choosing your box now?
Thanks, any suggestions?
From bad to worse…
– Relief trials: ‘happier’ is correct. ‘The same’ or ‘sadder’ is wrong – Regret trials: ‘sadder’ is correct. ‘The same’ or ‘happier’ is wrong
Extremes of the scale?
– Experiment 1
– Experiment 3
