Here’s the scenario: You have been hired by the new online mega-petstore G.O.P. (Guaranteed
Overnight Pets) to formulate a new fish food. Their current fish food produces rather slow
growth, requiring too many days to rear fish to an acceptable weight for being sent through the mail.
You’ve done the hard work — tested various kinds of insect larvae, nematodes, and algae,
you have created (you believe) the perfect formula, named it “Fish2Whale”, and
sent it off. You are waiting for your paycheck when instead you receive a rather terse note:
Sir/Madam:
Please be advised that we do not intend to make any payment for your formula,
“Fish2Whale”. We tried it out on Edgar, the company mascot, and he didn’t
seem to get very big.
Sincerely,
G.O.P.
You might be excused for feeling a bit put out at this point. Management at G.O.P. has not
followed the scientific method when they tested your formula. Below I list several statements
about fishfood. Pick out the one(s) that describe/s a satisfactory test of your fishfood. (This
is not the same as picking the statements that you think are the most likely explanations. All
we care about is that the idea is stated in a way that is testable.)
Good scientific procedure?
| yes no |
If you feed a group of fish with Fish2Whale and they get to be at least 10 cm long, that proves that Fish2Whale works. not a good procedure: there is no group to compare the Fish2Whale fish with. |
| yes no |
You should write back and tell the company that Fish2Whale is healthy for fish and will make their scales shine. This is not a scientific procedure, this sounds like an advertisement! Fish2Whale is healthy compared to what? And how will you compare their effects? |
| yes no |
You should feed one fish Fish2Whale and one fish normal food, and see which fish gets bigger. it’s good that you are comparing your treatment to a control, but what if the ‘treatment’ fish (the one you’re feeding Fish2Whale to) happens to be unwell? |
| yes no |
You should write back and tell them that its stupid to use the old food when they’ve already bought the new food. “Stupid” is a value judgement, not a testable proposition. This is not even a commercial, it’s just career suicide! |
| yes no |
You should feed normal fish food to one group of fish (the control group) and Fish2Whale to another group of fish (the treatment group) and see which group does better. This is pretty close. You have a comparison and replication, but you don’t have a way to measure which group is “doing better” (length? weight? number of surviving offspring? scale shininess?) |
| yes no |
You should feed normal fish food to one group of fish (the control group) and Fish2Whale to another group of fish (the treatment group) and see which group grows longer in length. This is a sound scientific procedure. You have a way to measure success, something to compare your success to, and replication to ensure that random effects don’t stuff up your experiment. |
| Answers: |
no, no, no, no, no, yes
|
Once again, make sure that when you design an experiment, you think in terms of
MEASURE-COMPARE-REPLICATE. And when you are analysing a statement above, look to
see how the authors used each of these elements.