So now that you know all the arcane details of performing a t-test, let’s take one last step back and look at the big picture. Remember that the number you are calculating is supposed to tell you how important your treatment was compared to the “random error” in your experiment. As you move the sliders below, you can see how the mean and standard deviation in each group affects the t-test outcome.
Before you start pushing sliders, think about which part of the fraction should change as you move the averages farther apart or closer together? How will that affect the bottom line (whether or not your results are significant)?
And what will happen if you increase the SD for one or both treatments?
And how much overlap is too much?