Hypothesis Testing Part 2 - A Visual Approach
28 Jan 2009 Rob Slazas 7 comments 850 views
Rob Slazas is back with some more Statistics goodness. Click on the link to see more of Rob’s work.
For those of us who consider ourselves “efficient thinkers” (outsiders might use the term “lazy”), here is a low-effort way of comparing the central tendencies between groups WITHOUT using any hypothesis test functions! I used a t-test in the last post to show that my mean long-jump was farther than Quan’s mean long-jump. Here we will get the same task done using a boxplot with the notch parameter. Let’s examine “notches” a little bit and then try an example.
Contents
Boxplot Bonus!
Have I mentioned that I really like boxplots? Well, one more reason to love them is found in the notch parameter. A notch is a V-shaped indentation in the box, centered on the median line. This feature is a quick way to decide if the medians in any two boxes are different from each other at the alpha = 5% significance level. If the notches overlap (looking horizontally across the boxes in the default orientation), they are not different. Separated notches are different. It’s that simple.

Looking at the figure above, I have put a blue shaded bar over the notched region of boxplot #1. Since the notched region of #1 overlaps the notch of #3, you can say with 95% confidence that the medians of #1 and #3 are not different. Also, since the bar does not overlap any part of the notch of #2, you can say with 95% confidence that medians #1 and #2 are different.
A Notched Example - Faster Cars
Since I’m an engineer, I like to break test things to see how they work. I want to know if a certain heat treatment will make the metal frame of my racecar stiffer (better handling). This will help me race Quan through the Hollywood hills. I happen to have an instrument that gives readings on the flexibility of metal in units called “flexons.” Since I want a stiffer frame, less “flexons” is better.

Using a traditional 2-sample t-test, I would gather two sets of metal to test. The first set would be my control - I would do nothing but measure how many “flexons” it has. The second group would be my experimental group - I would perform my new heat treatment on it AND THEN measure its “flexons”. The MATLAB command would look like this (keeping the default of alpha = 0.05):
load flexons.mat;
[h,p,ci] = ttest2(control,heattreat)
h =
1
p =
0.0014
ci =
8.6202
33.9798
…Which you recall means that we reject H0 in favor of H1; little-p is much less than our alpha of 0.05, so we have reached statistical significance; and the average difference between the control and experimental groups is between 8.6 and 34 flexons.
…OR, instead of thinking too much about setting up the hypothesis test and interpreting the results, I could take the same two sets of data and graph them in a boxplot with the “notch” parameter turned on. Like so:
boxplot([control,heattreat],'notch','on','labels',{'controls';'heattreated'}); % dress up the plot a little bit (optional) ylabel('Flexons'); title('Racecar Frame Flex Test');

Since the extents of the “V” shaped notches do not overlap each other, I can say with 95% confidence that the median flexibility after my new heat treatment is lower than without it! Lower “flexons” = stiffer frame = faster car = beating Quan to the bottom of the hill!
Wrapping Up
Hopefully this neat little feature comes in handy for you sometime. If there are others you know of and want to share, please post below in the comments. And as usual, your questions and other comments are welcome.
7 Responses to “Hypothesis Testing Part 2 - A Visual Approach”
Leave a Reply
Include MATLAB code in your comment by doing the following:
<pre lang="MATLAB">
%insert code here
</pre>


But what if you put the blue-shaded bar over the notched region of plot #3 — then the median of plot #1 will be outside this bar and suddenly they *are* different…
Marcus,
it is not about looking if the median of one group is contained inside the notched region for the other group (that’s what you suggest), it is about looking if the two notched intervals have any part in common. The results then do not change depending on whether you draw the shaded region around notches of group 1 or group 3.
Is there a way to use a boxplot to show the mean and standard deviation of a sample instead of the median and quartile ranges? This would at least make the boxplot more analogous to a t-test…
Hello Rob,
First off, you are my hero. Next:
If the notches are positioned such that it’s difficult to tell if they overlap, then this boxplot technique doesn’t seem too useful. I would have to say that for visual purposes, it’s a great technique, but I’m a numbers guy personally, and sometimes there is more than meets the eye!
A question I’ve always had is the following:
Lets assume alpha =0.05
If little p = 0.04 in one case, and if little p = 0.01 in another, how much “stronger” is our rejection in the second case? Is there a way to quantify it?
Looking forward to the next post,
Quan
@ Marcus,
Posa is correct (thanks Posa for helping out!): if any part of the notches overlap, then the two medians are NOT different. Perhaps I should update the figure in the post to better show that. It’s an unfortunate coincidence that #3’s median is right under the edge of the bar from #1’s notch. Sorry about the confusion.
@ DP,
You are reading my mind. I haven’t found anything in the boxplot function arguments to change for that. If there isn’t anything in the file exchange already, then this might be an opportunity to publish ‘boxplot2′, which uses the mean and SD to construct the box instead of median and iqr. Let me know if you want to collaborate on that.
@ Quan,
You always ask questions outside the box (pun intended)! Yes, this is a graphical analysis technique and is subject to the same advantages (easy to use, understand, and explain to others) and inaccuracies (visual inspections / judgements). If the result is very important to you, and the notches are so close that you find yourself squinting at the screen and zooming in, then I recommend using the traditional method. Some folks just find graphical methods more convenient.
As far as your question about little-p, this takes some more explanation… perhaps in a future post. I get LOTS of questions like this about little-p! For now just focus on your alpha - if little-p is less than alpha then you have reached significance. If little-p was very low and you want to try to prove the difference to a higher standard, then lower your alpha or beta and test again (don’t forget to adjust sample size if necessary!).
HTH,
Rob
hi
would be interesting to know how the notches are exactly computed? Don’t like this black box approach so much unless it is explicitly stated how they are derived.
thanks
Hi again
that’s the part within the boxplot function where the notches are computed. p25, p50, p75 are the 25, 50 and 75 percentiles of the distribution.
explanations in :
http://www.mathworks.com/matlabcentral/newsreader/view_thread/78341
cheers
s.