forked from saundersg/Statistics-Notebook
-
Notifications
You must be signed in to change notification settings - Fork 3
/
ANOVA.html
1372 lines (1297 loc) · 51.3 KB
/
ANOVA.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta http-equiv="X-UA-Compatible" content="IE=EDGE" />
<title>Analysis Of Variance (ANOVA)</title>
<script src="site_libs/header-attrs-2.14/header-attrs.js"></script>
<script src="site_libs/jquery-3.6.0/jquery-3.6.0.min.js"></script>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link href="site_libs/bootstrap-3.3.5/css/cerulean.min.css" rel="stylesheet" />
<script src="site_libs/bootstrap-3.3.5/js/bootstrap.min.js"></script>
<script src="site_libs/bootstrap-3.3.5/shim/html5shiv.min.js"></script>
<script src="site_libs/bootstrap-3.3.5/shim/respond.min.js"></script>
<style>h1 {font-size: 34px;}
h1.title {font-size: 38px;}
h2 {font-size: 30px;}
h3 {font-size: 24px;}
h4 {font-size: 18px;}
h5 {font-size: 16px;}
h6 {font-size: 12px;}
code {color: inherit; background-color: rgba(0, 0, 0, 0.04);}
pre:not([class]) { background-color: white }</style>
<script src="site_libs/navigation-1.1/tabsets.js"></script>
<script src="site_libs/navigation-1.1/codefolding.js"></script>
<style type="text/css">
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
span.underline{text-decoration: underline;}
div.column{display: inline-block; vertical-align: top; width: 50%;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
</style>
<link rel="stylesheet" href="styles.css" type="text/css" />
<style type = "text/css">
.main-container {
max-width: 940px;
margin-left: auto;
margin-right: auto;
}
img {
max-width:100%;
}
.tabbed-pane {
padding-top: 12px;
}
.html-widget {
margin-bottom: 20px;
}
button.code-folding-btn:focus {
outline: none;
}
summary {
display: list-item;
}
details > summary > p:only-child {
display: inline;
}
pre code {
padding: 0;
}
</style>
<style type="text/css">
.dropdown-submenu {
position: relative;
}
.dropdown-submenu>.dropdown-menu {
top: 0;
left: 100%;
margin-top: -6px;
margin-left: -1px;
border-radius: 0 6px 6px 6px;
}
.dropdown-submenu:hover>.dropdown-menu {
display: block;
}
.dropdown-submenu>a:after {
display: block;
content: " ";
float: right;
width: 0;
height: 0;
border-color: transparent;
border-style: solid;
border-width: 5px 0 5px 5px;
border-left-color: #cccccc;
margin-top: 5px;
margin-right: -10px;
}
.dropdown-submenu:hover>a:after {
border-left-color: #adb5bd;
}
.dropdown-submenu.pull-left {
float: none;
}
.dropdown-submenu.pull-left>.dropdown-menu {
left: -100%;
margin-left: 10px;
border-radius: 6px 0 6px 6px;
}
</style>
<script type="text/javascript">
// manage active state of menu based on current page
$(document).ready(function () {
// active menu anchor
href = window.location.pathname
href = href.substr(href.lastIndexOf('/') + 1)
if (href === "")
href = "index.html";
var menuAnchor = $('a[href="' + href + '"]');
// mark it active
menuAnchor.tab('show');
// if it's got a parent navbar menu mark it active as well
menuAnchor.closest('li.dropdown').addClass('active');
// Navbar adjustments
var navHeight = $(".navbar").first().height() + 15;
var style = document.createElement('style');
var pt = "padding-top: " + navHeight + "px; ";
var mt = "margin-top: -" + navHeight + "px; ";
var css = "";
// offset scroll position for anchor links (for fixed navbar)
for (var i = 1; i <= 6; i++) {
css += ".section h" + i + "{ " + pt + mt + "}\n";
}
style.innerHTML = "body {" + pt + "padding-bottom: 40px; }\n" + css;
document.head.appendChild(style);
});
</script>
<!-- tabsets -->
<style type="text/css">
.tabset-dropdown > .nav-tabs {
display: inline-table;
max-height: 500px;
min-height: 44px;
overflow-y: auto;
border: 1px solid #ddd;
border-radius: 4px;
}
.tabset-dropdown > .nav-tabs > li.active:before {
content: "";
font-family: 'Glyphicons Halflings';
display: inline-block;
padding: 10px;
border-right: 1px solid #ddd;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open > li.active:before {
content: "";
border: none;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open:before {
content: "";
font-family: 'Glyphicons Halflings';
display: inline-block;
padding: 10px;
border-right: 1px solid #ddd;
}
.tabset-dropdown > .nav-tabs > li.active {
display: block;
}
.tabset-dropdown > .nav-tabs > li > a,
.tabset-dropdown > .nav-tabs > li > a:focus,
.tabset-dropdown > .nav-tabs > li > a:hover {
border: none;
display: inline-block;
border-radius: 4px;
background-color: transparent;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open > li {
display: block;
float: none;
}
.tabset-dropdown > .nav-tabs > li {
display: none;
}
</style>
<!-- code folding -->
<style type="text/css">
.code-folding-btn { margin-bottom: 4px; }
</style>
</head>
<body>
<div class="container-fluid main-container">
<div class="navbar navbar-default navbar-fixed-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-bs-toggle="collapse" data-target="#navbar" data-bs-target="#navbar">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="index.html">Statistics Notebook</a>
</div>
<div id="navbar" class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
R Help
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="RCommands.html">R Commands</a>
</li>
<li>
<a href="RMarkdownHints.html">R Markdown Hints</a>
</li>
<li>
<a href="RCheatSheetsAndNotes.html">R Cheatsheets & Notes</a>
</li>
<li>
<a href="DataSources.html">Data Sources</a>
</li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
Describing Data
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="GraphicalSummaries.html">Graphical Summaries</a>
</li>
<li>
<a href="NumericalSummaries.html">Numerical Summaries</a>
</li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
Making Inference
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="MakingInference.html">Making Inference</a>
</li>
<li>
<a href="tTests.html">t Tests</a>
</li>
<li>
<a href="WilcoxonTests.html">Wilcoxon Tests</a>
</li>
<li>
<a href="Kruskal.html">Kruskal-Wallis Test</a>
</li>
<li>
<a href="ANOVA.html">ANOVA</a>
</li>
<li>
<a href="LinearRegression.html">Linear Regression</a>
</li>
<li>
<a href="LogisticRegression.html">Logistic Regression</a>
</li>
<li>
<a href="ChiSquaredTests.html">Chi Squared Tests</a>
</li>
<li>
<a href="PermutationTests.html">Randomization</a>
</li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
Analyses
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="./Analyses/StudentHousing.html">Good Example Analysis</a>
</li>
<li>
<a href="./Analyses/StudentHousingPOOR.html">Poor Example Analysis</a>
</li>
<li>
<a href="./Analyses/Rent.html">Rent</a>
</li>
<li>
<a href="./Analyses/Stephanie.html">Stephanie</a>
</li>
<li>
<a href="./Analyses/t Tests/HighSchoolSeniors.html">High School Seniors</a>
</li>
<li>
<a href="./Analyses/Wilcoxon Tests/RecallingWords.html">Recalling Words</a>
</li>
<li>
<a href="./Analyses/ANOVA/MyTwoWayANOVA.html">My Two-way ANOVA</a>
</li>
<li>
<a href="./Analyses/Kruskal-Wallis Test/Food.html">Food</a>
</li>
<li>
<a href="./Analyses/Linear Regression/MySimpleLinearRegression.html">My Simple Linear Regression</a>
</li>
<li>
<a href="./Analyses/Linear Regression/CarPrices.html">Car Prices</a>
</li>
<li>
<a href="./Analyses/Logistic Regression/MyLogisticRegression.html">My Logistic Regression</a>
</li>
<li>
<a href="./Analyses/Chi Squared Tests/MyChiSquaredTest.html">My Chi-sqaured Test</a>
</li>
</ul>
</li>
</ul>
<ul class="nav navbar-nav navbar-right">
</ul>
</div><!--/.nav-collapse -->
</div><!--/.container -->
</div><!--/.navbar -->
<div id="header">
<div class="btn-group pull-right float-right">
<button type="button" class="btn btn-default btn-xs btn-secondary btn-sm dropdown-toggle" data-toggle="dropdown" data-bs-toggle="dropdown" aria-haspopup="true" aria-expanded="false"><span>Code</span> <span class="caret"></span></button>
<ul class="dropdown-menu dropdown-menu-right" style="min-width: 50px;">
<li><a id="rmd-show-all-code" href="#">Show All Code</a></li>
<li><a id="rmd-hide-all-code" href="#">Hide All Code</a></li>
</ul>
</div>
<h1 class="title toc-ignore">Analysis Of Variance (ANOVA)</h1>
</div>
<script type="text/javascript">
function showhide(id) {
var e = document.getElementById(id);
e.style.display = (e.style.display == 'block') ? 'none' : 'block';
}
function openTab(evt, tabName) {
var i, tabcontent, tablinks;
tabcontent = document.getElementsByClassName("tabcontent");
for (i = 0; i < tabcontent.length; i++) {
tabcontent[i].style.display = "none";
}
tablinks = document.getElementsByClassName("tablinks");
for (i = 0; i < tablinks.length; i++) {
tablinks[i].className = tablinks[i].className.replace(" active", "");
}
document.getElementById(tabName).style.display = "block";
evt.currentTarget.className += " active";
}
</script>
<hr />
<p>An ANOVA is for testing the equality of several means simultaneously.
A single <em>quantitative</em> response variable is required with one or
more <em>qualitative</em> explanatory variables, i.e., factors.</p>
<div style="font-size:0.8em;">
Note: A factor is defined as a qualitative variable containing at least
two categories. The categories of the factor are referred to as the
“levels” of the factor.
</div>
<hr />
<div id="one-way-anova"
class="section level3 tabset tabset-fade tabset-pills">
<h3 class="tabset tabset-fade tabset-pills">One-way ANOVA</h3>
<div style="float:left;width:125px;" align="center">
<p><img src="Images/QuantYQualXg3plus.png" width=58px;></p>
</div>
<p>Each experimental unit is assigned to exactly one factor-level
combination. Another way to say this is “one measurement per individual”
(no repeated measures) and “equal numbers of individuals per group”.</p>
<div id="overview" class="section level4">
<h4>Overview</h4>
<div style="padding-left:125px;">
<p>An ANOVA is only appropriate when all of the following are
satisfied.</p>
<ol style="list-style-type: decimal">
<li><p>The sample(s) of data can be considered to be representative of
their population(s).</p></li>
<li><p>The data is normally distributed in each group. (This can safely
be assumed to be satisfied when the <em>residuals</em> from the ANOVA
can be assumed to be normally distributed when seen in a Q-Q
Plot.)</p></li>
<li><p>The population variance of each group can be assumed to be the
same. (This can be safely assumed to be satisfied when the
<em>residuals</em> from the ANOVA show constant variance, i.e., are
similarly vertically spread out in a Residuals versus fitted-values
plot.)</p></li>
</ol>
<p><strong>Hypotheses</strong></p>
<p>For a One-way ANOVA</p>
<p><span class="math display">\[
H_0: \mu_1 = \mu_2 = \ldots = \mu
\]</span> <span class="math display">\[
H_a: \mu_i \neq \mu \ \text{for at least one} \ i
\]</span></p>
<p><strong>Mathematical Model</strong></p>
<p>A typical model for a one-way ANOVA is of the form <span
class="math display">\[
Y_{ij} = \mu_i + \epsilon_{ij}
\]</span> where <span class="math inline">\(\mu_i\)</span> is the mean
for level (group) <span class="math inline">\(i\)</span>, and <span
class="math inline">\(\epsilon_{ij} \sim N(0,\sigma^2)\)</span> is the
error term for each point <span class="math inline">\(j\)</span> within
level (group) <span class="math inline">\(i\)</span>.</p>
<hr />
</div>
</div>
<div id="r-instructions" class="section level4">
<h4>R Instructions</h4>
<div style="padding-left:125px;">
<p><strong>Console</strong> Help Command: <code>?aov()</code></p>
<ul>
<li><code>myaov</code> is some name you come up with to store the
results of the <code>aov()</code> test.</li>
<li><code>Y</code> must be a “numeric” vector of the quantitative
response variable.</li>
<li><code>X</code> is a qualitative variable (should have
<code>class(X)</code> equal to <code>factor</code> or
<code>character</code>. If it does not, use <code>as.factor(X)</code>
inside the <code>aov(Y ~ as.factor(X),...)</code> command.</li>
<li><code>YourDataSet</code> is the name of your data set.</li>
</ul>
<p><strong>Perform the ANOVA</strong></p>
<p><code>myaov <- aov(Y ~ X, data=YourDataSet)</code></p>
<p><code>summary(myaov)</code></p>
<p><strong>Diagnose ANOVA Assumptions</strong></p>
<p><code>par(mfrow=c(1,2))</code></p>
<p><code>plot(myaov, which=1:2)</code></p>
<p><br/></p>
<p><strong>Example Code</strong></p>
<p>Hover your mouse over the example codes to learn more.</p>
<p><em>Perform the ANOVA</em></p>
<a href="javascript:showhide('oneWayAnova')">
<div class="hoverchunk">
<p><span class="tooltipr"> chick.aov <- <span class="tooltiprtext">
Saves the results of the ANOVA test as an object named
‘chick.aov’.</span> </span><span class="tooltipr"> aov( <span
class="tooltiprtext">‘aov()’ is a function in R used to perform the
ANOVA.</span> </span><span class="tooltipr"> weight <span
class="tooltiprtext">Y is ‘weight’, which is a numeric variable from the
chickwts dataset.</span> </span><span class="tooltipr"> ~ <span
class="tooltiprtext">‘~’ is the tilde symbol used to separate the Y and
X in a model formula.</span> </span><span class="tooltipr"> feed, <span
class="tooltiprtext">X is ‘feed’, which is a qualitative variable in the
chickwts dataset, or more specifically, a factor with six levels:
“casein”, “horsebean”, and so on… Use str(chickwts) to see this. </span>
</span><span class="tooltipr"> data = chickwts) <span
class="tooltiprtext"> ‘chickwts’ is a dataset in R.</span>
</span><br><span class="tooltipr"> summary( <span class="tooltiprtext">
‘summary()’ shows the results of the ANOVA.</span> </span><span
class="tooltipr"> chick.aov) <span class="tooltiprtext"> ‘chick.aov’ is
the name of the ANOVA.</span> </span><span class="tooltipr"> <br />
<span class="tooltiprtext">Press Enter to run the code if you have typed
it in yourself. You can also click here to view the output.</span>
</span><span class="tooltipr" style="float:right;font-size:.8em;">
Click to View Output <span class="tooltiprtext">Click to View
Output.</span> </span></p>
</div>
</a>
<div id="oneWayAnova" style="display:none;">
<pre><code>## Df Sum Sq Mean Sq F value Pr(>F)
## feed 5 231129 46226 15.37 5.94e-10 ***
## Residuals 65 195556 3009
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1</code></pre>
</div>
<p><br/></p>
<p><em>Diagnose the ANOVA</em></p>
<a href="javascript:showhide('oneWayAnovaCheck')">
<div class="hoverchunk">
<p><span class="tooltipr"> par( <span class="tooltiprtext">‘par’ is a R
function that can be used to set or query graphical parameters.</span>
</span><span class="tooltipr"> mfrow = c(1,2)) <span
class="tooltiprtext">The <code>mfrow</code> parameter controls “multiple
frames on a row”. In this case, the c(1,2) specifies 1 row of 2 plots.
This will cause the two diagnostic plots to be placed side-by-side.
</span> </span><br><span class="tooltipr"> plot( <span
class="tooltiprtext">‘plot’ is a R function for the plotting of R
objects.</span> </span><span class="tooltipr"> chick.aov, <span
class="tooltiprtext">‘chick.aov’ is the name of the ANOVA.</span>
</span><span class="tooltipr"> which = 1:2) <span
class="tooltiprtext">The <code>which=1:2</code> selects “which” of 6
available plots we want to have graphed. In this case, 1 shows the
Residuals vs Fitted, and 2 shows the Normal QQ-plot. Both are needed to
check the ANOVA assumptions.</span> </span><span class="tooltipr"
style="float:right;font-size:.8em;"> Click to View Output <span
class="tooltiprtext">Click to View Output.</span> </span></p>
</div>
</a>
<div id="oneWayAnovaCheck" style="display:none;">
<p><img src="ANOVA_files/figure-html/unnamed-chunk-2-1.png" width="672" /></p>
</div>
<hr />
</div>
</div>
<div id="explanation" class="section level4">
<h4>Explanation</h4>
<div style="padding-left:125px;">
<p>Analysis of variance (ANOVA) is often applied to the scenario of
testing for the equality of three or more means from (possibly) separate
normal distributions of data. The normality assumption is required. No
matter the sample size. If the distributions are skewed then a
nonparametric test should be applied instead of ANOVA.</p>
<p><br /></p>
<div id="one-way-anova-1" class="section level5">
<h5>One-Way ANOVA</h5>
<p>One-way ANOVA is when a completely randomized design is used with a
single factor of interest. A typical mathematical model for a one-way
ANOVA is of the form <span class="math display">\[
Y_{ik} = \mu_i + \epsilon_{ik} \quad (\text{sometimes written}\ Y_{ik}
= \mu + \alpha_i + \epsilon_{ik})
\]</span> where <span class="math inline">\(\mu_i\)</span> is the mean
of each group (or level) <span class="math inline">\(i\)</span> of a
factor, and <span class="math inline">\(\epsilon_{ik}\sim
N(0,\sigma^2)\)</span> is the error term. The plot below demonstrates
what these symbols represent. Note that the notation <span
class="math inline">\(\epsilon_{ik}\sim N(0,\sigma^2)\)</span> states
that we are assuming the error term <span
class="math inline">\(\epsilon_{ik}\)</span> is normally distributed
with a mean of 0 and a standard deviation of <span
class="math inline">\(\sigma\)</span>.</p>
<p><img src="ANOVA_files/figure-html/unnamed-chunk-4-1.png" width="672" /></p>
<div id="hypotheses" class="section level6">
<h6>Hypotheses</h6>
<p>The aim of ANOVA is to determine which hypothesis is more plausible,
that the means of the different distributions are all equal (the null),
or that at least one group mean differs (the alternative).
Mathematically, <span class="math display">\[
H_0: \mu_1 = \mu_2 = \ldots = \mu_m = \mu
\]</span> <span class="math display">\[
H_a: \mu_i \neq \mu \quad \text{for at least one}\ i\in\{1,\ldots,m\}.
\]</span> In other words, the goal is to determine if it is more
plausible that each of the <span class="math inline">\(m\)</span>
different samples (where each sample is of size <span
class="math inline">\(n\)</span>) came from the same normal distribution
(this is what the null hypothesis claims) or that at least one of the
samples (and possibly several or all) come from different normal
distributions (this is what the alternative hypothesis claims).</p>
</div>
<div id="visualizing-the-hypotheses" class="section level6">
<h6>Visualizing the Hypotheses</h6>
<p>The first figure below demonstrates what a given scenario might look
like when all <span class="math inline">\(m=3\)</span> samples of data
are from the same normal distribution. In this case, the null hypothesis
<span class="math inline">\(H_0\)</span> is true. Notice that the
variability of the sample means is smaller than the variability of the
points.</p>
<p><img src="ANOVA_files/figure-html/unnamed-chunk-6-1.png" width="672" /></p>
<p>The figure below shows what a given scenario might look like for
<span class="math inline">\(m=3\)</span> samples of data from three
different normal distributions. In this case, the alternative hypothesis
<span class="math inline">\(H_a\)</span> is true. Notice that the
variability of the sample means, i.e., <span
class="math inline">\((\bar{x}_1,\bar{x}_2,\bar{x}_3)\)</span>, is
greater than the variability of the points.</p>
<p><img src="ANOVA_files/figure-html/unnamed-chunk-7-1.png" width="672" /></p>
</div>
<div id="explaining-the-name" class="section level6">
<h6>Explaining the Name</h6>
<p>The above plots are useful in understanding the mathematical details
behind ANOVA and why it is called <em>analysis of variance</em>. Recall
that variance is a measure of the spread of data. When data is very
spread out, the variance is large. When the data is close together, the
variance is small. ANOVA utilizes two important variances, the between
groups variance and the within groups variance.</p>
<ul>
<li><p><strong>Between groups variance</strong>–a measure of the
variability in the sample means, the <span
class="math inline">\(\bar{x}\)</span>’s.</p></li>
<li><p><strong>Within groups variance</strong>–a combined measure of the
variability of the points within each sample.</p></li>
</ul>
<p>The plot below combines the information from the previous plots for
ease of reference. It emphasizes the fact that when the null hypothesis
is true, the points should have a large variance (be really spread out)
while the sample means are relatively close together. On the other hand,
when the points are relative close together <em>within</em> each sample
and the sample means have a large variance (are really spread out) then
the alternative hypothesis is true. This is the theory behind analysis
of variance, or ANOVA.</p>
<p><img src="ANOVA_files/figure-html/unnamed-chunk-8-1.png" width="672" /></p>
</div>
<div id="calculating-the-test-statistic-f" class="section level6">
<h6>Calculating the Test Statistic, <span
class="math inline">\(F\)</span></h6>
<p>The ratio of the “between groups variation” to the “within groups
variation” provides the test statistic for ANOVA. Note that the test
statistic of ANOVA is an <span class="math inline">\(F\)</span>
statistic.</p>
<p><span class="math display">\[
F = \frac{\text{Between groups variation}}{\text{Within groups
variation}}
\]</span></p>
<p>It would be good to take a minute and <a
href="MakingInference.html#fdist">review the <span
class="math inline">\(F\)</span> distribution</a>. The <span
class="math inline">\(p\)</span>-value for ANOVA thus comes from an
<span class="math inline">\(F\)</span> distribution with parameters
<span class="math inline">\(p_1 = m-1\)</span> and <span
class="math inline">\(p_2 = n-m\)</span> where <span
class="math inline">\(m\)</span> is the number of samples and <span
class="math inline">\(n\)</span> is the total number of data points.</p>
<p><br /></p>
</div>
<div id="a-deeper-look-at-variance" class="section level6">
<h6>A Deeper Look at Variance</h6>
<p>It is useful to take a few minutes and explain the word
<em>variance</em> as well as mathematically define the terms “<em>within
group variance</em>” and “<em>between groups variance</em>.”</p>
<p>Variance is a statistical measure of the <em>variability</em> in
data. The square root of the variance is called the <em>standard
deviation</em> and is by far the more typical measure of spread. This is
because standard deviation is easier to interpret. However,
mathematically speaking, the variance is the more important
measurement.</p>
<p>As mentioned previously, the variance turns out to be the key to
determining which hypothesis is the most plausible, <span
class="math inline">\(H_0\)</span> or <span
class="math inline">\(H_a\)</span>, when several means are under
consideration. There are two variances that are important for ANOVA, the
“within groups variance” and the “between groups variance.”</p>
<p>Recall that the formula for computing a <a
href="NumericalSummaries.html#variance">sample variance</a> is given by
<span class="math display">\[
s^2 = \frac{\sum_{i=1}^n(x_i - \bar{x})^2}{n-1} \quad\leftarrow
\frac{\text{sum of squares}}{\text{degrees of freedom}}
\]</span> This formula has a couple of important pieces that are so
important they have been given special names. The <span
class="math inline">\(n-1\)</span> in the denominator of the formula is
called the “degrees of freedom.” The other important part of this
formula is the <span class="math inline">\(\sum_{i=1}^n(x_i -
\bar{x})^2\)</span>, which is called the “sum of squared errors” or
sometimes just the “sum of squares” or “SS” for short. Thus, the sample
variance is calculated by computing a “sum of squares” and dividing this
by the “degrees of freedom.”</p>
<p>It turns out that this general approach works for many different
contexts. Specifically, it allows us to compute the “within groups
variance” and the “between groups variance.” To introduce the
mathematical definitions of these two variances, we need to introduce
some new notation.</p>
<div style="padding-left:15px;">
<p>Let <span class="math inline">\(\bar{y}_{i\bullet}\)</span> represent
the sample mean of group <span class="math inline">\(i\)</span> for
<span class="math inline">\(i=1,\ldots,m\)</span>.</p>
<p>Let <span class="math inline">\(n_i\)</span> denote the sample size
in group <span class="math inline">\(i\)</span>.</p>
<p>Let <span class="math inline">\(\bar{y}_{\bullet\bullet}\)</span>
represent the sample mean of all <span class="math inline">\(n =
n_1+n_2+\cdots+n_m\)</span> data points.</p>
</div>
<p>The mathematical calculations for each of these variances is given as
follows. <span class="math display">\[
\text{Between groups variance} = \frac{\sum_{i=1}^m
(\bar{y}_{i\bullet}-\bar{y}_{\bullet\bullet})^2}{m-1} \leftarrow
\frac{\text{Between groups sum of squares}}{\text{Between groups degrees
of freedom}}
\]</span> <span class="math display">\[
\text{Within groups variance} =
\frac{\sum_{i=1}^m\sum_{k=1}^{n_i}(y_{ik}-\bar{y}_{i\bullet})^2}{n-m}
\leftarrow \frac{\text{Within groups sum of squares}}{\text{Within
groups degrees of freedom}}
\]</span></p>
<p><br /></p>
</div>
<div id="a-fabricated-example" class="section level6">
<h6>A Fabricated Example</h6>
<p>The following table provides three samples of data:
<strong>A</strong>, <strong>B</strong>, and <strong>C</strong>. These
samples were randomly generated from normal distributions using a
computer. The true means <span class="math inline">\(\mu_1,
\mu_2\)</span>, and <span class="math inline">\(\mu_3\)</span> of the
normal distributions are thus known, but withheld from you at this point
of the example.</p>
<table>
<thead>
<tr class="header">
<th align="right">A</th>
<th align="right">B</th>
<th align="right">C</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="right">13.15457</td>
<td align="right">13.17463</td>
<td align="right">16.66831</td>
</tr>
<tr class="even">
<td align="right">12.65225</td>
<td align="right">12.16277</td>
<td align="right">15.54719</td>
</tr>
<tr class="odd">
<td align="right">13.73061</td>
<td align="right">12.76905</td>
<td align="right">16.63074</td>
</tr>
<tr class="even">
<td align="right">14.43471</td>
<td align="right">13.38524</td>
<td align="right">15.06726</td>
</tr>
<tr class="odd">
<td align="right">13.79728</td>
<td align="right">12.02690</td>
<td align="right">15.57534</td>
</tr>
<tr class="even">
<td align="right">13.88599</td>
<td align="right">13.24651</td>
<td align="right">15.99915</td>
</tr>
<tr class="odd">
<td align="right">12.77753</td>
<td align="right">12.58386</td>
<td align="right">15.58995</td>
</tr>
<tr class="even">
<td align="right">13.81536</td>
<td align="right">12.64615</td>
<td align="right">16.99429</td>
</tr>
<tr class="odd">
<td align="right">13.03635</td>
<td align="right">12.52055</td>
<td align="right">15.47153</td>
</tr>
<tr class="even">
<td align="right">14.26062</td>
<td align="right">14.03566</td>
<td align="right">16.13330</td>
</tr>
</tbody>
</table>
<p>An ANOVA will be performed with the sample data to determine which
hypothesis is more plausible: <span class="math display">\[
H_0: \mu_1 = \mu_2 = \mu_3 = \mu
\]</span> <span class="math display">\[
H_a: \mu_i \neq \mu \ \text{for at least one} \ i \in \{1,\ldots,m\}
\]</span></p>
<p>To perform an ANOVA, we must compute the between groups variance and
the within groups variance. This requires the Between groups sums of
squares, within groups sums of squares, between groups degrees of
freedom, and the within groups degrees of freedom. Note that to get the
sums of squares, we first had to calculate <span
class="math inline">\(\bar{y}_{1\bullet}\)</span>, <span
class="math inline">\(\bar{y}_{2\bullet}\)</span>, <span
class="math inline">\(\bar{y}_{3\bullet}\)</span>, and <span
class="math inline">\(\bar{y}_{\bullet\bullet}\)</span> where the 1, 2,
3 corresponds to Samples A, B, and C, respectively. After some work, we
find these values to be <span class="math display">\[
\bar{y}_{1\bullet} = 13.55, \quad \bar{y}_{2\bullet} = 12.86 \quad
\bar{y}_{3\bullet} = 15.97
\]</span> and <span class="math display">\[
\bar{y}_{\bullet\bullet} = \frac{13.55+12.86+15.97}{3} = 14.13
\]</span> Using these values we can then compute the between groups sum
of squares and the within groups sum of squares according to the
formulas stated previously. This process is very tedious and will not be
demonstrated. Only the results are shown in the following table which
summarizes all the important information.</p>
<table>
<colgroup>
<col width="11%" />
<col width="27%" />
<col width="22%" />
<col width="13%" />
<col width="12%" />
<col width="12%" />
</colgroup>
<thead>
<tr class="header">
<th> </th>
<th>Degrees of Freedom</th>
<th>Sum of Squares</th>
<th>Variance</th>
<th>F-value</th>
<th>p-value</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>Between groups</td>
<td>2</td>
<td>53.3</td>
<td>26.67</td>
<td>70.2</td>
<td>2e-11</td>
</tr>
<tr class="even">
<td>Within groups</td>
<td>27</td>
<td>10.3</td>
<td>0.38</td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
</div>
<div id="anova-table" class="section level6">
<h6>ANOVA Table</h6>
<p>In general, the ANOVA table is created by</p>
<table>
<colgroup>
<col width="11%" />
<col width="27%" />
<col width="22%" />
<col width="13%" />
<col width="12%" />
<col width="12%" />
</colgroup>
<thead>
<tr class="header">
<th> </th>
<th>Degrees of Freedom</th>
<th>Sum of Squares</th>
<th>Variance</th>
<th>F-value</th>
<th>p-value</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>Between groups</td>
<td><span class="math inline">\(m-1\)</span></td>
<td><span class="math inline">\(\sum_{i=1}^m
n_i(\bar{y}_{i\bullet}-\bar{y}_{\bullet\bullet})^2\)</span></td>
<td><span class="math inline">\(\frac{\text{sum of
squares}}{\text{degrees of freedom}}\)</span></td>
<td><span class="math inline">\(\frac{\text{Between groups
variance}}{\text{Within groups variance}}\)</span></td>
<td><span class="math inline">\(F\)</span>-distribution tail
probability</td>
</tr>
<tr class="even">
<td>Within groups</td>
<td><span class="math inline">\(n-m\)</span></td>
<td><span
class="math inline">\(\sum_{i=1}^m\sum_{k=1}^{n_i}(y_{ik}-\bar{y}_{i\bullet})^2\)</span></td>
<td><span class="math inline">\(\frac{\text{sum of
squares}}{\text{degrees of freedom}}\)</span></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
<p><br /></p>
</div>
</div>
<div id="residuals" class="section level5">
<h5>ANOVA Assumptions</h5>
<p>The requirements for an analysis of variance (the assumptions of the
test) are two-fold and concern only the error terms, the <span
class="math inline">\(\epsilon_{ik}\)</span>.</p>
<ol style="list-style-type: decimal">
<li><p>The errors are normally distributed.</p></li>
<li><p>The variance of the errors is constant.</p></li>
</ol>
<p>Both of these assumptions were stated in the mathematical model where
we assumed that <span class="math inline">\(\epsilon_{ik}\sim
N(0,\sigma^2)\)</span>.</p>
<div id="checking-anova-assumptions" class="section level6">
<h6>Checking ANOVA Assumptions</h6>
<p>To check that the ANOVA assumptions are satisfied, it is required to
check the data in each group for normality using QQ-Plots. Also, the
sample variance of each group must be relatively constant. The fastest
way to check these two assumptions is by analyzing the
<em>residuals</em>.</p>
<ul>
<li>An ANOVA <strong>residual</strong> is defined as the difference
between the observed value of <span
class="math inline">\(y_{ik}\)</span> and the mean <span
class="math inline">\(\bar{y}_{i\bullet}\)</span>. Mathematically, <span
class="math display">\[
r_{ik} = y_{ik} - \bar{y}_{i\bullet}
\]</span> One QQ-Plot of the residuals will provide the necessary
evidence to decide if it is reasonable to assume that the error terms
are normally distributed. Also, the constant variance can be checked
visually by using what is known as a residuals versus fitted values
plot. For the <strong>Fabricated Example</strong> above, the QQ-Plot and
residuals versus fitted values plots show the two assumptions of ANOVA
appear to be satisfied.</li>
</ul>
<p><img src="ANOVA_files/figure-html/unnamed-chunk-10-1.png" width="672" /></p>
<p><br /></p>
<hr />
</div>
</div>
</div>
</div>
</div>
<div id="section" class="section level3">
<h3></h3>
<div style="padding-left:125px;">
<p><strong>Examples:</strong> <a
href="./Analyses/ANOVA/Examples/chickwtsOneWayANOVA.html">chickwts</a>
(One-way)</p>
</div>
<hr />
</div>
<div id="two-way-anova"
class="section level3 tabset tabset-fade tabset-pills">
<h3 class="tabset tabset-fade tabset-pills">Two-way ANOVA</h3>
<div style="float:left;width:125px;" align="center">
<p><img src="Images/QuantYMultQualX.png" width=80px;></p>
</div>
<div id="overview-1" class="section level4">
<h4>Overview</h4>
<div style="padding-left:125px;">
<p>A two-way ANOVA is only appropriate when all of the following are
satisfied.</p>
<ol style="list-style-type: decimal">
<li><p>The sample(s) of data can be considered to be representative of
their population(s).</p></li>
<li><p>The data is normally distributed in each group. (This can safely
be assumed to be satisfied when the <em>residuals</em> from the ANOVA
are normally distributed.)</p></li>
<li><p>The population variance of each group can be assumed to be the
same. (This can be safely assumed to be satisfied when the
<em>residuals</em> from the ANOVA show constant variance, i.e., are
similarly vertically spread out.)</p></li>
</ol>
<p><strong>Hypotheses</strong></p>
<p>With a two-way ANOVA there are three sets of hypotheses. Writing out
the hypotheses can be very involved depending on whether you use the
official “effects model” notation (very mathematically correct) or a
simplified “means model” notation (which isn’t very mathematically
correct, but gets the idea across in an acceptable way).</p>
<div class="tab">
<p><button class="tablinks" onclick="openTab(event, 'meansModel')">Means
Model</button>
<button class="tablinks" onclick="openTab(event, 'effectsModel')">Effects
Model</button></p>
</div>
<div id="meansModel" class="tabcontent" style="display:block;">
<p>
<ol style="list-style-type: decimal">
<li>The first set of hypotheses are a “one-way” set of hypotheses for
the first <em>factor</em> of the ANOVA. Factor: <code>X1</code> with
say, levels <span class="math inline">\(A\)</span> and <span
class="math inline">\(B\)</span>.</li>
</ol>
<p><span class="math display">\[
H_0: \mu_A = \mu_B = \mu
\]</span> <span class="math display">\[
H_a: \mu_A \neq \mu_B
\]</span></p>
<ol start="2" style="list-style-type: decimal">
<li><p>The second set of hypotheses are also a “one-way” set of