Introduction to Biostatistics

	\(H_0\) false	\(H_0\) true	Total
Significant test	True positive	False positive	\(Total\ signif\ tests\)
NS test	False negative	True negative	\(Total\ NS\ tests\)
	\(Total\) \(H_0\ false\)	\(Total\) \(H_0\ true\)	\(Total\ \#\ of\ Tests\)

	\(H_0\) false	\(H_0\) true	Total
Significant test	True positive	False positive	\(\color{red}{Total\ signif\ tests}\)
NS test	False negative	True negative	\(\color{red}{Total\ NS\ tests}\)
	\(Total\) \(H_0\ false\)	\(Total\) \(H_0\ true\)	\(\color{red}{Total\ \#\ of\ tests}\)

	\(H_0\) false	\(H_0\) true	Total
Significant test	\(\color{blue}{True}\) \(\color{blue}{positive}\)	\(\color{blue}{False}\) \(\color{blue}{positive}\)	\(\color{red}{Total\ signif\ tests}\)
NS test	False negative	True negative	\(\color{red}{Total\ NS\ tests}\)
	\(Total\) \(H_0\ false\)	\(Total\) \(H_0\ true\)	\(\color{red}{Total\ \#\ of\ Tests}\)

Test	No correction	Bonferroni	Holm	FDR (B-H)
1	.04	.12	.04	.04
2	.02	.06	.04	.03
3	.01	.03	.03	.03

1 / 53

Introduction to Biostatistics

Introduction to Biostatistics
1 Learning objectives
1.1 Learning objectives
2 Multiple comparisons
2.1 Multiple comparisons
2.2 How science “works” sometimes
2.3 How science “works” sometimes
2.4 How science “works” sometimes
2.5 How science “works” sometimes
2.6 How science “works” sometimes
2.7 How science “works” sometimes
2.8 How science “works” sometimes
2.9 Problem: Inflated type I error
2.10 Two approaches to corrections
3 Family-wise error rate (FWER)
3.1 Type I error rate
3.2 FWER by \(\alpha\) and number of tests
3.3 Independent vs correlated tests
3.4 Independent vs correlated tests
3.5 Bonferroni
3.6 Bonferroni: Adjust \(\alpha\)
3.7 Bonferroni: Adjust \(p\)-values
3.8 Holm
3.9 Holm: Adjust \(\alpha\)
3.10 Holm: Adjust \(p\)-value
3.11 Adjustments in R: \(p\)-values
4 False discovery rate (FDR)
4.1 False discovery rate (FDR)
4.2 False discovery rate
4.3 False discovery rate: What do we know?
4.4 False discovery rate: What is FDR?
4.5 Distributions of \(p\)-values
4.6 Combined distribution of \(p\)-values
4.7 Ordered \(p\)-values
4.8 Ordered \(p\)-values
4.9 No correction: FDR = 50/143 = 0.35
4.10 Corrected B-H FDR = .05
4.11 Corrected B-H FDR = .10
4.12 How many significant tests?
4.13 B-H FDR: Adjust \(\alpha\) (/critical \(p\)-value)
4.14 B-H FDR: Adjust \(p\)-value
4.15 Adjustments in R: \(p\)-values
4.16 Adjustments in R: \(p\)-values
4.17 Adjustments in R: \(p\)-values
4.18 Adjustments in R: \(p\)-values
5 Summary
5.1 Summary of results
5.2 Which should I use? FWER or FDR?
5.3 What FDR level? It depends
5.4 What do you correct for?
5.5 Critique, commentary, and nuance
6 In-class activities
6.1 In-class activities