404 – Statistical Computing and Programming
405 – Data Management
Tools for data acquisition, transformation and analysis, data visualization, machine learning and tools for reproducible data analysis, collaboration and model deployment used by data scientists in practice. Advanced R packages, analytical databases, high-performance machine learning libraries, big data tools.
Despite the new name and the recent hype, Data Science is hardly new, it has in fact solid foundations in statistics and computing technology that go back several decades. The 405 – Data Management course lies down the foundations necessary for most data science tasks (data acquisition, exploratory data analysis, data cleaning and transformation, databases, data visualization and data mining) while the 404 – Statistical Computing and Programming course prepares students to use R, the most widely used tool for data science. This proposed course will build on all this knowledge and it will discuss more advanced topics and present more tools (R packages, scalable machine learning libraries, high-performance analytical databases, big data technologies) that are used by data scientists in daily practice. The course will also discuss software engineering tools/techniques that are important when working on data science projects or when the results of the projects (models, data visualization dashboards etc.) are deployed in production. While this instructor is a great proponent of a good balance of theory and practice, and furthermore in the context of data science advocates a good balance of statistics and computing technology, this course will try to complement the other existing courses in the MAS program, and therefore it might appear as overly tilted towards software systems. Therefore, it is important to restate here that good statistical and theoretical foundations (e.g. cognitive science for data visualization or a good understanding of machine learning algorithms etc.) are also crucial when conducting data science in practice.
Week 1: Overview of data science. The elements of a data science project. Overview of tools (R/Python, databases, machine learning libraries, big data tools etc)
Week 2: Tools for reproducible research/productive data analysis and collaboration (Rmarkdown, Jupyter notebooks, git/Github).
Week 3: Tools for data visualization: ggplot2, shiny (interactive web applications with R), shiny dashboards
Week 4: The Unix toolbox for manipulating files/text and automating tasks. Cloud computing for scaling up data science (EC2). Virtualization for quick experimentation and/or for reproducible deployment (vagrant, docker).
Week 5: Foundations for supervised learning (classification/regression): basic algorithms, overfitting, train and test sets, cross-validation, bias-variance tradeoff, regularization, ROC curve for binary classification (various R packages)
Week 6: Tools for supervised learning 1 (GLM, Lasso, random forest, gradient boosted machines) (R packages, Vowpal Wabbit, xgboost, H2O)
Week 7: Tools for supervised learning 2 (support vector machines, neural networks, deep learning, ensembles) (R packages, H2O, libraries for deep learning on GPUs)
Week 8: Tools for unsupervised learning (K-means clustering, hierarchical clustering) (R packages)
Week 9: Analytical databases (columnar/MPP relational databases), SQL. NoSQL databases (key-value stores, document databases). “Big data” technologies (Hadoop, HDFS, Map-reduce, Hive, Impala, Spark, EMR etc.)
Week 10: Student project presentations. Course discussions/Q&A session. Summary of the course, conclusions, final thoughts etc.
40% assignment #1: exploratory data analysis and data visualization project
40% assignment #2: machine learning project
20% student presentations
It is imperative for these assignments that students already have a good knowledge of R (404 course) and a good understanding of the elements of data analysis (405 course) before taking this current course.
David Donoho: 50 years of Data Science
Sean Kandel, Andreas Paepcke, Joseph M. Hellerstein, and Jeffrey Heer: Enterprise Data Analysis and Visualization: An Interview Study
Rexer Analytics: 2013 Data Miner Survey (Summary Report)
Leo Breiman: Statistical Modeling: The Two Cultures
Rich Caruana, Alexandru Niculescu-Mizil: An Empirical Comparison of Supervised Learning Algorithms
John M. Chambers: Software for Data Analysis: Programming with R, Springer, 2008
Hadley Wickham: Advanced R, Chapman & Hall/CRC, 2015
W.N. Venables, B.D. Ripley, Modern Applied Statistics with S, Springer, 4th ed., 2003
William S. Cleveland, The Elements of Graphing Data, Hobart Press, 1994
William S. Cleveland, Visualizing Data, Hobart Press, 1993
Edward R. Tufte, The Visual Display of Quantitative Information, Graphics Press, 2nd ed., 2001
Stephen Few, Show Me the Numbers: Designing Tables and Graphs to Enlighten, 2nd ed., Analytics Press, 2012
Hadley Wickham, ggplot2: Elegant Graphics for Data Analysis, Springer, 2009
Dorian Pyle: Data Preparation for Data Mining, Morgan Kaufmann, 1999
Micheline Kamber, Jiawei Han, Data Mining: Concepts and Techniques, Morgan Kaufmann, 2nd ed., 2005
Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani: An Introduction to Statistical Learning with Applications in R
Trevor Hastie, Robert Tibshirani, Jerome Friedman: The Elements of Statistical Learning, 2nd. ed.