4-DataVisualization.qmd

---
title: 'Worksheet 4: Data Visualization'
author: 'Megha Khanna'
date: '02/23/2024'
---

_This is the fourth in a series of worksheets for History 8510 at Clemson University. The goal of these worksheets is simple: practice, practice, practice. The worksheet introduces concepts and techniques and includes prompts for you to practice in this interactive document. When you are finished, you should change the author name (above), knit your document, and upload it to canvas. Don't forget to commit your changes as you go and push to github when you finish the worksheet._

## Charts and Graphs with `ggplot2()`

An essential element of working with data is trying to make sense of it and communicate your findings. Frequently, the way to do that is through some kind of data visualization. This week we are going to think about how to represent information graphically. How can we highlight patterns and trends in data in a way that helps ourselves and our readers/users understand historical data?

R has many visualization packages but the most powerful of those is `ggplot()` which builds on the concept of a grammar of graphics. To quote Hadley Wickham, "A grammar of graphics is a tool that enables us to concisely describe the components of a graphic. Such a grammar allows us to move beyond named graphics (e.g., the `scatterplot`) and gain insight into the deep structure that underlies statistical graphics." In other words, `ggplot()` provides a set of tools to map data to visual elements on a plot, to specify the type of plot, and to control the fine details of how that plot will be displayed. 

What does that mean in practice? 

Any type of plot in R has 3 necessary layers. 

**1) Data:** A data frame with one or more variables, each one with one or more observations. 

**2) Aesthetic:** A mapping of one or more variables to one or more visual elements on the graph. For example, you could map a variable to the x-axis, another variable to the y-axis, and a categorical variable to color so that different categories get plotted with different colors.

**3) Geometry:** The type or shape of the visual elements on the graph. For example, this could be a point in the case of a scatter plot, a bar in the case of a bar plot, or a line in the case of a line plot.

Lets load all the libraries we'll use in this worksheet: 
```{r message=FALSE, warning=FALSE}
library(gapminder) #this is a dataset
library(ggplot2) 
library(tidyverse)
library(DigitalMethodsData)
```
To begin, we're going to use one of R's built in datasets. First lets take a look at the data: 
```{r}
data(gapminder)
```

(@) Take a look at the data. What can you say about it? 

> According to what I can observe, the dataset consists of 1,704 observations (rows) and 6 columns: country, continent, year, Life expectancy, population, and GDP per capita. When I searched the other abbreviations, it says (<fct>) is factors denoted to country and continent which indicates categorical data. The year is an integer (<int>), while lifeExp (life expectancy), pop (population), and gdpPercap (GDP per capita) are numeric (<dbl>), with lifeExp and gdpPercap being double-precision floating-point numbers.

This data includes variables for life exptenency and per capita GDP. Lets say we want to plot these two variables against each other for all country-years within the dataset. First we have to tell `ggplot()` what our data is. 
```{r}
p <- ggplot(data = gapminder)
```

What happened here? A new variable, `p` was created but we didn't get a plot yet. That is because we've told `ggplot` what data we're going to use but we haven't mapped anything to a plot yet. We need to tell `ggplot` what variables in the data should be represented by which visual elements in the plot. We also haven't told it what kind of plot we want. 

Mappings in `ggplot` are defined using the `aes()` function like this: 
```{r}
p <- ggplot(data = gapminder, mapping = aes(x = gdpPercap, y=lifeExp))
```

In this code we gave `ggplot()` two arguments instead of one - data and mapping. This tells ggplot that we want the data about GDP to be mapped to the x-axis and data about life expectancy on the y-axis. The `mapping = aes()` argument is **linking variables in our data to things you will see on the plot**.

There are numerous types of aesthetic mappings. x and y values are the most common and obvious but we could also define things like color, shape, size, and line type (solid vs dashed etc.)

We can view the plot that we've created by calling the variable that we stored our plot in, `p`. What we get is an empty plot. You'll notice that ggplot has created both the x and y-axis for us but there are not points on our plot. That is because we need to add a layer to the plot that includes the data. `geom_point()` helps us do that. It takes the x and y values and plots them in a scatterplot. 

```{r}
p + geom_point()
```

To summarize what we did above, there were 3 steps we went through. First, tell `ggplot()` what our data is. Second, add `mapping = aes()` to map our variables to the plot. And, third, choose a geom. In this case our geom was `geom_point()` which created the points/data layer on our plot. A fourth step could have included adding a scale, legend, title or some other element to the graph. These are typically referred to as labs. 

(@) Modify the above plot so that the variables it draws on are life expectancy and population. Then, use a comment to annotate the code and mark each of the 3 parts described above.
```{r}
p <- ggplot(data = gapminder, mapping = aes(x = lifeExp, y = pop))
p + geom_point()
p + geom_point() + labs(title = "Life Expectancy vs. Population", x = "Life Expectancy", y = "Population")

```

We can make this plot even more complex. Adding another value, color, to `aes()` we can change the color of each dot to correspond to a year. We also might want to rename the x and y-axis so that the labels have a more useful description of the variable. We can do so using the labs function. 

```{r}
ggplot(data = gapminder, mapping = aes(x = gdpPercap, y=lifeExp, color=year)) + geom_point() + labs(x="GDP Per Capita", y="Life Expectency in Years", title="Life Expectency and Per Capita GDP by Year") 
```

(@) Using the same plot above, can you edit the code to add a trend line? (Hint: You'll need to add another geom)
```{r}
ggplot(data = gapminder, mapping = aes(x = gdpPercap, y = lifeExp, color = year)) +
  geom_point() + geom_smooth(method = "lm", se = FALSE) + labs(x = "GDP Per Capita", y = "Life Expectancy in Years", title = "Life Expectancy and Per Capita GDP by Year")
```


(@) The file below contains data about the population of cities in 1930 and their recreation expenditures. Can you create a scatter plot from this data? 
```{r}
rec <- read.csv(file = "https://raw.githubusercontent.com/regan008/DigitalMethodsData/main/raw/Recreation-Expenditures.csv")

library(ggplot2)
rec <- read.csv(file = "https://raw.githubusercontent.com/regan008/DigitalMethodsData/main/raw/Recreation-Expenditures.csv")
ggplot(data = rec, mapping = aes(x = population, y = total_expenditures)) +  geom_point() +  labs(x = "Population in 1930", y = "Recreation Expenditures", title = "Population vs. Recreation Expenditures in 1930")

```

The `geom_point()` function allows us to create scatterplots but its not the only geom within `ggplot2()`.

### Histogram 

A histogram is a graph used to represent the frequency distribution of a few data points of one variable. Histograms often classify data into various “bins” or “range groups” and count how many data points belong to each of those bins.

```{r}
recreational.data <- read.csv("https://raw.githubusercontent.com/regan008/DigitalMethodsData/main/raw/Recreation-Expenditures.csv")

ggplot(recreational.data, aes(x = population)) +
  geom_histogram()
```

(@) Use the Boston Women's voters dataset to create a histogram of the age of voters. 
```{r}
library(DigitalMethodsData)
library(ggplot2)
data("BostonWomenVoters")

ggplot(BostonWomenVoters, aes(x = Age)) +
   geom_histogram(binwidth = 2, color = "purple", fill = "pink") +
   labs(x = "Age", y = "Frequency", title = "Boston Women Voters' Age Histogram")
```

(@) By default, `geom_histogram()` is choosing the number of bins. Can you change the number of bins or the bin width? (Hint: How might you look up the options for this function?)
```{r}
ggplot(BostonWomenVoters, aes(x = Age)) +
  geom_histogram(bins = 20, binwidth = 3) +
labs(x = "Age", y = "Frequency", title = "Boston Women Voters' Age Histogram")

```

(@) Explain, what does changing the number of bins do? 

> What I found is that changing the number of bins in a histogram affects how the data is grouped and displayed. Each bin represents a range of values, and the height of each bin shows the number of observations (frequency) that fall in that range. 

### Barplots 

A barplot (or barchart) is one of the most common types of graphic. Whereas a histogram shows the number of entries in each "bin" - a bar chart shows the relationship between a numeric and a categorical variable. Each entity in the categorical variable is represented as a bar. The size of the bar represents its numeric value. 

```{r}
sc.rec <- recreational.data %>% group_by(state) %>% summarize(state.expenditures = sum(total_expenditures))
ggplot(sc.rec, aes(x = state, y = state.expenditures )) + 
  geom_col()
```

(@) Using the `gayguides` data, create a bar chart that shows the number of locations in 1980 in South Carolina, Georgia, and North Carolina. 
```{r}
library(DigitalMethodsData)
library(ggplot2)
data("gayguides")
filtered_data <- gayguides %>%
   filter(Year == 1980, state %in% c("SC", "GA", "NC")) %>%
  group_by(state) %>%
  summarize(locations = n())

ggplot(filtered_data, aes(x = state, y = locations)) +
  geom_col(fill = "skyblue") +  
  labs(x = "State", y = "Number of Locations", title = "Number of Locations in 1980")

```

(@) Using the `gayguides` data, plot the number of religious institutions in each state.
```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
library(stringr)
data("gayguides")
religious.institutions.data <- gayguides %>%
   filter(str_detect(description, "church|mosque|synagogue|temple")) %>%
  group_by(state) %>%
  summarize(churches = n())
ggplot(religious.institutions.data, aes(x = state, y = churches)) +
  geom_col(bewidth = 3, color = "Blue", fill = "coral") +
  labs(x = "State", y = "Number of Religious Institutons", title = "Number of Religious Institutions per State") 
```

```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
library(stringr)
data("gayguides")
rel.ins.gayguides <- gayguides %>%
  filter(type == "Religious Institution") %>%  
  group_by(state) %>% 
  summarise(religious_institutions = n())  
ggplot(rel.ins.gayguides, aes(x = state, y = religious_institutions)) +  
  geom_col()
  
```
(@) Formulate a question based on a dataset of your choice. That question should be something that would be appropriately answered by a bar chart. What is the question you want to explore?

< What is the average age of Boston women voters within each occupational sector?

(@) Now write the code to answer that question and use comments (`#`) to explain your thinking as you go.
```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
data("BostonWomenVoters")  #loading all the libraries
occupation_age <- BostonWomenVoters %>%
  group_by(Occupation) %>%
  summarize(average_age = mean(Age, na.rm = TRUE))
ggplot(occupation_age, aes(x = Occupation, y = average_age)) +
  geom_col(bewidth = 3, color = "purple", fill = "plum") +
  labs(x = "Occupation", y = "Average Age", title = "Average Age of Boston Women Voters by Occupation") 
```

### Line Charts

Line charts excel at demonstrating trends over time. 

```{r}
data(gayguides)
gg.byyear <- gayguides %>% group_by(Year) %>% summarize(count = n())
ggplot(gg.byyear, aes(x = Year, y = count)) +
  geom_line()
```

(@) Create a line chart that shows the number of religious institutions in the gay guides data over time.
```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
library(stringr)

data("gayguides")

religious_institutions_over_time <- gayguides %>%
  filter(str_detect(description, "church|mosque|synagogue|temple")) 
group_by(Year) %>%
  summarize(number_of_institutions = n())

ggplot(religious_institutions_over_time, aes(x = Year, y = number_of_institutions)) +
  geom_line() +
  geom_point()
  labs(x = "Year", y = "Number of Religious Institutions", title = "Number of Religious Institutions Over Time") +
  theme_minimal()
```

```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
library(stringr)

data("gayguides")
rel.ins.over.time <- gayguides %>%
  filter(type == "Religious Institution") %>%  
  group_by(Year) %>% 
  summarise(religious_institutions = n())  
ggplot(rel.ins.over.time, aes(x = Year, y = religious_institutions)) +  
  geom_line() +
  geom_point()
  labs(x = "Year", y = "Number of Religious Institutions", title = "Number of Religious Institutions Over Time") +
  theme_minimal()

```
(@) In the gayguides data, danger or risk is indicated in several different ways. Sometimes it through the use of an amenity categorization of (HOT) or (AYOR) which meant that these locations were frequented by police and should be used at your own risk. Other times, locations were listed as "inquire locally" in either the description or address field. There are many reasons a location would have requested to have been listed this way. Many times, it was due to harassment and the location deemed it too dangerous to publicly list its location. Can you create a subset of the gay guides data about risk and then use it to create a chart about the usage of these indicators over time? 
```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
library(stringr)
data("gayguides") #loading all the libraries
risk_indicators_subset <- gayguides %>%
  filter(str_detect(amenityfeatures, "HOT|AYOR") | 
           str_detect(description, "inquire locally") | 
           str_detect(streetaddress, "inquire locally")) %>%
  mutate(indicator = case_when(
    str_detect(amenityfeatures, "HOT") ~ "HOT",
    str_detect(amenityfeatures, "AYOR") ~ "AYOR",
    str_detect(description, "inquire locally") | str_detect(streetaddress, "inquire locally") ~ "Inquire Locally",
    TRUE ~ "Other"
  )) %>%
  group_by(Year, indicator) %>%
  summarize(count = n())

head(risk_indicators_subset)
```


### Faceting
Unlike the previous exmaples, `facet_wrap` is not a geom.  It splits the chart window in several small parts (a grid), and displays a similar chart in each section. Each section usually shows the same graph for a specific group of the dataset. For example, remember the line chart above that graphed the number of locations in the `gayguides` data across each year? We can facet that by state to see the number of locations in each state in each year. 

```{r}
gg.byyear <- gayguides %>% filter(state == "SC" | state == "NC" | state == "GA") %>% group_by(state, Year) %>% summarize(count = n())

ggplot(gg.byyear, aes(x = Year, y = count)) +
  geom_line() + facet_wrap(~state)
```

(@) Using the state recreation data, can you create a chart that outlines the total expenditures in South Carolina by city?
```{r}
library(DigitalMethodsData)
data(recreational.data)
sc_expenditures_by_city <- recreational.data %>%
  filter(state == "SC") %>%
  group_by(city) %>%
  summarize(total_expenditures)
ggplot(sc_expenditures_by_city, aes(x = reorder(city, total_expenditures), y = total_expenditures)) +
  geom_col(fill = "turquoise") +
  labs(x = "City", y = "Total Expenditures", title = "Total Recreational Expenditures in South Carolina by City") 

```

(@) Using the gayguides data can you select 5 types of locations and chart the number of each type between 1965 and 1980? 
```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
library(stringr)
data("gayguides") 
selected_types <- c("Bars/Clubs", "Restaurant", "Hotel", "Cruising Areas", "Baths") 
type_counts <- gayguides %>%
  filter(Year >= 1965, Year <= 1980, type %in% selected_types) %>%
  group_by(type, Year) %>%
  summarize(count = n())
ggplot(type_counts, aes(x = Year, y = count, color = type)) +
  geom_line() +
  geom_point() +
  labs(x = "Year", y = "Number of Locations", title = "Number of Selected Location Types Between 1965 and 1980") +
  theme_minimal() +
  scale_color_brewer(palette = "Set2")

```

(@) Using this recreation dataset from last week, can you find the average number of volunteer and paid workers over time? 
```{r}
rec.workers <- read.csv("https://raw.githubusercontent.com/regan008/DigitalMethodsData/main/raw/RecreationData-Wide.csv")

library(tidyr)
library(dplyr)
rec.workers <- read.csv("https://raw.githubusercontent.com/regan008/DigitalMethodsData/main/raw/RecreationData-Wide.csv")

long_rec_workers <- rec.workers %>%
  pivot_longer(cols = starts_with("X"), names_to = "Year", values_to = "Count") %>%
  mutate(Year = as.numeric(sub("X", "", Year)))
long_rec_workers <- long_rec_workers %>%
  separate(type_of_worker, into = c("Gender", "Status"), sep = " ") %>%
  mutate(Status = ifelse(Status == "paid", "Paid", "Volunteer"))
average_workers_by_status <- long_rec_workers %>%
  group_by(Status) %>%
  summarize(AverageCount = mean(Count, na.rm = TRUE)) 
average_workers_by_status

```


### Practice
In the slots below use the datasets for class (or a dataset of your choice) to create a plot. The plot you create in each of these three examples should try to make some kind of historical observation. After you finish creating the plot, write a short paragraph that explains what kind of observation you were trying to make and what your plot demonstrates. 

(@) Plot 1
```{r}
library(DigitalMethodsData)
library(ggplot2)
library(dplyr)
data("gayguides")
establishments_over_time <- gayguides %>%
  group_by(Year) %>%
  summarize(number_of_establishments = n())
ggplot(establishments_over_time, aes(x = Year, y = number_of_establishments)) +
  geom_line() +
  labs(x = "Year", y = "Number of Establishments", title = "Growth of Gay-Friendly Establishments Over Time") +
  theme_minimal()
```

(@) Plot 1 explanation: 
> I through this plot aimed to observe the historical growth of gay-friendly establishments as listed in the gayguides dataset. The line chart shows the number of such places recorded each year, which can give us an idea of how the visibility and acceptance of LGBTQ+ spaces have evolved. The increasing trend in the plot gives us a clue that there was greater social acceptance and community support.

(@) Plot 2
```{r}
library(DigitalMethodsData)
library(ggplot2)
data <- data.frame(recreational.data)
library(ggplot2)

data <- recreational.data

# Create a ggplot bar chart
plot <- ggplot(data, aes(x = city, y = type_of_worker)) +  
  geom_col(color = "lightgreen", fill = "yellow") +  
  labs(x = "City", y = "Population", title = "Population Trend by City") +  
  theme_minimal()  

# Print the plot
print(plot)

```

(@) Plot 2 explanation: 
> Through this bar chart I intended a design to compare the workers' type sizes of various cities. Each bar represents a different city, and the height of the bar indicates the type of workers size of that city. Though, it looks a bit chaotic, but it helped me identify the population trend.

(@) Plot 3. For this one, try a different kind of plot. [Check out a useful gallery here.](https://github.com/regan008/DigitalMethodsData/blob/main/raw/RecreationData-Wide.csv)
(For this week, avoid choosing a map. We'll talk more about maps and geographic data over the next two weeks.)
```{r}
library(DigitalMethodsData)
library(ggplot2)
data(rec)
ggplot(rec, aes(x = state, y = total_expenditures)) +
  geom_boxplot() +
  labs(y = "Expenditures", title = "Distribution of Recreational Expenditures by State")
```

(@) Plot 3 explanation: 
> With the help of boxplot, I aimed to explore the distribution of recreational expenditures by state, revealing disparities in spending levels across different locations. We can infer about the equity of recreational investments and identify states with exceptionally high or low spending.