---
title: "Listing Statistics Canada data tables"
description: >
  Data discovery and pinpointing the best data to use is an important and often challenging aspect of analysis. The package offers several methods to programmatically search through data available via the StatCan NDM.
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Listing Statistics Canada data tables}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  collapse=TRUE,
  comment = "#>",
  cache = FALSE,
  fig.width = 7,
  fig.height = 5,
  eval = nzchar(Sys.getenv("COMPILE_VIG"))
)
```

This vignette details how to use the internal table search functions in the `cansim` package with a simple example using employment data for economic regions in British Columbia. 

The list of available tables is cached in the current R session to avoid repeated downloading of data.

### Listing and filtering tables

Calling `list_cansim_cubes` returns a data frame with useful metadata for available tables. There are 21 fields of metadata for each table including title, in English and French, keyword sets, notes, and table numbers. 
```{r}
library(cansim)

names(list_cansim_cubes())
```
The appropriate table can be found by subsetting or filtering on the properties we want to use to find the appropriate tables. 
```{r}
library(dplyr, warn.conflicts = FALSE)

list_cansim_cubes() %>% 
  filter(grepl("Labour force characteristics",cubeTitleEn), 
         grepl("economic region",cubeTitleEn)) %>% 
  select(cansim_table_number,cubeTitleEn)
```
The search came up with two tables. In this example we are interested in the unemployment rate for 2015 onward for the Lower Mainland, Vancouver Island, and Okanagan economic regions from the Labour Force Characteristics table. We use the `tidyr` package here to reshape data from a long format to a wider format. 
```{r}
library(tidyr)

selected_table <- "14-10-0293"

data <-get_cansim(selected_table) %>% 
  filter(grepl("Mainland|Vancouver Island|Okanagan", GEO),
         Date>=as.Date("2015-01-01"),
         `Labour force characteristics`=="Unemployment rate") %>%
  select(Date, GEO, Statistics, val_norm) %>%
  spread(key="Statistics", value=val_norm)
```
We can visualize then results with `ggplot2`. 

```{r fig.alt="Vignette example plot, unemployent rate"}
library(ggplot2)
ggplot(data, aes(x=Date, group = GEO,y=Estimate)) +
  geom_ribbon(aes(ymin=Estimate - `Standard error of estimate`,
                  ymax=Estimate + `Standard error of estimate`, fill=""),
              alpha=0.8) +
  geom_line(aes(color=GEO)) +
  scale_y_continuous(labels=scales::percent) +
  scale_fill_manual(name = "", values="grey80", label="Standard error") +
  theme_bw() + 
  labs(title = "Comparison of unemployment rate by economic region",
       y = "Unemployment Rate", 
       x = "",
       color = "",
       caption=paste0("CANSIM ", selected_table))
```