From Rolling Quarters to Monthly Estimates: SIDRA Mensalization Guide

Why Rolling Quarters Are Problematic

Each published “quarter” is actually a 3-month moving average:

• “2019-Q1” = average of Jan, Feb, Mar 2019
• “2019-Q2” = average of Feb, Mar, Apr 2019
• “2019-Q3” = average of Mar, Apr, May 2019

Rolling quarters overlap: each ‘quarter’ shares 2 months with its neighbors

When unemployment jumps sharply in a single month, the rolling quarter spreads that spike across multiple overlapping periods. The mensalization algorithm inverts this averaging process to recover the true monthly values.

Discovering Available Series

Use get_sidra_series_metadata() to explore all 86+ available series:

meta <- get_sidra_series_metadata()

# View series organized by theme
meta[, .N, by = .(theme, theme_category)]

# Filter to specific theme categories
meta[theme_category == "employment_type", .(series_name, description)]

The metadata uses a hierarchical taxonomy: theme (top level, e.g., “labor_market”), theme_category (e.g., “employment_type”), and optionally subcategory (e.g., “levels”, “rates”).

Step 1: Fetching Rolling Quarter Data

fetch_sidra_rolling_quarters() downloads data from five SIDRA tables:

rq <- fetch_sidra_rolling_quarters(verbose = TRUE)

# Inspect structure
dim(rq)
names(rq)[1:20]

Key columns: anomesfinaltrimmovel (end month of rolling quarter, YYYYMM), mesnotrim (month position 1/2/3), plus one column per series.

Step 2: The Mensalization Transform

monthly <- mensalize_sidra_series(rq, verbose = TRUE)

# Compare dimensions
cat("Rolling quarters:", nrow(rq), "rows\n")
cat("Monthly data:", nrow(monthly), "rows\n")

The row count is approximately the same (one per month), but the meaning changes from “rolling quarter ending in month X” to “exact estimate for month X”.

Step 3: Using Monthly Estimates

# --- VIGNETTE CODE: plot-unemployment ---
library(ggplot2)

monthly[, date := as.Date(paste0(substr(anomesexato, 1, 4), "-",
                                  substr(anomesexato, 5, 6), "-01"))]

ggplot(monthly, aes(x = date, y = m_taxadesocup)) +
  geom_line(color = "#1976D2", linewidth = 0.8) +
  labs(title = "Monthly Unemployment Rate",
       x = NULL, y = "Unemployment Rate (%)")

Population Data for Weighting

For analyses requiring monthly population estimates separately:

pop <- fetch_monthly_population()
head(pop)

Returns a data.table with ref_month_yyyymm and m_populacao columns.

Fetching by Theme

Instead of fetching all 86+ series, filter by theme or theme category:

# Only employment type series
employment <- fetch_sidra_rolling_quarters(theme_category = "employment_type")

# Only wage mass series
wages <- fetch_sidra_rolling_quarters(theme_category = "wage_mass")

# Only labor market theme (includes participation, unemployment, employment types, etc.)
labor <- fetch_sidra_rolling_quarters(theme = "labor_market")

Fetching Specific Series

For maximum efficiency, request only the series you need:

# Only unemployment-related series
unemp <- fetch_sidra_rolling_quarters(
  series = c("popdesocup", "taxadesocup", "popnaforca")
)

Excluding Derived Series

Some series are rates computed from other series. To fetch only “base” series:

# Exclude computed rates (only population and income levels)
base_only <- fetch_sidra_rolling_quarters(exclude_derived = TRUE)

Selecting Output Columns

After mensalization, select columns as needed:

monthly <- mensalize_sidra_series(rq)

# Select specific series
labor_market <- monthly[, .(
  anomesexato,
  employed = m_popocup,
  unemployed = m_popdesocup,
  unemp_rate = m_taxadesocup,
  participation = m_taxapartic
)]

The Core Concept

Rolling quarters are 3-month moving averages. If we denote the true monthly value for month \(t\) as \(y_t\), then the rolling quarter value \(x_t\) is:

\[x_t = \frac{y_{t-2} + y_{t-1} + y_t}{3}\]

The mensalization algorithm inverts this relationship to recover \(y_t\) from the sequence of \(x_t\) values.

The Mensalization Formula

Step 1: Compute first differences

\[d3_t = x_t - x_{t-1}\]

Step 2: Identify month position (mesnotrim)

Each month has a position within its quarter: - Position 1: Jan, Apr, Jul, Oct - Position 2: Feb, May, Aug, Nov - Position 3: Mar, Jun, Sep, Dec

Step 3: Cumulative sum by position

For each position separately, compute the cumulative sum of first differences, starting from a calibrated “starting point” \(y_0\):

\[y_t = y_0 + \sum_{s \in \text{same position}, s \leq t} d3_s\]

Mensalization process: rolling quarters (blue) vs monthly estimates (red)

The Role of Starting Points (\(y_0\))

The starting point \(y_0\) is crucial. It determines the level of all subsequent monthly estimates. The package includes pre-computed starting points for 53 series, calibrated during the stable 2013-2019 period.

Starting points are computed by:

1. Processing PNADC microdata to get “true” monthly aggregates (\(z\) values)
2. Comparing these to rolling quarters
3. Finding the \(y_0\) that makes \(y_0 + \text{cumsum}(d3)\) match the microdata

Assumptions and Limitations

• Monthly values within each position evolve smoothly
• The calibration period (2013-2019) reflects “normal” conditions
• Cannot recover intra-month variation
• Starting points are calibrated to national totals (not regional breakdowns)

API Caching

The package caches SIDRA API responses in memory during your R session:

# First call: fetches from API (~10 seconds)
rq1 <- fetch_sidra_rolling_quarters()

# Second call with use_cache = TRUE: uses cached data (instant)
rq2 <- fetch_sidra_rolling_quarters(use_cache = TRUE)

# Clear all cached data (force fresh fetch on next call)
clear_sidra_cache()

The cache persists until you call clear_sidra_cache() or restart R.

Common Errors

# Check if series exists
meta <- get_sidra_series_metadata()
"taxadesocup" %in% meta$series_name  # TRUE

Data Quality Notes

COVID-19 disruptions (2020): IBGE suspended in-person interviews during the pandemic. Some indicators show unusual patterns in 2020-Q2.

CNPJ series availability: Series based on CNPJ registration (empregadorcomcnpj, contapropriacomcnpj, etc.) are only available from October 2015, when V4019 was introduced.

Option A: All-in-One Function

# Load your stacked PNADC microdata (with pnadc_apply_periods weights)
stacked <- readRDS("my_calibrated_pnadc.rds")

# Compute starting points
custom_y0 <- compute_starting_points_from_microdata(
  data = stacked,
  calibration_start = 201301L,
  calibration_end = 201912L,
  verbose = TRUE
)

# Use custom starting points
monthly <- mensalize_sidra_series(rq, starting_points = custom_y0)

Option B: Step-by-Step

# Step 1: Build crosswalk and calibrate
crosswalk <- pnadc_identify_periods(stacked)
calibrated <- pnadc_apply_periods(
  stacked, crosswalk,
  weight_var = "V1028",
  anchor = "quarter",
  calibration_unit = "month"
)

# Step 2: Compute z_ aggregates (monthly totals from microdata)
z_agg <- compute_z_aggregates(calibrated)

# Step 3: Fetch rolling quarters for comparison
rq <- fetch_sidra_rolling_quarters()

# Step 4: Compute starting points
y0 <- compute_series_starting_points(
  monthly_estimates = z_agg,
  rolling_quarters = rq,
  calibration_start = 201301L,
  calibration_end = 201912L
)

# Step 5: Use custom starting points
result <- mensalize_sidra_series(rq, starting_points = y0)

CNPJ-based series automatically use a later calibration period (2016-2019) when use_series_specific_periods = TRUE (the default in compute_series_starting_points()).

Validating Custom Starting Points

bundled <- pnadc_series_starting_points

# Merge and compare
comp <- merge(custom_y0, bundled,
              by = c("series_name", "mesnotrim"),
              suffixes = c("_custom", "_bundled"))

comp[, rel_diff := abs(y0_custom - y0_bundled) / abs(y0_bundled) * 100]
comp[rel_diff > 1]  # Flag series with >1% difference