Package {AIGovernance}

Type:	Package
Title:	Statistical Auditing, Drift Monitoring and Governance Reporting for AI Decision Systems
Version:	0.2.0
Description:	Provides statistical auditing, risk documentation, and reporting tools to support AI governance workflows for employment, hiring and other AI-assisted decision systems. Implements the EEOC four-fifths adverse impact rule (Equal Employment Opportunity Commission, 1978, https://www.ecfr.gov/current/title-29/subtitle-B/chapter-XIV/part-1607), NYC Local Law 144 bias audit requirements (New York City, 2023, https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page), and the AI Risk Management Framework checklist items from the National Institute of Standards and Technology (2023, <doi:10.6028/NIST.AI.100-1>). Optionally supports EU AI Act high-risk classification (European Parliament and Council, 2024, https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai). It also audits decision drift over time: the change in a decision rate is decomposed into case-mix, allocation (routing of cases to decision units) and rule-change components in the spirit of Kitagawa (1955) <doi:10.1080/01621459.1955.10501299>; net and absolute drift are estimated over a fixed reference case mix with overlap trimming and cluster bootstrap uncertainty; categorical decisions and unit-level drift with noise-corrected heterogeneity are supported; and a simulator with known drift mechanisms is included. The package does not provide legal advice or certify legal compliance; it is a statistical and documentation support tool.
License:	MIT + file LICENSE
Encoding:	UTF-8
LazyData:	true
Depends:	R (≥ 4.1.0)
Imports:	cli, rlang, stats, utils, tibble, dplyr, nnet
Suggests:	ggplot2, knitr, rmarkdown, testthat (≥ 3.0.0)
VignetteBuilder:	knitr
Language:	en-US
Config/testthat/edition:	3
URL:	https://github.com/subirhait/AIGovernance
BugReports:	https://github.com/subirhait/AIGovernance/issues
NeedsCompilation:	no
Packaged:	2026-11-11 23:01:57 UTC; subir
Author:	Subir Hait [aut, cre]
Maintainer:	Subir Hait <haitsubi@msu.edu>
Config/roxygen2/version:	8.1.1
Repository:	CRAN
Date/Publication:	2026-11-12 14:50:34 UTC

EEOC Adverse Impact Analysis (4/5ths Rule)

Description

Computes selection rates by group and applies the EEOC Uniform Guidelines 4/5ths (80%) rule to assess adverse impact in employment selection procedures (EEOC, 1978).

The Adverse Impact Ratio (AIR) for each group is:

AIR_g = \frac{\text{selection rate}_g}{\text{selection rate}_{\text{ref}}}

A group is flagged for adverse impact when AIR < 0.80. The function also reports the two-proportion Z-test and Fisher's exact test p-values as supplementary statistics.

Note: The 4/5ths rule is a rule of thumb, not a bright-line legal standard. Small sample sizes reduce reliability. This function does not provide legal advice.

Usage

aigov_adverse_impact(gov, min_n = 30L)

Arguments

Value

The input gov object with gov$results$adverse_impact appended, a tibble containing:

group

Group label.

n

Total applicants in group.

n_selected

Number selected.

selection_rate

Proportion selected.

AIR

Adverse Impact Ratio vs reference group.

fourfifths_flag

Logical: TRUE if AIR < 0.80.

z_stat

Two-proportion Z statistic.

p_value

Two-sided p-value from Z test.

fisher_p

Fisher's exact test p-value.

small_n_flag

Logical: TRUE if n < min_n.

References

Equal Employment Opportunity Commission (1978). Uniform guidelines on employee selection procedures. Federal Register, 43(166), 38295–38309.

Examples

data(hiring_sim)
gov <- aigov_build(hiring_sim, selected, race_ethnicity, ref_group = "White")
gov <- aigov_adverse_impact(gov)
gov$results$adverse_impact

NIST AI Risk Management Framework (AI RMF 1.0) Audit

Description

Implements a structured checklist audit aligned with the NIST AI Risk Management Framework (AI RMF 1.0, NIST, 2023). The RMF organises AI risk management into four core functions: GOVERN, MAP, MEASURE, and MANAGE.

This function presents the checklist items most relevant to employment AI systems and records user-supplied responses (or defaults to NA for items that cannot be verified from data alone).

Usage

aigov_audit_nist(gov, responses = list())

Arguments

Value

The input gov object with gov$results$nist_rmf appended, containing:

checklist

Named list with TRUE/FALSE/NA per item.

scores

Per-function completion scores (proportion of confirmed items).

overall_score

Overall confirmed proportion.

verdict

One of "GREEN" (>= 0.75), "AMBER" (0.50–0.74), or "RED" (< 0.50).

References

National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. doi:10.6028/NIST.AI.100-1

Examples

data(hiring_sim)
gov <- aigov_build(hiring_sim, selected, race_ethnicity, ref_group = "White")
gov <- aigov_audit_nist(gov, responses = list(
  GOVERN_1_1 = TRUE,
  GOVERN_1_2 = TRUE,
  MAP_1_1    = TRUE
))

NYC Local Law 144 Bias Audit Module

Description

Implements the statistical components of a NYC Local Law 144 (2023) bias audit for Automated Employment Decision Tools (AEDTs). The law requires employers using AEDTs to:

1. Conduct an annual independent bias audit.

2. Publish a summary of audit results on their website.

3. Provide advance notice to candidates/employees.

This function computes the required impact ratio statistics for each race/ethnicity and sex category and generates a publication-ready summary table matching the format expected in public disclosures.

The NYC LL144 impact ratio is:

IR_g = \frac{\text{selection rate}_g}{\text{selection rate of most-selected category}}

Note: LL144 uses the most-selected category (not a user-specified reference group) as the denominator – this differs from the EEOC approach. This function implements both.

Usage

aigov_audit_nyc(gov, use_most_selected = TRUE)

Arguments

Value

The input gov object with gov$results$nyc_ll144 appended, containing:

impact_table

Tibble with selection rates and impact ratios.

most_selected_group

The reference category used.

disclosure_table

Formatted table for public disclosure.

checklist

LL144 procedural checklist (named logical vector).

verdict

Statistical verdict: "PASS" or "REVIEW".

References

New York City Local Law 144 of 2021 (effective January 1, 2023). NYC Department of Consumer and Worker Protection (DCWP). https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page

Examples

data(hiring_sim)
gov <- aigov_build(hiring_sim, selected, race_ethnicity, ref_group = "White",
                  frameworks = c("NYC_LL144"))
gov <- aigov_audit_nyc(gov)

Build an AIGovernance audit object

Description

Constructs an aigov object from employment decision data. This is the entry point for all auditing, classification, and reporting functions in the AIGovernance package.

Disclaimer: AIGovernance provides statistical and documentation support tools only. It does not provide legal advice and does not certify compliance with any law or regulation.

Usage

aigov_build(
  data,
  outcome,
  group,
  ref_group,
  frameworks = c("EEOC", "NYC_LL144", "NIST_RMF"),
  org_name = NULL,
  system_name = NULL,
  audit_date = NULL
)

Arguments

Value

An object of class "aigov" containing:

data

The input data frame.

outcome

Name of the outcome column (character).

group

Name of the group column (character).

ref_group

Reference group label.

frameworks

Active frameworks.

org_name

Organisation name.

system_name

AI system name.

audit_date

Audit date.

group_levels

All observed group labels.

n_total

Total number of records.

Examples

data(hiring_sim)
gov <- aigov_build(
  data        = hiring_sim,
  outcome     = selected,
  group       = race_ethnicity,
  ref_group   = "White",
  frameworks  = c("EEOC", "NYC_LL144", "NIST_RMF"),
  org_name    = "Acme Corp",
  system_name = "Resume Screening Tool v1.0"
)
print(gov)

Display Governance Checklist for a Framework

Description

Returns all checklist item names and descriptions for a given framework. Use the returned item names as keys in the responses argument of aigov_audit_nist.

Usage

aigov_checklist(gov, framework)

Arguments

Value

A tibble with columns item_id, function_area, and description.

Examples

data(hiring_sim)
gov <- aigov_build(hiring_sim, selected, race_ethnicity, ref_group = "White")
aigov_checklist(gov, "NYC_LL144")
aigov_checklist(gov, "NIST_RMF")

Classify AI System Risk Level

Description

Classifies the AI system under two frameworks:

1. EU AI Act (2024) – Assigns one of four risk tiers: Unacceptable, High, Limited, or Minimal risk. Employment/worker-management AI is listed in Annex III as High Risk.

2. NIST AI RMF – Assigns a risk tier (1–4) based on impact on individuals' rights and opportunities.

Usage

aigov_classify(
  gov,
  domain = "employment",
  makes_final_decision = TRUE,
  human_oversight = NA
)

Arguments

Value

The input gov object with gov$results$risk_class appended, containing EU AI Act and NIST risk classifications with explanatory text.

References

European Parliament and Council (2024). Regulation (EU) 2024/1689 (EU AI Act). https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai

NIST (2023). AI RMF 1.0. doi:10.6028/NIST.AI.100-1

Examples

data(hiring_sim)
gov <- aigov_build(hiring_sim, selected, race_ethnicity, ref_group = "White")
gov <- aigov_classify(gov, domain = "employment",
                       makes_final_decision = TRUE,
                       human_oversight = FALSE)

Audit Decision Drift Between Two Periods

Description

Separates a change in decision rates between two periods into changes in the cases being decided and changes in how comparable cases are decided. Single-period audits such as the four-fifths rule or the NYC Local Law 144 impact ratio cannot make this distinction.

Let D be the decision, X the recorded case characteristics (covariates), G the group and t the period. The function estimates the system-level decision rule m_t(x, g) = P_t(D = k \mid X = x, G = g) in each period and reports:

Decomposition

The raw change in the rate of each decision category is split into a case-mix component and a rule-change component evaluated at the current case mix. When unit is given (for example recruiter, office, judge or court), the change is split three ways: case mix, allocation of cases to units, and within-unit rule change. The components add to the raw change exactly in the analysis sample.

Net drift

\Delta_g = E_{Q_g}[m_1(X, g) - m_0(X, g)] for the focal category, over a fixed reference case mix Q_g.

Absolute drift

A_g = E_{Q_g}[\frac{1}{2}\sum_k |m_1(X,g)_k - m_0(X,g)_k|], the mean total-variation change in the decision rule. For a binary decision this is E_{Q_g}|m_1 - m_0|. Increases and decreases for different cases do not cancel, so A_g can be large when \Delta_g is near zero ("masked drift").

Unit drift

With unit, the drift of each unit over a common reference case mix, with delta-method standard errors, a noise-corrected between-unit variance \tau^2, empirical-Bayes shrunken estimates and Benjamini-Hochberg q-values.

Cases outside the region where both periods have comparable cases are removed before estimation (overlap_trim). Uncertainty is estimated with a bootstrap that resamples units (or cluster) when given.

Interpretation. Drift is a change in decisions conditional on the recorded characteristics. It does not identify the causal effect of an AI system or of human discretion, and a change in an unrecorded decision-relevant characteristic appears as drift. The absolute-drift estimate is biased upward by estimation noise; abs_drift_bc reports a bootstrap bias-corrected value, and alerts for absolute drift compare the lower confidence bound with tolerance rather than with zero. The decomposition and drift estimates are model-based standardizations: they are accurate when the decision models are adequately specified. When the true rule is nonlinear in the covariates (for example because cases are routed to units according to their characteristics), add flexible terms with extra_terms and compare. These procedures are approximate and should be reviewed with the overlap and sample-size diagnostics. This function does not provide legal advice.

Usage

aigov_drift(
  gov,
  period,
  covariates,
  extra_terms = NULL,
  unit = NULL,
  decision = NULL,
  baseline = NULL,
  current = NULL,
  reference = c("pooled", "baseline", "current"),
  overlap_trim = 0.02,
  tolerance = 0.05,
  focal = NULL,
  cluster = NULL,
  B = 200L,
  conf_level = 0.95,
  min_unit_n = 20L,
  max_ref = 2000L,
  seed = NULL,
  verbose = TRUE
)

Arguments

Value

The input gov object with gov$results$drift, a list containing:

decomposition

Tibble by group and decision category: rates, raw change, case-mix, allocation (multilevel only), rule-change and net drift, with bootstrap percentile intervals.

drift

Tibble by group: net drift for the focal category, absolute drift, bias-corrected absolute drift, intervals, p-values and alert. net_p tests \Delta_g = 0; net_p_tol is a one-sided test of |\Delta_g| \le tolerance (normal approximation). Alerts use interval nulls: "net drift" when the whole interval for \Delta_g lies beyond \pm tolerance; "masked drift" when the lower bound for A_g exceeds tolerance while |\hat\Delta_g| \le tolerance; "conditional drift" when the lower bound for A_g exceeds tolerance otherwise; "none" otherwise.

overlap

Cases kept and trimmed by group.

units

Unit-level drift table and summary (multilevel only).

settings

Analysis settings.

verdict

"PASS" if no alert and no flagged unit, otherwise "REVIEW".

References

Kitagawa, E. M. (1955). Components of a difference between two rates. Journal of the American Statistical Association, 50(272), 1168–1194. doi:10.1080/01621459.1955.10501299

Cai, T. T., Namkoong, H., & Yadlowsky, S. (2023). Diagnosing model performance under distribution shift. arXiv. doi:10.48550/arXiv.2303.02011

See Also

aigov_monitor, aigov_simulate_drift

Examples

data(hiring_panel)
gov <- aigov_build(hiring_panel, selected, group, ref_group = "A")
gov <- aigov_drift(gov, period = period,
                   covariates = c("qualification", "degree"),
                   baseline = 2024, current = 2026, B = 30, seed = 1)
gov$results$drift$drift


# three-way decomposition with recruiters as decision units
gov <- aigov_drift(gov, period = period,
                   covariates = c("qualification", "degree"),
                   unit = recruiter, baseline = 2024, current = 2026,
                   B = 100, seed = 1)
gov$results$drift$units$summary

Monitor Decision Drift Across Several Periods

Description

Runs aigov_drift for a sequence of prespecified comparisons (each period against the baseline, or each period against the previous one) and adjusts the net-drift p-values for the number of comparisons. A net-drift alert is kept only if the adjusted one-sided p-value for |\Delta| \le tolerance is below (1 - conf_level)/2. This supports the periodic monitoring of deployed decision systems.

Note: the adjustment covers the comparisons made in one call. It does not make repeated, data-dependent looks at accumulating data valid; that requires sequential (anytime-valid) methods.

Usage

aigov_monitor(
  gov,
  period,
  covariates,
  unit = NULL,
  decision = NULL,
  compare = c("baseline", "previous"),
  adjust = "holm",
  tolerance = 0.05,
  B = 200L,
  seed = NULL,
  verbose = TRUE,
  ...
)

Arguments

Value

The input gov object with gov$results$monitor, a list with table (one row per comparison and group, with adjusted p-values and alerts), fits (the individual drift results) and verdict.

Examples

data(hiring_panel)
gov <- aigov_build(hiring_panel, selected, group, ref_group = "A")
gov <- aigov_monitor(gov, period = period,
                     covariates = c("qualification", "degree"),
                     B = 50, seed = 1)
gov$results$monitor$table

Generate an AI Governance Audit Report

Description

Produces a self-contained HTML (or plain-text) governance audit report from a completed aigov object. The report is suitable for internal documentation, legal review, or public disclosure (e.g., NYC LL144 website posting requirement).

Usage

aigov_report(gov, format = "html", output_file = NULL, open = TRUE)

Arguments

Value

Invisibly returns the path to the generated file.

Examples

## Not run: 
data(hiring_sim)
gov <- aigov_build(hiring_sim, selected, race_ethnicity, ref_group = "White")
gov <- aigov_adverse_impact(gov)
gov <- aigov_audit_nyc(gov)
gov <- aigov_audit_nist(gov)
aigov_report(gov, format = "html")

## End(Not run)

Determine Applicable Governance Frameworks

Description

Returns a summary of which governance frameworks apply to the AI system described in the aigov object, based on domain and jurisdiction.

Usage

aigov_scope(gov, domain = "employment", us_state = NULL)

Arguments

Value

A tibble with columns framework, applies, jurisdiction, and note.

Examples

data(hiring_sim)
gov <- aigov_build(hiring_sim, selected, race_ethnicity, ref_group = "White")
aigov_scope(gov, domain = "employment", us_state = "NY")

Simulate Decision Data With Known Drift Mechanisms

Description

Generates multi-period decision records from a known data-generating process so that the targets of aigov_drift are known exactly. Used for testing, teaching and simulation studies of audit procedures.

Cases have a continuous qualification score, a binary degree, a group ("A", "B", "C") and an unrecorded binary characteristic z. Each case is routed to one of n_units decision units (recruiter). The decision follows a logistic (binary) or cumulative-logistic (three-category) model:

\eta = -0.2 + qualification + 0.6\,degree - z + \gamma_g + \alpha_j + v_j s_t + \delta_t(x, g),

where s_t rises from 0 in the first period to 1 in the last.

Mechanisms (scenario; several may be combined):

"stable"

Nothing changes.

"case_mix"

Mean qualification falls by 0.5 SD.

"rule"

All groups face a stricter rule (\delta = -effect).

"offsetting_groups"

Group B gains and group A loses by effect.

"offsetting_cases"

Within every group, cases with a degree gain and cases without lose by effect.

"unobserved"

The share of cases with the unrecorded z rises from 0.3 to 0.6; the full-information rule does not change.

"reallocation"

High-qualification cases are routed to more lenient units; no unit changes its rule.

"unit_offsetting"

Half of the units become more lenient and half stricter by effect; the average unit does not change.

"category_shift"

(Requires categorical = TRUE.) The threshold between "review" and "advance" rises by effect while the share passing the first screen is unchanged.

Usage

aigov_simulate_drift(
  n = 2000L,
  n_periods = 2L,
  n_units = 30L,
  scenario = "stable",
  effect = 0.8,
  categorical = FALSE,
  first_period = 2024L,
  seed = NULL
)

Arguments

Value

A data frame with columns case_id, period, group, qualification, degree, recruiter, selected and, if categorical = TRUE, decision. The attribute "truth" is a list with drift (true net and absolute drift of selected, by group, between the first and last period, over the pooled cases conditional on the recorded characteristics; with categorical = TRUE also the true net drift of "advance" and the true total-variation drift of decision), decomposition (true case-mix, allocation and within-unit rule-change components of the change in selected, evaluated on the realized cases) and units (true unit drift over the pooled cases).

Examples

sim <- aigov_simulate_drift(n = 500, scenario = "offsetting_cases", seed = 1)
attr(sim, "truth")$drift

Simulated Multi-Period Hiring Decisions With Known Drift

Description

A synthetic panel of screening decisions made by 24 recruiters over three annual periods, generated with aigov_simulate_drift. It is built so that a single-period audit and a drift audit tell different stories: applicant qualifications fall over time (case-mix change), the rule changes in opposite directions for applicants with and without a degree (masked drift), stronger applicants are increasingly routed to more lenient recruiters (allocation change), and the threshold between "review" and "advance" rises (category shift). All records are fictional.

Usage

hiring_panel

Format

A data frame with 4,500 rows and 8 variables:

case_id

Integer case identifier.

period

Integer period (2024, 2025, 2026).

group

Character group label ("A", "B", "C").

qualification

Numeric standardized qualification score.

degree

Integer (0/1) degree indicator.

recruiter

Character recruiter (decision unit) identifier.

selected

Integer (0/1): passed the first screen (decision is "review" or "advance").

decision

Factor: "reject", "review" or "advance".

Details

The attribute "truth" holds the true net and absolute drift between 2024 and 2026 by group and the true drift of each recruiter. See data-raw/hiring_panel.R in the package sources for the generating call.

Source

Simulated with aigov_simulate_drift(n = 1500, n_periods = 3, n_units = 24, scenario = c("case_mix", "offsetting_cases", "reallocation", "category_shift"), categorical = TRUE, seed = 2026).

Examples

data(hiring_panel)
with(hiring_panel, tapply(selected, list(group, period), mean))
attr(hiring_panel, "truth")$drift

Simulated Employment Screening Dataset

Description

A synthetic dataset of 500 job applicants processed by a hypothetical automated resume-screening tool. Generated for illustrative and testing purposes only. All individuals are fictional.

Usage

hiring_sim

Format

A data frame with 500 rows and 6 variables:

applicant_id

Integer applicant identifier.

race_ethnicity

Character. One of "White", "Black", "Hispanic", "Asian", "Other".

gender

Character. "Male" or "Female".

years_experience

Numeric. Years of relevant experience.

score

Numeric. AI screening score (0–100).

selected

Integer (0/1). Whether the applicant was advanced to the next stage (1 = selected, 0 = not selected).

Details

Selection probabilities were set to produce a realistic adverse impact pattern across race/ethnicity groups, consistent with published empirical ranges. The data are purely synthetic and do not represent any real organisation or hiring process.

Source

Simulated by the package authors for illustration purposes.

Examples

data(hiring_sim)
head(hiring_sim)
table(hiring_sim$race_ethnicity, hiring_sim$selected)