Adapter cookbook: from split_spec to native resamples

What this vignette is for

splitGraph ends at a split_spec object. It deliberately knows nothing about rsample, tidymodels, or any other resampling engine. The handoff contract is the sample_data table inside the spec plus a few scalar fields (group_var, block_vars, time_var, stratum_var, ordering_required, recommended_resampling), together with provenance the adapter can inspect to choose a strategy (constraint_mode, constraint_strategy).

You do not always have to write this glue yourself. The reference downstream consumer, bioLeak, takes a split_spec directly — bioLeak::as_leaksplits(spec, data, outcome) builds an executable, leakage-audited split plan from it. The released bioLeak (0.3.8) reads the subject, batch, study and time modes; for the others, including composite, hand it the grouping column instead (bioLeak::make_split_plan(joined, outcome, mode = "subject_grouped", group = "group_id")), which produces the same split by a different route.

This cookbook is for the other case: when you want to feed a split_spec into a different engine, or understand exactly what a consumer has to honor. It shows three small, self-contained adapters that turn a split_spec into something a downstream workflow can use:

A base-R adapter that returns a list of (train, test) row-index pairs — runnable here, no extra dependencies.
An rsample::group_vfold_cv() adapter for grouped cross-validation keyed to group_id.
An rsample::rolling_origin() adapter for ordered evaluation keyed to order_rank.

Adapters 2 and 3 are evaluated when rsample is installed (it is in Suggests, never in Imports: splitGraph itself has no resampling dependency) and shown as code otherwise.

The same pattern works for any other resampling library you happen to use.

Build a split_spec to work with

meta <- data.frame(
  sample_id    = c("S1", "S2", "S3", "S4", "S5", "S6"),
  subject_id   = c("P1", "P1", "P2", "P2", "P3", "P3"),
  batch_id     = c("B1", "B2", "B1", "B2", "B1", "B2"),
  timepoint_id = c("T0", "T1", "T0", "T1", "T0", "T1"),
  time_index   = c(0, 1, 0, 1, 0, 1),
  outcome_id   = c("ctrl", "case", "ctrl", "case", "case", "ctrl"),
  stringsAsFactors = FALSE
)

g <- graph_from_metadata(meta, graph_name = "cookbook")
subject_constraint <- derive_split_constraints(g, mode = "subject")
spec <- as_split_spec(subject_constraint, graph = g)
spec
#> <split_spec> subject 
#>   Samples: 6 
#>   Groups: 3 
#>   Block vars: batch_group 
#>   Time var: order_rank  
#>   Stratum var: stratum 
#>   Recommended resampling: grouped_cv

The sample_data table is the contract:

as.data.frame(spec)[, c("sample_id", "group_id", "batch_group", "order_rank")]
#>   sample_id   group_id batch_group order_rank
#> 1        S1 subject:P1          B1          1
#> 2        S2 subject:P1          B2          2
#> 3        S3 subject:P2          B1          1
#> 4        S4 subject:P2          B2          2
#> 5        S5 subject:P3          B1          1
#> 6        S6 subject:P3          B2          2

Adapter 1 — base R: leave-one-group-out folds

This is the simplest meaningful adapter. It groups by whatever split_spec$group_var says is the split unit, and returns one held-out group per fold.

logo_folds <- function(spec, observation_data, sample_id_col = "sample_id") {
  stopifnot(inherits(spec, "split_spec"))
  if (!sample_id_col %in% names(observation_data)) {
    stop("`observation_data` must contain a `", sample_id_col, "` column.")
  }

  joined <- merge(
    observation_data,
    spec$sample_data[, c("sample_id", spec$group_var)],
    by.x = sample_id_col, by.y = "sample_id", sort = FALSE
  )
  joined$.row <- seq_len(nrow(joined))
  groups <- split(joined$.row, joined[[spec$group_var]])

  lapply(names(groups), function(g) {
    list(
      group   = g,
      train   = unlist(groups[setdiff(names(groups), g)], use.names = FALSE),
      assess  = groups[[g]]
    )
  })
}

# Pretend we have an observation frame keyed by sample_id.
set.seed(1)
obs <- data.frame(
  sample_id = meta$sample_id,
  x = rnorm(nrow(meta)),
  y = rbinom(nrow(meta), 1, 0.5)
)

folds <- logo_folds(spec, obs)
length(folds)
#> [1] 3
folds[[1]]
#> $group
#> [1] "subject:P1"
#> 
#> $train
#> [1] 3 4 5 6
#> 
#> $assess
#> [1] 1 2

That is the entire downstream contract: take spec, take an observation frame, return train/assess index lists. Anything more complicated is specific to a resampling library.

Honoring block variables

group_var is the primary split unit, but split_spec also advertises coarser block_vars — dependency axes that should ideally not straddle a fold even when they are not the grouping unit. They are per-sample columns aligned to sample_id, so an adapter reads them exactly like group_var:

spec$block_vars
#> [1] "batch_group"
head(spec$sample_data[, c("sample_id", spec$group_var, spec$block_vars)])
#>   sample_id   group_id batch_group
#> 1        S1 subject:P1          B1
#> 2        S2 subject:P1          B2
#> 3        S3 subject:P2          B1
#> 4        S4 subject:P2          B2
#> 5        S5 subject:P3          B1
#> 6        S6 subject:P3          B2

A block-aware adapter can pass these to a resampler’s blocking/strata argument, or simply audit its folds. Here we check whether any batch straddles the train/assess boundary — a leak a subject-only split does not prevent, and exactly what carrying batch_group on the spec lets a consumer catch:

block <- spec$block_vars[[1]]
block_of <- setNames(spec$sample_data[[block]], spec$sample_data$sample_id)

do.call(rbind, lapply(folds, function(f) {
  data.frame(
    held_out_group    = f$group,
    straddling_batches = paste(
      intersect(block_of[obs$sample_id[f$train]],
                block_of[obs$sample_id[f$assess]]),
      collapse = ", "
    )
  )
}))
#>   held_out_group straddling_batches
#> 1     subject:P1             B1, B2
#> 2     subject:P2             B1, B2
#> 3     subject:P3             B1, B2

Every batch appears on both sides, because grouping by subject does not also block by batch. Whether that matters is a scientific decision — the point is that the spec carries enough information for the adapter to make it.

Honoring the stratum annotation

stratum_var names one more column: the outcome level each sample carries. splitGraph records it and stops there — it never balances folds itself — so stratification is the adapter’s job, and the spec hands it the input.

spec$stratum_var
#> [1] "stratum"
spec$sample_data[, c("sample_id", spec$group_var, spec$stratum_var)]
#>   sample_id   group_id stratum
#> 1        S1 subject:P1    ctrl
#> 2        S2 subject:P1    case
#> 3        S3 subject:P2    ctrl
#> 4        S4 subject:P2    case
#> 5        S5 subject:P3    case
#> 6        S6 subject:P3    ctrl

There is a catch worth knowing before you wire it into a resampler’s strata argument. The annotation is per sample, while the split unit is the group, and the two need not agree: here every subject contributes one case and one ctrl, so no group has a single stratum at all.

tapply(spec$sample_data$stratum, spec$sample_data$group_id,
       function(x) length(unique(x)) == 1L)
#> subject:P1 subject:P2 subject:P3 
#>      FALSE      FALSE      FALSE

A grouped resampler that stratifies needs one stratum per group, and says so plainly when it does not get one:

joined_s <- merge(obs, spec$sample_data[, c("sample_id", "group_id", "stratum")],
                  by = "sample_id", sort = FALSE)

tryCatch(
  rsample::group_vfold_cv(joined_s, group = "group_id", v = 3, strata = "stratum"),
  error = function(e) conditionMessage(e)
)
#> [1] "strata must be constant across all members of each group."

So the adapter has to decide how to get there — summarise the annotation to the group level (the group’s only label when it has one, a majority label otherwise), stratify on a blocking variable that is constant within the group, or accept unbalanced folds. splitGraph deliberately does not pick for you; it records what each sample is so the choice is visible instead of silent.

Adapter 2 — `rsample::group_vfold_cv()`

Grouped CV keyed to group_id. The downstream package would typically ship something like this; the adapter is short enough that you can paste it into your own analysis script.

spec_to_group_vfold <- function(spec, observation_data,
                                v = NULL,
                                sample_id_col = "sample_id") {
  stopifnot(inherits(spec, "split_spec"))
  if (!requireNamespace("rsample", quietly = TRUE)) {
    stop("Install rsample to use this adapter.")
  }

  joined <- merge(
    observation_data,
    spec$sample_data[, c("sample_id", spec$group_var)],
    by.x = sample_id_col, by.y = "sample_id", sort = FALSE
  )

  n_groups <- length(unique(joined[[spec$group_var]]))
  if (is.null(v)) v <- n_groups

  rsample::group_vfold_cv(
    data  = joined,
    group = !!spec$group_var,
    v     = v
  )
}

v = NULL (the default above) gives leave-one-group-out, which is the right default when splitGraph has already grouped samples by their deepest leakage-relevant unit (e.g. subject). Pick a smaller v for k-fold-style grouped CV.

grouped <- spec_to_group_vfold(spec, obs)
grouped
#> # Group 3-fold cross-validation 
#> # A tibble: 3 × 2
#>   splits        id       
#>   <list>        <chr>    
#> 1 <split [4/2]> Resample1
#> 2 <split [4/2]> Resample2
#> 3 <split [4/2]> Resample3
# Every assessment set is exactly one subject's samples:
vapply(grouped$splits, function(s) {
  paste(sort(unique(rsample::assessment(s)$group_id)), collapse = ", ")
}, character(1))
#> [1] "subject:P2" "subject:P1" "subject:P3"

Adapter 3 — `rsample::rolling_origin()`

When spec$ordering_required is TRUE (or spec$time_var is set), the right downstream object is an ordered split rather than a grouped one.

spec_to_rolling_origin <- function(spec, observation_data,
                                   sample_id_col = "sample_id",
                                   initial = NULL,
                                   assess = 1L) {
  stopifnot(inherits(spec, "split_spec"))
  if (is.null(spec$time_var)) {
    stop("This split_spec has no `time_var`; ordered evaluation is not available.")
  }
  if (!requireNamespace("rsample", quietly = TRUE)) {
    stop("Install rsample to use this adapter.")
  }

  joined <- merge(
    observation_data,
    spec$sample_data[, c("sample_id", spec$time_var)],
    by.x = sample_id_col, by.y = "sample_id", sort = FALSE
  )
  ordered <- joined[order(joined[[spec$time_var]]), , drop = FALSE]

  if (is.null(initial)) initial <- max(1L, floor(nrow(ordered) * 0.6))
  rsample::rolling_origin(ordered, initial = initial, assess = assess)
}

rolling <- spec_to_rolling_origin(spec, obs, initial = 3, assess = 1)
rolling
#> # Rolling origin forecast resampling 
#> # A tibble: 3 × 2
#>   splits        id    
#>   <list>        <chr> 
#> 1 <split [3/1]> Slice1
#> 2 <split [4/1]> Slice2
#> 3 <split [5/1]> Slice3
# No analysis sample comes after any assessment sample:
vapply(rolling$splits, function(s) {
  max(rsample::analysis(s)$order_rank) <= min(rsample::assessment(s)$order_rank)
}, logical(1))
#> [1] TRUE TRUE TRUE

The key idea: splitGraph puts ordering information on the spec; the adapter is just a thin shim that consumes it.

Two things to check before trusting an ordered adapter

order_rank has ties, and row-wise slicing ignores them. order_rank is a rank over timepoints, not over rows, so every sample collected at the same timepoint shares a value — here six samples carry just two distinct ranks. A resampler that slices by row position will therefore put part of a timepoint in the analysis set and the rest in the assessment set:

length(unique(spec$sample_data$order_rank))   # distinct ranks
#> [1] 2
nrow(spec$sample_data)                        # rows
#> [1] 6

vapply(rolling$splits, function(s) {
  shared <- intersect(rsample::analysis(s)$order_rank,
                      rsample::assessment(s)$order_rank)
  paste(shared, collapse = ", ")
}, character(1))
#> [1] ""  "2" "2"

Slices 2 and 3 share rank 2: one T1 sample is being predicted while another T1 sample is in training. If your evaluation requires that a whole timepoint be held out, cut on rank boundaries rather than row counts — choose initial so it falls at a change of order_rank, or group the rows by rank first. The spec gives you the rank; only you know whether ties are acceptable.

rolling_origin() is superseded. It still works and is not deprecated, but rsample now steers users to sliding_window() / sliding_index() / sliding_period(), where active development happens. The equivalent call is:

joined  <- merge(obs, spec$sample_data[, c("sample_id", spec$time_var)],
                 by = "sample_id", sort = FALSE)
ordered <- joined[order(joined[[spec$time_var]]), , drop = FALSE]

sliding <- rsample::sliding_window(
  ordered,
  lookback     = Inf,   # cumulative analysis window, like rolling_origin()
  assess_stop  = 1,
  complete     = FALSE,
  skip         = 2      # start where `initial = 3` did
)

identical(
  lapply(sliding$splits, function(s) rsample::assessment(s)$sample_id),
  lapply(rolling$splits, function(s) rsample::assessment(s)$sample_id)
)
#> [1] TRUE

Either way the adapter is the same shim; only the rsample entry point changes.

Going across language boundaries via JSON

If the downstream consumer is not in R, write the spec to JSON and let the consumer interpret it. The on-disk format is a formal, versioned contract: it has a JSON Schema (Draft 2020-12) shipped in inst/schema/<schema_version>/, each file names it via a $schema key, and validate_split_spec_json() checks a file against it before you consume it.

tmp <- tempfile(fileext = ".json")
write_split_spec(spec, tmp)

# The file opens with its $schema reference and schema_version.
cat(readLines(tmp, n = 5), sep = "\n")
#> {
#>   "$schema": "https://raw.githubusercontent.com/selcukorkmaz/splitGraph/main/inst/schema/0.3.0/split_spec.schema.json",
#>   "splitGraph_object": "split_spec",
#>   "schema_version": "0.3.0",
#>   "group_var": "group_id",

# Validate the file against the shipped JSON Schema, then read it back exactly.
validate_split_spec_json(tmp)$valid
#> [1] TRUE
spec2 <- read_split_spec(tmp)
identical(spec$sample_data$group_id, spec2$sample_data$group_id)
#> [1] TRUE

unlink(tmp)

You do not have to write a JSON parser to consume this from Python: the package ships a pure-Python reference reader (inst/python/splitspec) that recovers the same grouping and ordering and drives scikit-learn GroupKFold / TimeSeriesSplit. The cross-language-handoff vignette walks the full R → JSON → Python → scikit-learn path:

vignette("cross-language-handoff", package = "splitGraph")

The same read/write pair exists for dependency_graph (write_dependency_graph() / read_dependency_graph(), validated with validate_graph_json()). Both formats are documented under ?write_split_spec and ?write_dependency_graph. Because schema_version follows a documented major-compatibility policy, a file written by an older splitGraph still loads; migrate_split_spec_json() upgrades it to the current version in place.

Letting the spec pick the adapter

The three adapters above cover different shapes of split. You do not have to choose between them by hand: split_spec carries recommended_resampling, so a single dispatcher can route each spec to the right one. This makes a pipeline that handles subject, batch, time, and composite specs uniformly.

recommend_adapter <- function(spec) {
  switch(
    spec$recommended_resampling,
    grouped_cv          = "group_vfold_cv (group = group_id)",
    blocked_cv          = "group_vfold_cv (group = group_id)",
    custom_grouped_cv   = "group_vfold_cv (group = group_id)",
    leave_one_group_out = "leave-one-group-out over group_id",
    ordered_split       = "rolling_origin (order by order_rank)",
    "group_vfold_cv (default)"
  )
}

# The subject spec recommends grouped CV; a time-mode spec recommends ordering.
recommend_adapter(spec)
#> [1] "group_vfold_cv (group = group_id)"
time_spec <- as_split_spec(derive_split_constraints(g, mode = "time"), graph = g)
recommend_adapter(time_spec)
#> [1] "rolling_origin (order by order_rank)"

Those five branches are the whole vocabulary: recommended_resampling takes one of exactly grouped_cv, blocked_cv, leave_one_group_out, ordered_split and custom_grouped_cv, whatever the mode. Eleven modes map onto them like this:

`recommended_resampling`	Modes that produce it
`grouped_cv`	`subject`, `region`, `assay`, `relatedness`, `spatial`, `composite` (rule-based)
`blocked_cv`	`batch`, `platform`
`leave_one_group_out`	`study`, `site`
`ordered_split`	`time`
`custom_grouped_cv`	`composite` (strict)

# One graph carrying every direct relation, so each mode has something to group.
g_all <- graph_from_metadata(transform(
  meta,
  site_id     = c("N", "N", "N", "B", "B", "B"),
  region_id   = "ctx",
  platform_id = "il",
  assay_id    = "rna"
))

modes <- c("subject", "batch", "study", "time",
           "site", "region", "platform", "assay")
vapply(modes, function(m) {
  as_split_spec(derive_split_constraints(g_all, mode = m),
                graph = g_all)$recommended_resampling
}, character(1))
#>               subject                 batch                 study 
#>          "grouped_cv"          "blocked_cv" "leave_one_group_out" 
#>                  time                  site                region 
#>       "ordered_split" "leave_one_group_out"          "grouped_cv" 
#>              platform                 assay 
#>          "blocked_cv"          "grouped_cv"

# And the two composite strategies, which supply the fifth value.
c(
  strict = as_split_spec(derive_split_constraints(
    g_all, mode = "composite", strategy = "strict",
    via = c("Subject", "Batch")), graph = g_all)$recommended_resampling,
  rule_based = as_split_spec(derive_split_constraints(
    g_all, mode = "composite", strategy = "rule_based",
    priority = c("subject", "batch")), graph = g_all)$recommended_resampling
)
#>              strict          rule_based 
#> "custom_grouped_cv"        "grouped_cv"

So a dispatcher with those five branches cannot be surprised by a spec, and the default arm above is belt and braces rather than a real fallback. recommended_resampling is still only a hint — your adapter is free to override it — but it lets one entry point serve every constraint mode without inspecting the graph.

When you need a custom adapter

The only assumptions an adapter has to honor:

split_spec$sample_data is keyed by sample_id (character).
split_spec$group_var is the column that holds the splitting unit.
split_spec$block_vars are present-but-coarser blocking columns. Depending on the graph these include batch_group, study_group, site_group, region_group, platform_group, and assay_group — an adapter can block or stratify on any that are present.
split_spec$time_var, when non-NULL, defines the ordering. Its values are ranks over timepoints, so ties are expected and meaningful.
split_spec$stratum_var, when non-NULL, names the per-sample outcome level. It is an annotation: splitGraph never balances folds, and the value is not guaranteed constant within a group.
split_spec$recommended_resampling is a hint, not a contract — your adapter is free to ignore it. It is one of five documented strings. constraint_mode / constraint_strategy are available if you want to branch (e.g. treat "time" specs as ordered).

That is the whole interface, and it is stable: bioLeak::as_leaksplits() consumes exactly these fields, and a contract test in splitGraph pins the seam so it cannot drift. As long as those fields are honored, anything is a valid downstream consumer.

Adapter cookbook: from split_spec to native resamples

Selçuk Korkmaz

2026-09-17

What this vignette is for

Build a split_spec to work with

Adapter 1 — base R: leave-one-group-out folds

Honoring block variables

Honoring the stratum annotation

Adapter 2 — `rsample::group_vfold_cv()`

Adapter 3 — `rsample::rolling_origin()`

Two things to check before trusting an ordered adapter

Going across language boundaries via JSON

Letting the spec pick the adapter

When you need a custom adapter

Adapter cookbook: from split_spec to native resamples

Selçuk Korkmaz

2026-09-17

What this vignette is for

Build a split_spec to work with

Adapter 1 — base R: leave-one-group-out folds

Honoring block variables

Honoring the stratum annotation

Adapter 2 — rsample::group_vfold_cv()

Adapter 3 — rsample::rolling_origin()

Two things to check before trusting an ordered adapter

Going across language boundaries via JSON

Letting the spec pick the adapter

When you need a custom adapter

Adapter 2 — `rsample::group_vfold_cv()`

Adapter 3 — `rsample::rolling_origin()`