A Kolmogorov-Arnold network computes, in each layer, \(h^{(l+1)}_j = \sum_i \phi_{j,i}(h^{(l)}_i)\). The variants published since Liu et al. (2024) differ only in how an edge function \(\phi_{j,i}\) is written: a Chebyshev, Legendre or Jacobi expansion of \(\tanh x\), Gaussian radial basis functions (FastKAN), Mexican-hat wavelets (Wav-KAN), safe Pade rational functions, a hybrid SiLU + Chebyshev + RBF sum, or squared ReLU bumps (ReLU-KAN). thangldKANr keeps one engine (training loop with both backends, preprocessing, early stopping, prediction, edge plots, persistence, validation) and treats the edge type as an argument.
kan_bases()[, c("name", "label")]
#> name label
#> 1 chebyshev Chebyshev polynomials (ChebyKAN)
#> 2 rbf Gaussian radial basis functions (FastKAN)
#> 3 legendre Legendre polynomials
#> 4 wavelet Mexican-hat wavelets (Wav-KAN)
#> 5 rational Safe Pade rational functions
#> 6 hybrid SiLU + Chebyshev + RBF (hybrid)
#> 7 jacobi Jacobi polynomials (JacobiKAN)
#> 8 relu Squared ReLU bumps (ReLU-KAN)Layers are stacked; interactions between inputs arise through composition, so a one-layer network is an additive model and should be reported as such.
Inputs and response are standardized with constants estimated on the
training rows; predictions are returned on the original scale. The base
R backend (backend = "r") is used here so the vignette
builds without libtorch; backend = "torch" (the default)
uses R ‘torch’ autograd and follows the same trajectory in double
precision.
set.seed(1)
n <- 400
x <- matrix(runif(2 * n, -2, 2), n, 2, dimnames = list(NULL, c("x1", "x2")))
f <- function(x) x[, "x1"] * x[, "x2"] + sin(x[, "x1"])
y <- f(x) + rnorm(n, sd = 0.05)
train <- 1:240
valid <- 241:300
test <- 301:400
fit_one <- function(basis, ...) {
thangld_kan_fit(thangld_kan(c(2, 4, 1), basis = basis, ..., seed = 1),
x[train, ], y[train],
validation_data = list(x = x[valid, ], y = y[valid]),
epochs = 150, batch_size = 32, lr = 0.02, patience = 30,
seed = 1, backend = "r")
}
fits <- list(chebyshev = fit_one("chebyshev", degree = 4),
rbf = fit_one("rbf"),
relu = fit_one("relu"))
t(sapply(fits, function(m) thangld_kan_metrics(y[test], predict(m, x[test, ]))))
#> rmse mae r2
#> chebyshev 0.1258298 0.09181367 0.9934847
#> rbf 0.5212386 0.35118095 0.8882005
#> relu 0.1213449 0.08555169 0.9939409The epoch with the lowest validation loss is restored; the test rows were not used for any choice. This small example is not a benchmark: the ranking of bases depends on the data, the width, the basis size and the training budget.
An edge function is one component of the fitted network. In a multi-layer network it is not the partial effect of a predictor, and it is never a causal effect.
basis may have one entry per layer. Basis-specific
arguments go to the layers whose basis uses them; options
sets values for one basis only.
m <- thangld_kan(c(2, 4, 3, 1), basis = c("rbf", "chebyshev", "jacobi"),
degree = 3, options = list(jacobi = list(alpha = 0.5)),
seed = 2)
m
#> <thangld_kan> Kolmogorov-Arnold network
#> widths : 2 -> 4 -> 3 -> 1
#> layers : RBF G=8 d=0.571 +LN +base; Chebyshev K=3; Jacobi K=3 (alpha=0.5, beta=1)
#> params : 140
#> fitted : no
kan_layer_bases(m)
#> [1] "rbf" "chebyshev" "jacobi"Other packages add their edge type with
register_kan_basis(); every function above then works with
it, including thangld_kan_validate(). The interface is
described in ?register_kan_basis. A toy linear edge:
lin <- list(
label = "linear (toy)",
layer_specs = function(widths) rep(list(list()), length(widths) - 1L),
describe = function(spec) "linear",
param_dims = function(p, q, spec) list(w = c(p, q)),
init = function(p, q, spec) list(w = matrix(rnorm(p * q, sd = 0.3), p, q)),
check_params = function(par, spec) invisible(TRUE),
forward_r = function(h, par, spec) list(out = h %*% par$w, cache = h),
backward_r = function(cache, g, par, spec)
list(grads = list(w = crossprod(cache, g)), dh = g %*% t(par$w)),
forward_torch = function(h, par, spec) torch::torch_matmul(h, par$w),
edge_r = function(x, par, spec, i, j) data.frame(phi = par$w[i, j] * x))
register_kan_basis("toy_linear", lin)
res <- thangld_kan_validate("r", basis = "toy_linear", mixed = FALSE)
res[, c("group", "check", "pass")]
#> group check pass
#> 1 basis basis-specific checks NA
#> 2 forward oracle vs naive edge-by-edge loops NA
#> 3 gradient parameters vs central difference O4 (h=0.0001) TRUE
#> 4 gradient input vs central difference O4 (h=0.0001) TRUE
#> 5 gradient parameters vs central difference O4 (h=1e-05) TRUE
#> 6 gradient input vs central difference O4 (h=1e-05) TRUE
#> 7 gradient parameters vs central difference O4 (h=1e-06) TRUE
#> 8 gradient input vs central difference O4 (h=1e-06) TRUE
#> 9 save/load predictions before vs after reload TRUE
unregister_kan_basis("toy_linear")The missing basis-specific checks of the toy basis are reported as
skipped (NA), not as passed.
f <- tempfile(fileext = ".rds")
thangld_kan_save(fits$relu, f)
all.equal(predict(fits$relu, x[test, ]), predict(thangld_kan_load(f), x[test, ]))
#> [1] TRUE
res <- thangld_kan_validate("r")
aggregate(pass ~ basis, res, function(p) sprintf("%d/%d", sum(p), length(p)))
#> basis pass
#> 1 chebyshev 12/12
#> 2 hybrid 12/12
#> 3 jacobi 14/14
#> 4 legendre 13/13
#> 5 mixed 8/8
#> 6 rational 12/12
#> 7 rbf 13/13
#> 8 relu 14/14
#> 9 wavelet 16/16thangld_kan_validate("both") with the torch runtime
installed adds the torch forward, autograd and training checks; without
the runtime they are reported as skipped, never as passed.