Skip to contents

Cost functions

cost-functions.Rmd

This chapter lists the cost functions built into rupturesRcpp, their options, and how to supply your own cost as an R function. A cost function measures how badly a single model fits a segment; every segmentation class and costFactory takes one as a costFunc object.

Built-in cost functions

Create a costFunc object by specifying the desired (supported) cost function and optional parameters:

costFuncObj = costFunc$new("L2")
costFuncObj$pass() #output attributes corresponding to the specified cost function.
#> $costFunc
#> [1] "L2"

The following table shows the list of supported cost functions (pre-implemented ones are PELT-compatible; see Segmentation methods). Here, n is the segment length, and the time complexity is that of one segment query. For "LinearL1", p is the number of features (columns of tsMat), J the number of regression coefficients (covariates plus the intercept) and maxIter the maximum number of IRLS iterations: every query refits the regression on the segment’s rows, so it grows linearly in n.

Cost function Description Parameters/active bindings Dimension Time complexity
"L1" Sum of L1 distances to the segment-wise median; robust to outliers. costFunc multi O(nlog(n))
"L2" Sum of squared L2 distances to the segment-wise mean; faster but less robust than L1. costFunc multi O(1)
"SIGMA" Log-determinant of the empirical covariance (divided by n, no bias correction); models varying mean&variance. costFunc, addSmallDiag, epsilon multi O(1)
"LinearL1" Sum of L1 residuals from a linear regression model, fit via Iteratively Reweighted Least Squares (IRLS); robust to outliers. costFunc, intercept, tol, maxIter multi O(p·n·J²·maxIter)
"LinearL2" Sum of squared residuals from a linear regression model with constant noise variance. costFunc, intercept multi O(1)
"LinearSIGMA" Log-determinant of the residual covariance (divided by n, no degrees-of-freedom correction) from a linear regression model; models varying noise covariance around a regression mean. costFunc, intercept, addSmallDiag, epsilon multi O(1)
"VAR" Sum of squared residuals from a vector autoregressive model with constant noise variance. costFunc, pVAR multi O(1)
"Custom" User-defined cost, supplied as a plain R function (see User-defined cost functions below). costFunc, evalFun, paramFun multi depends on evalFun

Every covariance in the package, in the "SIGMA"/"LinearSIGMA" costs and in the cov returned by $get_params()/$segments(), is the biased maximum-likelihood estimate: divided by the segment length n, with no Bessel (n - 1) or degrees-of-freedom (n - q for q regression coefficients) correction.

If active binding costFunc is modified by assigning to costFuncObj$costFunc and the required parameters are missing, the default parameters will be used. This does not apply to "Custom": there is no sensible default evalFun, so $pass()/$fit() will error until one is set (see below).

costFuncObj$costFunc = "VAR"
costFuncObj$pass()
#> $costFunc
#> [1] "VAR"
#> 
#> $pVAR
#> [1] 1

User-defined cost functions

If none of the built-in cost functions fit, costFunc = "Custom" lets you supply your own as a plain R function, with no C++ required. It takes two active bindings:

  • evalFun (required): called as evalFun(segment, a, b), where segment is the raw matrix of rows (a+1):b of the fitted tsMat, and a/b are the same 0-indexed (a,b] bounds $eval(a, b) uses. Must return a single numeric value.
  • paramFun (optional, default NULL): same calling convention, used by internal parameter reporting; if omitted, "Custom" simply reports no parameters.

Passing a/b through, not just segment, is what makes this more than a convenience wrapper: evalFun can use them to align segment against any other externally-captured, position-indexed data (e.g. an exogenous series or weight vector) that the package itself is never told about. Because each call crosses back into R, it is substantially slower per call than the built-in costs, so prefer one of those when it fits.

For example, "L2" re-implemented as a "Custom" cost:

myL2eval = function(segment, a, b){
  segment = as.matrix(segment)
  cm = colMeans(segment)
  sum(sweep(segment, 2, cm, FUN = "-")^2)
}

customCF = costFunc$new("Custom", evalFun = myL2eval)
customCF$pass()
#> $costFunc
#> [1] "Custom"
#> 
#> $evalFun
#> function (segment, a, b) 
#> {
#>     segment = as.matrix(segment)
#>     cm = colMeans(segment)
#>     sum(sweep(segment, 2, cm, FUN = "-")^2)
#> }
#> 
#> $paramFun
#> NULL

Risk of data mismatch

The segmentation logic of existing modules only depends on being able to compute the cost for an arbitrary segment (a,b]; it does not depend on how the data are stored. Therefore, with a custom cost function, a mismatch can occur if the function relies on external data that are not part of the object passed to $fit(). The Custom cost functions case study shows an example.

Using a cost function

A costFunc object is passed to a segmentation class or to costFactory: