State-Space Model

From GM-RKB
(Redirected from State space representation)
Jump to navigation Jump to search

A State-Space Model is a mathematical model that represents a system in terms of a set of input variables, output variables, and state variables (to represent the system's current status), all related by differential equations or difference equations.



References

2024

  • (Wikipedia, 2024) ⇒ https://en.wikipedia.org/wiki/State-space_representation Retrieved:2024-1-5.
    • In control engineering and system identification, a state-space representation is a mathematical model of a physical system specified as a set of input, output and variables related by first-order (not involving second derivatives) differential equations or difference equations. Such variables, called state variables, evolve over time in a way that depends on the values they have at any given instant and on the externally imposed values of input variables. Output variables’ values depend on the values of the state variables and may also depend on the values of the input variables.

      The state space or phase space is the geometric space in which the variables on the axes are the state variables. The state of the system can be represented as a vector, the state vector, within state space.

      If the dynamical system is linear, time-invariant, and finite-dimensional, then the differential and algebraic equations may be written in matrix form. The state-space method is characterized by significant algebraization of general system theory, which makes it possible to use Kronecker vector-matrix structures. The capacity of these structures can be efficiently applied to research systems with modulation or without it. The state-space representation (also known as the "time-domain approach") provides a convenient and compact way to model and analyze systems with multiple inputs and outputs. With [math]\displaystyle{ p }[/math] inputs and [math]\displaystyle{ q }[/math] outputs, we would otherwise have to write down [math]\displaystyle{ q \times p }[/math] Laplace transforms to encode all the information about a system. Unlike the frequency domain approach, the use of the state-space representation is not limited to systems with linear components and zero initial conditions. The state-space model can be applied in subjects such as economics, statistics, computer science and electrical engineering, and neuroscience. In econometrics, for example, state-space models can be used to decompose a time series into trend and cycle, compose individual indicators into a composite index, [1] identify turning points of the business cycle, and estimate GDP using latent and unobserved time series. Many applications rely on the Kalman Filter or a state observer to produce estimates of the current unknown state variables using their previous observations. [2]

  1. James H. Stock & Mark W. Watson, 1989. "New Indexes of Coincident and Leading Economic Indicators," NBER Chapters, in: NBER Macroeconomics Annual 1989, Volume 4, pages 351-409, National Bureau of Economic Research, Inc.
  2. Harvey, Andrew C. (1990). Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge: Cambridge University Press. doi:10.1017/CBO9781107049994

2023

2021

  • (Gu et al., 2021) ⇒ Albert Gu, Karan Goel, and Christopher Ré. (2021). “Efficiently Modeling Long Sequences with Structured State Spaces.” In: arXiv preprint arXiv:2111.00396.
    • NOTE: This paper addresses the challenge of sequence modeling, especially for long-range dependencies in sequences exceeding 10,000 steps. While RNNs, CNNs, and Transformers offer specialized variants for long dependencies, they face scalability issues. The paper discusses the use of the state space model (SSM) \( x'(t) = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t) \) and proposes a new model, the Structured State Space sequence model (S4). The S4 model, with its novel parameterization of the SSM, is computationally efficient and theoretically robust, effectively handling long-range dependencies. It applies a low-rank correction to the state matrix \( A \), leading to a stable diagonalization and reducing the SSM to a computation involving a Cauchy kernel. The model demonstrates strong empirical results across various benchmarks, including high accuracy in sequential CIFAR-10, closing the gap with Transformers in image and language modeling tasks, and achieving state-of-the-art results in the Long Range Arena benchmark. Notably, it solves the Path-X task of length 16k, where prior models struggled, while maintaining efficiency.

2013

  • (Chen & Brown, 2013) ⇒ Z. Chen, and E.N. Brown. (2013). “State-Space Models.” In: Scholarpedia.
    • QUOTE: State space model (SSM) refers to a class of probabilistic graphical model (Koller and Friedman, 2009) that describes the probabilistic dependence between the latent state variable and the observed measurement. The state or the measurement can be either continuous or discrete. The term “state space” originated in 1960s in the area of control engineering (Kalman, 1960). SSM provides a general framework for analyzing deterministic and stochastic dynamical systems that are measured or observed through a stochastic process. The SSM framework has been successfully applied in engineering, statistics, computer science and economics to solve a broad range of dynamical systems problems. Other terms used to describe SSMs are hidden Markov models (HMMs) (Rabiner, 1989) and latent process models. The most well studied SSM is the Kalman filter, which defines an optimal algorithm for inferring linear Gaussian systems.

1998

  • (Kitagawa, 1998) ⇒ G. Kitagawa. (1998). “A Self-Organizing State-Space Model.” In: Journal of the American Statistical Association, JSTOR.
    • NOTE: This paper discusses the limitations of traditional state-space models in handling certain problems and introduces several types of nonlinear non-Gaussian state-space models, including the dynamic generalized linear model.

1994

  • (Hamilton, 1994) ⇒ J.D. Hamilton. (1994). “State-Space Models.” In: Handbook of Econometrics. Elsevier.
    • NOTE: This chapter reviews models of changes in regime and develops the parallel between such models and linear state-space models.