What is the Kalman filter?
The Kalman filter is a means of state estimation named after the one of the many developers of the concept - Rudolf Kalman. It is described as a means of using mutliple noisy measurements of a
Rudolf E. Kalman
Kalman was a Hungarian born American engineer and mathematician known for the filter named after him and the very foundations of its application - the state space representation theory.
quantity, usually a “state” measurement to make a best guess estimate of the probable state value at a future time.
A state measurement can be interpreted quite literally as the state of a system, which for a human being from a physiological perspective be things like their temperature, their heart rate, the insulin levels in their blood stream, and even how they might be feeling emotionally - happy, content, sad, bored, etc. And while temperature and insulin levels are continuous values, heart rate
Continuous variables
can assume any two infintesimally different values which might for example be similar up to the \(10^{-100 th}\) place, and can in general assume any of the uncountable number of values within its range.
(and emotional status to some extent) is a discrete state variable, it
Discrete variables
can assume only a fixed set of values and cannot take intermediate values in between two consecutive discrete ones.
can only assumea quantized set of values -integers in this case, and there can be no intermediate fractional values.
Now, discrete or continuous, these “state” values or variables have something in common. They are all values of which we can measure across time and put down in a sequential form as a graph or a trend. And many times engineers and scientists have to deal with systems which are either very complex like the physiology of the animal body, or are such that measurements of some important states can’t be easily obtained, like tracking the position and velocity of a spacecraft in real time to make course corrections to follow a planned trajectory.
How does it work?
The Wiener filter was the predecessor of the kalman filter, named after scientist Norbert Wiener, which is a technique of estimating the state of a system or a variable which has a linear response and is stationary.
LTI systems
Time invariant systems are those systems whose “dynamics” do not vary over time. They are such that the update rules that take the system from one time step to the next are governed by a fixed set of equations, and given the state of the system at a certain time step, we can repeatedly apply these update rules to get the state of the system at any future or past time.
LTI - stands for linear time invariant systems, and they have output responses which are some linear combination of time delayed input signals.
The Wiener filter is a means of true state estimation upon corruption with noise under the conditions that the noise is additive and we know the statistical properties of the noise, such as its mean and its variance.
Fig 1. shows the effects of a simple denoising wiener filter applied to a noisy picture and its output when applied as an average over 7x7 pixel groups. When the pixel grouping and the statistical parameters of the noise are estimated optimally, the filter performs quite well, but the results are not optimal. For example, the denoised filter in the Fig 1. example had as MSE of 4422.45 compared
Mean Squared Error
Mean squared error is a metric of divergence from a reference quantity calculated as the mean of the element wise squares of difference in component values.
mse = \(\frac{1}{n} \sum_{n} ( x_{reference} - x_{being measured} )^2\)
to the original figure compared to the noisy image with an MSE of 4490.06. And while this is an improvement, it is a slight one and was not satisfactory for applications that the kalman filter was historically first used for, such as the circumlunar trajectory guidance and navigation for the Apollo mission’s onboard computers 1.
The Kalman filter on the other hand, does not need to know the statistical properties of the noise or any adversarial effects distorting the signal to be estimated. It starts with a guess of the mean and the variance of the “noise” and updates its estimates along with its estimate of the state as more noisy observations are obtained.
The pseudo-code for the kalman filter is given below: