On the slow convergence of EM and VBEM in low-noise linear models

Kaare Brandt Petersen; Ole Winther; Lars Kai Hansen

doi:10.1162/0899766054322991

On the slow convergence of EM and VBEM in low-noise linear models

Kaare Brandt Petersen, Ole Winther, Lars Kai Hansen

27 Citations (Scopus)

Abstract

We analyze convergence of the expectation maximization (EM) and variational Bayes EM (VBEM) schemes for parameter estimation in noisy linear models. The analysis shows that both schemes are inefficient in the low-noise limit. The linear model with additive noise includes as special cases independent component analysis, probabilistic principal component analysis, factor analysis, and Kalman filtering. Hence, the results are relevant for many practical applications.

Original language	English
Journal	Neural Computation
Volume	17
Issue number	9
Pages (from-to)	1921-1926
Number of pages	6
ISSN	0899-7667
DOIs	https://doi.org/10.1162/0899766054322991
Publication status	Published - Sept 2005
Externally published	Yes

Keywords

Algorithms
Artifacts
Bayes Theorem
Factor Analysis, Statistical
Linear Models
Principal Component Analysis

Access to Document

10.1162/0899766054322991

Cite this

@article{9285a0635aa74c598357c24dd8db34c1,

title = "On the slow convergence of EM and VBEM in low-noise linear models",

abstract = "We analyze convergence of the expectation maximization (EM) and variational Bayes EM (VBEM) schemes for parameter estimation in noisy linear models. The analysis shows that both schemes are inefficient in the low-noise limit. The linear model with additive noise includes as special cases independent component analysis, probabilistic principal component analysis, factor analysis, and Kalman filtering. Hence, the results are relevant for many practical applications.",

keywords = "Algorithms, Artifacts, Bayes Theorem, Factor Analysis, Statistical, Linear Models, Principal Component Analysis",

author = "Petersen, \{Kaare Brandt\} and Ole Winther and Hansen, \{Lars Kai\}",

year = "2005",

month = sep,

doi = "10.1162/0899766054322991",

language = "English",

volume = "17",

pages = "1921--1926",

journal = "Neural Computation",

issn = "0899-7667",

publisher = "MIT Press",

number = "9",

}

TY - JOUR

T1 - On the slow convergence of EM and VBEM in low-noise linear models

AU - Petersen, Kaare Brandt

AU - Winther, Ole

AU - Hansen, Lars Kai

PY - 2005/9

Y1 - 2005/9

N2 - We analyze convergence of the expectation maximization (EM) and variational Bayes EM (VBEM) schemes for parameter estimation in noisy linear models. The analysis shows that both schemes are inefficient in the low-noise limit. The linear model with additive noise includes as special cases independent component analysis, probabilistic principal component analysis, factor analysis, and Kalman filtering. Hence, the results are relevant for many practical applications.

AB - We analyze convergence of the expectation maximization (EM) and variational Bayes EM (VBEM) schemes for parameter estimation in noisy linear models. The analysis shows that both schemes are inefficient in the low-noise limit. The linear model with additive noise includes as special cases independent component analysis, probabilistic principal component analysis, factor analysis, and Kalman filtering. Hence, the results are relevant for many practical applications.

KW - Algorithms

KW - Artifacts

KW - Bayes Theorem

KW - Factor Analysis, Statistical

KW - Linear Models

KW - Principal Component Analysis

UR - https://www.scopus.com/pages/publications/22944474285

U2 - 10.1162/0899766054322991

DO - 10.1162/0899766054322991

M3 - Journal article

C2 - 16106590

SN - 0899-7667

VL - 17

SP - 1921

EP - 1926

JO - Neural Computation

JF - Neural Computation

IS - 9

ER -

On the slow convergence of EM and VBEM in low-noise linear models

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this