Distortion-invariant recognition via jittered queries

Computer Vision and Pattern Recognition (CVPR-2000), June 2000
Distortion-invariant recognition via jittered queries
Dennis DeCoste, M.C. Burl

This paper presents a new approach for achieving distortion-invariant recognition and classification. A test example to be classified is viewed as a query intended to find similar examples in the training set (or class models derived from the training set). The key idea is that instead of querying with a single pattern, we construct a more robust query, based on the family of patterns formed by distorting the test example.

Although query execution is slower than if the invariances were successfully pre-compiled during training, there are significant advantages in several contexts:

(i) providing invariances in memory-based learning,

(ii) in model selection, where reducing training time at the expense of test time is a desirable trade-off, and

(iii) in enabling robust, ad hoc searches based on a single example. Preliminary tests for memory-based learning on the NIST handwritten digit database with a limited set of shearing and translation distortions produced an error rate of 1.35%.

Another publication from the same category: Machine Learning and Data Science

IEEE Computing Conference 2018, London, UK

Regularization of the Kernel Matrix via Covariance Matrix Shrinkage Estimation

The kernel trick concept, formulated as an inner product in a feature space, facilitates powerful extensions to many well-known algorithms. While the kernel matrix involves inner products in the feature space, the sample covariance matrix of the data requires outer products. Therefore, their spectral properties are tightly connected. This allows us to examine the kernel matrix through the sample covariance matrix in the feature space and vice versa. The use of kernels often involves a large number of features, compared to the number of observations. In this scenario, the sample covariance matrix is not well-conditioned nor is it necessarily invertible, mandating a solution to the problem of estimating high-dimensional covariance matrices under small sample size conditions. We tackle this problem through the use of a shrinkage estimator that offers a compromise between the sample covariance matrix and a well-conditioned matrix (also known as the "target") with the aim of minimizing the mean-squared error (MSE). We propose a distribution-free kernel matrix regularization approach that is tuned directly from the kernel matrix, avoiding the need to address the feature space explicitly. Numerical simulations demonstrate that the proposed regularization is effective in classification tasks.