Learning Hierarchical Partially Observable Markov Decision Processes for Robot Navigation

IEEE Conference on Robotics and Automation , (ICRA01), 2001, Seoul, South Korea
Learning Hierarchical Partially Observable Markov Decision Processes for Robot Navigation
Georgios Theocharous, Khashayar Rohanimanesh, Sridhar Mahadevan

We propose and investigate a general framework for hierarchical modeling of partially observable environments, such as oce buildings, using Hierarchical Hidden Markov Models (HHMMs). Our main goal is to explore hierarchical modeling as a basis for designing more ecient methods for model construction and useage.

As a case study we focus on indoor robot navigation and show how this framework can be used to learn a hierarchy of models of the environment at dierent levels of spatial abstraction. We introduce the idea of model reuse that can be used to combine already learned models into a larger model.

We describe an extension of the HHMM model to includes actions, which we call hierarchical POMDPs, and describe a modied hierarchical Baum-Welch algorithm to learn these models. We train dierent families of hierarchical models for a simulated and a real world corridor environment and compare them with the standard \at" representation of the same environment.

We show that the hierarchical POMDP approach, combined with model reuse, allows learning hierarchical models that t the data better and train faster than at models.

Another publication from the same category: Machine Learning and Data Science

IEEE Computing Conference 2018, London, UK

Regularization of the Kernel Matrix via Covariance Matrix Shrinkage Estimation

The kernel trick concept, formulated as an inner product in a feature space, facilitates powerful extensions to many well-known algorithms. While the kernel matrix involves inner products in the feature space, the sample covariance matrix of the data requires outer products. Therefore, their spectral properties are tightly connected. This allows us to examine the kernel matrix through the sample covariance matrix in the feature space and vice versa. The use of kernels often involves a large number of features, compared to the number of observations. In this scenario, the sample covariance matrix is not well-conditioned nor is it necessarily invertible, mandating a solution to the problem of estimating high-dimensional covariance matrices under small sample size conditions. We tackle this problem through the use of a shrinkage estimator that offers a compromise between the sample covariance matrix and a well-conditioned matrix (also known as the "target") with the aim of minimizing the mean-squared error (MSE). We propose a distribution-free kernel matrix regularization approach that is tuned directly from the kernel matrix, avoiding the need to address the feature space explicitly. Numerical simulations demonstrate that the proposed regularization is effective in classification tasks.