Learning Hierarchical Partially Observable Markov Decision Processes for Robot Navigation

IEEE Conference on Robotics and Automation , (ICRA01), 2001, Seoul, South Korea
Learning Hierarchical Partially Observable Markov Decision Processes for Robot Navigation
Georgios Theocharous, Khashayar Rohanimanesh, Sridhar Mahadevan

We propose and investigate a general framework for hierarchical modeling of partially observable environments, such as oce buildings, using Hierarchical Hidden Markov Models (HHMMs). Our main goal is to explore hierarchical modeling as a basis for designing more ecient methods for model construction and useage.

As a case study we focus on indoor robot navigation and show how this framework can be used to learn a hierarchy of models of the environment at dierent levels of spatial abstraction. We introduce the idea of model reuse that can be used to combine already learned models into a larger model.

We describe an extension of the HHMM model to includes actions, which we call hierarchical POMDPs, and describe a modied hierarchical Baum-Welch algorithm to learn these models. We train dierent families of hierarchical models for a simulated and a real world corridor environment and compare them with the standard \at" representation of the same environment.

We show that the hierarchical POMDP approach, combined with model reuse, allows learning hierarchical models that t the data better and train faster than at models.

Another publication from the same category: Machine Learning and Data Science

WWW '17 Perth Australia April 2017

Drawing Sound Conclusions from Noisy Judgments

David Goldberg, Andrew Trotman, Xiao Wang, Wei Min, Zongru Wan

The quality of a search engine is typically evaluated using hand-labeled data sets, where the labels indicate the relevance of documents to queries. Often the number of labels needed is too large to be created by the best annotators, and so less accurate labels (e.g. from crowdsourcing) must be used. This introduces errors in the labels, and thus errors in standard precision metrics (such as P@k and DCG); the lower the quality of the judge, the more errorful the labels, consequently the more inaccurate the metric. We introduce equations and algorithms that can adjust the metrics to the values they would have had if there were no annotation errors.

This is especially important when two search engines are compared by comparing their metrics. We give examples where one engine appeared to be statistically significantly better than the other, but the effect disappeared after the metrics were corrected for annotation error. In other words the evidence supporting a statistical difference was illusory, and caused by a failure to account for annotation error.