## Tour de Heuristics: Policy Iteration

Policy Iteration is the most available option for dealing with infinite horizon DEC-POMDP’s. In this space, it is sub-optimal. It can be, however, epsilon-optimal. Epsilon optimality means that based on the starting point and a decay factor, we can plan a controller out for enough steps that the expected discounted Continue Reading

## Tour de Heuristics: MAA*

Multiagent A* is a heuristic that takes the commonly used A* algorithm and applies it to Dec-POMDP’s. Let’s investigate how it works. The Algorithm def estimated_state_value(belief, action): """ The cornerstone of the A* algorithm is to have an optimistic estimator. Because an MDP assumes more information, it will always have Continue Reading

## Multi-Agent Systems: Finite Horizons

In our previous post, we covered the basics of what a Dec-POMDP is. Let’s actually look at what a policy is and how we can generate one. The Example In this example, there are two robots that are trying to meet anywhere on the map. They want to do this Continue Reading

## Multi-Agent Systems: What is a Dec-POMDP?

This is the first post in a study on all things using Decentralized Partially Observable Markov Decision Processes (Dec-POMDP) with my professor, Prithviraj Dasgupta, who runs the CMantic Lab at the University of Nebraska, Omaha. I intend to write the summaries of what I find as blog posts, so be Continue Reading

## Nightmare before Christmas Code

A common thing seen in the enterprise world is the “implementation” of new technology using the same methods as old technology. It is often done by doing things “the old way” with the new tech. One example “santa tech” that has been recently abused is the term “REST”. Many people Continue Reading

## Stopping SD Card Corruption on Raspberry Pi’s Raspbian

The following are instructions for minimizing SD card writes for Raspberry Pi’s “Raspbian” Distribution. If you’re like me, you’ve run into a corrupted SD card too many times to not become hell-bent on making it never happen again. I have the following setup, and it seems to be working well Continue Reading

## Using the Power of the Markov Assumption

I have hear the words “Markov” more than a few times, but it is only recently that I can appreciate exactly what this simplification buys me. Markov was a guy who liked to model things by using only the current state. This simplification is often very appropriate and often offers Continue Reading

## Bending the Rules of RequireJS to Deal with a Legacy CDN

RequireJS is an awesome library that lets you as a developer write very explicitly modular javascript code that every other language had right out of the box. Unfortunately, because there is such a thing as history, there are a lot of code bases that have a large number of globals. Continue Reading

## Rock the RequireJS in Maven

After a considerable amount of trial and error, I have discovered the following setup steps for enabling requirejs in a maven project. Integrating Maven The requirejs maven plugin world has been marked by two packages: brew, and requirejs-maven-plugin. Brew takes the approach of “we’ll do everything and try to support Continue Reading