Light

Ongoing workDecision-making under uncertainty

Bayesian Decision Making in Unknown Dynamics

Illustration on real winds (ERA5 reanalysis, September 2017). The balloon only changes altitude; each altitude has its own wind. (The route shown was computed with the wind known to show the setting; it is not a result from the paper, in fact it's less impressive!)

A weather balloon 🎈 cannot steer itself. It can only go up or down ↕️, and the wind 🌬️ at each height carries it somewhere different. To reach a target 🎯 it has to ride the tide 🌊, but it doesn't know where the flow is going! (well maybe it pays attention to weather forecasts πŸ“‘ before going out): it only knows the wind where it has already been.

So it has two jobs at once: learn the wind, and get to where it is going. Should it take a detour to learn more (explore 🧭)? or head for the target on a shaky guess (exploit 🎲)?

Illustration on real winds (ERA5 reanalysis, September 2017). The balloon only changes altitude; each altitude has its own wind. (The route shown was computed with the wind known to show the setting; it is not a result from the paper, in fact it's less impressive!)

This is ongoing work on making decisions in settings with exploration-exploitation tradeoffs, when you don’t know the underlying dynamics and constraints, and can only get partial observations. More soon.

The simulation environment behind this work is zermelo.