Underactuated Robotics

Algorithms for Walking, Running, Swimming, Flying, and Manipulation

Russ Tedrake

© Russ Tedrake, 2022
Last modified .
How to cite these notes, use annotations, and give feedback.

Note: These are working notes used for a course being taught at MIT. They will be updated throughout the Spring 2022 semester. Lecture videos are available on YouTube.

Previous Chapter Table of contents Next Chapter

Output Feedback (aka Pixels-to-Torques)

In this chapter we will start considering systems of the form: \begin{gather*} \bx[n+1] = {\bf f}(\bx[n], \bu[n], \bw[n], n) \\ \by[n] = {\bf g}(\bx[n], \bu[n], \bv[n], n).\end{gather*} In other words, we'll finally start addressing the fact that we have to make decisions based on sensor measurements -- most of our discussions until now have tacitly assumed that we have access to the true state of the system for use in our feedback controllers (and that's already been a hard problem).

In some cases, we will see that the assumption of "full-state feedback" is not so bad -- we do have good tools for state estimation from raw sensor data. But even our best state estimation algorithms do add some dynamics to the system in order to filter out noisy measurements; if the time constants of these filters is near the time constant of our dynamics, then it becomes important that we include the dynamics of the estimator in our analysis of the closed-loop system.

In other cases, it's entirely too optimistic to design a controller assuming that we will have an estimate of the full state of the system. Some state variables might be completely unobservable, others might require specific "information-gathering" actions on the part of the controller.

For me, the problem of robot manipulation is the application domain where more direct approaches to output feedback become critically important. Imagine you are trying to design a controller for a robot that needs to button the buttons on your dress shirt. If step one is to estimate the state of the shirt (how many degrees of freedom does my shirt have?), then it feels like we're not going to be successful. Or if you want to program a robot to make a salad -- what's the state of the salad? Do I really need to know the positions and velocities of every piece of lettuce in order to be successful?

The Classical Perspective

To some extent, this idea of calling out "output feedback" as a special, advanced topic is a new problem. Before state space and optimization-based approaches to control ushered in "modern control", we had "classical control". Classical control focused predominantly (though not exclusively) on linear time-invariant (LTI) systems, and made very heavy use of frequency-domain analysis (e.g. via the Fourier Transform/Laplace Transform). There are many excellent books on the subject; Hespanha09+Astrom10 are nice examples of modern treatments that start with state-space representations but also treat the frequency-domain perspective.

What's important for us to acknowledge here is that in classical control, basically everything was built around the idea of output feedback. The fundamental concept is the transfer function of a system, which is a input-to-output map (in frequency domain) that can completely characterize an LTI system. Core concepts like pole placement and loop shaping were fundamentally addressing the challenge of output feedback that we are discussing here. Sometimes I feel that, despite all of the things we've gain with modern, optimization-based control, I worry that we've lost something in terms of considering rich characterizations of closed-loop performance (rise time, dwell time, overshoot, ...) and perhaps even in practical robustness of our systems to unmodeled errors.

Add a few examples here that capture it.

Observer-based Feedback

Luenberger Observer

Linear Quadratic Regulator w/ Gaussian Noise (LQG)

Partially-observable Markov Decision Processes

 Defer the rest of the discussion to the state estimation chapter
Russ Tedrake 11:28 AM so we have three broad approaches to searching for linear dynamical controllers (for linear dynamical plants): 1) gradient descent in the original parameters. you have asymptotic convergence, but not a guaranteed rate. 2) convex reparameterizations (e.g. from scherer). in here there is a rank condition is trivially satisfied, and therefore disappears, when dim(xc) >= dim(x) 3) Youla/SLS, which in the time-domain is a finite-time synthesis but can be used to recover the LDC to arbitrary precision (e.g. via Ho-Kalman). is that a reasonable summary? jack 11:36 AM I think this is reasonable :slightly_smiling_face: I would probably add the Riccati solutions of the famous DGKF paper https://authors.library.caltech.edu/3087/1/DOYieeetac89.pdf (if you wanted “completeness”) gradient descent in original parameters; no suboptimal stationary points for minimal controllers finite dimensional convex reparametrizations from Scherer, as you wrote DGKF solution: solve two Riccati equations Disturbance feedback: Youla/Q/SLS, etc. Synthesis problem is convex, but the decision variable(s) are infinite dimensional, but stable, transfer functions. Various approximations can be used (most common these days, for academic examples and learning theorists, is FIR). 11:37 I would be a bit cautious about recovering the LDC with Ho-Kalman. Nik Matni and James Anderson had a paper https://ieeexplore.ieee.org/abstract/document/8262844 about recovering state-space controllers from FIRs, in the SLS setting. According to James, I don’t think they meant it as a serious paper (i.e. something that you would ever actually implement), just an investigation. I suppose I should re-read that paper, but there was no dimensionality reduction going on, when recovering a state-space representation of the FIR

Static Output Feedback

For Linear Systems

 Bilinear alternations with SOS, Policy search with SGD

Disturbance-based feedback

State-space models. ARX Models.

System-Level Synthesis
Task-relevant variables

References

  1. Joao P. Hespanha, "Linear Systems Theory", Princeton Press , 2009.

  2. Karl Johan {\AA}str{\"o}m and Richard M Murray, "Feedback systems: an introduction for scientists and engineers", Princeton university press , 2010.

Previous Chapter Table of contents Next Chapter