Final Project Information

The final project will be your opportunity to take what you have learned in the class and put it towards an application of your choice.

In previous years, it has been very popular to choose a problem that you are passionate about (e.g. a dolphin surfing on waves, controlling a yo-yo, brachiation as upside-down walking, ...), then find the absolute simplest way to formulate a model/problem for this system, then apply some of the tools from class to that problem (trajectory optimization, region of attraction, control design, ...).

Here are some guidelines:

• Apart from rare exceptions, your problem should have apply to a system with dynamics, and should take an algorithmic approach to analyzing/controlling those dynamics.
• You are welcome to work on the project in small teams (2-3 people). If you plan on doing a team project, please make this clear in your proposal. Our expectations for your project will be proportional to the number of people on the team.
• The project is only over 4-5 weeks, so scope your project carefully.
• Developing novel algorithms/systems is great, but reimplementing published work and/or applying a published algorithm to a model system from class is also great. You will be evaluated based on whether the project forced you to understand new concepts, not based on the novelty of the approach.
• Do not underestimate the time it can take to set up a simulation in place. Using/modifying an existing example from the textbook is great. Using Drake is not a requirement, but the staff can help you more if you choose this route.
• Some of you will be able to do a project that is closely related to your current research. This is completely acceptable, as long as the subproblem you are solving for this class is clearly defined. If you are using part or all of your work for research or as part of another class, you must state this explicitly in your final report. We expect you to do something new that you would not have done if you hadn’t taken this class (e.g., I was already working on robot X, but I was using RRT for trajectory planning, and here I tried trajectory optimization).
• The goal of the project is to deepen each student's mastery of some topic from class. If you do work in a similar field, and submit an incremental paper in that line of work that makes no connections to the class material, then that does not constitute a successful project.
• Please don't underestimate the time it could take to get a basic simulation of your system in place. Using/modifying an existing example from the textbook is a great idea. Using Drake is not a requirement, but the staff can help you more if you choose this route.
• We encourage to build on code and tools of the community, but remember that you must clearly distinguish your contribution and acknowledge the work of others to avoid plagiarism. If you do improve an open-source tool or example, continue contributing back!
• You can find some great final projects from previous semesters here. I would love to add your work to the list!

You can find some project ideas at the bottom of this page. The course staff will provide you with feedback on the scope on direction of your proposal shortly after we receive it.

Project Proposal

The project proposal is intended as a forcing function for you to crystalize a project idea. Moreover, it gives us a chance to offer you a feedback and make sure that your plan is plan is feasible in the time alloted to the project. It should be approximately one page (at most, say, 500 words) detailing your idea for the project. It should:

• Define exactly what the project deliverable is.
• Briefly describe why the project is interesting.
• List the topics that we have studied (or that we will study) in this class that are covered by your project. For example: dynamic programming, Lyapunov theory, legged robots, reinforcement learning etc.
• Discuss any related prior work you have found that is relevant. If this project is related to your research or a project you are doing concurrently for another class, tell us about that now (and read the guidelines for the final report below)..
• Define specific goals that you expect to have accomplished before each of the progress updates.

It does not have to be extremely polished, but it does need to provide us with enough information to understand what you're hoping to do. Otherwise, we won't be able to help you!

Progress Updates

The short update will consist of two parts. In the first part, we will expect a 1-paragraph general description in order to keep us up to date with how you have progressed. In the second part, we will expect a specific description on your progress in setting up simulation environment. If you have already set up your simulation, a picture of the setup along with descriptions on the dynamics/environment would be great. If you are still in the process, please describe your plan for building the simulation environment (e.g. relevant toolboxes in drake) and potential technical barriers you expect.

Final Project Video

The presentation video must have maximum duration of 3 minutes for individual projects, 4 minutes for 2 two person projects, and 5 minutes for 3 person projects. These videos will be uploaded on YouTube, more details on this will be given on Piazza.

Final Report

Using the IEEE template for conference proceedings (probably the Latex one, unless you really enjoy using MS Word), write a summary of what you accomplished during your project. Write it, as much as possible, like a conference paper. You should include:

• An abstract.
• An introduction of your project and why you think it is interesting.
• A related work section (at your discretion -- this part will depend a lot on the nature of the work you do).
• Your results (partial or work-in-progress results is expected, and fine!).
• A discussion of your results and potential next steps.
• For team projects: please describe the contributions made by each team member.
• For projects related to your research or a project in another class, please clearly denote what parts of the project overlap (or not) with the other efforts.

For projects that do overlap with your research, please make connections to the topics in the class. A report that might have been written if you had not taken the class does not constitute a successful project. The goal of the project is to demonstrate some level of mastery of the material from class.

We don't have any requirements on length, but realize that this is worth a very large portion of your overall grade, so be as thorough as possible in demonstrating mastery of the course material. For reference, reasonable lengths could be anywhere between 3 and 8 pages, with figures and a few citations where relevant. Videos are encouraged as well, as a way to showcase simulation/hardware results and provide visual explanation.

Project Ideas

Here are a set ideas we believe might be worth investigating, but do not feel the need to grab one of these!

• Simulating 3D passive-dynamic walkers with curved feet. Drake has a novel contact model called "hydroelastic contact" which allows simulation for even non-convex collision geometries. It would be great to see how well this can capture those curved feet. Even if there is little or no control, the project could use trajectory optimization to find stable limit cycles.
• Balancing control / footstep placement / walking control for Boston Dynamics' Spot. Spot is now a commercial product and is appearing in research labs. It comes with an amazing API for robust walking / door-opening, etc. But it's hard to simulate because we don't have access to those controllers. I would love to offer up a rough approximation for people to use broadly in the research community.
• Trajectory optimization with Graphs of Convex Sets. This is a new planning paradigm and we're only beginning to explore the possible applications. Pick a system, try it on your problem, tell us what works, what doesn't (and maybe contribute new ideas!).
• Explore different control parameterizations for gradient-based policy search. How well do deep nets solve our favorite benchmarks vs e.g. polynomial feedback?
• System identification / model learning. Try comparing multibody methods vs linear or deep models for a task of your choice.
• Value iteration on systems with discontinuous value functions. How well can we make neural value iteration (example from the text) work? Are ReLU networks really a natural representation for this type of problem? Are there better representations? Or study some of the more classical (non-neural) algorithms like upwind discretization or variable-resolution value iteration.
• Racecar. A number of groups have been converging on a model car driving quickly around a small track as a benchmark problem for e.g. model-predictive control. This paper could be a starting point.
• Bicycle/Airplane/Quadrotor driving/flying through a random forest/building. Via online model-predictive control? Or could you find a simple feedback controller that does not depend on knowing/estimating the map?
• Simple legged-robot walking on rough terrain. The RABBIT model from Grizzle might be a nice place to start.
• System-level Synthesis. Implement SLS feedback control in place of LQR or time-varying LQR for one or more of the model systems.
• Can you find an optimization-based interpretation of a classical controller? E.g. Mark Spong's swing-up controllers? Marc Raibert's hopping control? For instance, the vertical dynamics of the hopper (from Assignment 4) are hybrid linear; is there a natural problem formulation for which mixed-integer MPC produces the same control inputs? If not, how are they different?
• Robust trajectory optimization. We've talked about trajectory optimization and robust control, mostly separately. There are a few approaches that attempt to incorporate robustness into the trajectory design (search google for e.g. Robust MPC). Try one of these algorithms on a model system.