CMSC 818B: Decision-Making for Robotics - Fall 2020

Class announcements, assignments, and due dates will be posted on ELMS. Use Piazza for any course-related discussion and for messaging the course staff. Piazza allows you to post a private message that only the instructor and the TA can see. Use that functionality instead of e-mail. If its a question that the rest of the class can benefit, then please make it public.


This course will focus on the principles of decision-making in robotics. Motion planning focuses on the problem of how to go from point A to point B. In this course, we will examine higher-level algorithms that make decisions as to what points A and B should be depending on the task at hand. In standard motion planning, the goal location is given as input. In constrast, in the decision-making problems we will consider in this course, the goal locations(s) will be the output. In fact, most scenarios we will study require making sequential decisions where the output is a series of goal location(s) (or actions or tasks and so on) and where decisions we make in the past may influence the options available at a future instance.

Most of the lectures will be based on research papers and discussion of open research problems. The students will also work on a research-oriented course project. The goal of the course is to understand, critique, and implement research papers, identify open problems, and make progress towards solving them. An ideal outcome is for students to write a research paper at the end of the course.

We will have a mix of traditional lectures conducted by the instructor synchronously over Zoom. The lectures will be recorded and posted on ELMS. However, a significant portion of the course will revolve around discussion of recent research papers (led by the students). Some of the discussion will happen asynchronously via Piazza and some live during lectures. We are also planning to have guest lectures from academia and industry and try some atypical lecture formats (stay tuned for more).

The lectures will be conducted on Zoom. The lectures will be recorded and posted on ELMS for viewing and downloading for other students in the class. If you use your audio or video during the recording, your consent is implied. If you are not willing to have your voice or face recorded, you may opt to use only the chat feature.
This is a qualifying course in artificial intelligence.


No required textbook. Course materials will be drawn from research papers and notes that will be posted online on ELMS.


Class schedule given here.

Topics that will be covered include (but are not limited to): Markov Decision Processes; Partially Observable Markov Decision Processes; Monte-Carlo Tree Search; Deep reinforcement learning; Proximal Policy Optimization; Graph Neural Networks; Multi-Robot Task Assignment and Matching; Vehicle Routing Problems; Information-Theoretic Path Planning (mutual information, entropy, submodularity); Active Perception.


This is still tentative and subject to change. Evaluation will be composed of the following:

Details of each are given below.


This will be in the style of a take-home exam. You will be given 1-2 days (subject to change) to solve the exam and upload your solutions. The exam will likely be in November.

Research Project

This is a research-oriented course. An ideal outcome of this course is a research paper that can be submitted to a top-tier robotics venue. During the course, each student is expected to identify a research question and make progress towards solving it. The course project will be the medium through which we will facilitate that. It is perfectly okay if you cannot solve the problem in its entirety during the class, however, you must demonstrate sufficient effort and progress.

The project can be done in groups of up to four students. Students are encouraged to work on a project that connects the material covered in this course with their existing research.

There will be several milestones during the course which will all count towards the points reserved for the project.


There will be two mini-projects. You are expected to work on these for roughly 4 weeks each. The mini-projects must be done in groups of 2 students.


The assignments will be a mix of hands-on implementation exercises as well theory questions. We will have 3-4 assignments and typically due within 10 days of posting.

Paper Presentations

Each student will lead the discussion on research papers during the semester. You may choose a paper from the list given here.

Class Participation

While the lectures will be conducted synchronously, the participation points are reserved only for the asynchronous component. These points will be awarded for being active in the class discussion forum (Piazza), scribing lecture notes, and peer review of project proposals and mini-projects.


This is a seminar-style course that will feature a discussion of research papers. An ability to read and critique research papers is required.

There are no explicit prerequisites. However, students should be very comfortable in at least two of the following areas: Probability and Bayesian Statistics; Algorithms; Machine Learning/AI; Optimization.

We will mostly use Python for implementation. Some level of comfort using Python would be good but is not a requirement. It is expected that you will be able to pick up the basics needed for the assignments.

Prior background in robotics is helpful, but not required.

Academic Integrity

The CS Department takes academic integrity seriously. Information on how the CS Department views and handle academic integrity matters can be found at Academic Integrity.

Note that academic dishonesty includes not only cheating, fabrication, and plagiarism, but also includes helping other students commit acts of academic dishonesty by allowing them to obtain copies of your work. In short, all submitted work must be your own. Cases of academic dishonesty will be pursued to the fullest extent possible as stipulated by the Office of Student Conduct.

It is very important for you to be aware of the consequences of cheating, fabrication, facilitation, and plagiarism. For more information on the Code of Academic Integrity or the Student Honor Council, please visit

On any graded exam or assignments, you are NOT allowed to collaborate or exchange code. We compare each student's code with every other student's code to check for similarities. Every semester, we catch an embarrassingly high number of students that engage in cheating and we have to take them to the Honor Council.

You may post your project code to private Github (or similar service) repos only. As a student, you can make a private repo for free. Just remember that your free premium subscription has an expiry date, and your code becomes public once it expires. The Honor Council can retroactively give an XF (even to students who have already graduated) if your code is then used by another student to cheat. So just be careful. Posting graded code meant for an assignment to a public repo will give you a free ticket to the Honor Council.

We encourage students to talk about course material and help each other out in group chats and discussion forums. However, this does NOT include graded assignments. Talking about release tests is NOT okay.

Excused Absences

Any student who needs to be excused for an absence from a single lecture, recitation, or lab due to a medically necessitated absence shall make a reasonable attempt to inform the instructor of his/her illness prior to the class. Upon returning to the class, present their instructor with a self-signed note attesting to the date of their illness. Each note must contain an acknowledgment by the student that the information provided is true and correct. Providing false information to University officials is prohibited under Part 9(i) of the Code of Student Conduct (V-1.00(B) University of Maryland Code of Student Conduct) and may result in disciplinary action.

Students are not obligated to provide documentation from the University Health Center, from their physician, or from another source to verify an absence or missed due date for an assignment. The student should send an e-mail or a Piazza note to the instructor, explaining the absence, the dates missed, and a statement that the information is accurate and that they recognize the expectations in the University of Maryland Code of Student Conduct (V-1.00[B]). Students may use this template to certify their absence. The take-home mid-semester exam is the only major graded event. Other assignments and projects have a longer lead-time for which extensions may be provided to make up for lost time. Instructors may consider re-weighting of course assessments, along with providing alternative assignments and make-up work, as ways to accommodate missed exam or other graded work to allow a student to meaningfully proceed through the course.

Services for Students with Disabilities

Any student who feels that they may need an accommodation because of a disability (learning disability, attention deficit disorder, psychological, physical, etc.), please make an appointment to see the instructor during office hours or contact via e-mail or private Piazza post.