ROBOTICS 101 Xian (Shawn) Li | Northeastern University

Robotics 101

A free place to start learning robotics. Try short interactive concepts, read practical handouts, and follow a roadmap from your first circuit to a robot that maps and navigates on its own.

Maintained by Prof. Xian (Shawn) Li, Department of Electrical and Computer Engineering, Northeastern University.

What every robot does

Almost every robot, from a vacuum to a Mars rover, runs the same loop many times per second. It senses, it thinks, and it acts. Each concept on this site is one piece of that loop.

Sense

Measure the world and the robot itself, with wheel encoders, IMUs, cameras, and LiDAR.

Think

Work out where the robot is and what to do next, even when every sensor is noisy.

Act

Turn a decision into motion, through wheels, motors, and joints.

Acting changes the world, so the robot senses again, and the loop repeats.

Concept explainers

Short interactive pages that show one robotics idea at a time. Move the sliders and watch what changes. 265 explainers in 19 sections.

No explainers match your search and level filter.

Math Foundations

18 explainers
Math Foundations · 01

Vectors, Dot and Cross Products

Drag two vectors and watch the projection, angle, and cross product update, then use them to tell if a target is ahead or to the left.

  • Level Beginner
Math Foundations · 02

Coordinate Frames and Rotations

Move and rotate a robot frame inside a world frame, and see how a point seen by the robot maps to world coordinates.

  • Level Beginner
Math Foundations · 03

3D Rotation Matrices

Rotate a 3D frame about x, y, and z and see that changing the order of the same rotations gives a different result.

  • Level Intermediate
Math Foundations · 04

Euler Angles and Gimbal Lock

Turn the yaw, pitch, and roll rings of a gimbal until two of them line up and one degree of freedom disappears.

  • Level Intermediate
Math Foundations · 05

Axis-Angle and Rodrigues' Formula

Pick a rotation axis and an angle and watch a vector sweep around the axis, then see the rotation matrix from Rodrigues' formula.

  • Level Intermediate
Math Foundations · 06

Quaternions and Interpolation (Slerp)

Blend between two orientations with slerp and with naive Euler-angle blending, and compare the paths the nose takes.

  • Level Intermediate
Math Foundations · 07

Homogeneous Transforms and Transform Chains

Chain world, base, arm, and camera frames and follow a point seen by the camera through every transform to the world.

  • Level Intermediate
Math Foundations · 08

Rigid Body Motion: Twists and Screws

Pick any end pose for a rigid body and see that the motion is a single rotation about one fixed point, the planar version of a screw axis.

  • Level Advanced
Math Foundations · 09

Lie Groups SO(3) and SE(3) for Robotics

Drive a robot with heading noise and see why its uncertainty is a banana, not an ellipse, and how the Lie group exponential map captures it.

  • Level Advanced
Math Foundations · 10

Linear Least Squares and the Pseudoinverse

Drag noisy points, fit a line or curve, and see the squared residuals the fit makes as small as possible.

  • Level Intermediate
Math Foundations · 11

Nonlinear Least Squares: Gauss-Newton and Levenberg-Marquardt

Fit a curve with Gauss-Newton and Levenberg-Marquardt from the same start, and watch where Gauss-Newton overshoots and LM stays safe.

  • Level Advanced
Math Foundations · 12

Gradient Descent and Optimization Landscapes

Drop a ball on a cost surface, change the step size until it diverges, and see how momentum helps in a narrow valley.

  • Level Intermediate
Math Foundations · 13

Gaussians and Covariance Ellipses

Stretch and rotate a 2D Gaussian and see the 1, 2, and 3 sigma ellipses and how many samples fall inside each.

  • Level Beginner
Math Foundations · 14

Bayes' Rule and Belief Updates

A robot checks whether a door is open with a noisy sensor. See how each reading updates its belief, drawn as areas you can see.

  • Level Beginner
Math Foundations · 15

Sampling and Monte Carlo

Estimate the free area of a cluttered room by throwing random points, and watch the error shrink as more points land.

  • Level Beginner
Math Foundations · 16

Numerical Integration: Euler versus RK4

Simulate the same pendulum with three integrators and see Euler gain energy while RK4 stays accurate.

  • Level Intermediate
Math Foundations · 17

Sampling, Aliasing, and the Nyquist Rate

Film a spinning wheel at different frame rates and see it appear to slow down, stop, or turn backward.

  • Level Beginner
Math Foundations · 18

Frequency and the Fourier Transform

Build a sensor signal from sine waves and noise, see its spectrum, and remove the noise with a low-pass cutoff.

  • Level Intermediate

Sensors

19 explainers
Sensors · 01

Wheel Encoders and Quantization

Spin a wheel past encoder ticks, read the A and B signals, and see how resolution and sample time limit speed estimates.

  • Level Beginner
Sensors · 02

Accelerometers and Gyroscopes

Tilt and spin a virtual IMU and see what its accelerometer and gyroscope report, including gravity and noise.

  • Level Beginner
Sensors · 03

IMU Bias, Noise, and Drift

Integrate a gyroscope and an accelerometer with bias and noise, and watch heading error grow linearly and position error grow even faster.

  • Level Intermediate
Sensors · 04

Complementary Filter for Tilt

Blend a noisy accelerometer and a drifting gyroscope with one slider, and see the fused tilt beat both.

  • Level Beginner
Sensors · 05

Magnetometers and Calibration

Spin a robot in a circle near iron parts and fix its distorted compass readings with hard-iron and soft-iron calibration.

  • Level Intermediate
Sensors · 06

GPS and GNSS

Move satellites across the sky and see how their geometry changes position error, then turn on RTK corrections.

  • Level Intermediate
Sensors · 07

Ultrasonic and Sonar Range Sensors

Sweep an ultrasonic sensor around a room and see its wide beam, missed echoes from slanted walls, and false readings from multipath.

  • Level Beginner
Sensors · 08

LiDAR and Time of Flight

Sweep a 2D LiDAR around a room and see how angular resolution, range, glass, and dark surfaces shape the scan.

  • Level Beginner
Sensors · 09

3D LiDAR and Point Clouds

Spin a multi-beam LiDAR above a street scene and see its rings on the ground, and how few points land on a person far away.

  • Level Intermediate
Sensors · 10

The Pinhole Camera Model

Move a 3D object in front of a camera and see where it lands in the image as the focal length and distance change.

  • Level Beginner
Sensors · 11

Lens Distortion and Camera Calibration

Bend a checkerboard grid with radial and tangential lens distortion, then recover the distortion from straight lines alone.

  • Level Intermediate
Sensors · 12

Stereo Vision and Disparity

Move an object in depth in front of two cameras and see its disparity shrink, and the depth error grow, with distance.

  • Level Intermediate
Sensors · 13

Depth Cameras: Structured Light and Time of Flight

Compare how a structured light camera and a time of flight camera see the same scene, and where each one fails: sunlight, glass, dark surfaces, and corners.

  • Level Intermediate
Sensors · 14

Radar

See how an FMCW radar turns a frequency chirp into range and speed, and how bandwidth decides whether two close objects can be told apart.

  • Level Intermediate
Sensors · 15

Event Cameras

Watch a fast spinning bar through a normal camera and an event camera, and see blur in one and a clean stream of brightness changes in the other.

  • Level Advanced
Sensors · 16

Force, Torque, and Tactile Sensors

Press on a tactile skin to see contact location and force maps, and push on a tool tip to see the six numbers a wrist force-torque sensor reports.

  • Level Intermediate
Sensors · 17

Encoders and Joint Sensing on Arms

Compare an absolute and an incremental encoder on a robot joint, then switch the power off while the arm sags and see which one still knows the angle.

  • Level Beginner
Sensors · 18

Time Sync, Latency, and Sensor Timestamps

Fuse a camera and a LiDAR on a turning robot and see how a camera delay that nobody accounts for makes the two disagree.

  • Level Intermediate
Sensors · 19

Extrinsic Calibration: Camera to LiDAR and Hand to Eye

Find the pose of a camera relative to a LiDAR by lining up a target both sensors can see, first by hand and then by solving for it.

  • Level Advanced

Actuators and Mechanisms

10 explainers
Actuators and Mechanisms · 01

DC Motor Model and Torque-Speed Curve

Load a DC motor and move along its torque-speed, current, power, and efficiency curves.

  • Level Beginner
Actuators and Mechanisms · 02

Gears, Gear Ratios, and Backlash

Change the gear ratio to trade speed for torque, then add backlash and watch the output lag when the input reverses.

  • Level Beginner
Actuators and Mechanisms · 03

Servo Motors

Command a hobby servo with a pulse width and see its internal position loop, its speed limit, and what happens when the load is too heavy.

  • Level Beginner
Actuators and Mechanisms · 04

Stepper Motors and Microstepping

Step a stepper motor in full steps and microsteps, and see it miss steps when the speed or load is too high.

  • Level Beginner
Actuators and Mechanisms · 05

Brushless Motors and Field-Oriented Control

Rotate the stator field of a brushless motor with six-step commutation and with field-oriented control, and compare the torque ripple.

  • Level Advanced
Actuators and Mechanisms · 06

Series Elastic and Quasi-Direct-Drive Actuators

Drive a robot link into a wall with a stiff gearbox, a spring in series, and a low-ratio motor, and compare the impact force.

  • Level Advanced
Actuators and Mechanisms · 07

Linkages and Four-Bar Mechanisms

Change the four link lengths of a four-bar linkage and see whether it is a crank-rocker, a double crank, or a double rocker, with the curve its coupler traces.

  • Level Beginner
Actuators and Mechanisms · 08

Parallel Mechanisms: Delta Robots

Move the platform of a delta robot and watch its three arms solve their angles, inside the workspace the arms can reach.

  • Level Intermediate
Actuators and Mechanisms · 09

Hydraulic and Pneumatic Actuation

Lift a load to a target height with an electric screw, a hydraulic cylinder, and a pneumatic cylinder, and compare force, stiffness, and overshoot.

  • Level Intermediate
Actuators and Mechanisms · 10

Soft Actuators

Inflate the air chambers of a soft finger and watch it curl, then let it wrap around objects of different sizes.

  • Level Intermediate

Manipulator Kinematics

9 explainers
Manipulator Kinematics · 01

Forward and Inverse Kinematics of a Two-Link Arm

Turn the joints of a planar two-link arm to see forward kinematics, or click a target to have the arm solve inverse kinematics and reach it.

  • Level Beginner to intermediate
Manipulator Kinematics · 02

Denavit-Hartenberg Parameters

Build a three-joint arm one Denavit-Hartenberg row at a time and see where each row places the next frame.

  • Level Intermediate
Manipulator Kinematics · 03

Product of Exponentials

Build forward kinematics from each joint's screw axis and the home pose, and check that it gives the same answer as the DH table.

  • Level Advanced
Manipulator Kinematics · 04

Workspace and Reachability

Sweep the joint limits of a three-link arm and see its reachable workspace, and the smaller dexterous workspace where the tool can point any direction.

  • Level Beginner
Manipulator Kinematics · 05

The Jacobian and Velocity Kinematics

Spin each joint of a two-link arm and see the hand velocity it creates, and how the Jacobian adds them up.

  • Level Intermediate
Manipulator Kinematics · 06

Singularities and Manipulability Ellipsoids

Drive a two-link arm toward fully straight and fully folded, and watch its velocity ellipse flatten as it loses the ability to move one way.

  • Level Intermediate
Manipulator Kinematics · 07

Numerical Inverse Kinematics

Click a target and watch pseudoinverse and damped least squares inverse kinematics step a three-link arm toward it, or struggle near singularities.

  • Level Intermediate
Manipulator Kinematics · 08

Redundancy and Null-Space Motion

Hold the hand of a three-link arm still and move its elbow anyway, using the motions that leave the hand unchanged.

  • Level Advanced
Manipulator Kinematics · 09

Leg Kinematics

Place the foot of a two-joint robot leg and see both knee solutions, then watch the leg follow a walking step.

  • Level Intermediate

Dynamics

11 explainers
Dynamics · 01

Newton's Laws for a Mobile Robot

Push a robot with its motors up a slope against friction, and see how mass, force, and incline set its acceleration and top speed.

  • Level Beginner
Dynamics · 02

Friction Models

Pull a block through a spring at steady speed and compare Coulomb, viscous, and stiction friction, including the jerky stick-slip that stiction causes.

  • Level Beginner
Dynamics · 03

The Pendulum and Energy

Swing a pendulum and watch potential and kinetic energy trade back and forth, with and without damping, along with its phase portrait.

  • Level Beginner
Dynamics · 04

The Cart-Pole and Inverted Pendulum

Try to balance a pole on a cart by pushing it left and right, then let an LQR controller do it and poke the pole to test it.

  • Level Intermediate
Dynamics · 05

Arm Dynamics: Mass Matrix, Coriolis, Gravity

Move a two-link arm along a swinging path at different speeds and see which part of the joint torque dominates: inertia, Coriolis and centrifugal, or gravity.

  • Level Advanced
Dynamics · 06

Contact, Collisions, and Restitution

Drop two balls with the same bounciness, one with an instant bounce rule and one with a spring-and-damper contact, and see bounce heights and sinking into the floor.

  • Level Intermediate
Dynamics · 07

Physics Simulation and Time Steps

Rest a box on stiff ground in a simulator and raise the time step until it starts to jitter, bounce, and finally explode.

  • Level Intermediate
Dynamics · 08

Wheel Slip and Tire Forces

Accelerate a wheel on asphalt, wet road, and ice, see the slip curve that sets how much grip it has, and turn on traction control.

  • Level Intermediate
Dynamics · 09

Center of Mass and the Support Polygon

Reach out with a mobile manipulator's arm, add a payload, park it on a slope, and see when its center of mass leaves its wheelbase and it tips over.

  • Level Beginner
Dynamics · 10

Quadrotor Dynamics and Motor Mixing

Command thrust, roll, pitch, and yaw and see how the mixer turns them into four propeller speeds, and how the drone tilts and turns.

  • Level Intermediate
Dynamics · 11

Underwater and Marine Vehicle Dynamics

Trim an underwater robot's buoyancy, drive it into a current, and see how drag, added mass, and currents make underwater control hard.

  • Level Advanced

Control

24 explainers
Control · 01

Open-Loop versus Closed-Loop Control

Send two robots to the same target, one following a fixed plan and one using feedback, then add a disturbance and a wheel error.

  • Level Beginner
Control · 02

Bang-Bang Control

Drive a robot to a target using only full forward and full reverse, and see the chattering that hysteresis can reduce.

  • Level Beginner
Control · 03

PID Control

Tune the P, I, and D gains and watch a robot drive to a target. See overshoot, oscillation, and steady-state error appear and disappear.

  • Level Beginner to intermediate
Control · 04

Step Response: Rise Time, Overshoot, Settling

Give a system a sudden new target and read rise time, overshoot, and settling time off its response as its damping and speed change.

  • Level Beginner
Control · 05

PID Tuning and Integral Windup

Tune a PID controller with the Ziegler-Nichols method, then hit the motor limit and see integral windup cause a huge overshoot, and how anti-windup fixes it.

  • Level Intermediate
Control · 06

Cascade Control

Nest a velocity loop inside a position loop on a motor, and see why the inner loop must be several times faster than the outer one.

  • Level Intermediate
Control · 07

Feedforward plus Feedback

Make a robot arm joint follow a swinging path with feedback alone, then add gravity and inertia feedforward and watch the tracking error drop.

  • Level Intermediate
Control · 08

Transfer Functions, Poles, and Zeros

Drag poles on the complex plane and see the step response change.

  • Level Intermediate
Control · 09

Frequency Response and Bode Plots

Shake a system at different frequencies and see gain and phase.

  • Level Intermediate
Control · 10

Stability and Root Locus

Raise a gain and watch the poles move toward instability.

  • Level Advanced
Control · 11

State-Space Models

Simulate a system as states, inputs, and outputs in matrix form.

  • Level Intermediate
Control · 12

Controllability and Observability

Remove an actuator or a sensor and see which states you lose.

  • Level Advanced
Control · 13

Pole Placement

Place closed-loop poles and see the response you designed.

  • Level Advanced
Control · 14

Linear Quadratic Regulator (LQR)

Trade state error against effort and balance the cart-pole.

  • Level Advanced
Control · 15

Lyapunov Stability

See an energy-like function decrease along every trajectory.

  • Level Advanced
Control · 16

Feedback Linearization and Computed Torque

Cancel an arm's nonlinear dynamics and track a trajectory.

  • Level Advanced
Control · 17

Model Predictive Control (MPC)

Watch a controller plan ahead over a horizon and replan every step.

  • Level Advanced
Control · 18

Impedance and Admittance Control

Make an arm feel like a soft or stiff spring when you push it.

  • Level Advanced
Control · 19

Force Control and Hybrid Position/Force Control

Wipe a surface while keeping a set contact force.

  • Level Advanced
Control · 20

Adaptive and Robust Control

Change the payload mid-run and see an adaptive controller adjust.

  • Level Advanced
Control · 21

Discrete-Time Control and Loop Rate

Lower the control rate and see delay cause oscillation.

  • Level Intermediate
Control · 22

Pure Pursuit Path Tracking

Change the look-ahead distance and see corner cutting or wobble.

  • Level Beginner
Control · 23

Stanley Controller

Track a path with heading and cross-track error, compared with pure pursuit.

  • Level Intermediate
Control · 24

Quadrotor Control: Attitude and Position Cascade

Fly a drone to a waypoint through nested attitude and position loops.

  • Level Intermediate

State Estimation

14 explainers
State Estimation · 01

Moving Average and Low-Pass Filters

Smooth a noisy signal and see the lag each filter adds.

  • Level Beginner
State Estimation · 02

Bayes Filters and Grid Localization

Localize a robot in a 1D hallway with a histogram filter.

  • Level Beginner
State Estimation · 03

The Kalman Filter in 1D

A robot moves along a line and gets noisy position readings. Watch the Kalman filter blend its motion model with each reading to track the robot.

  • Level Intermediate
State Estimation · 04

The Kalman Filter in 2D

Track a moving robot and see its covariance ellipse grow and shrink.

  • Level Intermediate
State Estimation · 05

The Extended Kalman Filter (EKF)

Linearize a turning robot's motion and see where linearization fails.

  • Level Advanced
State Estimation · 06

The Unscented Kalman Filter (UKF)

Push sigma points through a nonlinear function and compare with the EKF.

  • Level Advanced
State Estimation · 07

Particle Filter Localization

Hundreds of guesses about where a robot might be, each scored against range readings to landmarks. Watch the cloud collapse onto the true pose.

  • Level Intermediate
State Estimation · 08

Probabilistic Motion and Measurement Models

Sample odometry noise and see beam and likelihood-field sensor models.

  • Level Intermediate
State Estimation · 09

Sensor Fusion: Odometry, IMU, and GPS

Turn sensors on and off and see the fused estimate degrade.

  • Level Intermediate
State Estimation · 10

Outlier Rejection and Gating

Inject bad readings and reject them with a Mahalanobis gate.

  • Level Intermediate
State Estimation · 11

Data Association

Match landmark observations to the map and see what a wrong match does.

  • Level Advanced
State Estimation · 12

RANSAC

Fit a line through data full of outliers by random sampling and voting.

  • Level Intermediate
State Estimation · 13

Factor Graphs and Smoothing

Build a graph of poses and measurements and optimize it all at once.

  • Level Advanced
State Estimation · 14

Visual-Inertial Odometry

Combine camera features and IMU data and see scale and drift.

  • Level Advanced

Mapping and SLAM

16 explainers
Mapping and SLAM · 01

Occupancy Grid Mapping

Drive a LiDAR robot and watch log-odds cells fill in.

  • Level Intermediate
Mapping and SLAM · 02

Scan Matching with ICP

Align two LiDAR scans step by step and see bad initial guesses fail.

  • Level Intermediate
Mapping and SLAM · 03

NDT Scan Matching

Align scans against a grid of Gaussians instead of points.

  • Level Advanced
Mapping and SLAM · 04

EKF SLAM with Landmarks

See robot and landmark uncertainty shrink together when landmarks are re-seen.

  • Level Advanced
Mapping and SLAM · 05

FastSLAM

Give each particle its own small map and compare with EKF SLAM.

  • Level Advanced
Mapping and SLAM · 06

Graph SLAM and Loop Closure

Drive a loop, close it, and watch the whole map snap into place.

  • Level Advanced
Mapping and SLAM · 07

Place Recognition

Match the current view to past views with a bag of visual words.

  • Level Advanced
Mapping and SLAM · 08

Visual Odometry

Track features across frames and recover the camera's motion.

  • Level Advanced
Mapping and SLAM · 09

Bundle Adjustment

Start with rough camera poses and 3D point guesses, then refine all of them together to shrink the reprojection error, one iteration at a time.

  • Level Advanced
Mapping and SLAM · 10

Structure from Motion

Rebuild the layout of a scene from bearings seen by a moving camera, step by step, and see why the overall scale cannot be known from images alone.

  • Level Advanced
Mapping and SLAM · 11

Visual SLAM

Follow a camera robot around a loop as it tracks features, adds keyframes and map points, drifts, and then snaps its map into place when it recognizes where it started.

  • Level Advanced
Mapping and SLAM · 12

LiDAR SLAM

Split each LiDAR scan into sharp edge points and flat plane points by curvature, the way LOAM does, and build a map from those features.

  • Level Advanced
Mapping and SLAM · 13

Map Representations

Show the same room as a point cloud, a voxel grid, a quadtree, a line mesh, and a signed distance field, and compare how much memory each needs.

  • Level Intermediate
Mapping and SLAM · 14

Neural Maps: NeRF and 3D Gaussian Splatting

Render a scene two ways, by marching rays through a density field as NeRF does and by blending sorted Gaussian blobs as 3D Gaussian splatting does, and compare them to the true view.

  • Level Advanced
Mapping and SLAM · 15

Semantic Maps and 3D Scene Graphs

Label a floor plan with rooms and objects, build the scene graph that links them, and answer questions like where is the mug or which room has the sofa.

  • Level Advanced
Mapping and SLAM · 16

Multi-Robot Mapping

Let two robots map different parts of a building in their own frames, then merge their maps once they meet and work out the transform between them.

  • Level Advanced

Perception and Computer Vision

20 explainers
Perception and Computer Vision · 01

Images as Arrays and Color Spaces

Hover over the pixels of a robot camera image to read their numbers, then compare RGB and HSV for picking out a colored object as the light changes.

  • Level Beginner
Perception and Computer Vision · 02

Convolution, Filtering, and Edges

Slide a small kernel across an image to blur it, sharpen it, or find its edges, and watch the window move one pixel at a time.

  • Level Beginner
Perception and Computer Vision · 03

Thresholding and Color Segmentation

Tune hue, saturation, and value thresholds to pick out a red ball, then dim the light or cast a shadow and compare with a plain RGB threshold.

  • Level Beginner
Perception and Computer Vision · 04

Feature Detection and Matching

Find corners in two views of the same scene and match them by their surrounding patch, and see how a ratio test removes the wrong matches.

  • Level Intermediate
Perception and Computer Vision · 05

Homography and Image Mosaicking

Drag four corners to warp one image onto another with a homography, the same tool used to stitch photos into a panorama or map a floor from above.

  • Level Intermediate
Perception and Computer Vision · 06

Epipolar Geometry

Click a point in one camera image and see the line in the other image where its match must lie, the epipolar line, from the fundamental matrix.

  • Level Advanced
Perception and Computer Vision · 07

Optical Flow

Move a camera forward, sideways, or turn it, and see the pattern of motion it creates across the image.

  • Level Intermediate
Perception and Computer Vision · 08

Fiducial Markers and PnP Pose

Move and tilt an AprilTag-style marker in front of a camera and watch the camera recover its full 6D pose from the four corners.

  • Level Intermediate
Perception and Computer Vision · 09

Point Cloud Basics: Downsampling and Normals

Thin a dense point cloud with a voxel grid filter and estimate a surface normal at each point from its neighbors.

  • Level Intermediate
Perception and Computer Vision · 10

Plane and Object Segmentation in Point Clouds

Remove the table plane from a depth scan with RANSAC, then group the remaining points into separate objects by clustering.

  • Level Intermediate
Perception and Computer Vision · 11

Neural Networks 101

Train a tiny neural network to separate two classes of points and watch its decision boundary bend into shape as it learns.

  • Level Beginner
Perception and Computer Vision · 12

Convolutional Neural Networks

Draw a shape and pass it through a small convolutional network to see what each filter responds to, layer by layer.

  • Level Intermediate
Perception and Computer Vision · 13

Attention and Transformers

Pick a word and watch attention weights link it to the image patches that match it, then change the temperature to sharpen or blur the focus.

  • Level Advanced
Perception and Computer Vision · 14

Object Detection: Boxes, IoU, and NMS

Score candidate boxes from a detector, set the confidence threshold, and remove duplicate boxes with non-maximum suppression using IoU.

  • Level Intermediate
Perception and Computer Vision · 15

Semantic and Instance Segmentation

Label every pixel of a cluttered table by class, then by individual object, and see how blurry mask edges lower the overlap score.

  • Level Intermediate
Perception and Computer Vision · 16

Object Tracking

Track objects across frames by predicting where each one will move and matching new detections to the predictions, and count identity swaps when they cross.

  • Level Intermediate
Perception and Computer Vision · 17

6D Object Pose Estimation

Fit a known object model to a cluttered scan to recover its position and orientation, and see where the grasp attached to the model lands.

  • Level Advanced
Perception and Computer Vision · 18

Monocular Depth Estimation

Compare a depth map guessed from a single image with stereo depth, and fix the single-image depth's unknown scale and shift from a few known points.

  • Level Intermediate
Perception and Computer Vision · 19

Human Pose and Hand Tracking

Track the keypoints of a moving person, then add occlusion and fast motion and see where tracking breaks, and how smoothing helps or lags.

  • Level Intermediate
Perception and Computer Vision · 20

Vision Foundation Models and Open-Vocabulary Perception

Find objects by typing a word, and see how shared image and text features, like those from CLIP, score every object against the query.

  • Level Advanced

Motion Planning

23 explainers
Motion Planning · 01

Configuration Space

Move a two-link arm among obstacles and see the same scene in its configuration space, where the whole arm becomes a single point.

  • Level Intermediate
Motion Planning · 02

A* Path Planning

Draw walls on a grid and watch A*, Dijkstra, and greedy search find a path. See how a good heuristic cuts down the search.

  • Level Beginner to intermediate
Motion Planning · 03

Replanning with D* Lite

Send a robot across a map with hidden walls, let it discover them as it drives, and watch its path repair itself by updating only the part of the search that changed.

  • Level Intermediate
Motion Planning · 04

Visibility Graphs and Cell Decomposition

Build a roadmap from obstacle corners that can see each other, or by splitting free space into cells, and compare the paths each one finds.

  • Level Intermediate
Motion Planning · 05

Potential Fields

Let a robot slide downhill on a field that pulls it toward the goal and pushes it away from obstacles, and find where it gets stuck in a local minimum.

  • Level Beginner
Motion Planning · 06

Probabilistic Roadmaps (PRM)

Scatter random samples through free space, connect each to its nearest neighbors, and search the resulting roadmap for a path, including through a narrow passage.

  • Level Intermediate
Motion Planning · 07

RRT and RRT*

Grow a random tree from the start until it reaches the goal, then switch to RRT* and watch rewiring straighten the path as more samples arrive.

  • Level Intermediate
Motion Planning · 08

Collision Checking

Drag and rotate a robot among obstacles and see how cheap bounding shapes rule out most collisions before any exact test is needed.

  • Level Intermediate
Motion Planning · 09

Path Smoothing and Shortcutting

Take a jagged path from a grid planner, cut its corners with random shortcuts, then round it off into a smooth curve a robot can drive.

  • Level Beginner
Motion Planning · 10

Trajectory Generation: Polynomials and Splines

Fit a trajectory through draggable waypoints with straight segments, cubic splines, and quintic minimum-jerk segments, and compare their speed, acceleration, and jerk.

  • Level Intermediate
Motion Planning · 11

Velocity Profiles and Time Scaling

Turn a move of a set distance into a trajectory with a trapezoidal or an S-curve velocity profile, within limits on speed, acceleration, and jerk.

  • Level Intermediate
Motion Planning · 12

Dubins and Reeds-Shepp Paths

Find the shortest path between two poses for a car that can only turn with a limited radius, first forward only, then allowed to reverse.

  • Level Intermediate
Motion Planning · 13

Hybrid A* for Car-Like Robots

Plan a parking maneuver for a car with a steering limit using Hybrid A*, which searches over position and heading with real driving arcs.

  • Level Advanced
Motion Planning · 14

Kinodynamic Planning

Drive a fast robot along a path with a sharp corner, see why a speed limit and an acceleration limit make the geometric path impossible, and plan a feasible speed profile.

  • Level Advanced
Motion Planning · 15

Trajectory Optimization

Start with a straight path through obstacles and bend it away by minimizing a cost that rewards smoothness and penalizes getting close to obstacles, as in CHOMP and TrajOpt.

  • Level Advanced
Motion Planning · 16

Costmaps and Inflation

Inflate the obstacles on a map by the robot's radius plus a soft cost that fades with distance, and see how the planned path moves away from walls.

  • Level Beginner
Motion Planning · 17

Local Planning: The Dynamic Window Approach

Watch a robot choose its next speed and turn rate by simulating many short possible motions, scoring them, and picking the best, while dodging moving obstacles.

  • Level Intermediate
Motion Planning · 18

The Navigation Stack: Global and Local Planners

Watch a global planner draw a route across a map while a local planner follows it and dodges a person who steps into the way, and see the global plan update when the way is blocked.

  • Level Intermediate
Motion Planning · 19

Frontier-Based Exploration

Send a robot into an unknown building and watch it map as it goes, always heading for the nearest frontier between known free space and unknown space.

  • Level Intermediate
Motion Planning · 20

Coverage Path Planning

Plan a lawnmower path that sweeps every part of an area around an obstacle, and compare it with random bouncing for the same time.

  • Level Beginner
Motion Planning · 21

Multi-Robot Collision Avoidance

Send robots across a circle to swap places, first ignoring each other, then avoiding each other with velocity obstacles and the reciprocal version that shares the work.

  • Level Advanced
Motion Planning · 22

Social Navigation

Send a robot across a room full of walking people and compare the shortest route with one that respects personal space.

  • Level Intermediate
Motion Planning · 23

Planning Under Uncertainty: MDPs and POMDPs

Plan in a gridworld where moves can slip and see value iteration find a policy that avoids risky edges, then decide whether a robot unsure of its location should look before it acts.

  • Level Advanced

Manipulation

11 explainers
Manipulation · 01

Grippers and Grasp Types

Match parallel-jaw, suction, three-finger, and soft grippers against a box, a ball, a plate, a bag of chips, and a mug, and see why each one works or fails.

  • Level Beginner
Manipulation · 02

Friction Cones and Force Closure

Place two or three fingers on an object, set the friction, and test whether the grasp can resist a push from any direction, called force closure.

  • Level Intermediate
Manipulation · 03

Grasp Detection

Score candidate parallel-jaw grasps on a depth image of a cluttered bin and keep the best ones, the way learned grasp detectors do.

  • Level Intermediate
Manipulation · 04

The Pick and Place Pipeline

Run a full pick and place, from seeing the object to letting it go, one stage at a time, and see what happens when the object moves partway through.

  • Level Beginner
Manipulation · 05

Visual Servoing

Steer a camera on a robot arm with what it sees, using image-based control that moves features straight across the image, and compare with pose-based control.

  • Level Advanced
Manipulation · 06

Contact-Rich Assembly: Peg in Hole

Push a peg into a tight hole that is slightly off from where the robot thinks it is, with stiff position control and with compliance, and compare the forces.

  • Level Advanced
Manipulation · 07

Nonprehensile Manipulation: Pushing

Push a box toward a goal with a single point contact and see how where you push decides whether it slides straight or spins away.

  • Level Intermediate
Manipulation · 08

Dexterous In-Hand Manipulation

Rotate an object inside a three-finger hand by finger gaiting: lift one finger at a time, move it, and put it back down while the other two keep the grip.

  • Level Advanced
Manipulation · 09

Deformable Objects: Cloth and Cables

Drag the end of a simulated cable around pegs and see why a floppy object, with a shape that changes everywhere at once, is so much harder to manipulate than a rigid one.

  • Level Advanced
Manipulation · 10

Bimanual Manipulation

Coordinate two arms to carry a rigid bar together, keeping both hands locked to it, then pass an object from one hand to the other.

  • Level Advanced
Manipulation · 11

Mobile Manipulation

Reach a target with an arm mounted on a moving base, and set how much the robot prefers to move its wheels versus its arm.

  • Level Intermediate

Legged Robots and Other Platforms

18 explainers
Legged Robots and Other Platforms · 01

Static and Dynamic Stability

Walk a four-legged robot with a slow crawl that always keeps three feet down, then switch to a trot that is always slightly falling and must keep moving to stay up.

  • Level Beginner
Legged Robots and Other Platforms · 02

Zero Moment Point (ZMP)

Move and accelerate a walking robot's upper body and watch its zero moment point slide along the foot, and see the robot tip when the ZMP reaches the edge.

  • Level Intermediate
Legged Robots and Other Platforms · 03

The Linear Inverted Pendulum Model for Walking

Push a walking robot modeled as an inverted pendulum and recover by stepping to its capture point, or watch it fall if it does not step.

  • Level Advanced
Legged Robots and Other Platforms · 04

Gaits and Central Pattern Generators

Switch a four-legged robot between walk, trot, pace, bound, and gallop by changing the timing of its legs, and see coupled oscillators settle into each pattern.

  • Level Intermediate
Legged Robots and Other Platforms · 05

Hopping and the Raibert Controller

Balance and steer a one-legged hopping robot by choosing where to put its foot down on each hop, the way Marc Raibert's hoppers did in the 1980s.

  • Level Advanced
Legged Robots and Other Platforms · 06

Whole-Body Control with Task Priorities

Give a three-link robot three tasks at once, reach a target, keep the elbow high, and hold a comfortable pose, and see which one gives way when they conflict.

  • Level Advanced
Legged Robots and Other Platforms · 07

Learned Locomotion with Reinforcement Learning

Train a small balancing policy by trial and error in simulation, with and without randomized physics, and test both on a

  • Level Advanced
Legged Robots and Other Platforms · 08

Motion Retargeting from Humans to Humanoids

Map a recorded human arm motion onto a robot arm with different link lengths and joint limits, by copying joint angles or by matching hand positions.

  • Level Advanced
Legged Robots and Other Platforms · 09

Humanoid Loco-Manipulation

Make a humanoid walk while carrying a box held out in front, and see how it must lean back and shorten its steps as the box gets heavier.

  • Level Advanced
Legged Robots and Other Platforms · 10

Autonomous Driving Pipeline

Follow one moment of a self-driving car's software from sensor detections through predictions of other cars, to a chosen trajectory, to the steering command.

  • Level Intermediate
Legged Robots and Other Platforms · 11

Fixed-Wing Flight Basics

Fly a small fixed-wing drone with throttle and pitch, find the angle where its wing stalls, and bank it to turn.

  • Level Intermediate
Legged Robots and Other Platforms · 12

Underwater Robots and Acoustic Navigation

Navigate an underwater robot without GPS, first by dead reckoning with a Doppler velocity log, then corrected by ranges to acoustic beacons.

  • Level Advanced
Legged Robots and Other Platforms · 13

Swarm Robotics and Consensus

Let many simple robots each copy the heading of their neighbors and watch the whole swarm come to agree, or break into groups when their sensing range is short.

  • Level Intermediate
Legged Robots and Other Platforms · 14

Exoskeletons and Wearable Robots

Assist a walking leg with an exoskeleton motor, and see that the help only helps when its timing matches the gait cycle.

  • Level Intermediate
Legged Robots and Other Platforms · 15

Prosthetics and Myoelectric Control

Drive a prosthetic hand from a muscle signal: rectify it, smooth it, and threshold it, then see the trade-off between noise and delay.

  • Level Intermediate
Legged Robots and Other Platforms · 16

Surgical and Medical Robots

Control a surgical instrument through a fixed incision point, scale the surgeon's hand motion down, and filter out tremor.

  • Level Advanced
Legged Robots and Other Platforms · 17

Continuum and Snake Robots

Bend a two-section tendon-driven continuum robot into curved shapes, then switch to a snake robot that slithers forward with a traveling wave.

  • Level Advanced
Legged Robots and Other Platforms · 18

Field Robots: Agriculture, Mining, and Space

Drive a robot remotely across rough terrain with a communication delay, from a farm next door to Mars, and see why far-away robots need more autonomy.

  • Level Intermediate

Robot Learning

27 explainers
Robot Learning · 01

Supervised Learning, Overfitting, and Validation

Fit curves of growing complexity to noisy sensor data and watch the training error keep falling while the validation error turns back up.

  • Level Beginner
Robot Learning · 02

Imitation Learning: Behavior Cloning

Copy expert demonstrations of a reaching task with supervised learning, then roll out the learned policy and watch small errors carry it off the demonstrated path.

  • Level Intermediate
Robot Learning · 03

Distribution Shift and DAgger

Fix behavior cloning's compounding errors by running the learned policy, asking the expert what it should have done in the states it actually reached, and retraining.

  • Level Intermediate
Robot Learning · 04

Teleoperation and Robot Data Collection

Record the same demonstration with a leader-follower arm, a VR controller, and hand tracking, and compare how smooth, how accurate, and how fast each interface is.

  • Level Beginner
Robot Learning · 05

Action Chunking (ACT)

Compare a policy that predicts one action at a time with one that predicts a short chunk of future actions, with and without blending overlapping chunks, and see the jitter disappear.

  • Level Advanced
Robot Learning · 06

Diffusion Policy

Generate a robot's path by starting from random noise and removing the noise step by step, and see why this handles two equally good ways of doing a task better than averaging.

  • Level Advanced
Robot Learning · 07

Flow Matching Policies

Generate robot paths by following a learned velocity field from noise straight to data, and compare how many steps it needs against diffusion.

  • Level Advanced
Robot Learning · 08

Reinforcement Learning: MDPs, Rewards, and Returns

Score a robot's route along a cliff in a gridworld, step by step, and see how the discount and the chance of slipping change which route earns the most.

  • Level Beginner
Robot Learning · 09

Q-Learning

Watch a Q-table fill in as an agent explores a cliff gridworld by trial and error, and see the greedy policy appear from the learned values.

  • Level Intermediate
Robot Learning · 10

Exploration versus Exploitation

Play a five-armed bandit where each arm pays off with a hidden probability, and compare greedy, epsilon-greedy, and optimistic UCB strategies by their regret.

  • Level Beginner
Robot Learning · 11

Policy Gradients and PPO

Train a simple stochastic policy by nudging it toward actions that earned more than average, then compare plain policy gradients with PPO's clipped, safer updates at a high learning rate.

  • Level Advanced
Robot Learning · 12

Reward Design and Reward Hacking

Write a reward for a cleaning robot, let an optimal agent maximize it, and watch it find a loophole, then fix the reward so the behavior you wanted is also the best one.

  • Level Intermediate
Robot Learning · 13

Sim-to-Real and Domain Randomization

Tune a gripper's squeeze force in simulation, then test it on real objects whose friction differs, and see how training across many randomized simulators closes the gap.

  • Level Intermediate
Robot Learning · 14

Model-Based RL and Learned Dynamics

Learn a model of how a robot responds to commands from a little data, then plan with that model, and compare how few real trials it needs against trial and error alone.

  • Level Advanced
Robot Learning · 15

Offline Reinforcement Learning

Learn a policy only from a fixed log of past robot behavior, and see why plain Q-learning trusts actions it never saw, while a conservative method stays with what the data supports.

  • Level Advanced
Robot Learning · 16

Learning from Human Preferences

Teach a robot what a good path through a room looks like by repeatedly choosing the better of two paths, and watch a reward model learn your hidden priorities.

  • Level Intermediate
Robot Learning · 17

Learning from Human Videos

Watch where many people's hands touch everyday objects across simulated videos, build an affordance map, and let a robot choose its grasp from it.

  • Level Advanced
Robot Learning · 18

Cross-Embodiment Learning

Pool demonstration data from many different robots into one shared action space and see how much less data a new robot needs to learn a task.

  • Level Advanced
Robot Learning · 19

Vision-Language Models for Robots

Ask a vision-language model questions about a robot's camera view: name the objects, point to one, plan the next step, or judge whether a task succeeded.

  • Level Intermediate
Robot Learning · 20

Vision-Language-Action (VLA) Models

Follow an image and an instruction through a vision-language-action model to robot actions, and compare actions output as discrete tokens with smooth continuous actions.

  • Level Advanced
Robot Learning · 21

Dual-System VLAs: Fast and Slow Control

Track a moving target with a big, slow vision-language model alone, then split the job between a slow planner and a small fast controller, and compare how well each keeps up.

  • Level Advanced
Robot Learning · 22

World Models

Let a learned model imagine how a bouncing ball will move, compare its imagined future with reality, and see the prediction error grow the further ahead it dreams.

  • Level Advanced
Robot Learning · 23

World-Action Models

Have a model first imagine the future states that would achieve a goal, then recover the actions that connect them with an inverse dynamics model, and execute them.

  • Level Advanced
Robot Learning · 24

Synthetic Data and Data Augmentation

Train a classifier on a few bright, real images of red and orange balls, then test it in dim light, and see how augmentation and synthetic images keep it working.

  • Level Intermediate
Robot Learning · 25

Fine-Tuning Robot Foundation Models

Adapt a pretrained model to a new task with a small dataset, by full fine-tuning, a low-rank adapter, or training only the last layer, and see how much each forgets and how replaying old data helps.

  • Level Advanced
Robot Learning · 26

Evaluating Robot Policies

Estimate two policies' success rates from a limited number of real trials, put confidence intervals on them, and find how many trials it takes to tell them apart.

  • Level Intermediate
Robot Learning · 27

Safe Reinforcement Learning

Train a delivery robot to move as fast as it can while keeping its risk of getting too close to people under a limit, using a learned penalty that adjusts itself, and compare with a safety shield.

  • Level Advanced

Human-Robot Interaction and Safety

7 explainers
Human-Robot Interaction and Safety · 01

Robot Safety Standards and Collaborative Modes

Walk a person toward an industrial robot and see how each of the four collaborative operating modes keeps them safe: monitored stop, hand guiding, speed and separation monitoring, and power and force limiting.

  • Level Beginner
Human-Robot Interaction and Safety · 02

Emergency Stops and Safety-Rated Systems

Press the emergency stop on a moving robot arm using each stop category and compare how far it travels before it halts, then inject a fault into a single-channel and a dual-channel safety circuit.

  • Level Beginner
Human-Robot Interaction and Safety · 03

Teleoperation and Shared Autonomy

Steer a robot toward one of several objects while it guesses which one you want and blends in help, more strongly as it grows confident.

  • Level Intermediate
Human-Robot Interaction and Safety · 04

Haptics and Force Feedback

Push a remote robot against a wall through a teleoperation link, with and without force feedback to the operator's hand, and add network delay to see why feedback can become unstable.

  • Level Intermediate
Human-Robot Interaction and Safety · 05

Gesture and Intent Recognition

Draw a gesture for the robot and see it matched to the closest template, then watch the robot predict which object a person is reaching for before the hand gets there.

  • Level Intermediate
Human-Robot Interaction and Safety · 06

Trust, Transparency, and Explainable Behavior

Watch a robot reach for one of two nearby objects with a predictable motion and with a legible motion that exaggerates its intent, and see how soon an observer can tell which one it wants.

  • Level Intermediate
Human-Robot Interaction and Safety · 07

Robot Ethics and Responsibility

Work through realistic robot deployment scenarios, see who is affected, which concerns arise, what design choices address them, and who is responsible when something goes wrong.

  • Level Beginner

Robot Software and Systems

12 explainers
Robot Software and Systems · 01

The Robot Software Stack

Follow one LiDAR reading up through the layers of a robot's software, from driver to perception to planning, and the resulting command back down to the motors, with the time each layer adds.

  • Level Beginner
Robot Software and Systems · 02

ROS 2 Nodes, Topics, and Publish/Subscribe

Connect ROS 2 nodes through topics, watch messages flow from publishers to every subscriber, and see what happens when a subscriber is too slow for the publishing rate.

  • Level Beginner
Robot Software and Systems · 03

ROS 2 Services, Actions, and Parameters

Call a service and wait for its answer, send a long-running action goal and watch its feedback, cancel it, and change a node's parameter while it runs.

  • Level Beginner
Robot Software and Systems · 04

TF2 and the Transform Tree

Move the frames of a mobile robot with an arm and a camera, then ask TF2 for the transform between any two frames and see which branches of the tree it chains together.

  • Level Intermediate
Robot Software and Systems · 05

Quality of Service and DDS

Send ROS 2 messages over a lossy wireless link and compare best-effort with reliable delivery, then let a late subscriber join and see what durability gives it.

  • Level Intermediate
Robot Software and Systems · 06

URDF and Robot Description

Build a small robot arm from links and joints, change each joint's type, axis, and limits, and watch the URDF file and the robot update together.

  • Level Beginner
Robot Software and Systems · 07

Simulators and Physics Engines

Stack boxes in a simple physics engine and change the time step and the number of solver iterations to see when the stack sits still, jitters, sinks, or explodes.

  • Level Beginner
Robot Software and Systems · 08

Digital Twins

Run a simulated twin alongside a real robot arm, compare what the twin predicts with what the robot actually does, and catch an unexpected payload when the two disagree.

  • Level Intermediate
Robot Software and Systems · 09

Real-Time Systems, Latency, and Jitter

Run a 1 kHz control loop on an ordinary operating system and on a real-time one, look at the spread of loop periods, and see how missed deadlines disturb a balancing controller.

  • Level Intermediate
Robot Software and Systems · 10

Message Synchronization

Fuse a 30 Hz camera with a 10 Hz LiDAR by pairing their messages by timestamp, compare exact and approximate matching, and see how a mismatched pair misplaces a moving object.

  • Level Intermediate
Robot Software and Systems · 11

Logging, Bags, and Replay

Record a robot run into a bag file, then scrub through it, replay it at different speeds, and replay a failure exactly to find its cause.

  • Level Beginner
Robot Software and Systems · 12

Edge Computing and Inference Budgets

Place a robot's perception, planning, and language models on the onboard computer, a nearby GPU server, or the cloud, and check whether the end-to-end delay fits the control loop's budget.

  • Level Intermediate

Handouts

Longer guides you can read on screen or print. Every link inside is clickable.

Cover page of Becoming a Robotics Engineer

Becoming a Robotics Engineer

A six-month self-study roadmap with curated resources, practice projects, and milestones. It assumes no prior electronics background and about two to three hours of work per day.

  • Level Beginner to intermediate
  • Length 24 pages
  • Format PDF

Adapted from work by Ronin (@deronin_) on X. The original ideas and structure are the original author's work.

A six-month path

Ready to go further? The roadmap handout breaks robotics into six monthly steps, each with a goal and a signature build. Keep every build in its own GitHub repository, with a photo, your measured results, and a note on what broke and how you fixed it.

  1. Month 1

    Electronics, Python, Git

    Read a schematic, build a working circuit, and find faults.

  2. Month 2

    Microcontrollers, motors, sensors

    Build a robot that moves, senses, and corrects itself.

  3. Month 3

    CAD and fabrication

    Design a part, make it, and have it fit.

  4. Month 4

    ROS 2 and simulation

    Simulate a robot that maps and navigates on its own.

  5. Month 5

    Control, kinematics, perception

    Understand the math under the libraries.

  6. Month 6

    Robot learning and career

    Pick a direction, finish a portfolio piece, and apply.