**Reinforcement learning**

- Part 1: Q-learning and exploration
- Part 2: SARSA vs Q-learning
- Part 3: Egocentric learning
- Part 4: Combining egocentric and allocentric

**Control theory**

Operational space control

- Part 1: Forward transformation matrices
- Part 2: Jacobians, velocity, and force
- Part 3: Accounting for mass and gravity
- Part 4: Operation space control
- Part 5: Controlling in the null space
- Part 6: Handling singularities
- Part 7: OSC of a 3-link arm
- Part 8: Force control of task-space orientation
- Deriving a robot’s transform matrices
- Full body obstacle collision avoidance
- Velocity limiting in operational space control

OSC with 6DOF robot and spiking camera

- Part 1: Getting access to the arm through Python
- Part 2: Deriving the Jacobian
- Part 3: Tracking a target using spiking cameras

Inverse kinematics

LQR, iLQR, and related methods

- Linear-Quadratic Regulation for non-linear systems using finite differences
- The iterative Linear Quadratic Regulator algorithm
- Simultaneous perturbation vs finite differences for linear dynamics estimation and control signal optimization

Deep learning

Likelihood calculus (Stochastic differential operators)

- Part 1: Controlling variability
- Part 2: Neuro-mechanical control using differential stochastic operators
- Part 3: Distributed control of uncertain systems using superpositions of linear operators

Miscellaneous

**Dynamic movement primitives**

- Part 1: The basics
- Part 2: Controlling end-effector trajectories
- Part 3: Rhythmic movements
- Part 4: Avoiding obstacles – update

**Computational Neuroscience**

Modelling

- Building a spiking neural model of adaptive arm control
- Improving neural models by compensating for discrete rather than continuous filter dynamics when simulating on digital systems
- Setting up an arm simulation interface in Nengo 2
- Nengo model – Low pass derivative filter
- Nengo scripting: absolute value

Theory

- Online Goal Babbling – motor learning paper review
- Dynamic primitives of motor behavior – paper review
- The role of phasic dopamine in the basal ganglia
- The basal ganglia for action selection

**Programming**

Python

- Natural policy gradient in TensorFlow
- ABR Jaco2 repo public release!
- ABR Control repo public release!
- Quick calculations with SymPy and Cython
- Using SymPy’s lambdify for generating transform matrices and Jacobians
- Using VREP for simulation of force-controlled models

C++

- Using the same random number generator in C++ and Python
- C++ debugging with GDB and Valgrind quickstart
- Getting function run time in C++ on Linux
- N-dimensional Matrices in C++

Cython

- Part 1: The Basics
- Part 2: Including Eigen
- Part 3: Wrapping C code and passing arrays
- Wrapping MapleSim C code for Python

Visualizations

- Arm visualization with PyGame
- Arm simulation visualization with Matplotlib
- Matplotlib legends for mean and confidence interval plots

Miscellaneous

**Math**