Shortcourse: Using Algorithmic Differentiation to Compute Derivatives for Sensitivity Analysis, Uncertainty Quantification and Optimization

Instructors:

Jean Utke (ANL)

Sri Hari Krishna Narayanan (ANL)

Algorithmic differentiation (AD) is a process for producing derivative computations from computer programs. The resulting derivatives are accurate to machine precision with respect to the original computation and can be used in many contexts, including uncertainty quantification, numerical optimization, nonlinear partial differential equation solvers, or the solution of inverse problems using least squares.

Established tools provide AD for various programming languages, including Fortran, C, Matlab, and C++. The tools differ in their support for the basic algorithmic approaches for AD, known as tangent-linear and adjoint mode, and also their capabilities for advanced uses such as sparsity exploitation and the computation of higher order derivatives. AD tools generally adopt one of two implementations approaches: operator overloading and source transformation. Depending on the code/problem that requires derivatives, the choice between the basic algorithmic approach, the AD tool, and the exploitation of higher-level properties such as sparsity are important to obtain the most efficiently computed derivatives especially for computationally complex numerical models.

The course will introduce the AD theory needed to motivate the choices mentioned above. The majority of the course will explore the use and illustrate the theoretical concept using four AD tools. The tools ADIC2, ADOL-C, OpenAD, and Rapsodia have all been co-authored by the presenters. Each tool is appropriate for a particular class of code/problem requiring derivatives. The goal of the course is to allow an attendee to to identify opportunities to use AD. The attendee should come away with the ability to apply a tool that we present, know the basic aspects of the AD theory that have a practical impact on the derivative computation, and understand how these theoretical aspects are related to applying an AD tool.

The course will consist of the following sections:
1) An introduction to why AD is possible
2) Operator overloading AD with an ADOL-C
3) Source transformation AD with ADIC2 and OpenAD
4) Reverse mode operator overloading AD with ADOL-C
5) Reverse mode source to source transformation with OpenAD and ADIC2
6) Exploiting sparsity in Jacobians with ADOL-C and ADIC2
7) Higher order derivatives with Rapsodia

Each section will feature an explanation of relevant AD theory and offer the chance for participants to run tools on examples provided. Attendees will be required to install a virtual machine image containing pre-installed software on their laptops in order to run the examples. Practical issues such as pre-processing input code and writing drivers will also be discussed.