Automated Differentiation System

An Automated Differentiation System is a function differentiation system that is an automated system.

• Context:
  • It can (typically) implement an Automated Differentiation Algorithm (to solve an automated differentiation task).
  • …
• See: torch.autograd.

---

---

References

2019

• https://pytorch.org/docs/stable/autograd.html
  • QUOTE: ... torch.autograd provides classes and functions implementing automatic differentiation of arbitrary scalar valued functions. It requires minimal changes to the existing code - you only need to declare Tensor s for which gradients should be computed with the requires_grad=True keyword. …

2019

• https://program-transformations.github.io
  • QUOTE: ... Machine learning researchers often express complex models as a program, relying on program transformations to add functionality. New languages and transformations (e.g., TorchScript and TensorFlow AutoGraph) are becoming core capabilities of ML libraries. However, existing transformations, such as automatic differentiation (AD or autodiff), inference in probabilistic programming languages (PPLs), and optimizing compilers are often built in isolation, and limited in scope …

---