|
9 | 9 | AD! Don't nest [[gradient]] calls inside [[emmy.env/D]] calls."
|
10 | 10 | (:refer-clojure :exclude [compare zero?])
|
11 | 11 | (:require #?(:clj [clojure.pprint :as pprint])
|
12 |
| - [emmy.differential :as d] |
| 12 | + [emmy.dual :as d] |
13 | 13 | [emmy.function :as f]
|
14 | 14 | [emmy.generic :as g]
|
15 | 15 | [emmy.matrix :as matrix]
|
|
26 | 26 | ;;
|
27 | 27 | ;; This namespace develops an implementation of "reverse-mode" automatic
|
28 | 28 | ;; differentation, in contrast with the forward mode AD implementation
|
29 |
| -;; in [[emmy.differential]]. These two are closely related, and the |
| 29 | +;; in [[emmy.dual]]. These two are closely related, and the |
30 | 30 | ;; implementations could merge once I get around to reading and
|
31 | 31 | ;; implementing [the YOLO paper](https://arxiv.org/abs/2204.10923).
|
32 | 32 | ;;
|
|
58 | 58 | ;; First, wrap all inputs to the function in an instance of a type called
|
59 | 59 | ;; a [[TapeCell]]. This type holds
|
60 | 60 | ;;
|
61 |
| -;; - a "tag", used as in [[emmy.differential]] to prevent confusion |
| 61 | +;; - a "tag", used as in [[emmy.dual]] to prevent confusion |
62 | 62 | ;; between [[TapeCell]] instances created for different nested runs of
|
63 | 63 | ;; differentation
|
64 | 64 | ;; - a unique ID, used to de-duplicate work in the reverse pass
|
|
331 | 331 |
|
332 | 332 | [<tag> <tape-or-dual-number>]
|
333 | 333 |
|
334 |
| - containing the tag and instance of [[emmy.differential/Dual]] or [[TapeCell]] |
| 334 | + containing the tag and instance of [[emmy.dual/Dual]] or [[TapeCell]] |
335 | 335 | associated with the inner-most call to [[with-active-tag]] in the current call
|
336 | 336 | stack.
|
337 | 337 |
|
|
358 | 358 |
|
359 | 359 | (defn deep-primal
|
360 | 360 | "Version of [[tape-primal]] that will descend recursively into any perturbation
|
361 |
| - instance returned by [[tape-primal]] or [[emmy.differential/primal]] until |
| 361 | + instance returned by [[tape-primal]] or [[emmy.dual/primal]] until |
362 | 362 | encountering a non-perturbation.
|
363 | 363 |
|
364 | 364 | Given a non-perturbation, acts as identity."
|
|
535 | 535 | ;;
|
536 | 536 | ;; TODO [[->partials]] really should depend on `fmap`, as should so many other
|
537 | 537 | ;; things in the library. Without this we can't generically support output
|
538 |
| -;; values like maps or quaternions. [[emmy.differential/extract-tangent]] does |
| 538 | +;; values like maps or quaternions. [[emmy.dual/extract-tangent]] does |
539 | 539 | ;; this for forward-mode, I believe.
|
540 | 540 |
|
541 | 541 | (declare ->partials)
|
|
643 | 643 | ;; output-walking.
|
644 | 644 | ;;
|
645 | 645 | ;; I am sure we are going to need to glom on to the [[replace-tag]] machinery as
|
646 |
| -;; well. [[emmy.differential/extract-tangent]] is similar to what we want. Maybe |
| 646 | +;; well. [[emmy.dual/extract-tangent]] is similar to what we want. Maybe |
647 | 647 | ;; we can share?
|
648 | 648 |
|
649 | 649 | (defn ^:no-doc interpret
|
|
718 | 718 | ;; ## Lifted Functions
|
719 | 719 |
|
720 | 720 | ;; [[lift-1]] and [[lift-2]] "lift", or augment, unary or binary functions with
|
721 |
| -;; the ability to handle [[emmy.differential/Dual]] and [[TapeCell]] instances |
| 721 | +;; the ability to handle [[emmy.dual/Dual]] and [[TapeCell]] instances |
722 | 722 | ;; in addition to whatever other types they previously supported.
|
723 | 723 | ;;
|
724 |
| -;; Forward-mode support for [[emmy.differential/Dual]] is an implementation of |
| 724 | +;; Forward-mode support for [[emmy.dual/Dual]] is an implementation of |
725 | 725 | ;; the single and multivariable Taylor series expansion methods discussed at the
|
726 |
| -;; beginning of [[emmy.differential]]. |
| 726 | +;; beginning of [[emmy.dual]]. |
727 | 727 | ;;
|
728 | 728 | ;; To support reverse-mode automatic differentiation, When a unary or binary
|
729 | 729 | ;; function `f` encounters a [[TapeCell]] `x` (and `y` in the binary case) it
|
|
745 | 745 |
|
746 | 746 | ;; There is a subtlety here, noted in the docstrings below. [[lift-1]]
|
747 | 747 | ;; and [[lift-2]] really are able to lift functions like [[clojure.core/+]] that
|
748 |
| -;; can't accept [[emmy.differential/Dual]] and [[TapeCell]]s. But the |
| 748 | +;; can't accept [[emmy.dual/Dual]] and [[TapeCell]]s. But the |
749 | 749 | ;; first-order derivatives that you have to supply _do_ have to be able to take
|
750 | 750 | ;; instances of these types.
|
751 | 751 | ;;
|
752 |
| -;; This is because, for example, the [[emmy.differential/tangent]] of [[Dual]] |
753 |
| -;; might still be a [[Dual]], and will hit the first-order derivative via the |
| 752 | +;; This is because, for example, the [[emmy.dual/tangent]] of [[emmy.dual/Dual]] |
| 753 | +;; might still be a [[emmy.dual/Dual]], and will hit the first-order derivative via the |
754 | 754 | ;; chain rule.
|
755 | 755 | ;;
|
756 | 756 | ;; Magically this will all Just Work if you pass an already-lifted function, or
|
|
764 | 764 | single argument
|
765 | 765 |
|
766 | 766 | Returns a new unary function that operates on both the original type of
|
767 |
| - `f`, [[TapeCell]] and [[emmy.differential/Dual]] instances. |
| 767 | + `f`, [[TapeCell]] and [[emmy.dual/Dual]] instances. |
768 | 768 |
|
769 | 769 | If called without `df:dx`, `df:dx` defaults to `(f :dfdx)`; this will return
|
770 | 770 | the derivative registered to a generic function defined
|
771 | 771 | with [[emmy.util.def/defgeneric]].
|
772 | 772 |
|
773 | 773 | NOTE: `df:dx` has to ALREADY be able to handle [[TapeCell]]
|
774 |
| - and [[emmy.differential/Dual]] instances. The best way to accomplish this is |
| 774 | + and [[emmy.dual/Dual]] instances. The best way to accomplish this is |
775 | 775 | by building `df:dx` out of already-lifted functions, and declaring them by
|
776 | 776 | forward reference if you need to."
|
777 | 777 | ([f]
|
|
804 | 804 | - a function `df:dy`, similar to `df:dx` for the second arg
|
805 | 805 |
|
806 | 806 | Returns a new binary function that operates on both the original type of
|
807 |
| - `f`, [[TapeCell]] and [[emmy.differential/Differential]] instances. |
| 807 | + `f`, [[TapeCell]] and [[emmy.dual/Dual]] instances. |
808 | 808 |
|
809 | 809 | NOTE: `df:dx` and `df:dy` have to ALREADY be able to handle [[TapeCell]]
|
810 |
| - and [[emmy.differential/Dual]] instances. The best way to accomplish this is |
| 810 | + and [[emmy.dual/Dual]] instances. The best way to accomplish this is |
811 | 811 | by building `df:dx` and `df:dy` out of already-lifted functions, and declaring
|
812 | 812 | them by forward reference if you need to."
|
813 | 813 | ([f]
|
|
860 | 860 | first and second args in the binary case
|
861 | 861 |
|
862 | 862 | Returns a new any-arity function that operates on both the original type of
|
863 |
| - `f`, [[TapeCell]] and [[emmy.differential/Dual]] instances. |
| 863 | + `f`, [[TapeCell]] and [[emmy.dual/Dual]] instances. |
864 | 864 |
|
865 | 865 | NOTE: The n-ary case of `f` is populated by nested calls to the binary case.
|
866 | 866 | That means that this is NOT an appropriate lifting method for an n-ary
|
|
890 | 890 | `differential-op` (defaults to `(lift-1 generic-op)`)
|
891 | 891 |
|
892 | 892 | installs an appropriate unary implementation of `generic-op` for `::tape` and
|
893 |
| - `:emmy.differential/dual` instances." |
| 893 | + `:emmy.dual/dual` instances." |
894 | 894 | ([generic-op]
|
895 | 895 | (defunary generic-op (lift-1 generic-op)))
|
896 | 896 | ([generic-op differential-op]
|
|
905 | 905 | `differential-op` (defaults to `(lift-2 generic-op)`)
|
906 | 906 |
|
907 | 907 | installs an appropriate binary implementation of `generic-op` between
|
908 |
| - `::tape`, `::emmy.differential/dual` and `::v/scalar` instances." |
| 908 | + `::tape`, `::emmy.dual/dual` and `::v/scalar` instances." |
909 | 909 | ([generic-op]
|
910 | 910 | (defbinary generic-op (lift-2 generic-op)))
|
911 | 911 | ([generic-op differential-op]
|
|
0 commit comments