du failing on simple function

[barak]

The directional derivative operator du doesn't seem to handle the simplest case.

$ ghci
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.

Prelude> :m + Numeric.AD

Prelude Numeric.AD> let f [x0,x1,x2] = [x0*x1^2*x2^3, x0^3*x1^2*x2]

Prelude Numeric.AD> :t f
f :: Num t => [t] -> [t]

Prelude Numeric.AD> :t du f

<interactive>:1:4:
    Couldn't match type `[AD
                            s (Numeric.AD.Internal.Forward.Forward a0)]'
                  with `AD s (Numeric.AD.Internal.Forward.Forward a0)'
    Expected type: [AD s (Numeric.AD.Internal.Forward.Forward a0)]
                   -> AD s (Numeric.AD.Internal.Forward.Forward a0)
      Actual type: [AD s (Numeric.AD.Internal.Forward.Forward a0)]
                   -> [AD s (Numeric.AD.Internal.Forward.Forward a0)]
    In the first argument of `du', namely `f'
    In the expression: du f

[barak]

(I understand that duF can do it. But that is something of a wart! These things should be unified. Maybe the type system isn't up to it, but if so, we should whine about the deficient type system when explaining the plethora of suffixes for various shapes of input and output.)

[cartazio]

Prelude Numeric.AD> :t du
du
  :: (Num a, Functor f) =>
     (forall s.
      f (AD s (Numeric.AD.Internal.Forward.Forward a))
      -> AD s (Numeric.AD.Internal.Forward.Forward a))
     -> f (a, a) -> a

if your function had the type

foo :: Num a => [a]-> a
foo [x,y]= x*y

then everything works fine

Prelude Numeric.AD> let foo [x,y]= x*y
Prelude Numeric.AD> du foo [(1,0),(2,8)]
8

[cartazio]

Are you suggesting that perhaps every value thats not wrapped in a Functor should be considered to be implicitly lifted into the identity functor? Thats what it sounds like, But in that case du would be the wrong type, and duF would become the default, right?

[barak]

I guess that's one way to think about it.

In more mathematical terms, the "tangent space" is always a vector space. I directional derivative maps a point on the input tangent space to the output tangent space. Which is clean and unified. Having the tangent spaces come in different "shapes" with different derivative-taking operators to deal with them depending on shape is sort of annoying. And worse, we could have something like

f :: Num a => ([[[a]]],[[a]],[a]) -> (a,[a],[[[[a]]]])

which makes complete sense to take a directional derivative of, but which doesn't fit into any of our cases.

[cartazio]

false, it fits in that regime like so

Prelude Numeric.AD> :t duF
duF
  :: (Num a, Functor g, Functor f) =>
     (forall s.
      f (AD s (Numeric.AD.Internal.Forward.Forward a))
      -> g (AD s (Numeric.AD.Internal.Forward.Forward a)))
     -> f (a, a) -> g a

just writing down the defn for duF for now

newtype F a = F ([[[a]]],[[a]],[a])
   deriving Functor 
newtype G a = G  (a,[a],[[[[a]]]])
   deriving Functor 

f:: Num a => F a -> G a

this f using G and F is trivially a wrapping of your proposed problematic f,
BUT duF works just fine for that example

[cartazio]

Or maybe I'm not understanding what you mean by directional derivative!

[cartazio]

what would a direction derive look like in that context/example? I'm imagining it being semantically equivalent to a big flattened version of a list

[ekmett]

I frankly don't see a plausible way to do unify these things.

Yes, types get in the way some times, and in this case they force a distinction.

You could argue that duF should actually be du, and that everyone who wants the simpler case should have to pay for an Identity wrapper or some nonsense, but we do carefully distinguish between the cases that take and/or return vectors throughout the API, and I don't see that changing.

To go far enough to get you what you want we have to throw out the entire numeric tower, because Num mixes (+) in with (*), etc.

That isn't happening, not with keeping any users.

[ekmett]

The only way this is going to change is if we adopt something like #2, which would have pretty far reaching consequences on the API. I'll leave this issue open as a sort of adjunct / example for that one.

[barak]

We could use fancy types, let the type constraint Tang t tt mean that the tangent type of t is tt, have things like (Tang t tt, Tang u uu) => Tang (t,u) (tt,uu) and Tang t tt => Tang [t] [tt] and such. This works fine except when you get to numbers, where it gets tricky because if someone was silly and defined something like Num a => Num [a] then you'd lose, so the system lets you ground the definitions in concrete things like Double but not in Num.

(Naturally this would need an API, the obvious one would be zero :: Tang t tt => t -> tt to get a zero element of the tangent space, and addition and scalar product inside the vector space, and maybe something to "take a step in this direction" with type t -> s -> tt -> t, where s is a step size of the appropriate numeric type. Thinking about that makes one think the type constraint should have a third slot for the base numeric type. Or else something more complicated to allow a mixture of numeric types...)

[ekmett]

Losing access to AD over arbitrary Num types isn't really an option. A number of the users of this library use it at some pretty exotic types!

You can avoid the base case issue by doing what we do with the AD wrapper when we want to work abstractly over the choice of Mode, we can't make the instance to know m a is a Num for all modes m, but something like AD m a can have such an instance. Such a beast would also have the slot for the base numeric type.

The problem is I don't think the technique you propose and having the rank-2 guard against infinitesimal confusion can't work together. There isn't a regular 'place' in the type for the region/infinitesimal parameter, as it is changing position all over the place moving under more functors layers, etc.

[barak]

Wouldn't that "brand" type go in the constraint but not the instance, so it could be the same in the various constraints on the LHS of the instance declaration. (With the appropriate extension enabled.)

[ekmett]

The "brand" can't float in the constraint without actually showing up in a type somewhere. You could almost kind of fake it with implicit parameters, but then they'd yield totally bogus results when it comes to 'diff of diff', rather than just making you insert autos to get it to compile.