Feature Request: choice for levels of caching and/or derivatives

[adwilson10]

From [email protected] on February 28, 2013 16:25:19

Since the fourth derivatives of g and vb (as seen in _structure_changed() in frames.py) end up being very large for systems described by deep trees---a depth of 30 results in about 4GB of memory used---it would be very useful to implement an option that allows one to choose the level of caching and the level of available derivatives, in order to be able to simulate larger systems without prohibitively large memory requirements.

Original issue: http://code.google.com/p/trep/issues/detail?id=18

[adwilson10]

From [email protected] on February 28, 2013 14:39:39

I'd prefer a more transparent solution than requiring the user to disable levels of caching. This could be done easily by initializing the cache pointers to None and allocating them in the C code when building a cache set. A method could be exposed to delete all the caches in case a user wants to calculate an expensive value and then release the memory. I'd prefer to avoid a completely cache-less implementation because that requires a separate code path each of the derivative types. I'll try an initial auto-allocating implementation tonight.

Owner: [email protected]

[adwilson10]

From [email protected] on February 28, 2013 19:58:34

I just committed some changes that should fix it. Give it a good testing and let me know if that helps.

Status: Fixed

[adwilson10]

From [email protected] on March 04, 2013 08:14:21

Thanks, it seems to be working great as long as I don't specifically ask to calculate the higher order derivatives for large systems. This makes sense, because those higher order objects need to be calculated and cached if I am asking for the first (or second derivative) of the dynamics. I wonder, however, if there is a way to "force" a trade between memory and CPU usage for such cases, since using swap is simply way too slow and dedicated hardware is not a very elegant solution. Thoughts?

[adwilson10]

From [email protected] on March 04, 2013 23:37:26

You could try doing a non-caching implementation of those derivatives. I don't think it will be much faster in general because you have to traverse the frame tree all the way back to the root frame for each of the NxNxNxN derivatives. You can reduce a lot of the work by taking advantage of symmetry like the current implementation does, but not enough to make it fast. For the special case of a single non-branching sub-tree like that of your whiskers, there is almost certainly a more elegant solution. Especially if you are willing to violate the current abstraction between the dynamics/derivatives and kinematics, but thinking about how to approach that makes my brain hurt.