dump_callback etc.

[jtlz2]

Hi - I'm trying to pass the current sample values to a python wrapper to plot them using pyplot as MultiNEST runs, rather than using show as in your demo script, because I hope it's more efficient than using show -> pdf.

I have a function to do the plot update, but I can't figure out how to extract the current sample values. Should I use dump_callback, or progressPrinter, or go via an ascii file..? How should I do that if so?

Thanks v much.

[jtlz2]

PS: I suppose I could add a routine to the prior or likelihood functions to do this - is that the best way forward?

[JohannesBuchner]

Yes, the dump_callback would be the right way to do it. However, since I have not used it before, the implementation is not really tested. You will receive nsamples, nlive, n, and then what is in the postdist file as a 2-dimensional double array (nsamples x n). I would be happy if you could test it.

The benefit of writing to a pdf file and watching it with a viewer is that the output is permanent (does not go away if your code crashed), and that you are independent from graphical displays.

The second option is progressPrinter, which just reads the MultiNest output every so often. Note that there can be a partial output and so it might fail once in some read-outs, but it should be stable against that and just try again. You can build on e.g. ProgressPrinter, and read self.live using numpy.loadtxt

The third option is as you say, just output on every evaluation. This will probably make your code very slow, because also points that will be rejected are used then.

[jtlz2]

Thanks for the speedy reply. Happy to test - I want to try dump_callback -
could you give a simple example of the syntax? Something like:

pymultinest.run(..., dump_callback=func)

func(???):
print ???
return

What would be an example of the syntax for ProgressPrinter (sounds a bit
more complicated).

[JohannesBuchner]

You can always start with

def func(*args): 
   print args 

but in this case, using ctypes to convert the array

def func(nsamples, nlive, n, postdist):
   print nsamples, nlive, n, postdist
   arr_type = ((ctypes.c_float * nsamples) * n)
   values = arr_type(postdist)
   print nsamples, nlive, n, values

[JohannesBuchner]

something along those lines anyways, you will have to try around a bit

[jtlz2]

Thanks - so I get an error with dump_wrapper:

evidence_tolerance=0.5,mode_tolerance=-1e90,dump_callback=func)

File
"/Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/pymultinest-0.1-py2.7.egg/pymultinest/run.py",
line 187, in run
lib.set_dumper(dumper_type(dump_wrapper))
NameError: global name 'dump_wrapper' is not defined

I tried setting the name of func to dump_wrapper, but get the same.

[JohannesBuchner]

That should be dump_callback instead of dump_wrapper

[JohannesBuchner]

(bug in https://github.com/JohannesBuchner/PyMultiNest/blob/master/pymultinest/run.py#L199 )

[jtlz2]

Now I get (1000 live points):

Acceptance Rate: 0.847836
Replacements: 1900
Total Samples: 2241
ln(Z): -88079.423159
Acceptance Rate: 0.838710
Replacements: 1950
Total Samples: 2325
ln(Z): -77446.214381
converting for access ...
--> DumperConverter called with: 2000 1000 4
converting for access ... done.
dumper set to 0x1002fff60
calling client.
calling 0x1002fff60
Illegal instruction

Could this take a while to debug? :-S

[JohannesBuchner]

Try as a starting point

def func(nsamples, nlive, n, postdist):
   print 'python dumper callback called!'
   sys.stdout.flush()
   print nsamples, nlive, n
   sys.stdout.flush()

[jtlz2]

Here you go (n_iter_before_update=10):

generating live points
live points generated, starting sampling
Acceptance Rate: 0.999049
Replacements: 1050
Total Samples: 1051
ln(Z): **************
converting for access ...
--> DumperConverter called with: 1100 1000 4
converting for access ... done.
dumper set to 0x100598f60
calling client.
calling 0x100598f60
Segmentation fault

[JohannesBuchner]

In the multinest bridge try simplifying

#define DUMPERTYPE(f) void (f)(int nsamples, int nlive, int npar, \
double ** physlive, double ** posterior, \
double *mean, double *std, double *best, double *map, \
double maxloglike, double logz, double logzerr)

to

#define DUMPERTYPE(f) void (f)(int nsamples, int nlive, int npar, \
double ** physlive)

and

p.Dumper(*nsamples, *nlive, n,
(double**)postdist, (double**)pLivePts,
mean, std, best, map, *maxloglike, *logz, *logzerr);

to

p.Dumper(*nsamples, *nlive, n, (double**)postdist);

[jtlz2]

Same problem:

---

Starting MultiNest
generating live points
live points generated, starting sampling
Acceptance Rate: 0.998099
Replacements: 1050
Total Samples: 1052
ln(Z): **************
converting for access ...
--> DumperConverter called with: 1100 1000 4
converting for access ... done.
dumper set to 0x100598f60
calling client.
calling 0x100598f60
Segmentation fault

[JohannesBuchner]

Try putting a few printf in before the p.Dumper() call
to check whether *nsamples, *nlive, n, and postdist contain reasonable values (e.g. not NULL).
You can also try replacing

dumper_type = CFUNCTYPE(c_void_p,
c_int, c_int, c_int, POINTER(c_double))

with

dumper_type = CFUNCTYPE(c_void_p, 
c_int, c_int, c_int, c_void_p)

in run.py

[jtlz2]

*nsamples, *nlive and n are fine, so the problem must be with postdist (and
hence with posterior)? It might be an overflow, but I'm not really a C
programmer(!)...

[JohannesBuchner]

You can check by replacing postdist in the p.Dumper call by NULL for the moment.
printf("%p\n", postdist) would tell you the address.