nuclear-multimessenger-astronomy · tylerbarna · Feb 13, 2022 · Feb 13, 2022 · Feb 14, 2022 · Feb 14, 2022
diff --git a/.github/workflows/deploy-github-pages.yml b/.github/workflows/deploy-github-pages.yml
@@ -12,7 +12,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.7]
+        python-version: [3.9]
 
     steps:
     - uses: actions/checkout@v2
@@ -36,6 +36,7 @@ jobs:
     - name: Install dependencies
       run: |
         #sudo apt-get install gfortran swig libhdf5-serial-dev libmpich-dev
+        python -m pip install --upgrade git+https://github.com/bitranox/wrapt_timeout_decorator.git
         python -m pip install coverage cpp-coveralls flake8 pytest
         python -m pip install .[doc]
 

diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml
@@ -53,8 +53,9 @@ jobs:
       run: |
         # sudo apt-get install gfortran swig libhdf5-serial-dev libmpich-dev
         sudo apt install -y openmpi-bin libopenmpi-dev gfortran build-essential libblas3 libblas-dev liblapack3 liblapack-dev libatlas-base-dev texlive texlive-latex-extra texlive-fonts-recommended dvipng cm-super
-        python -m pip install pytest pytest-cov flake8 pytest-aiohttp sqlparse freezegun PyJWT pathlib joblib
-        python -m pip install .[doc]
+        python -m pip install --upgrade git+https://github.com/bitranox/wrapt_timeout_decorator.git
+        python -m pip install pytest pytest-cov flake8 pytest-aiohttp sqlparse freezegun PyJWT joblib tensorflow afterglowpy
+        python -m pip install .
         git clone https://github.com/JohannesBuchner/MultiNest && cd MultiNest/build && cmake .. && make && cd ../..
         pwd
     - name: Lint with flake8
@@ -70,6 +71,8 @@ jobs:
         psql -U nmma -h localhost -c "GRANT ALL PRIVILEGES ON DATABASE nmma TO nmma;" nmma
     - name: Test with pytest
       run: |
+        pwd
+        ls
         py.test nmma/tests/*
       env:
         LD_LIBRARY_PATH: .:/home/runner/work/nmma/nmma/MultiNest/lib # for Linux

diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,5 @@
+
+
+
+svdmodels/Bu2019lm_lbol.pkl
+svdmodels/Bu2019lm_mag.pkl
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -6,14 +6,14 @@ repos:
       - id: end-of-file-fixer
       - id: trailing-whitespace
   - repo: https://github.com/python/black
-    rev: 20.8b1
+    rev: 22.3.0
     hooks:
       - id: black
         pass_filenames: true
-        exclude: nmma|nb|.ipynb_checkpoints|data|dev|letsencrypt|logs|dask-worker-space
+        exclude: nb|.ipynb_checkpoints|data|dev|letsencrypt|logs|dask-worker-space
   - repo: https://gitlab.com/pycqa/flake8
     rev: 3.8.4
     hooks:
       - id: flake8
         pass_filenames: true
-        exclude: nmma|nb|.ipynb_checkpoints|data|dev|letsencrypt|logs|dask-worker-space|__init__.py
+        exclude: nb|.ipynb_checkpoints|data|dev|letsencrypt|logs|dask-worker-space|__init__.py
diff --git a/LICENSE b/LICENSE
diff --git a/LICENSE.md b/LICENSE.md
diff --git a/README.md b/README.md
@@ -2,3 +2,12 @@ NMMA
 ====
 
 A pythonic library for probing nuclear physics and cosmology with multimessenger analysis
+
+
+Please check our official documentation here: https://nuclear-multimessenger-astronomy.github.io/nmma/
+
+
+How to contribute: https://nuclear-multimessenger-astronomy.github.io/nmma/contributing.html
+
+
+A tutorial about how to produce simulations of lightcurves is given in nmma/tutorials/tutorial-lightcurve_simulation.ipynb
diff --git a/doc/Cluster_Resources.md b/doc/Cluster_Resources.md
@@ -0,0 +1,37 @@
+## Cluster Resources (Expanse)
+
+One might also want to submit bulk jobs while using NMMA. Here, we have 
+included an example script for job submission (called as jobscript.sh) in SLURM. This job was submitted on SDSC's
+Expanse (ACCESS) cluster:
+
+	#!/bin/bash
+	#SBATCH --job-name=gw170817_gp_test.job
+	#SBATCH --output=logs/gw170817_gp_test.out
+	#SBATCH --error=logs/gw170817_gp_test.err
+	#SBATCH -p compute
+	#SBATCH --nodes=1
+	#SBATCH --ntasks-per-node=10
+	#SBATCH --mem=249325M
+	#SBATCH --time=00:30:00
+	#SBATCH --mail-type=ALL
+	#SBATCH --mail-user= your_full_email
+	#SBATCH -A <<project*>>
+	#SBATCH --export=ALL
+	module purge
+	module load sdsc
+	module load cpu/0.15.4 gcc/10.2.0 intel-mpi/2019.8.254
+	source /home/username/anaconda3/bin/activate nmma_env
+	export PATH=$PATH:/home/username/anaconda3/lib/
+	export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/username/anaconda3/lib/
+	mpiexec -n 10 light_curve_analysis --model Me2017 --outdir outdir --label injection --prior /home/username/nmma/priors/Me2017.prior --tmin 0.1 --tmax 20 --dt 0.5 --error-budget 1 --nlive 512 --Ebv-max 0 --injection /home/username/nmma/injection.json --injection-num 0 --injection-outfile outdir/lc.csv --generation-seed 42 --filters u,g,r,i,z,y,J,H,K --plot --remove-nondetections  
+
+To submit the job, run:
+
+	sbatch jobscript.sh
+
+To check the job allotment, you can run:
+
+	squeue -u username
+
+
+Test runs on other clusters are currently in progress. Further examples on other cluster resources will be subsequently added.
diff --git a/doc/GW-EM-resampling.md b/doc/GW-EM-resampling.md
@@ -0,0 +1,53 @@
+## Connecting Electromagnetic Signals to Source Properties 
+
+In NMMA, it is possible to use the results from GW inferences together with kilonova inferences or joint KN+GRB afterglow inferences to get estimates on the binary properties. 
+A binary system can be a binary neutron star (BNS) merger or a neutron-star-black-hole (NSBH) merger. We can connect observed electromagnetic signals to potential 
+source properties via phenomenological relations, i.e., via fits based on numerical-relativity relations, see [(Pang et al. 2022)](https://arxiv.org/pdf/2205.08513.pdf) and Refs. therein for further details. 
+
+For estimating the source properties, the following input files are required:
+
+* `EMsamples` - is the posterior sample file from a previous Bayesian inference (e.g. Kilonova+GRB inference) on electromagnetic (EM) signals,
+* `EMprior` - is the prior file that was used for the EM inference 
+* `EOS` - number of equation of state files which will be used in the resampling
+* `EOSpath` - path to the folder of all EOS files
+* `GWsamples` - some fiducial randomly generated posterior samples for masses $m_{1,2}$ , chirp mass $\mathcal{M}_c$, mass ratio $q$, luminosity distance $D_L$, and EOS samples,
+* `GWprior` - a prior file for gravitational wave sources
+
+## Estimating BNS properties 
+
+Here, we take the observed gamma-ray burst [GRB211211A](https://arxiv.org/abs/2204.10864) as an example and assume that associated electromagnetic signals 
+originated from a BNS merger. For this signal, a joint inference (kilonova + GRB) can be carried out and will provide you with the required `EMsamples`.
+For the `GWsamples` input file, we need to generate some fiducial dummy GW samples. A script for the generation can be found [here](https://github.com/nuclear-multimessenger-astronomy/nmma/blob/main/example_files/tools/gwem_resampling/gwsamples_generation.py).
+The GW sample generation is based on the EOS set `15nsat_cse_uniform_R14` which can be found on [Zenodo](https://zenodo.org/record/6106130#.Y1pdM9JBxhG). 
+The `EMprior` file is the same as used for the KN+GRB inference (see [priors](https://github.com/nuclear-multimessenger-astronomy/nmma/tree/main/priors)) and the GWprior file should be adjusted to the GWsamples. 
+
+Finally, we can use this command:
+
+    gwem_resampling --outdir outdir --GWsamples example_files/tools/gwem_resampling/GWsamples.dat --GWprior priors/GWBNS.prior --EMsamples example_files/tools/gwem_resampling/GRB211211A_posterior_samples.dat --EOSpath 15nsat_cse_uniform_R14/macro/ --Neos 5000 --EMprior priors/Bu2019lm_TrPi2018GRB211211A.prior --nlive 1024  
+
+The result will be a posterior file containing information on:
+
+* the chirp mass $\mathcal{M}_c$,
+* mass ratio $q$,
+* Tidal deformability $\tilde{\Lambda}$,
+* dynamical ejecta mass error $α$,
+* dynamical ejecta mass fraction $ζ$, and
+* equation of state EOS
+
+A corner plot is shown below:
+
+![GWEMcornerplot](images/corner_samples.png)
+
+
+## Estimating NSBH properties 
+
+In order to estimate the properties of a NSBH system, you need to adjust the `GWsamples` and `GWprior` file accordingly and run the resampling
+with the argument `withNSBH` (otherwise, you will run for a BNS system).
+
+    gwem_resampling --outdir outdir --GWsamples example_files/tools/gwem_resampling/GWsamples_NSBH.dat --GWprior priors/GWNSBH.prior --withNSBH --EMsamples example_files/tools/gwem_resampling/GRB211211A_NSBH_posterior_samples.dat --EOSpath 15nsat_cse_uniform_R14/macro/ --Neos 5000 --EMprior priors/Bu2019nsbh_TrPi2018_GRB211211A.prior --nlive 1024  
+
+
+
+
+
+
diff --git a/doc/combined_analysis.md b/doc/combined_analysis.md
@@ -0,0 +1,91 @@
+## Perform combined analyses
+
+NMMA is capable of performing combined analyses to constrain the neutron star equation of state (EOS) and Hubble Constant. In the following, we will take as an example the EOS analysis.
+
+### Generate a simulation set
+
+First of all, you need to create an output directory, this output will host all the data that will be used to constrain the EOS.
+
+	mkdir -p ./output
+
+Running the following command line will generate a json file (injection.json)  with the BILBY processing of compact binary merging events. We take here binaries of type BNS, NSBH is also an option. This injection contents a simulation set of parameters : luminosity_distance, log10_mej_wind, KNphi, inclination_EM, KNtimeshift, geocent_time for the Bu2019lm model. This creates an injection.json file in the ./output directory.
+
+	nmma_create_injection --prior-file ./priors/Bu2019lm.prior --eos-file ./example_files/eos/ALF2.dat --binary-type BNS -f ./output/injection --n-injection 100 --original-parameters --extension json
+
+
+### lightcurve posterior
+
+EMdata will house the posteriors of the electromagnetic data you will produce: in particular the lc.csv (./example_files/csv_lightcurve/outdir/macroeventID, where macroeventID in range(0, 100)) lightcurves. We now compute posteriors using NMMA on this simulated set of 100 events, of which we assume a fraction is detectable by ZTF. The result can be find at  ./output/EMdata
+
+	for macroeventID in {0..99}
+
+	do
+	  mkdir -p ./output/EMdata/outdir/$macroeventID/
+	  light_curve_analysis --model Bu2019lm --svd-path ./svdmodels --interpolation_type tensorflow --outdir ./output/EMdata/outdir/$macroeventID --label injection_Bu2019lm --prior ./priors/Bu2019lm.prior --tmin 0 --tmax 7 --dt 0.5 --error-budget 1.0 --nlive 256 --Ebv-max 0 --injection ./output/injection.json --injection-num $macroeventID --injection-detection-limit 22,22,22 --injection-outfile ./output/EMdata/outdir/$macroeventID/lc.csv --generation-seed 42 --filters g,r,i --ztf-sampling --ztf-uncertainties --plot --remove-nondetections --photometry-augmentation --photometry-augmentation-filters u,g,r,i,z,y,J,H,K --photometry-augmentation-N-points 100
+	done
+
+
+### Download GW posteriors
+
+The gravitational wave samples can be can be downloaded at https://zenodo.org/record/6045029#.YgZzwITMKV5. At this link there are simulated posteriors for a number of gravitational-wave waveform models, here, we take the PhenDNRTv2 files.
+This only  concern  the PhenDNRTv2 files on this link. These we can directly download by using this command line:
+
+Create an outdir directory to put GW data that you will upload.
+
+	mkdir -p ./output/GWdata/outdir
+
+Go to the GWdata directory
+
+	cd ./output/GWdata/outdir
+
+Running the next command line:
+
+	xargs -n 1 curl -# -O < ../../../example_files/zenodo/gw_posteriors.txt
+
+or use this one :
+
+	for url in `cat ../../../example_files/zenodo/gw_posteriors.txt`
+	do
+	  curl -#  -O $url
+	done
+
+
+The gw_posteriors.txt contains a list of all links to the PhenDNRTv2 files. When the download is complete, return to the main directory
+
+        cd ../../..
+
+### Download the EOS repository
+
+All of the NMMA EOS simulation sets are kept in a separate github repository here:
+https://github.com/diettim/NMMA
+
+For this particular example, we have put the simulation set in Zenodo here:
+https://zenodo.org/record/6094691#.YgwA8YTMI5k
+
+### EoS from GW + EM
+
+This command line combines the EOS measurements for each simulation. As stated above, we assume only a fraction is detectable by ZTF (based on simulations of the associated kilonova brightnesses). The indices of the 26 detectable events are {0,  3,  5,  7,  8, 10, 12, 13, 14, 15, 17, 19, 21, 22, 23, 24, 26, 27, 28, 31, 32,34, 36, 37, 38, 39}.
+
+	for macroeventID in 0 3 5 7 8 10 12 13 14 15 17 19 21 22 23 24 26 27 28 31 32 34 36 37 38 39
+
+	do
+	  mkdir -p ./output/GW_EMdata/$macroeventID/
+	  gwem_resampling --outdir ./output/GW_EMdata/$macroeventID --EMsamples ./output/EMdata/outdir/$macroeventID/injection_Bu2019lm_posterior_samples.dat --GWsamples  ./output/GWdata/outdir/inj_PhD_posterior_samples_$macroeventID.dat --EOS ./example_files/eos/eos_sorted --nlive 8192 --GWprior ./priors/aligned_spin.priors --EMprior ./priors/EM.prior --total-ejecta-mass --Neos 5000
+
+	done
+
+
+### A combined EOS analysis
+
+We provide a helper function to combine the EOS results.
+
+First of all create a foder to put the final data about EoS:
+
+	mkdir -p ./output/Figures
+
+Then run the last one command line
+
+	 combined_EOS --outdir ./output/Figures --label ZTF --gwR14trend ./example_files/ --GWEMsamples ./output/GW_EMdata --detections-file ./example_files/csv_lightcurve/detectable.txt --EOS-prior ./example_files/eos/EOS_sorted_weight.dat --EOSpath ./example_files/eos/eos_sorted  --pdet ./example_files/eos/pdet_of_Mmax.dat --R14_true 11.55 --Neos 5000 --seed 42  --cred-interval 0.95
+
+
+This should return a EoS plot, R14_trend_GW_EM_ZTF.pdf, and  GW_EM_R14trend_ZTF.dat  at  ./output/Figures.
diff --git a/doc/contributing.md b/doc/contributing.md
@@ -1,12 +1,12 @@
-# How to Contribute
+## How to Contribute
 
-## License
+### License
 
 NMMA is released under the MIT license, which means that you are allowed to modify the code for your own purposes, as long as you retain our [copyright notice](https://github.com/nuclear-multimessenger-astronomy/nmma/blob/master/LICENSE.txt).
 
 However, we would love to grow the NMMA community, and integrate improvements directly into our [code repository on GitHub](https://github.com/nuclear-multimessenger-astronomy/nmma).
 
-## Including your changes
+### Including your changes
 
 To make a code contribution to the project, follow these steps (which
 are outlined in more detail in [this GitHub
@@ -26,7 +26,7 @@ For a more detailed explanation of the open contribution process, see
 the [scikit-image contributors' guide](http://scikit-image.org/docs/stable/contribute.html).
 We follow a very similar process; some guidance follows below.
 
-## Bug Reports
+### Bug Reports
 
 While we appreciate code changes, it is also very helpful simply to
 know when NMMA does not function correctly.  Please [file any
@@ -39,7 +39,7 @@ If possible, provide:
 2. A minimal way to reproduce the problem you see; these can be either
    a set of instructions, or a script.
 
-## Process guidelines
+### Process guidelines
 
 Because many developers work on NMMA, and PRs sometimes come in
 at a rapid pace, we have guidelines to streamline review and