This repo houses the code used to produce the work "dm2gal: Mapping Dark Matter to Galaxies with Neural Networks". This paper has been accepted to the NeurIPS Machine Learning and the Physical Sciences Workshop 2020.
Maps of cosmic structure produced by galaxy surveys are one of the key tools for answering fundamental questions about the Universe. Accurate theoretical predictions for these quantities are needed to maximize the scientific return of these programs. Hydrodynamic simulations are one of the most powerful techniques to accomplish this; unfortunately, these simulations are very computationally expensive. Alternatively, gravity-only simulations are cheaper, but do not predict the locations and properties of galaxies in the cosmic web. In this work, we use convolutional neural networks to paint galaxy stellar masses on top of the dark matter field generated by gravity-only simulations. Our model outperforms the state-of-the-art benchmark model and allows the generation of fast and accurate models of the observed galaxy distribution.
We use data from the state-of-the-art magneto-hydrodynamic simulation IllustrisTNG100-1, and its gravity-only counterpart, IllustrisTNG100-1_DM, at present time (z=0). Those simulations contain, among other things, the position and mass of all particles in the simulations. Each simulation also contains a catalogue of dark matter halos with their properties (e.g. mass and position). We use the Cloud-in-Cell (CIC) mass assignment interpolation scheme to construct the 3D stellar mass and dark matter mass fields from the particle positions and masses of the hydrodynamic and gravity-only simulations, respectively. Since galaxies are expected to reside in dark matter subhalos, we facilitate the training of the network by using also a 3D field with the mass-weighted subhalo field, that we construct from the gravity-only simulation. The 3D fields span a volume of (75 h^{-1} Mpc)^3 (one Mpc corresponds to 3.26 million light-years) and they contain 2048^3 voxels.
To reproduce this project, please see the following steps:
| Step | Description | File Path | File Name |
|---|---|---|---|
| 1 | Download raw snapshot(s) from the IllustrisTNG site and make .hdf5 files | /dat/illustris_tng/ | - |
| 2 | Prepare .npy files of dark matter and stellar mass density field input-target pairs | /dat/sampling | make_samples.py |
| 3 | Create mass bins used for weighted sampling of distribution for even training | /src | dataloader.py |
| 4 | Train models | /src | predict_stellar_mass.py & submit_jobs.py |
| 5 | Generate the held-out test field with the trained model | /src | test_cube.py |
| 6 | Prepare the stellar mass density field from the HOD algorithm | /src | hod.py |
| 7 | Calculate and plot power spectra and bispectra [2] | /src | create_plots.py |
| 8 | Further analysis on the outputs | /Notebooks | - |
[1] With Python 3.7.7, PyTorch 1.5.0, and Pytorch-Lightning 0.7.5. See env.yaml for more details
| Power Spectrum | Bispectrum |
|---|---|
 |
 |
| Density Fields |
|---|
 |
https://github.com/siyucosmo/ML-Recon
https://github.com/jhtyip/From-Dark-Matter-to-Galaxies-with-Convolutional-Neural-Networks