Spectrum Fitting¶
In this first basic example we will perform a basic spectral analysis of the Crab nebula using the public H.E.S.S. data release (easily accessable trough gammapy) and follow the gammapy documentation on spectral analysis closely so we can compare the results.
Let’s get started by importing all the relevant stuff.
[1]:
%matplotlib inline
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import Angle, SkyCoord
from astromodels.core.model import Model
from astromodels.core.units import get_units
from astromodels.functions import Log_parabola, Log_uniform_prior, Uniform_prior
from astromodels.sources import PointSource
from gammapy.data import DataStore
from gammapy.datasets import Datasets, SpectrumDataset
from gammapy.makers import (
ReflectedRegionsBackgroundMaker,
SafeMaskMaker,
SpectrumDatasetMaker,
)
from gammapy.maps import MapAxis, RegionGeom, WcsGeom
from gammapy.modeling import Fit
from gammapy.modeling.models import LogParabolaSpectralModel, SkyModel
from regions import CircleSkyRegion
from threeML import BayesianAnalysis, plot_spectra
from threeML.data_list import DataList
from gammapy_plugin.converter import AstromodelConverter
from gammapy_plugin.gammapy_like import GammapyLike
from gammapy_plugin.test.utils import get_close
Let’s start by laoding the relevant data
[2]:
datastore = DataStore.from_dir("$GAMMAPY_DATA/hess-dl3-dr1/")
obs_ids = [23523, 23526, 23559, 23592]
observations = datastore.get_observations(obs_ids)
We perform a standard gammapy-workflow of creating a dataset
setting as target
excluding the target region
creating our energy axis
[3]:
target_position = SkyCoord(ra=83.63, dec=22.01, unit="deg", frame="icrs")
on_region_radius = Angle("0.11 deg")
on_region = CircleSkyRegion(center=target_position.galactic, radius=on_region_radius)
exclusion_region = CircleSkyRegion(
center=SkyCoord(183.604, -8.708, unit="deg", frame="galactic"),
radius=0.5 * u.deg,
)
skydir = target_position.galactic
geom = WcsGeom.create(
npix=(150, 150), binsz=0.05, skydir=skydir, proj="TAN", frame="galactic"
)
exclusion_mask = ~geom.region_mask([exclusion_region])
energy_axis = MapAxis.from_energy_bounds(
0.1, 40, nbin=10, per_decade=True, unit="TeV", name="energy"
)
energy_axis_true = MapAxis.from_energy_bounds(
0.05, 100, nbin=20, per_decade=True, unit="TeV", name="energy_true"
)
geom = RegionGeom.create(region=on_region, axes=[energy_axis])
dataset_empty = SpectrumDataset.create(geom=geom, energy_axis_true=energy_axis_true)
We now have everything to create our makers and run them to get the result the gammapy-plugin can use - the DL4 Dataset. In this example we use a ReflectedRegionsBackgroundMaker to estimate the background.
[4]:
dataset_maker = SpectrumDatasetMaker(
containment_correction=True, selection=["counts", "exposure", "edisp"]
)
bkg_maker = ReflectedRegionsBackgroundMaker(exclusion_mask=exclusion_mask)
safe_mask_maker = SafeMaskMaker(methods=["aeff-max"], aeff_percent=10)
datasets = Datasets()
for obs_id, observation in zip(obs_ids, observations):
dataset = dataset_maker.run(dataset_empty.copy(name=str(obs_id)), observation)
dataset_on_off = bkg_maker.run(dataset, observation)
dataset_on_off = safe_mask_maker.run(dataset_on_off, observation)
datasets.append(dataset_on_off)
Now we have Dataset objects stored away in a Datasets object.
Let’s continue with the threeML steps: First lets choose and set up a model - we first initalize the spectral shape and assign it to the PointSource so that the units are already set in astromodels.
[5]:
logp = Log_parabola()
ps = PointSource(
source_name="crab",
ra=target_position.ra.deg,
dec=target_position.dec.deg,
spectral_shape=logp,
)
logp.K.prior = Log_uniform_prior(lower_bound=1e-22, upper_bound=1e-18) # this is in keV
logp.K = 1e-11 * u.Unit("TeV-1 cm-2 s-1")
logp.piv = 1e9 # this is in keV
logp.piv.free = False
logp.alpha.prior = Uniform_prior(lower_bound=-5, upper_bound=5)
logp.alpha = -2
logp.beta.prior = Uniform_prior(lower_bound=-2, upper_bound=2)
logp.beta = 0.1
model = Model(ps)
Let’s take a look at it:
[6]:
display(model)
| N | |
|---|---|
| Point sources | 1 |
| Extended sources | 0 |
| Particle sources | 0 |
Free parameters (3):
| value | min_value | max_value | unit | |
|---|---|---|---|---|
| crab.spectrum.main.Log_parabola.K | 0.0 | 0.0 | 100000.0 | keV-1 s-1 cm-2 |
| crab.spectrum.main.Log_parabola.alpha | -2.0 | None | None | |
| crab.spectrum.main.Log_parabola.beta | 0.1 | None | None |
Fixed parameters (3):
(abridged. Use complete=True to see all fixed parameters)
Properties (0):
(none)
Linked parameters (0):
(none)
Independent variables:
(none)
Linked functions (0):
(none)
Perfect we have one PointSource with a Log_parabola spectrum. Please be aware of the different definitions of a Logparabolas in astromodels and gammapy.
Now we convert this model. Alternatively we can skip that part and let the GammapyLike deal with it by not supplying the converted_model in the set_model() method.
[7]:
conv = AstromodelConverter(model)
Let’s create our “hess” plugin and already set the sources (optional) to “crab”. When setting the datsets we can either stick with the default “individual” mode or we can use the “stacked” mode - which will stack the dataset for us. That is equivalent to the gammapy joint vs. stacked analysis.
[8]:
gl = GammapyLike("hess", sources="crab")
gl.set_datasets(datasets, "stacked")
gl.set_model(model, converted_model=conv)
Nice :) We now initialize an BayesianAnalysis that links the model to the data and handles the sampling. We use emcee as a sampler which is always installed when installing 3ML.
[9]:
ba = BayesianAnalysis(model, DataList(gl))
ba.set_sampler("emcee")
ba.sampler.setup(n_iterations=400, n_burn_in=100)
ba.sample()
res = ba.results
res
Maximum a posteriori probability (MAP) point:
| result | unit | |
|---|---|---|
| parameter | ||
| crab.spectrum.main.Log_parabola.K | (4.36 -0.24 +0.20) x 10^-20 | 1 / (keV s cm2) |
| crab.spectrum.main.Log_parabola.alpha | -2.35 -0.13 +0.16 | |
| crab.spectrum.main.Log_parabola.beta | (1.8 -0.7 +0.9) x 10^-1 |
Values of -log(posterior) at the minimum:
| -log(posterior) | |
|---|---|
| hess | 39.877646 |
| total | 39.877646 |
Values of statistical measures:
| statistical measures | |
|---|---|
| AIC | -72.711814 |
| BIC | -69.867782 |
| DIC | -73.895942 |
| PDIC | 2.919632 |
[9]:
<threeML.analysis_results.BayesianResults at 0x7a14a1b16ba0>
Perfect! It seems like it succeeded - let’s take a look at the parameter distributions:
[10]:
res.corner_plot()
[10]:
Since threeML v2.6.0 we can also use display_spectrum_model_counts to see the modeled and observed count rate and the residuals. Please be aware that we calculate the Li & Ma significance.
The units of the count rates might be a bit unusual for IACT astronomers/astrophysicists ;)
[11]:
from threeML.io.plotting.post_process_data_plots import display_spectrum_model_counts
display_spectrum_model_counts(ba)
[11]:
Please be aware that this is not equivalent to the following, as the 3ML one is plotting the total model as differential count flux (including background) while the gammapy .plot_fit() is dispalying the predicted signal excess. Additionally the residuals for 3ML are given as deviance resiudals in units of \(\sigma\).
[12]:
ax_spectrum, ax_residuals = gl.datasets[0].plot_fit()
ax_spectrum.set_xlim(0.1, 40)
plt.show()
Comparing it to the results and plots from the `gammapy documentation <https://docs.gammapy.org/2.1/tutorials/analysis-1d/spectral_analysis.html#fit-quality-and-model-residuals>`__ it looks reasonably well :)
And finally let’s take a look at the spectrum with its 68% Confidence Interval:
[13]:
plot_spectra(
ba.results,
energy_unit="TeV",
ene_min=0.1 * u.TeV,
ene_max=100 * u.TeV,
num_ene=100,
flux_unit=u.Unit("TeV m-2 s-1"),
)
[13]: