DJA x SourceXtractor++

Introduction

This project looks into implementing SourceXtractor++ on the DAWN JWST Archive (DJA). The DJA is a repository of public JWST galaxy data reduced and ready for science. This project aims at expanding the DJA with model fitting and more precise measurements on the different sets of photometric images.

Installation

This code can be installed via PyPI: pip install dja_sepp --upgrade You will still need to download the config folder and store it somewhere accessible. Having it outside the package will allow you to make changes to the models and to the parameters.

Usage

There are two possibilites to run SourceXtractor++ in this project: in Jupyter Notebooks, or via scripts that can even start AWS EC2 instances autonomously.

Scripts

This part allows to run SourceXtractor++ on a complete field from the DJA, with Sérsic and Bulge+Disk model, and then combine them in a complete catalog.

• On AWS: Autonomously start AWS EC2 instances and run the scripts directly in them. This requires an access to EC2 instances, a S3 bucket to save the files, and a launch template with an AMI image of an instance with SExtractor, PSFEx and SourceXtractor++ installed.

  0. Setup (decompress images from DJA, calculate PSF, tile the field) : autorun_setup.sh
  1. Run SE++ on the tiles (one time with sersic_rg4 model, one time with B+D) : autorun_full.sh. This script will start one EC2 instance for every tile generated before. It is therefore necessary to know how many tiles have been created (a mosaic image is generated by the autorun_setup.sh script). Some tiles can fail to run in two scenarios :
     • Failure during SE++ segmentation (percentage stuck before 100%, CPU usage (can be monitored on the EC2 dashboard) stuck at a very low percentage (6% for c6a.4xlarge)). This issue is similar to a known issue with SE++ and doesn't have a solution. This tile can't be run.
     • Crash during SE++ modeling (instance stops, catalog and model images are generated, but not the residual images). The solution is generally to run SE++ with a larger instance (autorun_tile.sh allows to choose the instance type).
  2. Merge tiles (catalog and images, one time for each model) : 07_MergeTiles.ipynb. This step can be executed on an AWS EC2 instance (m5d.4xlarge) for faster files download and more available RAM.
  3. Merge DJA and SE++ catalogs : 06.2_DJA-SE++CatMerge.ipynb
  4. Analysis and science plots : 06.3_DJA-SE++Analysis.ipynb

• Locally: The scripts can be run locally for a few images or tiles. Take a look at the AWS scripts above for that as they're just starting an AWS instance and then running scripts from this repository on them. The scripts should be fairly easy to understand and adapt to your uses if required.

Jupyter notebooks (old way)

0. (optional) Cutouts in the image : 00.2_Cutout.ipynb
1. Detect point-like sources with the F200W band + create the PSF for every band : 03.1_SingleBand-PSF.ipynb
2. Run SourceXtractor++ in detection mode : 04_SE++.ipynb
3. Compare the results to the DJA : 06_Validation.ipynb
4. View and analyse the results : 05_Analysis.ipynb

Dependencies

• SExtractor
• PSFEx
• SourceXtractor++
• Astropy

Utilization with AWS EC2

One of the goal of this project is also to run SourceXtractor++ on AWS EC2 to make it run in the background and faster than on your personal computer.

A benchmark is currently being run to find how to set the parameters in SE++ and AWS EC2 to minimize runtime depending on the size of your images and the number of bands wanted.

To know how to use easily AWS EC2 in combination with Jupyter, please refer to this series of tutorial.