changes to the docstrings

docs/api.rst

@@ -1,10 +1,12 @@
 API documentation
 =================
 
+.. automodapi:: tessilator.fixedconstants
 .. automodapi:: tessilator.tessilator
-.. automodapi:: tessilator.lc_analysis
+.. automodapi:: tessilator.maketable
+.. automodapi:: tessilator.aperture
 .. automodapi:: tessilator.contaminants
+.. automodapi:: tessilator.detrend_cbv
+.. automodapi:: tessilator.lc_analysis
 .. automodapi:: tessilator.makeplots
-.. automodapi:: tessilator.maketable
-.. automodapi:: tessilator.fixedconstants
 

docs/command_line.rst

@@ -1,4 +1,5 @@
 .. _command_line:
+
 Running tessilator from the command line
 ========================================
 
@@ -27,7 +28,7 @@ such that all files in sector 'N' are in a subdirectory named './sectorNN/', whe
 NN represents a two digit integer (i.e., a trailing zero for sectors 1-9). To run
 this program simply enter::
 
-    $ run_tess_sector
+    $ run_tess_sectors
 
 Note that these programs will only run after a set of input parameters are passed.
 This can be done by providing a set of :ref:`command-line arguments<input_parameters>`
@@ -39,10 +40,10 @@ An example of running the tessilator using the supplied shell scripts would be::
 
    $ run_tess_cutouts 1 1 1 cutouts targets.csv
 
-| The command-line arguments here instruct the tessilator to:
-| (a) Calculate the flux from contaminating sources
-| (b) Run the tessilator analysis for neighbouring contaminants
-| (c) Produce a plot for each lightcurve
-| (d) Group the output files by the name ``cutouts``
-| (e) Use ``cutout_targets.csv`` as the input table containing the targets
+The command-line arguments here instruct the tessilator to:
+a. Calculate the flux from contaminating sources
+b. Run the tessilator analysis for neighbouring contaminants
+c. Produce a plot for each lightcurve
+d. Group the output files by the name ``cutouts``
+e. Use ``cutout_targets.csv`` as the input table containing the targets
 

docs/input_parameters.rst

@@ -1,21 +1,22 @@
 .. _input_parameters:
+
 Required input parameters
 =========================
 
 Whether running from the command line or as part of a module, the input parameters are:
 
-| 1) "flux_con": the toggle for applying the contamination calculation
+1) "flux_con": the toggle for applying the contamination calculation
        (yes=1, no=0).
 
-| 2) either "LC_con", if using the cutout functions or "sector_num" if sectors are needed.       
-* "LC_con" determines if lightcurve/periodogram analyses should be carried out for neighbouring contaminants (yes=1, no=0).
-* "sector_num" prompts the user to enter the sector number needed. If command line arguments are not used, the program will ask if a specific Camera and CCD is needed (1=yes, 0=no). If not required, the whole sector is analysed. If this is a command line argument, if the user enters just the sector number (maximum 2 digits) the whole sector will be analysed, and if the Camera and CCD number are given right after the sector number with no spaces, then a specific Camera and CCD configuration will be used. E.G: if "sector_num = 8", the entire sector 8 is analysed, whereas if "sector_num = 814" then the program will analyse only Camera 1 and CCD 4 in sector 8.
+2) either "LC_con", if using the cutout functions or "sector_num" if sectors are needed.       
+    * "LC_con" determines if lightcurve/periodogram analyses should be carried out for neighbouring contaminants (yes=1, no=0).
+    * "sector_num" prompts the user to enter the sector number needed. If command line arguments are not used, the program will ask if a specific Camera and CCD is needed (1=yes, 0=no). If not required, the whole sector is analysed. If this is a command line argument, if the user enters just the sector number (maximum 2 digits) the whole sector will be analysed, and if the Camera and CCD number are given right after the sector number with no spaces, then a specific Camera and CCD configuration will be used. E.G: if "sector_num = 8", the entire sector 8 is analysed, whereas if "sector_num = 814" then the program will analyse only Camera 1 and CCD 4 in sector 8.
 
-| 3) "make_plots" gives the user the option to make plots (yes=1, no=0)
+3) "make_plots" gives the user the option to make plots (yes=1, no=0)
 
-| 4) "file_ref" is a string expression used to reference the files produced.
+4) "file_ref" is a string expression used to reference the files produced.
 
-| 5) "t_filename" is the name of the input file required for analysis.
+5) "t_filename" is the name of the input file required for analysis.
 
 Table input
 -----------

docs/install.rst

@@ -1,9 +1,10 @@
 .. _install:
+
 Installing tessilator software
 ==============================
 
-Direct installation using pip 
------------------------------
+1) **Direct installation using pip**
+------------------------------------
 
 The easiest way to install the last released version of tessilator is to install it with pip into your Python environment::
 
@@ -12,8 +13,8 @@ The easiest way to install the last released version of tessilator is to install
 This should automatically install your dependencies, if you don't have them yet, in particular `numpy <https://numpy.org>`_ 
 and `matplotlib <https://matplotlib.org>`_.
 
-Cloning the GitHub repository
------------------------------
+2) **Cloning the GitHub repository**
+------------------------------------
 
 Users with GitHub accounts are able to clone the repository directly. This can be done interactively using the GitHub online
 interface. `The link to the GitHub repository is here. <https://github.com/alexbinks/TESSilator>`_ Alternatively the repository

docs/notes.rst

@@ -1,44 +1,48 @@
-.. notes:
+.. _notes:
+
 tessilator notes
 ================
 
 This page lists provides notes about the methods and underlying assumptions made in the production of the tessilator.
 
 Aperture photometry
 -------------------
-1. The zero-point TESS magnitude :math:`G_0 = 20.44 \pm 0.05` mag (`TESS instrument handbook <https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/active-missions/tess/_documents/TESS_Instrument_Handbook_v0.1.pdf>`_)
-2. When performing aperture photometry, there are some cases where the background is actually brighter than source. Background-subtracted flux values, whilst not physically meaningful, may still be used to generate a lightcurve which can be used in Lomb-Scargle periodogram analyses. However, there are rare cases in which the raw flux (before background-subraction) values are negative. These are likely due to overestimated smear corrections when a bright/saturated star is very close to the top of the CCD and accumulates a large charge count in the smear correction. For the latter case we are unable to use these data.
-3. The aperture radius can either be a set to a constant value (default 1.0 pixels), or is determined from an algorithm which uses the relative brightness of surrounding pixels. The latter case is set as default, whereby if the median brightness of the 8 pixels that neighbour the pixel centered on the target is greater than (a default value of) 0.25, the aperture radius is increased by one pixel, and the next outer square ring of pixel values are assessed, and so on, until either the criteria is not matched, or a maximum (default) radius of 3 pixels is reached.
-4. The background flux is calculated using two annuli, with default values of 6.0 and 8.0 pixels centered on the target. To reduce computational time, the modal flux used to represent the background is calculated using Pearson's empirical relationship.
+1. The zero-point TESS magnitude :math:`G_0 = 20.44 \pm 0.05` mag (`TESS instrument handbook <https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/active-missions/tess/_documents/TESS_Instrument_Handbook_v0.1.pdf>`_).
+2. When performing aperture photometry, there are some cases where the background is actually brighter than source. These background-subtracted negative flux values, whilst not physically meaningful, may still be used to generate a lightcurve which can be used in Lomb-Scargle periodogram analyses. However, there are rare cases in which the raw flux (before background-subraction) values are negative. These are likely due to overestimated smear corrections when a bright/saturated star is very close to the top of the CCD and accumulates a large charge count in the smear correction. For the latter case we are unable to use these data.
+3. The aperture radius can either be a set to a constant value (default 1.0 pixels), or determined from an algorithm which uses the relative brightness of surrounding pixels. The latter case is set as default, whereby if the median brightness of the 8 pixels that surround the pixel centered on the target is greater than (a default value of) 0.25, the aperture radius is increased by one pixel, and the next outer square ring of pixel values are assessed, and so on, until either the criteria is not matched, or a maximum (default) radius of 3 pixels is reached.
+4. The background flux is calculated using two annuli, with default radii of 6.0 and 8.0 pixels. To reduce computational time, the modal flux used to represent the background is calculated using the Pearson Mode Skewness formula.
 
 Contamination
 -------------
-1. The FWHM of the TESS point-spread function, :math:`T_{\rm PSF}` is assumed to be 0.65 pixels. This value is used for the calculation of flux contamination and was calculated by fitting Gaussians to 15 `TESS Pixel Response Functions <https://heasarc.gsfc.nasa.gov/docs/tess/observing-technical.html>`_ in a number of sector/camera/ccd configurations. The mean FWHM values from these fits are :math:`0.65 \pm 0.05`. In future releases we may implement a more rigorous method to calculate :math:`T_{\rm PSF}` by interpolating X/Y positions in these images, however, we are confident that :math:`T_{\rm PSF} = 0.65` is fairly constant and has little effect on the contamination calculation and no effect on the aperture photometry.
+1. The full-width half-maximum of the TESS point-spread function, :math:`T_{\rm PSF}` is assumed to be 0.65 pixels. This value is used for the calculation of flux contamination and was calculated by fitting Gaussians to 15 `TESS Pixel Response Functions <https://heasarc.gsfc.nasa.gov/docs/tess/observing-technical.html>`_ in a number of sector/camera/ccd configurations. The mean FWHM values from these fits are :math:`0.65 \pm 0.05`. In future releases we may implement a more rigorous method to calculate :math:`T_{\rm PSF}` by interpolating X/Y positions in these images, however, we are confident that :math:`T_{\rm PSF} = 0.65` is fairly constant and has little effect on the contamination calculation and no effect on the aperture photometry.
 2. The default parameters to identify potential contaminants are a search radius of 5-pixels (:math:`105''`) of the target, and limited to sources with Gaia RP-band magnitudes brighter than 3 magnitudes fainter than the target. Whilst these are optimized values designed for a quick and efficient search, they are set as keywords, and be easily modified. The RP-band was chosen because the passband response function is very similar to that of TESS.
 3. There are three basic parameters calculated from the flux contamination procedures. These are the number of potential contaminants within the given search radius, and the (logarithmic) ratio of flux captured within the aperture radius from both `all` contaminants, and the strongest contaminator, compared to that of the target. The analytic calculation to measure flux contamination is given by equation 3b-10 from `Biser & Millman (1965) <https://books.google.co.uk/books?id=5XBGAAAAYAAJ>`_.
 4. If the user selects to run a periodogram analysis on possible contaminants, the tessilator will select (as a default), the 10 most significant contaminanting sources identified in the contamination functions. The idea is to run the tessilator on each contaminant and return a flag representing how likely it is that the period measured for the target, is in fact, that of a contaminating source. This may be modified in future releases to return a probability of unreliability rather than a discrete flag.
 
 Lightcurve analyses
 -------------------
-1. The lightcurve uses the background-subtracted counts, calculated from the aperture photmetry routines, as the input for flux. Whilst magnitudes could be used, negative counts have indeterminate magnitude values and can't be used as data points.
-2. In general, there are nine steps used to normalise, detrend and clean the lightcurves:
-   a. Normalise the original flux points, by dividing these by the median value
-   b. Split the lightcurve into groups, where groups are separated by time factors much larger than the median observing cadence.
-   c. Remove very sparse groups, typically those with less than 50 datapoints.
-   d. Run the detrending process to pass to the cleaning function. This step uses Aikake Information Criteria to determine whether the normalised fluxes should be detrended with a linear fit, and if so, whether the groups should be detrended individually, or as a full lightcurve. Parabolas, or higher-order polynomials, are not used because these may remove genuine periodic signals from the lightcurves. 
-   e. Remove flux outliers from the edges of each group.
-   f. Remove highly scattered fluxes from the edges of each group.
-   g. Remove extreme outliers.
-   h. Divide the detrended fluxes from each group by the median flux of (qualified datapoints) the group.
-   i. Store all results from the full lightcurve analysis to a dictionary.
+The lightcurve uses the background-subtracted counts, calculated from the aperture photmetry routines, as the input for flux. Whilst magnitudes could be used, negative counts have indeterminate magnitude values and can't be used as data points. In general, there are nine steps used to normalise, detrend and clean the lightcurves:
+
+a. Normalise the original flux points, by dividing them by the median flux.
+b. Split the lightcurve into groups, where groups are separated by time factors much larger than the median observing cadence.
+c. Remove very sparse groups, typically those with less than 50 datapoints.
+d. Run the detrending process to pass to the cleaning function. This step uses Aikaike Information Criteria to determine whether the normalised fluxes should be detrended with a linear fit, and if so, whether the groups should be detrended individually, or as a full lightcurve. Parabolas, or higher-order polynomials, are not used because these may remove genuine periodic signals from the lightcurves. 
+e. Remove flux outliers from the edges of each group.
+f. Remove highly scattered fluxes from the edges of each group.
+g. Remove extreme outliers.
+h. Divide the detrended fluxes from each group by the median flux of (qualified datapoints) the group.
+i. Store all results from the full lightcurve analysis to a dictionary.
+
 During the (removal) steps c, e, f and g procedure, a record of datapoints that are kept or rejected is made, allowing users to assess the amount of data loss. The reader is referred to the doc strings for additional information on each of these individual steps.
 
 Accounting/Correcting for systematic noise
 ------------------------------------------
 Systematic noise is unfortunately an all persisting issue with TESS lightcurves. Like any other space-based mission, TESS suffers from non-uniform instrument degradation, and the photometer is sensitive to scattered lunar and diurnal light. The tessilator has 3 different methods to (attempt to) remove systematic noise.
-1. Applying the so-called Cotrending Basis Vectors (CBVs) provided by the `TESS SPOC <https://archive.stsci.edu/tess/bulk_downloads/bulk_downloads_cbv.html>`_. For a given sector, CCD and camera, the CBV vectors are designed to capture the lightcurve pattern caused by noise, as a function of X/Y pixel position. By taking the dot-product of the CBVs with the lightcurve flux, a noise-cleaned lightcurve is, in principle, produced. The tessilator comes with an option to apply the CBV corrections, however, user discretion is strongly advised. Overfitting CBVs can often lead to worse results than the original, which seems to become more problematic for targets with faint/low signal-to-noise lightcurves. If the CBV correction is selected, the results of the original and CBV-corrected lightcurve are assessed using a point scoring system, based on variety of lightcurve quality parameters (e.g., intrinsic scatter, number of outliers, etc). If the CBV lightcurve scores highest, a periodogram analysis is made for both lightcurves, and another point system based on the periodogram quality parameters is made to determine which set of results to use.
+
+1. Applying the so-called Cotrending Basis Vectors (CBVs) provided by the `TESS SPOC <https://archive.stsci.edu/tess/bulk_downloads/bulk_downloads_cbv.html>`_. For a given sector, CCD and camera, the CBV vectors are designed to capture the lightcurve pattern caused by noise, as a function of X/Y pixel position. By taking the dot-product of the CBVs with the lightcurve flux, a noise-cleaned lightcurve is, in principle, produced. The tessilator comes with an option to apply the CBV corrections, however, user discretion is strongly advised since overfitting can often lead to worse results than the original. This seems to become more problematic for targets with faint/low signal-to-noise lightcurves. If the CBV correction is selected, the results of the original and CBV-corrected lightcurve are assessed using a point scoring system, based on variety of lightcurve quality parameters (e.g., intrinsic scatter, number of outliers, etc). If the CBV lightcurve scores highest, a periodogram analysis is made for both lightcurves, and another point system based on the periodogram quality parameters is made to determine which set of results to use.
 2. Dividing the lightcurve by a lightcurve which is generated by taking the median flux (at a given timestamp) of all lightcurves for targets with no clear periodic signal in a given sector/CCD/camera and within a certain Gaia G-band magnitude range. This method is still under construction, and is only partially available for certain sector/CCD/camera/G-magnitude configurations. Also, this method does not account for the X/Y pixel response, which may vary across the surface of each CCD.
 3. Dividing the lightcurve by a lightcurve which is generated by taking the median flux (at a given timestamp) of all lightcurves from neighbouring, potential contaminant sources (identified during the contamination procedure) that have no clear periodic signal. Whilst this method gives a better approximation of the noise in the vicinity of the target, the method depends on a local region with enough identified potential contaminant sources, and does not filter out any targets, because there are generally not enough data to fill higher parameter spaces.
+
 All 3 methods have their limitations, and it is very much down to the user to decide if they want to apply any of these corrections. If a noise correction is applied, it is vital that only ONE of the above methods is used. 
 
 Lomb-Scargle periodograms