Merge branch 'v0.4.0' into main

CHANGELOG.md

@@ -4,6 +4,30 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## [0.4.0] - 2022-02-08
+
+### Added
+- In DiffusionFitBase.fit function added new keyword argument `threshold_on` which sets how the peak and tail signal values for step 1 thresholding and termination of the fitting procedure computed. It has string options image, filter, line, and fit.
+- New optional flag `-threshold-on` for the CLI script to set the `threshold_on` keyword argument to the fit function.
+- Class variables `_threshold_on_options` and `_center_options` in ModeBase and some error checking to make sure the `center` and `threshold_on` keyword options  to `__init__` and `fit` are valid options.
+- New fitting class `AnisotropicGaussianFit` for fitting cases where the diffusion coefficient is different along the major x and y axes, so the diffusion cloud is not isotropic.
+- New optional flag `--anisotropic-gaussian` for the CLI script to use the `AnisotropicGaussianFit` class for fitting along with changes in the CLI script to adjust the DataFrame used to print values to the screen after all fitting is complete.
+- In the CLI script an additional output of the DataFrame storing the diffusion coefficients and R-squared values for each image file to a csv file named diffusion_fitting/diffusion_fitting_summary.csv.
+- In the CLI script and additional output of the input arguments to a text file diffusion_fitting/diffusionfit_commandline_args.txt.   
+- Additional documentation on the programmatic and command line usage in the README.  
+- New keyword argument for the `DiffusionFitBase.__init__` function `subtract_background` which sets whether a background image should be computed and subtracted during the fitting.  
+- New optional flag `--no-background` for the CLI script to set models not to try and compute or subtract background from the images. This corresponds to the `subtract_background` keyword argument to  `DiffusionFitBase.__init__`. 
+
+### Changed
+- In the export_to_csv function of DiffusionFitBase the addition of the linear fit column was changed to use the DataFrame `assign` function to get rid of the SettingWithCopyWarning. The column name was also changed from `Linear-Fit` to `LinearFit`.
+- In the DiffusionFitBase.fit function the variables named signal were changed to peak to match the peak-to-tail thresholding naming during step 1 fitting.
+- Updated the version numbers in the README.
+
+### Fixed
+- The issue with early termination due to lower than expected signal near the stimulation zone can be addressed using the newly added `threshold_on` option (`-threshold-on` from CLI). For example, using `threshold_on='fit'` will ensure smooth signal and tail regions consistent with the assumed intensity model.  
+- In `measure.akaike_ic` the incorrect addition of the parametric term and max maximum_loglikelihood was fixed to be subtraction. This was causing an issue in the estimation of the time resolved diffusion coefficient.
+
+
 ## [0.3.0] - 2021-12-20
 
 ### Added
@@ -28,7 +52,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - Changed the way the thresholding is done after step 1 fitting. Now it terminates when mean(peak-region) <= mean(tail-region) + peak-to-tail * std(tail-region) and uses radial selections from the image instead of computing values from the Line-ROI.
 - The required argument signal_to_noise in __main__.py arguments was changed to optional keyword argument -peak-to-tail with default value of 3. This is used in the new step 1 thresholding.
 - In DiffusionFitBase class the member function `model` was changed to `intensity_model`, `linear_model` was changed to `diffusion_model`, and the `_fit_step1` and `fit_step2` functions were changed to `_fit_intensity` and `_fit_diffusion`, respectively.
-- Updated the initial description, What's new in, License, and Documentation and Usage sections in the README. 
+- Updated the initial description, What's new in, License, and Documentation and Usage sections in the README.
 
 
 ## [0.2.0] - 2021-12-20

README.md

@@ -11,11 +11,11 @@ Additionally, we have extended the framework to provide a model for fitting data
 
 ### What's new in
 
-#### version 0.3.0
- * PointClarkFit fitting class for fitting the fluorescent signal of receptor-based peptide sensors during peptide point source diffusion.
- * Funtionality to estimate the loss rate of the diffusing species.
- * Additional options when running **diffusion-fit** with it's command line interface.
- * Functionality to export fitting data to csv and tiff files.
+#### version 0.4.0
+ * New options define how the peak and tail signal are computed when applying the thresholding for images to determine whether to terminate the fitting.
+ * New AnisotropicGaussianFit class for fitting images with anisotropic diffusion along the major (x,y) axes; the x and y axes can have different diffusion coefficients.
+ * Additional output file from the CLI to store the measured diffusion coefficients and some other fitting diagnostics.
+ * Additional output file from the CLI to store the input arguments used for the diffusionfit run.  
 
 See the [CHANGELOG](CHANGELOG.md) for additional details.  
 
@@ -30,7 +30,8 @@ See the [CHANGELOG](CHANGELOG.md) for additional details.
  3. [Change Log](#change-log)
  4. [Documentation and Usage](#documentation-and-usage)
      1. [Quick Overview](#quick-overview)
-     2. [Examples](#examples)
+     2. [Programmatic use](#programmatic-use)
+     3. [Command line Use](#command-line-use)
  5. [Contact](#contact)
  6. [Citing](#citing)  
 
@@ -55,11 +56,17 @@ Note that `diffusion-fit` has the following core dependencies:
    * [Numba](https://numba.pydata.org/)
 
 ### pip install
-You can install `diffusionfit` with `pip` sourced from the GitHub repo:
+You can install `diffusionfit` version 0.4.0 with `pip` sourced from the GitHub repo:
+
+##### with git installed:
 ```
-pip install -e git+https://github.com/NTBEL/diffusion-fit#egg=diffusionfit
+pip install git+https://github.com/NTBEL/diffusion-fit@v0.4.0
 ```
 
+##### without git installed:
+```
+pip install https://github.com/NTBEL/diffusion-fit/archive/refs/tags/v0.4.0.zip
+```
 ### Manual install
 First, download the repository. Then from the `diffusion-fit` folder/directory run
 ```
@@ -90,49 +97,96 @@ See: [CHANGELOG](CHANGELOG.md)
 # Documentation and Usage
 
 ### Quick Overview
-Currently, `diffusionfit` defines the **GaussianFit** and **PointClarkFit** classes,
-```python
-from diffusonfit import GaussianFit, PointClarkFit
-```
-which define models to fit the 2D fluorescence intensity distribution and estimate the diffusion coeffient of
-molecules released from a point-like source.
-
-  * **GaussianFit** assumes the fluorescence signal comes directly from the diffusing species and that the the diffusion cloud has an isotropic Gaussian distribution that expands in width over time as in the point source diffusion model.
+Currently, `diffusionfit` defines the **GaussianFit**, **PointClarkFit**, and **AnisotropicGaussianFit** classes, which define models to fit the 2D fluorescence intensity distribution and estimate the diffusion coefficient of
+molecules released from a point-like source:
+  * **GaussianFit** assumes the fluorescence signal comes directly from the diffusing species and that the diffusion cloud has an isotropic Gaussian distribution that expands in width over time as in the point source diffusion model.
   * **PointClarkFit** assumes the fluorescent signal comes from the diffusing species binding to a fluorescent receptor-based sensor such as the dLight or kLight peptide sensors. The diffusion is still assumed to follow the isotropic point source model but the resulting fluorescent signal follows the Clark equation for receptor-response with the diffusing molecule as the ligand and the fluorescent sensor as the receptor.
+  * **AnisotropicGaussianFit** is also based on the point source diffusion model and assumes the fluorescence signal comes directly from the diffusing species, but allows the diffusion cloud to have an anisotropic Gaussian distribution that expands in width at different rates along the x and y dimensions over time. This model decouples diffusion in the x and y directions allowing the two dimensions to have different diffusion coefficients.
 
-In both cases, the 2D fluorescence distribution of each image in the time lapse of fluorescence images is fitted with the 2D intensity model and then the diffusion coefficient is extracted in a second linear fitting step. This two-step fitting procedure was adpated from the methods used to analyze integrative optical imaging data as described in Nicholson and Tao 1993 [doi: 10.1016/S0006-3495(93)81324-9](https://doi.org/10.1016/S0006-3495(93)81324-9) and Hrabe and Hrabetova 2019 [doi: 10.1016/j.bpj.2019.08.031](https://doi.org/10.1016/j.bpj.2019.08.031).
+In all cases, the 2D fluorescence distribution of each image in the time lapse of fluorescence images is fitted with the 2D intensity model and then the diffusion coefficient is extracted in a second linear fitting step. This two-step fitting procedure was adapted from the methods used to analyze integrative optical imaging data as described in Nicholson and Tao 1993 [doi: 10.1016/S0006-3495(93)81324-9](https://doi.org/10.1016/S0006-3495(93)81324-9) and Hrabe and Hrabetova 2019 [doi: 10.1016/j.bpj.2019.08.031](https://doi.org/10.1016/j.bpj.2019.08.031).
 
 `diffusionfit` can be used both programatically or from the command line. The command line interface simplifies the process for quickly initiating analysis of multiple image files (tiff files), and it automatically generates plot images of the fits and exports the fitting parameters to a csv file for further analysis for each image file (dumped into a new output fold named diffusion_fitting). However, programmatic use offers more flexibility for generating customized fitting and analysis workflows.   
 
-### Examples
-
-#### Programmatic use
+### Programmatic use
+The fitting models can be directly imported from `diffusionfit`:
+```python
+from diffusonfit import GaussianFit, PointClarkFit, AnisotropicGaussianFit
+```
+A fitting model can be initialized with the time lapse image file (in tiff format) and parameters describing the image as in this example:
 ```python
 # Units:
 # timestep (s)
 # pixel_width (micron)
 # stimulation_radius (micron)
 gfit = GaussianFit('images.tif',
                     stimulation_frame=50, timestep=0.6,
-                    pixel_width=0.99, stimulation_radius=30)
-Dstar = gfit.fit(verbose=True, step1_threshold=3)                    
+                    pixel_width=0.99, stimulation_radius=30,
+                    center='image', subtract_background=True)                   
+```
+After initialization, the fitting process is performed by calling the `fit` function which returns an estimate of the effective diffusion coefficient:
+```python
+Dstar = gfit.fit(verbose=True, step1_threshold=3)
 ```
-To see the plots corresponding to each of the two fitting steps:
+The `display_image_fits` and `display_linear_fit` functions can be used to generate and output plots corresponding to each of the two fitting steps:
 ```python
 # Step 1
 gfit.display_image_fits(saveas='step1_fits.png')
 # Step 2
 gfit.display_linear_fit(saveas='step2_linfit.png')
 ```
-#### Command line use
+
+### Command line use
+
+When called from the command line diffusionfit will analyze and fit all time lapse image files in the current directory/folder using the same fitting settings. Image files are expected to be in the multi-frame tiff format. If different image files require different setting they can be moved into separate folders and diffusion fit can be called with the appropriate input arguments for each case.
+
+#### Input arguments
+There are four required positional arguments:
 ```
-python -m diffusionfit 0.25 2.486 50 75 --loss-rate
+python -m diffusionfit [timestep] [pixel_size] [stim_frame] [d_stim]
 ```
-To show the usage and the full set of command line options:
+which are:
+  * **timestep** - the time interval between the images of the time lapse given in seconds.
+  * **pixel_size** - the width of each image pixel in microns.
+  * **stim_frame** - the frame where release of the diffusing species (e.g., by photostimulation) occurs in the time lapse. Images before this point are used to calculate the background for background subtraction.
+  * **d_stim** - the diameter in microns of the stimulation zone when a tornado scan is used for photorelease. This zone is excluded from fitting due to the possibility of thermal damage or other effects during photorelease. If you don't need or want to account for the stimulation zone (e.g., analyzing IOI data) then you can set the value to `0`.
+
+Additionally, there a many optional input flags that can be used to tune the fitting and its outputs. They are:
+  * `-peak-to-tail [peak_to_tail]` : Set the peak/tail threshold during step 1 fitting for terminating the fitting analysis.
+  * `-center [center]` : Set how the center of the diffusion cloud is determined. Options are: image - (default), use the center pixel location of the image. intensity - centroid of intensity after stimulation. y,x - a specific pixel location.
+  * `--ignore-threshold`  :  The fitting model will ignore thresholding used to determine when during the time lapse to terminate the fitting early due to low signal in an image.
+  * `-end-frame [end_frame]` : You can specify the maximum frame of the time lapse to include in the analysis. It should be larger than stim_frame. This can used in combination with the --ignore-threshold option to define an exact stopping point for the fitting in cases such as when you want to fit a pre-defined time interval or you have already pre-analyzed the images to determine when to terminate the fitting analysis.
+  * `--write-tif` : An ImageJ compatible tiff time lapse image file will be written out with the 2D fit to the intensity of each image included in the analysis. This can be used for diagnostic purposes to visually inspect how the fits compare to the original images.
+  * `--time-resolved` : The fitter will compute an estimate of time-resolved diffusion coefficient and output a corresponding plot. This is adapted from the TR-IOI approach described in Hrabe and Hrabetova 2019 [doi: 10.1016/j.bpj.2019.08.031](https://doi.org/10.1016/j.bpj.2019.08.031).
+  * `--loss-rate` : The fitter will compute and output an estimate of the loss rate for the diffusing species from the decay of the maximum intensity fitting parameter using a model for its time course that is derived from the point source model.
+  * `--point-clark` : Fit the intensity using the Point-Clark model (PointClarkFit). The Point-Clark model is derived from a combination of the point source diffusion model and the Clark equation for receptor-response (Hill equation with Hill coefficient of 1). This model should provide a more accurate estimate of peptide diffusion where the intensity comes from a fluorescent receptor-based sensor.
+  * `-threshold-on [threshold_on]` : This option can be used to set how the peak and tail tail values for thresholding the fitting are computed. Options are: image - (default), compute on the (background subtracted) image. filter - compute on a Gaussian filtered version of the image. line - compute on the line ROI taken along the minimum image dimension. fit - compute on the fit of the image to the intensity model.
+  * `--anisotropic-gaussian` : Use the Anisotropic Gaussian model (AnisotropicGaussianFit) to fit the diffusion intensity for cases where diffusion along the x and y dimensions is different.
+  * `--no-background` : Don't compute or subtract any background from the images when fitting the intensity. This option can be used in cases where images are already pre-processed to remove the background flurorescence or when the input has been converted to the relative difference (dF/F) in flurorescence intensity (relevant for peptide sensors).     
+
+The usage amd full set of command line options can be accessed at the command line by using the `--help` flag:
 ```
 python -m diffusionfit --help
 ```
 
+#### Outputs
+The command line call to diffusionfit will generate a new folder `diffusion_fitting` where it will write output files.
+
+Two summary files are written out:
+  * **diffusionfit_commandline_args.txt** : A text file that stores the input parameters passed to diffusionfit. This can be used to recreate the fitting again later with the same settings or for archival purposes.
+  * **diffusion_fitting_summary.csv** : csv file of summary values for the fitting. It contains the image file name and corresponding mean value of root mean squared errors from step 1 fitting and its standard deviation, the effective time, the estimated diffusion coefficient(s), and the R-squared from the linear fitting in step 2. It will also contain the loss rate if the optional `--loss-rate` flag is used.
+
+The following output files are generated for each image file ([image_file_name].tif):  
+  * **[image_file_name]_step_1_fits.csv** : csv file with the fitting and diagnostic parameters from the intensity model as fitted to each image in the time lapse.
+  * **[image_file_name_step_2_fits.csv]** : csv file with the fitting parameters and linear fit used in step 2.
+  * **[image_file_name]_step1.png** : An image file with plots comparing the original image to its 2D fit, a line ROI comparison, and a Ring ROI comparison. To keep the size of the image somewhat reasonable only a subset of the time lapse are shown.
+  * **[image_file_name]_step2.png** : An image file with the plot(s) of the step 2 linear fitting.
+
+#### Example
+diffusionfit can be used from the command line as in the following example:
+```
+python -m diffusionfit 0.25 2.486 50 75 --loss-rate
+```
+
 ------
 
 # Contact

diffusionfit/__init__.py

@@ -1 +1 @@
-from .diffusionfit import GaussianFit, PointClarkFit
+from .diffusionfit import GaussianFit, PointClarkFit, AnisotropicGaussianFit