Skip to content

Commit

Permalink
Add files via upload
Browse files Browse the repository at this point in the history
  • Loading branch information
@dev-geof
dev-geof authored Sep 13, 2024
1 parent d6a3732 commit 69315fe
Showing 1 changed file with 129 additions and 37 deletions.
166 changes: 129 additions & 37 deletions docs/code/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@

<li class="md-tabs__item">
<a href="../input/" class="md-tabs__link">
Input Data Format
Input Data Preparation
</a>
</li>

Expand All @@ -227,8 +227,8 @@


<li class="md-tabs__item">
<a href="../configuration/" class="md-tabs__link">
Configuration File
<a href="../training/" class="md-tabs__link">
Training
</a>
</li>

Expand All @@ -239,8 +239,8 @@


<li class="md-tabs__item">
<a href="../preparation/" class="md-tabs__link">
Data Preparation
<a href="../validation/" class="md-tabs__link">
Performance Assessment
</a>
</li>

Expand All @@ -251,8 +251,8 @@


<li class="md-tabs__item">
<a href="../training/" class="md-tabs__link">
Training
<a href="../configuration/" class="md-tabs__link">
Configuration
</a>
</li>

Expand All @@ -263,8 +263,20 @@


<li class="md-tabs__item">
<a href="../validation/" class="md-tabs__link">
Validation
<a href="../troubleshooting/" class="md-tabs__link">
Troubleshouting
</a>
</li>







<li class="md-tabs__item">
<a href="../copyrights/" class="md-tabs__link">
Copyrights and Credits
</a>
</li>

Expand Down Expand Up @@ -369,17 +381,44 @@
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>

<li class="md-nav__item">
<a href="#get-the-code" class="md-nav__link">
Get the code
<a href="#source-code-and-installation" class="md-nav__link">
Source Code and Installation
</a>

</li>

<li class="md-nav__item">
<a href="#installation" class="md-nav__link">
Installation
<a href="#software-implementation" class="md-nav__link">
Software Implementation
</a>

<nav class="md-nav" aria-label="Software Implementation">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#main-tools-and-utilities" class="md-nav__link">
Main Tools and Utilities
</a>

</li>

<li class="md-nav__item">
<a href="#entry-points" class="md-nav__link">
Entry Points
</a>

</li>

<li class="md-nav__item">
<a href="#modularity" class="md-nav__link">
Modularity
</a>

</li>

</ul>
</nav>

</li>

</ul>
Expand Down Expand Up @@ -414,7 +453,7 @@

<li class="md-nav__item">
<a href="../input/" class="md-nav__link">
Input Data Format
Input Data Preparation
</a>
</li>

Expand All @@ -428,8 +467,8 @@


<li class="md-nav__item">
<a href="../configuration/" class="md-nav__link">
Configuration File
<a href="../training/" class="md-nav__link">
Training
</a>
</li>

Expand All @@ -443,8 +482,8 @@


<li class="md-nav__item">
<a href="../preparation/" class="md-nav__link">
Data Preparation
<a href="../validation/" class="md-nav__link">
Performance Assessment
</a>
</li>

Expand All @@ -458,8 +497,8 @@


<li class="md-nav__item">
<a href="../training/" class="md-nav__link">
Training
<a href="../configuration/" class="md-nav__link">
Configuration
</a>
</li>

Expand All @@ -473,8 +512,23 @@


<li class="md-nav__item">
<a href="../validation/" class="md-nav__link">
Validation
<a href="../troubleshooting/" class="md-nav__link">
Troubleshouting
</a>
</li>










<li class="md-nav__item">
<a href="../copyrights/" class="md-nav__link">
Copyrights and Credits
</a>
</li>

Expand Down Expand Up @@ -507,17 +561,44 @@
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>

<li class="md-nav__item">
<a href="#get-the-code" class="md-nav__link">
Get the code
<a href="#source-code-and-installation" class="md-nav__link">
Source Code and Installation
</a>

</li>

<li class="md-nav__item">
<a href="#installation" class="md-nav__link">
Installation
<a href="#software-implementation" class="md-nav__link">
Software Implementation
</a>

<nav class="md-nav" aria-label="Software Implementation">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#main-tools-and-utilities" class="md-nav__link">
Main Tools and Utilities
</a>

</li>

<li class="md-nav__item">
<a href="#entry-points" class="md-nav__link">
Entry Points
</a>

</li>

<li class="md-nav__item">
<a href="#modularity" class="md-nav__link">
Modularity
</a>

</li>

</ul>
</nav>

</li>

</ul>
Expand All @@ -542,23 +623,34 @@

<h1 id="the-code">The Code</h1>
<p><img alt="Final State Transformer Architecture" src="../img/architecture.png" /></p>
<h2 id="get-the-code">Get the code</h2>
<p>Start by cloning the <code>final-state-transformer</code> repository. If you benefit from an ssh key for GitLab, you can proceed with:
<h2 id="source-code-and-installation">Source Code and Installation</h2>
<p>The source code for the FST toolkit is hosted on GitHub. Users can access the repository by visiting the final-state-transformer GitHub page. The repository contains all the scripts, utilities and documentation needed to understand and utilize the toolkit. To install the FST toolkit, follow the following steps to ensure smooth setup. Before installation, ensure you have the following software installed on your system: Python 3.9.13 or higher, Git, pip (Python package installer).</p>
<p>Start by cloning the FST repository from GitHub to your local machine. Open your terminal and execute the following commands:
<div class="highlight"><pre><span></span><code>git clone git@github.com:dev-geof/final-state-transformer.git
cd final-state-transformer
</code></pre></div></p>
<h2 id="installation">Installation</h2>
<p>First ensure that you have Python 3.9.13 or higher version installed on your system. You can check your Python version by running the following command in your terminal:
<div class="highlight"><pre><span></span><code>python --version
</code></pre></div>
If Python is not installed or is an older version, please download and install the latest version from the official Python website.</p>
<p>You can install <code>final-state-transformer</code> within a virtual environement. The <code>venv</code> environement is a lightweight solution for creating virtual environement in python. Docker images are not yet available. To create a new virtual environement:
<p>It is recommended to install the FST toolkit within a virtual environment. The python "venv" environment is a lightweight solution to install the necessary dependencies.
<div class="highlight"><pre><span></span><code>python3 -m venv env
source env/bin/activate
python -m pip install -e . -r requirements.txt
</code></pre></div></p>
<p>To install <code>final-state-transformer</code> and its dependencies you can use the following command:
<div class="highlight"><pre><span></span><code>python -m pip install -e . -r requirements.txt
</code></pre></div></p>
<h2 id="software-implementation">Software Implementation</h2>
<p>The FST software features a modular python architecture designed to facilitate the implementation, the training and the evaluation of Transformer encoder networks. The core structure is organized into three main tools and several utilities.</p>
<h3 id="main-tools-and-utilities">Main Tools and Utilities</h3>
<p>The main tools provide essential functionalities for various stages of the machine learning pipeline. These stages include data preparation, model training, and evaluation. Data preparation tools ensure that raw data is cleaned, formatted, and features are engineered for optimal input into the model. Training tools encompass scripts and utilities for initiating and monitoring the training process, optimizing hyper- parameters, and supporting distributed training across CPUs or GPUs. Evaluation tools offer methods for calculating performance metrics, validating models, and interpreting model outputs.</p>
<p>The utilities support the main tools and enhance their functionality. The Model utilities define and manage the architecture and parameters of Transformer-based models, including saving and loading models. Plotting utilities provide tools for visualizing training progress, model performance, and generating interactive plots. Data utilities handle efficient loading, batching, and management of datasets. Configu- ration utilities manage settings and parameters through configuration files, ensuring consistent experiment setups. Finally, Callbacks offer functions for logging progress, early stopping, and checkpointing models during training, improving the training process’s efficiency and reliability.</p>
<h3 id="entry-points">Entry Points</h3>
<p>The setup configuration specifies entry points for the tree main tool scripts, enabling users to execute data preparation, model training, and evaluation directly from the command line. This facilitates streamlined workflows and simplifies the execution of common tasks. The toolkit leverages YAML configuration files to allow users to pilot and manage all these scripts and tools efficiently. These configuration files store settings for data paths, hyper-parameters, training schedules, model configurations, and other experimen- tal parameters. The configuration system parses these files, ensuring that the entire pipeline—from data preparation to model evaluation adheres to the specified settings.</p>
<div class="highlight"><pre><span></span><code>usage: fst-preparation [-h] [--configfile CONFIGFILE]
fst-training [-h] [--configfile CONFIGFILE]
fst-validation [-h] [--configfile CONFIGFILE]

optional arguments:
-h, --help Show this help message and exit
--configfile CONFIGFILE YAML configuration file path
</code></pre></div>
<h3 id="modularity">Modularity</h3>
<p>The FST toolkit is designed with extensibility in mind, allowing users to easily adapt and extend the framework for different high-energy physics tasks. The modular design enables the integration of custom models, additional data processing techniques, and new evaluation metrics without altering the core structure of the toolkit.</p>



Expand Down

0 comments on commit 69315fe

Please sign in to comment.