Release 0.4 (#114)

* Industrial Release 0.4

---------

Co-authored-by: v1docq 
Co-authored-by: vadim_potemkin <aspirantvadim@gmail.com>
Co-authored-by: valer1435 <valp998@gmail.com>

README.rst

@@ -81,7 +81,7 @@
 
 Fedot.Ind - это автоматизированный фреймворк машинного обучения,
 разработанный для решения промышленных задач, связанных с прогнозированием
-временных рядов, классификацией, регрессией и обнаружением аномалий.
+временных рядов, классификацией и регрессией.
 Он основан на `AutoML фреймворке FEDOT`_ и использует его функциональность
 для композирования и тюнинга пайплайнов.
 
@@ -127,15 +127,26 @@ Fedot.Ind предоставляет высокоуровневый API, кот
 - Метод ``FedotIndustrial.get_metrics()`` оценивает качество прогнозов с использованием выбранных метрик.
 
 В качестве источников входных данных можно использовать массивы NumPy или
-объекты DataFrame из библиотеки Pandas. В данном случае, `x_train`,
-`y_train` и `x_test` представлены в виде объектов `numpy.ndarray()`:
+объекты DataFrame из библиотеки Pandas. В данном случае, ``x_train / x_test`` и ``y_train / y_test`` – ``pandas.DataFrame()`` и ``numpy.ndarray`` соответственно:
 
 .. code-block:: python
 
-    model = Fedot(task='ts_classification', timeout=5, strategy='quantile', n_jobs=-1, window_mode=True, window_size=20)
-    model.fit(features=x_train, target=y_train)
-    prediction = model.predict(features=x_test)
-    metrics = model.get_metrics(target=y_test)
+    dataset_name = 'Epilepsy'
+    industrial = FedotIndustrial(problem='classification',
+                                 metric='f1',
+                                 timeout=5,
+                                 n_jobs=2,
+                                 logging_level=20)
+
+    train_data, test_data = DataLoader(dataset_name=dataset_name).load_data()
+
+    model = industrial.fit(train_data)
+
+    labels = industrial.predict(test_data)
+    probs = industrial.predict_proba(test_data)
+    metrics = industrial.get_metrics(target=test_data[1],
+                                     rounding_order=3,
+                                     metric_names=['f1', 'accuracy', 'precision', 'roc_auc'])
 
 Больше информации об использовании API доступно в `соответствующей секции <https://fedotindustrial.readthedocs.io/en/latest/API/index.html>`__ документации.
 

README_en.rst

@@ -80,7 +80,7 @@
 
 
 Fedot.Ind is a automated machine learning framework designed to solve industrial problems related
-to time series forecasting, classification, regression, and anomaly detection. It is based on
+to time series forecasting, classification, and regression. It is based on
 the `AutoML framework FEDOT`_ and utilizes its functionality to build and tune pipelines.
 
 
@@ -125,14 +125,26 @@ It provides a fit/predict interface:
 - ``FedotIndustrial.get_metrics()`` estimates the quality of predictions using selected metrics.
 
 NumPy arrays or Pandas DataFrames can be used as sources of input data.
-In the case below, ``x_train``, ``y_train`` and ``x_test`` are ``numpy.ndarray()``:
+In the case below, ``x_train / x_test``, ``y_train / y_test`` are ``pandas.DataFrame()`` and ``numpy.ndarray`` respectively:
 
 .. code-block:: python
 
-    model = Fedot(task='ts_classification', timeout=5, strategy='quantile', n_jobs=-1, window_mode=True, window_size=20)
-    model.fit(features=x_train, target=y_train)
-    prediction = model.predict(features=x_test)
-    metrics = model.get_metrics(target=y_test)
+    dataset_name = 'Epilepsy'
+    industrial = FedotIndustrial(problem='classification',
+                                 metric='f1',
+                                 timeout=5,
+                                 n_jobs=2,
+                                 logging_level=20)
+
+    train_data, test_data = DataLoader(dataset_name=dataset_name).load_data()
+
+    model = industrial.fit(train_data)
+
+    labels = industrial.predict(test_data)
+    probs = industrial.predict_proba(test_data)
+    metrics = industrial.get_metrics(target=test_data[1],
+                                     rounding_order=3,
+                                     metric_names=['f1', 'accuracy', 'precision', 'roc_auc'])
 
 More information about the API is available in the `documentation <https://fedotindustrial.readthedocs.io/en/latest/API/index.html>`__ section.
 

benchmark/abstract_bench.py

@@ -1,6 +1,12 @@
 import logging
 import os
 
+import matplotlib
+from matplotlib import pyplot as plt
+
+from fedot_ind.api.main import FedotIndustrial
+from fedot_ind.tools.loader import DataLoader
+
 
 class AbstractBenchmark(object):
     """Abstract class for benchmarks.
@@ -48,6 +54,32 @@ def run(self):
         """
         raise NotImplementedError()
 
+    def evaluate_loop(self, dataset, experiment_setup: dict = None):
+        matplotlib.use('TkAgg')
+        train_data, test_data = DataLoader(dataset_name=dataset).load_data()
+        experiment_setup['output_folder'] = experiment_setup['output_folder'] + f'/{dataset}'
+        model = FedotIndustrial(**experiment_setup)
+        model.fit(train_data)
+        prediction = model.predict(test_data)
+        model.save_best_model()
+        model.save_optimization_history()
+        model.plot_operation_distribution(mode='each')
+        model.plot_fitness_by_generation()
+        plt.close('all')
+        model.shutdown()
+        return prediction.squeeze(), model.predict_data.target
+
+    def finetune_loop(self, dataset, experiment_setup: dict = None):
+        train_data, test_data = DataLoader(dataset_name=dataset).load_data()
+        experiment_setup['output_folder'] = experiment_setup['output_folder'] + f'/{dataset}'
+        tuning_params = experiment_setup['tuning_params']
+        del experiment_setup['tuning_params']
+        model = FedotIndustrial(**experiment_setup)
+        model.load(path=experiment_setup['output_folder'] + '/0_pipeline_saved')
+        model.finetune(train_data, tuning_params=tuning_params)
+        prediction = model.finetune_predict(test_data)
+        return prediction, model.predict_data.target
+
     def collect_results(self, output_dir):
         """Collect the results of the benchmark.