analysis plugin development
- Introduction
- Write general analysis plugin
- Write yara-based analysis plugin
- Add tag based on analysis result
FACT Analysis plug-ins can work on either the binaries of an object or on results of other plug-ins, as well as on both. In general a plugin is executed on each single object which is extracted from the uploaded firmware container. If a plugin should be run only on the outer container, this can be specified in the plugin code. This is explained in 1. This allows to run the same set of analysis on each firmware layer. Results of each layer can be propagated by adding a summary to the plugin, thus making partial results visible at the firmware level. The ability to use results of other plugins, additionally allows to create incremental analysis workflows.
FACT detects plug-ins automatically as long as they are stored in src/plugins/analysis. A plug-in consists of folders and files following the following template.
.
├── __init__.py
├── install.sh [OPTIONAL]
├── code
│ ├── __init__.py
│ └── PLUGIN_NAME.py
├── internal [OPTIONAL]
│ └── ADDITIONAL_SOURCES_OR_CODE
├── signatures [OPTIONAL: JUST FOR YARA PLUGINS]
├── test [OPTIONAL]
│ ├── __init__.py
│ ├── test_PLUGIN_NAME.py
│ └── data
│ └── SOME DATA FILES TO TEST
└── view
└── PLUGIN_NAME.html [OPTIONAL]
With only the __init__.py files and the code/PLUGIN_NAME.py being mandatory for the plugin to work, let's have a quick look on the other files:
install.sh when provided is automatically triggered by FACT's installation script.
A basic install.sh shall look like this.
#!/usr/bin/env bash
# change cwd to current file's directory
cd "$( dirname "${BASH_SOURCE[0]}" )" || exit 1
echo "------------------------------------"
echo " SOME MEANINGFUL TITLE "
echo "------------------------------------"
[YOUR CODE HERE]
exit 0❗ Do not forget the "exit 0" at the end! If you miss it and something goes wrong, the installer will end-up in an infinite loop! The cd at the start makes shure you are in the plugin directory.
This optional file can can contain a customized view utilizing Jinja2 template features. In general a standard template is used to show the results of an analysis. This standard view just generates a table from the result dictonary including all entries. :warning: Due to the internal FACT processing, this directory should not contain more than one file. That means you cannot add dependencies, like .js or .css files to your view.
ℹ️ To get started you can copy our minimal functional hello world plug-in to start your own development.
Plugins may implement completly genuine analysis techniques, use available python libraries or merely wrap existing third party code.
As input a plugin is presented with a FileObject.
During analysis a plugin result should be added to that object.
Finally the object is returned again.
The relevant parts of the FileObject class (processed_analysis and binary) are described in the comments inside the code template.
Let's have a look on the plugin template:
This is the actual plug-in code. The following should mostly be self explanatory and can be used to write your own plug-in.
from analysis.PluginBase import AnalysisBasePlugin
class AnalysisPlugin(AnalysisBasePlugin):
'''
Some Description
'''
NAME = 'PLUGIN_NAME'
DESCRIPTION = 'a short description of the plugin'
MIME_BLACKLIST = [optional list of MIME types that should be ignored by the plugin]
DEPENDENCIES = [LIST_OF_PLUG-IN_NAMES_THAT_THIS_PLUGIN_RELYS ON]
VERSION = 'x.x'
def __init__(self, plugin_adminstrator, config=None, recursive=True):
'''
recursive flag: If True recursively analyze included files
'''
self.config = config
# additional init stuff can go here
super().__init__(plugin_adminstrator, config=config, recursive=recursive, plugin_path=__file__)
def process_object(self, file_object):
'''
This function must be implemented by the plug-in.
Analysis result must be a dict stored in "file_object.processed_analysis[self.NAME]"
CAUTION: Dict keys must be strings!
If you want to propagate results to parent objects store a list of strings in
"file_object.processed_analysis[self.NAME]['summary']".
File's binary is available via "file_object.binary".
File's local storage path is available via "file_object.file_path".
Results of other plug-ins can be accesd via "file_object.processed_analysis['PLUGIN_NAME']".
Do not forget to add these plug-ins to "DEPENDENCIES".
'''
return file_objectThe blacklisting feature of FACT is used to increase the system performance, by skipping irrelevant file types. For example compressed file types like zip-archives will not produce results for most analysis plug ins.
Instead of the optional MIME_BLACKLIST, you can also use MIME_WHITELIST to specify a list of allowed MIME types.
This allows plugins to target specific files. E.g. the exploit mitigations plugin will only work on elf-files so this can be whitelisted.
If your plug-in needs configuration options, you can add them to src/main.cfg in a section named as your plug-in. You can access a field (e.g. "option_a") of your configuration with the following Code:
self.config.get(self.NAME, "option_a")The black/whitelists can be set in the configuration file as well by adding mime_blacklist = mime1, mime2, ... (or mime_whitelist) to the section of the plugin.
An important configuration option is the threads value. Looking at the default config you see that most plugins have a value of 2 or higher. The values directly corresponds to the number of concurrent plugin instances FACT will run. Memory heavy plugins should set this value conservatively while other, fast plugins might set a value of 4, 8 or higher depending on your system. :warning: Default for threads is 1.
The recursive parameter in the plugin __init__ controls if a plugin runs on all included files.
If you set it to False, the plugin will only be executed on the firmware object.
The optional folder internal can provide additional code or other stuff needed for the plugin. You can use the following code to get the absolute path of the internal directory:
from pathlib import Path
import sys
INTERAL_DIRECTORY = sys.path.append(str(Path(Path(__file__).parent.parent, 'internal')))
sys.path.append(INTERNAL_DIRECTORY)The last line is used to add the internal directory to your python path. It is only needed if code is stored there. Once you have added the path like that, you can just import from modules inside the internal directory as from normal libraries.
This file contains your test code. In general, FACT has quite a high code coverage, so if you intend to contribute to the core, tests will be necessary. For internal development tests might not be needed, though they might be helpful in debugging sources of failure.
Since analysis plug-ins use a complex multithreading approach, instantiating a plugin in tests is not easy. You might want to use this template to simplify your testing. The AnalysisPluginTest class used by the template handles all the multithreading stuff and checks some general stuff (e.g. plugin registration success):
from test.unit.analysis.analysis_plugin_test_class import AnalysisPluginTest
from ..code.YOUR_PLUGIN import AnalysisPlugin
class TestAnalysisPluginYOURPLUGINNAME(AnalysisPluginTest):
PLUGIN_NAME = 'YOUR_PLUGIN_NAME'
def setUp(self):
super().setUp()
config = self.init_basic_config()
# additional config can go here
# additional setup can go here
self.analysis_plugin = AnalysisPlugin(self, config=config)
def tearDown(self):
super().tearDown()
# additional tearDown can go here
def test_your_test_code(self):
# your test codeUsing this template you can just call functions inside your plugin class from the instance given at self.analysis_plugin.
Note that by running the default tests (simply calling pytest from your FACT_core directory) all plugin tests will run as well.
This folder shall contain any additional files, that are needed for your tests. You can address this folder using the following code in your test code file.
from pathlib import Path
from common_helper_files import get_dir_of_file
TEST_DATA_DIR = Path(Path(__file__).parent / 'data')A lot of analysis is based on simple pattern matching. Therefore FACT provides a special "YaraBasePlugin" utilizing the Yara pattern matching system.
Yara-based plugins are quite easy to create once you have signature files:
- Store your Yara rule files to the signatures directory (as seen in 0)
- Customize the following plug-in template instead of the original template shown in 1
from analysis.YaraPluginBase import YaraBasePlugin
class AnalysisPlugin(YaraBasePlugin):
'''
A Short description
'''
NAME = 'PLUGIN_NAME'
DESCRIPTION = 'a short description of the plugin'
DEPENDENCIES = [LIST_OF_PLUG-IN_NAMES_THAT_THIS_PLUGIN_RELYS ON]
VERSION = 'x.x'
FILE = __file__
def __init__(self, plugin_adminstrator, config=None, recursive=True):
super().__init__(plugin_adminstrator, config=config, recursive=recursive, plugin_path=__file__)- Done!
Additional optional steps include configuration of threads, designing an own view and writing tests as for the general plugin development.
FACT has a simple tagging mechanism, that allows to set tags for single files as well as propagate these tags to appear on the outer container object. This allows to display critical information in a more visible way.
To set a tag in your plugin you can use two built-in/helper-functions as seen in this snipped:
from helperFunctions.tag import TagColor
# [...]
self.add_analysis_tag(
file_object=file_object,
tag_name='internal_tag_identifier',
value='Content shown on tag in GUI',
color=TagColor.ORANGE,
propagate=True
)
# [...]The analysis base plugin implements the add_analysis_tag method so you can use it with self.
The propagate flag controls if the tag should be shown in the outer container as well.
For very significant information this might be useful.
TagColor is an enum style class that contains the values GRAY, BLUE, GREEN, LIGHT_BLUE, ORANGE and RED.
