Algorithm-trading-environment is a simulating trading environment designed for reinforcement learning agents. It is running on tick-level quote and trade data to reproduce trading process incorprating with new simulated orders.
Like general reinforcement learning environment, algorithm-trading-environment can be represented by a tuple of (s, a, p, r), where:
- environment state s is a numerical vector that consists of current real quote (i.e. market state) and simulated order status (i.e. agent state);
- action a is a tuple whose shape likes (trading direction, level, size);
- transition probability p describes state transition in general reinforcement learning environment, but the transition is constant and sequential in this environment;
- reward r denotes transaction cost of trading strategies.
Download zip file or use git clone.
$ git clone https://github.com/Pangjing-Wu/algorithm-trading-strategy-environment.gitTODO: we will release it as a python module in the future.
Here is a quick start example for loading algorithm-trading-environment.
Train agent based on vwap trading strategies
nohup python -u vwap.py --mode train --env hard_constrain --agent linear --stock 600000 --side sell --level 1 --tranche_id 0 2>&1 >./logs/train/600000-hard-linear-0-8.log &nohup python -u vwap.py --mode train --env historical_hard_constrain --agent linear --stock 600000 --side sell --level 1 --tranche_id 0 2>&1 >./logs/train/600000-historical-hard-linear-0-8.log &nohup python -u vwap.py --mode train --env recurrent_hard_constrain --agent lstm --stock 600000 --side sell --level 1 --tranche_id 0 2>&1 >./logs/train/600000-recurrent-hard-lstm-0-8.log &Test agent based on vwap trading strategies
python -u vwap.py --mode test --env historical_hard_constrain --agent linear --stock 600000 --side sell --level 1 --tranche_id 0 >./logs/test/600000-historical-hard-linear-0-8.log- env.py is main file of algorithm-trading-environment, provides algorithmic trading environment interface for agent.
- tickdata.py creates a class for tick-level data, which provides abundant function for query and processing quote or trade records.
- utils/ contains some functions to support algorithmic trading.
- transaction.py is transaction matching module to executing simulated order by matching it with real quote and trade data.
- h2db.py is h2 database connection and query module.
Besides, tutorial/ contains some examples of guiding to use the envronment, test contains test configuration and test cases, doc/ contains documents of this program.
The core of this program is to match simulated order with real quote and trade data to evaluate transaction cost. To address this problem, we need to
- read tick data from H2 database and convert to some type that is easy to handle in Python, such as
pd.DataFrame. - preprocess real quote and trade data;
- issue simulated order and match it with real quote and trade data;
- sequencially execute transaction matching process.
Read data from H2 database and convert to some type that is easy to handle in Python
Since raw data is stored in H2 database, we firstly design a module h2bd.py to read the raw tick data. h2bd.py provides H2Connection class to get connection with H2 database under the protocol provided by psycopg2 library. It support automatically detect H2 service status and start H2 service for MacOS/Linux. Data query is conducted by executing H2Connection.query.
Preprocess real quote and trade data
These raw data are preprocessed and converted to TickData class which defined in tickdata.py. It provides abundant function for processing quote and trade data, such as get_quote or get_trade record(s) by timestamp, get pre_quote or next_quote by timestamp or quote record, and get_trade_between two quotes, even you can tranform quote from record form to quote board form by quote_board.
Issue simulated order and match it with real quote and trade data
We design transaction_matching function in transaction.py to match simulated order with real quote and trade data. It considers 4 transaction matching situations, which are
- when simulated order's direction is 'buy' and price level is 'ask', the order is closed directly.
- when simulated order's direction is 'buy' and price level is 'bid', the order waits in queue until previous quote orders are closed.
- when simulated order's direction is 'sell' and price level is 'bid', the order is closed directly.
- when simulated order's direction is 'sell' and price level is 'ask', the order waits in queue until previous quote orders are closed.
Sequencially execute transaction matching process
We design AlgorithmicTrading in env.py whose interface is like the famous reinforcement learning library gym. It provides reset to initiate or reset environment and step to execute agent's action in environment.
The detailed interface document are presented in interface.
Release first version.
No license, private program, DO NOT distribute.