Simple Metaculus forecasting bot

This repository contains a simple bot meant to get you started with creating your own bot for the AI Forecasting Tournament.

Quick start -> Fork and use Github Actions

The easiest way to use this repo is to fork it, enable github workflow/actions, and then set repository secrets. Then your bot will run every 15min, pick up new questions, and forecast on them. Automation is handled in the .github/workflows/ folder. The daily_run_simple_bot.yaml file runs the simple bot every 15 min and will skip questions it has already forecasted on.

1. Fork the repository: Go to the repository and click 'fork'.
2. Set secrets: Go to Settings -> Secrets and variables -> Actions -> New respository secret and set API keys/Tokens as secrets. You will want to set your METACULUS_TOKEN. This will be used to post questions to Metaculus, and so you can use our OpenAI proxy (reach out to support@metaculus.com with your bot description to apply for credits. We are giving credits fairly generously to encourage participation).
3. Enable Actions: Go to 'Actions' then click 'Enable'. Then go to the 'Regularly forecast new questions' workflow, and click 'Enable'. To test if the workflow is working, click 'Run workflow', choose the main branch, then click the green 'Run workflow' button. This will check for new questions and forecast only on ones it has not yet successfully forecast on.

The bot should just work as is at this point. You can disable the workflow by clicking Actions > Regularly forecast new questions > Triple dots > disable workflow

As a note GET_NEWS is disabled by default, and you will need to edit this in main.py to enable searching the web. If enabled, the default search provider is AskNews which requires 2 keys (add them to your secrets). There is a function you can use to call Perplexity if you would rather use this. For more information on how to set up AskNews see the tournament page. Beyond this you can use the below instructions to edit the code and run it locally.

Run the bot locally

Clone the repository. Find your terminal and run the following commands:

git clone https://github.com/Metaculus/metac-bot-template.git

If you forked the repository first, you have to replace the url in the git clone command with the url to your fork. Just go to your forked repository and copy the url from the address bar in the browser.

Installing dependencies

Make sure you have python and poetry installed (poetry is a python package manager).

Inside the terminal, go to the directory you cloned the repository into and run the following command:

poetry install

to install all required dependencies.

Setting environment variables

Running the bot requires various environment variables. If you run the bot locally, the easiest way to set them is to create a file called .env in the root directory of the repository (copy the .env.template).

Running the bot

To run the simple bot, execute the following command in your terminal:

poetry run python main.py

Make sure to set the environment variables as described above and to set the parameters in the code to your liking. In particular, to submit predictions, make sure that submit_predictions is set to True.