Merge pull request #515 from artoonie/fork/slhudson/chore/readme

startup tweaks and README additions (#512)

.gitignore

@@ -9,8 +9,10 @@ media/
 static/CACHE/
 node_modules/
 staticfiles/
+.DS_Store
 
 # Elastic Beanstalk Files
 .elasticbeanstalk/*
 !.elasticbeanstalk/*.cfg.yml
 !.elasticbeanstalk/*.global.yml
+

README.md

@@ -21,68 +21,34 @@ Visualize the results of ranked-choice voting elections.
 
 Learn more on our Medium post: [An Illustrated Guide to Ranked-Choice Voting](https://medium.com/@armin.samii/an-illustrated-guide-to-ranked-choice-voting-4ce3c5fe73f9).
 
-## Installation
-Install `python3`, `virtualenv`, and `npm` with your favorite package manager, then run `./scripts/install.sh`.
-
-Create a .env file with your secrets and configuration options:
-
-```bash
-export RCVIS_SECRET_KEY=''
-export RCVIS_DEBUG=True
-export RCVIS_HOST=localhost
-
-# Either have OFFLINE_MODE=True
-export OFFLINE_MODE=True
-
-# The following fields are optional, though you will
-# have limited functionality without them.
-
-# Or set up an AWS bucket and enter your credentials
-# export OFFLINE_MODE=False
-# export AWS_STORAGE_BUCKET_NAME=''
-# export AWS_S3_REGION_NAME=''
-# export AWS_ACCESS_KEY_ID=''
-# export AWS_SECRET_ACCESS_KEY=''
-
-# To send registration emails when OFFLINE_MODE is False:
-# export SENDGRID_USERNAME=''
-# export SENDGRID_PASSWORD=''
+## Examples
+Check out [rcvis.com](https://www.rcvis.com) for live examples, including:
 
-# To clear cloudflare cache when models update:
-# export CLOUDFLARE_ZONE_ID=''
-# export CLOUDFLARE_AUTH_TOKEN=''
+| Barchart | Round-by-Round |
+| --- | --- |
+| ![Barchart](static/visualizer/icon_interactivebar.gif "Interactive Barchart") | ![Round-by-Round](static/visualizer/icon_interactiveroundbyround.gif "Round-by-Round") |
 
-# To run the SauceLabs integration tests, you will need
-export SAUCE_USERNAME=''
-export SAUCE_ACCESS_KEY=''
+| Sankey | Tabular Summary |
+| --- | --- |
+| ![Sankey](static/visualizer/icon_sankey.jpg "Sankey") | ![Tabular Summaries](static/visualizer/icon_singletable.png "Tabular Summaries") |
 
-# To generate videos (and to run movie tests), you will need:
-export IMAGEIO_FFMPEG_EXE='/usr/bin/ffmpeg'
-export SQS_QUEUE_NAME='default-queue'
-# export MOVIE_FONT_NAME="Roboto"
-# export AWS_POLLY_STORAGE_BUCKET_NAME="bucket-name-on-s3"
+## Embedding
+RCVis implements the [oembed protocol](http://www.oembed.com) with discoverability, allowing you to embed files into your website with an iframe.
 
-# To subscribe users to mailchimp upon registration, you need:
-# export MAILCHIMP_API_KEY=''
-# export MAILCHIMP_LIST_ID=''
-# export MAILCHIMP_DC=''
 
-# If you are updating a template, you'll need to clear the cache every time or set:
-# export DISABLE_CACHE=True
+# Running RCVis Locally
 
-```
+## Installation
+Install `python3`, `virtualenv`, and `npm` with your favorite package manager, then run `./scripts/install.sh`. This script will initialize a `.env` file in the root directory for your secrets and configuration. You will need to supply a secret key in `.env` before proceeding.
 
-To get moviepy working for Ubuntu 16.04 LTS users, comment out the following statement in `/etc/ImageMagick-6/policy.xml`:
-```xml
-<policy domain="path" rights="none" pattern="@*"/>
-```
-or, simply run `sudo ./scripts/fix-moviepy-on-ubuntu-1604.sh`
 
 ## Running
-To begin serving the website at localhost:8000:
+
+You can begin serving the website at localhost:8000 using:
 ```bash
 ./scripts/serve.sh
 ```
+The first time you execute this script, you will be prompted to create a new admin user. You can skip this step for future executions by storing the email of the admin user as `OFFLINE_ADMIN` in `.env`.
 
 You may also need to run this whenever the npm dependencies change:
 ```bash
@@ -93,24 +59,8 @@ npm install  # this works for me
 python3 manage.py npminstall  # this is purported to work but doesn't
 ```
 
-To run workers to generate movies (optional - only needed to use the movie generation flow):
-```bash
-source .env
-source venv/bin/activate
-export DISPLAY=":0" # if not already set
-celery -A rcvis worker --loglevel info
-```
-
-## Examples
-Check out [rcvis.com](https://www.rcvis.com) for live examples, including:
-
-| Barchart | Round-by-Round |
-| --- | --- |
-| ![Barchart](static/visualizer/icon_interactivebar.gif "Interactive Barchart") | ![Round-by-Round](static/visualizer/icon_interactiveroundbyround.gif "Round-by-Round") |
-
-| Sankey | Tabular Summary |
-| --- | --- |
-| ![Sankey](static/visualizer/icon_sankey.jpg "Sankey") | ![Tabular Summaries](static/visualizer/icon_singletable.png "Tabular Summaries") |
+## Test Data
+Test data, including real and mock elections, can be found in the `testData` directory. Example input formats can be found on [RCVFormats](https://github.com/artoonie/rcvformats/tree/main/testdata/inputs).
 
 ## REST API
 The primary API documentation is in the form of [example code](visualizer/tests/testRestApiExampleCode.py), which is documented line-by-line.
@@ -120,7 +70,7 @@ Addition documentation is available at [rcvis.com/api/](https://www.rcvis.com/ap
 To get started with programmatic access to rcvis:
 
 1. Create an account on RCVis
-2. Email team@rcvis.com to enable API access
+2. Email team@rcvis.com to request API access
 3. Submit a POST request to [https://www.rcvis.com/api/auth/get-token](https://www.rcvis.com/api/auth/get-token) to obtain an API Key, e.g. `curl -X POST https://www.rcvis.com/api/auth/get-token -d username=yourUserName -d password=yourAmazingPassword`
 
 With your API key, you may access two endpoints:
@@ -129,5 +79,19 @@ With your API key, you may access two endpoints:
 
 For both endpoints, upload with POST and modify with PUT or PATCH. Authenticated users are limited to 1000 requests per hour.
 
-## oembed
-RCVis implements the [oembed protocol](http://www.oembed.com) with discoverability, allowing you to embed files into your website with an iframe.
+
+## Movies
+
+To get moviepy working for Ubuntu 16.04 LTS users, comment out the following statement in `/etc/ImageMagick-6/policy.xml`:
+```xml
+<policy domain="path" rights="none" pattern="@*"/>
+```
+or, simply run `sudo ./scripts/fix-moviepy-on-ubuntu-1604.sh`
+
+To run workers to generate movies (optional - only needed to use the movie generation flow):
+```bash
+source .env
+source venv/bin/activate
+export DISPLAY=":0" # if not already set
+celery -A rcvis worker --loglevel info
+```

electionpage/tests.py

@@ -304,8 +304,9 @@ def test_are_results_certified_initializes_correctly(self):
 
         # Wait for the object to be loaded
         self._ensure_eventually_asserts(
-            ScrapableElectionPage.objects.filter(slug='cuteslug').exists
-        )
+            lambda: self.assertTrue(
+                ScrapableElectionPage.objects.filter(
+                    slug='cuteslug').exists()))
 
         # Then ensure all scrapers have areResultsCertified
         scrapableElectionPage = ScrapableElectionPage.objects.get(slug='cuteslug')

infra/.env.template

@@ -0,0 +1,52 @@
+### The following fields must be provided to run RCVis locally and visualize results.
+
+# a user-generated string
+export RCVIS_SECRET_KEY=''
+
+# set to True to enable debug mode
+export RCVIS_DEBUG=True
+
+# host IP address
+export RCVIS_HOST=localhost
+
+# toggle offline mode
+export OFFLINE_MODE=True
+
+# email address for local admin in offline mode
+export OFFLINE_ADMIN='name@domain.com' 
+
+### The following fields are optional, though you will have limited functionality without them.
+### These limitations are suitable for local development, but must be set up when deploying -- including cache management, registration email management, and mailing list management.
+
+# As an alternative to using OFFLINE_MODE=True, you may uncomment the following block to use AWS storage instead of local storage
+# export OFFLINE_MODE=False
+# export AWS_STORAGE_BUCKET_NAME=''
+# export AWS_S3_REGION_NAME=''
+# export AWS_ACCESS_KEY_ID=''
+# export AWS_SECRET_ACCESS_KEY=''
+
+# To send registration emails when OFFLINE_MODE is False:
+# export SENDGRID_USERNAME=''
+# export SENDGRID_PASSWORD=''
+
+# To clear cloudflare cache when models update:
+# export CLOUDFLARE_ZONE_ID=''
+# export CLOUDFLARE_AUTH_TOKEN=''
+
+# To run the SauceLabs integration tests:
+export SAUCE_USERNAME=''
+export SAUCE_ACCESS_KEY=''
+
+# To generate videos (and to run movie tests):
+export IMAGEIO_FFMPEG_EXE='/usr/bin/ffmpeg'
+export SQS_QUEUE_NAME='default-queue'
+# export MOVIE_FONT_NAME="Roboto"
+# export AWS_POLLY_STORAGE_BUCKET_NAME="bucket-name-on-s3"
+
+# To subscribe users to mailchimp upon registration:
+# export MAILCHIMP_API_KEY=''
+# export MAILCHIMP_LIST_ID=''
+# export MAILCHIMP_DC=''
+
+# If you are updating a template, you'll need to clear the cache every time or set:
+# export DISABLE_CACHE=True

infra/requirements-core.txt

@@ -1,4 +1,4 @@
-boto3==1.17.100
+boto3==1.26.0
 csscompressor==0.9.5
 django-admin-cursor-paginator==0.1.3
 django-compressor==4.3.1

scripts/install.sh

@@ -8,6 +8,20 @@ pip3 install -r requirements.txt
 # Install npm deps
 npm i
 
+# initialize .env
+if [ -f .env ]; then
+    echo ""
+    echo ".env file already exists."
+    echo ""
+else
+    # Copy the template to .env
+    cp infra/.env.template .env
+    echo ""
+    echo ".env file has been initialized from the template."
+    echo "Please modify the .env file to include the necessary values."
+    echo ""
+fi
+
 # Run migrations to set up db
 source .env
 python3 manage.py migrate

scripts/serve.sh

@@ -2,9 +2,24 @@
 # runserver, along with everything that goes along with it
 set -e
 
+# Source the .env file to get the environment variables
 source .env
+
+# Activate the virtual environment
 source venv/bin/activate
 
+# Check if the admin user with the email specified in .env exists
+admin_exists=$(python manage.py shell -c "from django.contrib.auth import get_user_model; User = get_user_model(); print(User.objects.filter(email='$OFFLINE_ADMIN', is_superuser=True).exists())")
+
+# Prompt the user to create the admin if not
+if [ "$admin_exists" = "False" && "$OFFLINE_MODE" = "True" ]; then
+    echo "Admin user with email $OFFLINE_ADMIN does not exist. Please create one."
+    python manage.py createsuperuser --email $OFFLINE_ADMIN
+else
+    echo "Admin user with email $OFFLINE_ADMIN already exists."
+fi
+
+# Clear cache, compress and run the server
 python3 manage.py clearCache
 python3 manage.py compress
-python3 manage.py runserver
+python3 manage.py runserver