Skip to content

Commit

Permalink
added AWS_EB_ALERT_WHEN_EXECUTES_LONGER_THAN_SECONDS support
Browse files Browse the repository at this point in the history
  • Loading branch information
DataGreed committed Sep 21, 2020
1 parent ef25b02 commit 723f527
Show file tree
Hide file tree
Showing 4 changed files with 27 additions and 2 deletions.
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,11 @@ Amazon Secret Access Key, refer to [the docs](https://docs.aws.amazon.com/genera

If set to true, all tasks will be run locally and synchronnously instead of being sent to SQS Queue. Defaults to `False`

### AWS_EB_ALERT_WHEN_EXECUTES_LONGER_THAN_SECONDS

Set this to the maximum number of seconds the job is supposed to run. If the job finishes requires more time to finish
ADMINS will be notified by email.

## Security

Always set `AWS_EB_HANDLE_SQS_TASKS=False` on Web Tier Environment so the tasks could not be spoofed!
Expand Down
18 changes: 18 additions & 0 deletions eb_sqs_worker/views.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import json
import logging
import time
import uuid

from django.conf import settings
from django.core.mail import mail_admins
from django.shortcuts import render

# Create your views here.
Expand Down Expand Up @@ -67,6 +69,22 @@ def post(self, request):
print(f"{call_id} Finished {task.get_pretty_info_string()}. "
f"Result: {result}. Execution time: {execution_time}s.")

alert_threshold_seconds = getattr(settings, "AWS_EB_ALERT_WHEN_EXECUTES_LONGER_THAN_SECONDS", None)
if alert_threshold_seconds:
if execution_time > alert_threshold_seconds:
try:
print(f"{call_id} took to long too finish. Reporting to admins via email. ")
mail_admins(f"Task {task.task_name} was running for too long",
f"\nTask {task.get_pretty_info_string()} (call id {call_id}) finished in {execution_time}s "
f"and you have "
f"alert thresholds set to {alert_threshold_seconds}. \nPlease check if there is "
f"something wrong with the task execution."
f"\nTask result: {result}",
fail_silently=False, connection=None, html_message=None)
except Exception as e:

logging.error(f"Failed to send email about long-running task {call_id}: {e}", exc_info=True)

return JsonResponse(
{},
status=200
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

setuptools.setup(
name="django-eb-sqs-worker", # Replace with your own username
version="0.3.0",
version="0.4.0",
author="Alexey Strelkov",
author_email="datagreed@gmail.com",
description="Django Background Tasks for Amazon Elastic Beanstalk",
Expand Down
4 changes: 3 additions & 1 deletion test_project/test_project/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,4 +126,6 @@
# eb-sqs-worker settings
AWS_EB_DEFAULT_REGION = "us-west-1"
AWS_ACCESS_KEY_ID = "dummy"
AWS_SECRET_ACCESS_KEY = "dummy"
AWS_SECRET_ACCESS_KEY = "dummy"

AWS_EB_ALERT_WHEN_EXECUTES_LONGER_THAN_SECONDS = 0.00000000000000000000000000000000000000000000000000000000001

0 comments on commit 723f527

Please sign in to comment.