Soccer Payment Notification Aggregator

An automated system to track and verify soccer meetup payments through email notifications from Zelle/Venmo, storing payment data in DynamoDB and syncing with iOS Reminders for easy participant check-off on Apple Watch.
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Architecture
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Multi-Cloud Architecture
5. Implementation Plan
6. Security and Compliance
7. Roadmap
8. Contributing
9. License
10. Contact

About The Project

A serverless solution for automating payment tracking for soccer meetups. The system processes payment notifications from Zelle and Venmo via AWS SES, parses the information using Lambda functions, stores the data in DynamoDB, and syncs with iOS Reminders for easy participant management.

Architecture

• Email Processing: AWS SES receives payment notifications from Zelle/Venmo
• Data Extraction: Python Lambda function parses email content for payment details
• Data Storage: DynamoDB stores payment records and participant information
• Task Management: Integration with iOS Reminders for participant check-off
• Infrastructure: Terraform for AWS resource provisioning and management

(back to top)

Built With

• 
• 
• 
• 

(back to top)

Getting Started

Prerequisites

• AWS Account with appropriate permissions
• Python 3.8+
• Terraform
• AWS CLI configured
• iOS device for Reminders integration

(back to top)

Installation

1. Clone the repository

   git clone https://github.com/jonathan-d-nguyen/payment-notification-aggregator.git

2. Install Python dependencies

   pip install -r requirements.txt

3. Configure AWS credentials

   aws configure

4. Set up environment variables in .env

   AWS_REGION=your_region
   SES_EMAIL=your_ses_verified_email
   REMINDER_LIST_ID=your_ios_reminder_list_id
   

5. Initialize Terraform

   cd terraform
   terraform init

(back to top)

Usage

Current functionality:

• Email parsing for Zelle/Venmo notifications via AWS SES
• Python scripts for payment information extraction

In progress:

• DynamoDB integration
• iOS Reminders sync
• Terraform infrastructure setup

(back to top)

Multi-Cloud Architecture

Infrastructure Components

• Email Ingestion

  • Primary: AWS SES
  • Alternatives: Azure Communication Services, GCP Cloud Tasks
  • Multi-region failover capability

• Processing Layer

  • AWS Lambda (primary)
  • Azure Functions (secondary)
  • GCP Cloud Functions (tertiary)
  • Load balancing across cloud providers

• Storage Strategy

  • Hot Data: DynamoDB with global tables
  • Cold Storage: Multi-cloud blob storage (S3/Azure Blob/GCP Storage)
  • Automated archival policies

• Event Processing

  • AWS EventBridge for primary orchestration
  • Cross-cloud event synchronization
  • Dead letter queues for failed events

• Secrets Management

  • HashiCorp Vault as central secret store
  • Cloud KMS integration for key management
  • Automatic secret rotation

(back to top)

Key Design Principles

1. Cloud Agnostic Core

   • Provider-neutral business logic
   • Abstracted cloud services
   • Portable configurations

2. Event-Driven Architecture

   • Asynchronous processing
   • Loose coupling
   • Scalable message handling

3. Security-First Approach

   • Zero-trust architecture
   • Encryption everywhere
   • Least privilege access

4. Comprehensive Monitoring

   • Cross-cloud metrics aggregation
   • Centralized logging
   • Real-time alerting

5. Cost Optimization

   • Dynamic resource allocation
   • Multi-cloud cost analysis
   • Automated scaling policies

   (back to top)

Implementation Plan

1. Infrastructure Setup

# payment-notification-aggregator/
├── infrastructure
│   └── cloudformation
│       └── templates
│           └── template.yaml
├── src
│   ├── extractors
│   │   ├── venmo_extractor.py
│   │   └── zelle_extractor.py
│   ├── processors
│   │   ├── email_processor.py
│   │   ├── venmo_processor.py
│   │   └── zelle_processor.py
│   └── utils
│       └── output_formatter.py
├── .gitignore
├── Dockerfile
├── main.py
├── main.tf
├── README.md
└── requirements.txt

Prospective Infrastructure (click to expand)

# payment-notification-aggregator/
│
├── terraform/
│   ├── modules/
│   │   ├── core/                 # Cloud-agnostic modules
│   │   │   ├── vault/
│   │   │   │   ├── main.tf
│   │   │   │   ├── variables.tf
│   │   │   │   └── outputs.tf
│   │   │   └── monitoring/      # Shared monitoring setup
│   │   │
│   │   ├── aws/                 # AWS-specific resources
│   │   │   ├── ses/
│   │   │   │   ├── main.tf      # Email ingestion setup
│   │   │   │   ├── variables.tf
│   │   │   │   └── outputs.tf
│   │   │   ├── lambda/
│   │   │   └── dynamodb/
│   │   │
│   │   ├── azure/               # Future Azure implementation
│   │   │   ├── logic_apps/
│   │   │   ├── functions/
│   │   │   └── cosmos_db/
│   │   │
│   │   └── gcp/                 # Future GCP implementation
│   │       ├── cloud_functions/
│   │       ├── pub_sub/
│   │       └── firestore/
│   │
│   └── environments/
│       ├── dev/
│       │   ├── main.tf
│       │   ├── variables.tf
│       │   ├── outputs.tf
│       │   └── backend.tf
│       └── prod/
└── └── [same as dev]
│
├── src/
│   ├── core/
│   │   ├── interfaces/
│   │   │   ├── email_processor.py    # Abstract base classes
│   │   │   └── storage.py
│   │   │
│   │   └── models/
│   │       ├── transaction.py         # Data models
│   │       └── reminder.py
│   │
│   ├── processors/
│   │   ├── parsers/
│   │   │   ├── venmo_parser.py       # Venmo-specific parsing
│   │   │   └── zelle_parser.py       # Zelle-specific parsing
│   │   │
│   │   └── filters/
│   │       └── incoming_filter.py     # Filter for received money only
│   │
│   ├── services/
│   │   ├── vault_service.py          # HashiCorp Vault integration
│   │   ├── email_service.py          # Email processing service
│   │   └── reminder_service.py       # iOS Reminders integration
│   │
│   └── utils/
│       ├── config.py
│       ├── logging.py
│       └── error_handling.py
│
├── docker/
│   ├── Dockerfile               # Multi-stage build
│   ├── docker-compose.yml       # Local development setup
│   └── vault/
│       ├── config.hcl          # Vault configuration
│       └── policies/
│           └── app-policy.hcl  # Access policies
│
├── tests/
│   ├── unit/
│   │   ├── test_parsers.py
│   │   ├── test_filters.py
│   │   └── test_services.py
│   │
│   ├── integration/
│   │   ├── test_email_flow.py
│   │   └── test_reminder_creation.py
│   │
│   └── fixtures/
│       ├── sample_emails/      # Test email templates
│       │   ├── venmo/
│       │   └── zelle/
│       └── expected_outputs/   # Expected parsing results
│
├── scripts/
│   ├── deploy.sh              # Deployment automation
│   ├── vault-setup.sh         # Vault initialization
│   └── local-setup.sh         # Development environment setup
│
├── docs/
│   ├── architecture.md        # System design documentation
│   ├── setup.md              # Setup instructions
│   ├── deployment.md         # Deployment guide
│   └── cloud-specific/       # Provider-specific details
│       ├── aws.md
│       ├── azure.md
│       └── gcp.md
│
├── .github/
│   └── workflows/
│       ├── test.yml              # Unit and integration tests
│       ├── lint.yml              # Code quality checks
│       └── deploy.yml            # Multi-cloud deployment pipeline
│
├── .gitignore
├── requirements.txt          # Python dependencies
├── pyproject.toml           # Python project configuration
└── README.md               # Project overview and quickstart

(back to top)

2. Processing Pipeline

1. Email Reception

   • Multi-provider email ingestion
   • Unified filtering rules
   • Cross-cloud storage replication

2. Processing Function

   • Cloud-agnostic business logic
   • Cross-cloud message routing
   • Unified error handling

3. Data Storage

   • Multi-region data replication
   • Cross-cloud backup strategy
   • Automated data lifecycle

4. Reminder Integration

   • Resilient webhook system
   • Cross-platform compatibility
   • Failure recovery mechanism

(back to top)

Security and Compliance

Authentication & Authorization

• OAuth/OIDC integration
• Cross-cloud IAM strategy
• Zero-trust network design

Monitoring & Observability

• Unified logging strategy
• Cross-cloud metrics
• Centralized alerting

Error Handling

• Global retry policies
• Cross-cloud circuit breakers
• Unified error reporting

(back to top)

Roadmap

• AWS SES Configuration
• Python Email Parser Development
• Multi-Cloud Infrastructure Setup
• Cross-Cloud Event Processing
• Global Data Replication
• Security Hardening
• Monitoring Implementation
• Cost Optimization
• Compliance Framework
• Disaster Recovery Testing

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Contact

Jonathan Nguyen - @twitter_handle - jonathan@jdnguyen.tech

Project Link: https://github.com/jonathan-d-nguyen/payment-notification-aggregator

(back to top)