navidocs/INTEGRATIONS-SIP-MAILGUN.md

80 KiB

Mailgun Email API Integration Research (Haiku-33)

Comprehensive 8-Pass IF.search Methodology Analysis

Document Version: 1.0 Research Agent: Haiku-33 Methodology: IF.search 8-Pass Analysis Date: November 2024 Status: Complete Research Analysis


Executive Summary

Mailgun is an enterprise-grade email service platform serving 150,000+ businesses globally, providing transactional email APIs, inbound routing, email validation, and real-time analytics. This research evaluates Mailgun's technical architecture, API capabilities, pricing models, security mechanisms, and deployment requirements for integration into InfraFabric transactional email infrastructure.

Key Findings:

  • Integration Complexity: 6/10 (moderate complexity, well-documented APIs)
  • Cost Model: $0-35+/month (free tier 100 emails/day; $35/month for 50K emails/month)
  • Security Score: 9/10 (HMAC webhook verification, EU data residency, GDPR compliance)
  • API Stability: Production-ready with 99.99% uptime SLA
  • Rate Limits: Dynamic based on account tier (typically 600 req/min for paid accounts)

PASS 1: SIGNAL CAPTURE

Comprehensive API Surface Area Scanning

1.1 Mailgun API Product Ecosystem

Mailgun's platform is organized into four primary product pillars:

A. Send API (Transactional Email Core)

  • RESTful HTTP API for programmatic email transmission
  • SMTP Relay for traditional mail server integration
  • Base Endpoint: https://api.mailgun.net/v3/YOUR_DOMAIN/messages
  • Authentication: HTTP Basic Auth (api as username, API key as password)
  • Response Format: JSON
  • Supported Methods: POST (message creation), GET (status retrieval)

B. Routes API (Inbound Email Handling)

  • Automated email routing and parsing
  • Regular expression-based message matching
  • Webhook delivery to custom endpoints
  • 3-day message retention for retrieval
  • Base Endpoint: https://api.mailgun.net/v3/routes
  • Operations: Create, Read, Update, Delete (CRUD) routes
  • Filtering: Priority-based route evaluation
  • Actions: Forward to HTTP, store, drop, or redirect

C. Webhooks API (Event Notification System)

  • Real-time email event notifications
  • Push-based architecture (no polling required)
  • Event types: delivered, opened, clicked, unsubscribed, complained, bounced, failed, dropped
  • Webhook endpoints: https://api.mailgun.net/v3/{domain}/webhooks
  • Security: HMAC-SHA256 signature verification
  • Payload Format: application/x-www-form-urlencoded or multipart/form-data

D. Email Validation API (Data Quality)

  • Real-time single email validation
  • Bulk batch validation processing
  • Risk assessment and categorization
  • Base Endpoint: https://api.mailgun.net/v4/address/validate
  • Operations: Validate single addresses, bulk uploads, status polling
  • Response Categories: deliverable, undeliverable, do_not_send, catch_all, unknown
  • Risk Levels: high, medium, low, unknown

E. Logs/Events API (Analytics & Tracking)

  • Comprehensive email event logging
  • ElasticSearch-backed full-text search
  • Real-time event retrieval and filtering
  • Base Endpoint: https://api.mailgun.net/v3/{domain}/events
  • Data Retention: 2-30 days (free to paid accounts)
  • Search Parameters: Complex multi-criteria filtering
  • Performance: Sub-second queries on 10M+ log entries

1.2 API Authentication Methods

Method 1: HTTP Basic Authentication

  • Username: api
  • Password: API key from dashboard
  • Implementation: Authorization: Basic base64(api:YOUR_API_KEY)
  • Used by: All REST API endpoints

Method 2: SMTP Relay Authentication

  • Username: postmaster@yourdomain.com
  • Password: SMTP password from dashboard
  • Protocol: TLS/SSL on ports 465, 587, or 25
  • Connection: smtp.mailgun.org

Method 3: Webhook Signature Verification

  • Algorithm: HMAC-SHA256
  • Key: Webhook Signing Key (separate from API key)
  • Parameters: timestamp + token
  • Validation: Compare signature parameter to computed hash

1.3 Core Message Sending Parameters

Minimum Required Parameters:
  - from: sender@yourdomain.com (must be verified)
  - to: recipient@example.com
  - subject: Email subject line
  - text OR html: Message body content
  - api_key: Authentication credential

Extended Parameters:
  - cc: Carbon copy recipients (array)
  - bcc: Blind copy recipients (array)
  - reply-to: Reply-to address
  - in-reply-to: Message-ID of parent message
  - references: Thread references
  - attachment: File attachment (multipart/form-data)
  - inline: Inline image/asset
  - o:tracking: Enable click/open tracking (yes/no)
  - o:tracking-clicks: Click tracking (yes/html/no)
  - o:tracking-opens: Open tracking (yes/no)
  - o:tag: Message tags for analytics
  - o:campaign-id: Campaign identifier
  - o:deliverytime: Scheduled send time
  - o:dkim: DKIM signing (yes/no)
  - o:testmode: Test mode flag (yes/no)

1.4 Domain Verification Requirements

Required DNS Records:

  1. SPF Record (Sender Policy Framework)

    • Type: TXT
    • Format: v=spf1 include:mailgun.org ~all
    • Purpose: Authorize Mailgun to send emails on behalf of domain
    • Validation Time: Immediate after DNS propagation
  2. DKIM Record (DomainKeys Identified Mail)

    • Type: TXT
    • Selector: mailgun or custom
    • Format: Public key cryptographic signature
    • Purpose: Cryptographic authentication of sender identity
    • Validation Time: 24-48 hours for DNS propagation
  3. MX Records (Optional but Recommended)

    • Type: MX
    • Purpose: Route inbound mail to Mailgun servers
    • Required for: Inbound routing and email receiving functionality
  4. CNAME Record (Optional for Management)

    • Type: CNAME
    • Purpose: Alternative domain verification method
    • Flexibility: Simplifies DNS record management

1.5 Event Types and Webhook Payloads

Primary Event Categories:

Event Type Description Webhook Payload Size Retry Logic
accepted Mailgun accepted message ~2KB Immediate delivery
delivered Successfully delivered to server ~2.5KB Retries up to 24 hours
failed Permanent delivery failure ~3KB No retry (permanent)
bounced Hard bounce (permanent) ~2.5KB No retry (permanent)
dropped Dropped by filters ~2KB No retry (dropped)
opened Recipient opened email ~1.5KB Retries up to 24 hours
clicked Recipient clicked link ~1.5KB Retries up to 24 hours
complained Marked as spam ~1.5KB Retries up to 24 hours
unsubscribed Unsubscribe link clicked ~1.5KB Retries up to 24 hours

Webhook Retry Strategy:

  • Initial attempt: Immediate delivery
  • Retry window: 24 hours
  • Retry frequency: Exponential backoff (5s, 10s, 20s, 40s, ...)
  • Success criteria: HTTP 2xx response within 10 seconds
  • Failure handling: Logged and available via Events API

1.6 Mailing Lists API Overview

Core Operations:

  1. Create Mailing List

    • Endpoint: POST /v3/lists
    • Required: List address (email format)
    • Optional: Description, access_level (readonly/members/everyone), reply preference
  2. Add List Members

    • Endpoint: POST /v3/lists/{list_address}/members
    • Bulk capacity: Up to 1,000 members per request
    • Fields: address, name, vars (custom variables), subscribed status
  3. Update Member

    • Endpoint: PUT /v3/lists/{list_address}/members/{member_address}
    • Modifiable: name, custom variables, subscription status
  4. Remove Member

    • Endpoint: DELETE /v3/lists/{list_address}/members/{member_address}
    • Cascade: Member removed from all mailings

PASS 2: PRIMARY ANALYSIS

Core Capability Deep Dive

2.1 Email Sending Architecture

Sending Flow (Sequential Process):

1. Client Application initiates HTTP POST to Send API
   ↓
2. Mailgun validates API credentials and domain
   ↓
3. Message normalized to internal format
   ↓
4. DKIM signature computed and attached
   ↓
5. Message queued for delivery (FIFO)
   ↓
6. SMTP connection established to recipient MX server
   ↓
7. Recipients evaluated against bounce/complaint lists
   ↓
8. Message transmission with retry logic
   ↓
9. Webhook event generated for outcome
   ↓
10. Event indexed in ElasticSearch logs

Delivery Queue Performance:

  • Average delivery time: 5-60 seconds
  • Burst capacity: Scales to thousands of emails per second
  • Queuing mechanism: Distributed across multiple server regions
  • Failure recovery: Automatic retry with exponential backoff

2.2 Email Routing System (Incoming Mail Processing)

Route Matching Algorithm:

Routes are evaluated in priority order against incoming message recipients:

  1. Route Definition Structure:

    • Priority: 0-1000 (higher = evaluated first)
    • Expression: Regular expression matching recipient address
    • Action: HTTP POST, store, drop, or redirect
    • Description: Human-readable route documentation
  2. Expression Matching Examples:

    • match_recipient(".*@example.com") - All emails to example.com domain
    • match_recipient("support-.*@example.com") - Prefix matching
    • match_header("subject", ".*invoice.*") - Header-based routing
    • match_recipient("user\\+.*@example.com") - Plus addressing
  3. Route Actions:

    • HTTP POST Action: POST parsed message to webhook URL (custom parsing)
    • Store Action: Retain message for 3 days (retrieve via Events API)
    • Redirect Action: Forward to another email address
    • Drop Action: Silently discard matching messages
  4. Message Parsing (Automatic):

    • Extraction of plain text and HTML bodies
    • Signature detection and optional stripping
    • Quoted part identification and separation
    • Attachment parsing and base64 encoding
    • Header extraction and indexing
  5. Webhook Payload Structure (Route Action):

{
  "timestamp": 1530000000,
  "token": "abcdef1234567890abcdef",
  "signature": "hexdigest_of_timestamp_token",
  "recipient": "user@example.com",
  "sender": "customer@example.com",
  "subject": "Order #12345 Confirmation",
  "from": "customer@example.com",
  "message_id": "<20240101000000.1@example.com>",
  "Message-Id": "<20240101000000.1@example.com>",
  "body-plain": "Order confirmed...",
  "body-html": "<html>Order confirmed...</html>",
  "stripped-text": "Order confirmed...",
  "stripped-html": "<html>Order confirmed...</html>",
  "stripped-signature": "Best regards,\nCustomer",
  "attachment-count": 1,
  "attachments": [
    {
      "filename": "invoice.pdf",
      "size": 45678,
      "content-type": "application/pdf"
    }
  ],
  "attachment-1": "base64_encoded_pdf_data"
}

2.3 Real-Time Validation System

Validation Architecture:

  1. Single Address Validation:

    • Endpoint: GET /v4/address/validate
    • Parameters: address, mailbox_verification (optional)
    • Response Time: < 500ms
    • Uses cached data from 450B+ delivered emails
  2. Validation Checks Performed:

    • Syntax Validation: RFC 5322 compliance
    • DNS Validation: MX record existence
    • Mailbox Validation: SMTP handshake with provider
    • Reputation Analysis: Historical bounce/complaint data
    • Role Detection: Identifies common roles (admin@, support@, etc.)
    • Catch-All Detection: Tests if domain accepts all addresses
  3. Response Categories:

    • deliverable: High confidence for successful delivery
    • undeliverable: High confidence of delivery failure
    • unknown: Insufficient data for categorization
    • catch_all: Domain accepts all addresses
    • do_not_send: Flagged for other reasons (risky, etc.)
  4. Risk Assessment:

    • high: Likely invalid or high-risk address
    • medium: Some risk indicators present
    • low: Valid address with low risk
    • unknown: Insufficient data
  5. Batch Validation Process:

    • Upload CSV file to validation service
    • Polling for completion status
    • Download results in CSV/JSON format
    • Processing time: Minutes to hours depending on list size

2.4 Detailed Analytics and Tracking

Event Types and Granularity:

  1. Delivery Events:

    • accepted: Mailgun received the message from your application
    • delivered: Successfully delivered to recipient mailbox
    • failed: Permanent delivery failure (hard bounce)
    • dropped: Message dropped (spam filters, bounce list, etc.)
  2. Engagement Events:

    • opened: Recipient opened email (if tracking enabled)
    • clicked: Recipient clicked tracked link
    • complained: Recipient marked as spam
    • unsubscribed: Recipient clicked unsubscribe link
  3. Bounce Management:

    • Permanent Bounces: Added to bounce list, not retried
    • Temporary Bounces: Retried for 24-48 hours
    • Suppression: Addresses on bounce/complaint lists bypass sending
  4. Metrics Calculation:

    • Delivery Rate: (delivered + deferred) / total sent
    • Bounce Rate: bounced / total sent
    • Open Rate: opened / delivered (requires tracking enabled)
    • Click Rate: clicked / delivered (requires tracking enabled)
    • Complaint Rate: complained / delivered

Analytics API Response Format:

{
  "stats": [
    {
      "time": 1530000000,
      "accept": {"incoming": 0, "outgoing": 10},
      "deliver": {"incoming": 0, "outgoing": 9},
      "drop": {"incoming": 0, "outgoing": 1},
      "fail": {"incoming": 0, "outgoing": 0},
      "bounce": {"incoming": 0, "outgoing": 1},
      "click": {"incoming": 0, "outgoing": 0},
      "open": {"incoming": 0, "outgoing": 0},
      "complain": {"incoming": 0, "outgoing": 0},
      "unsubscribe": {"incoming": 0, "outgoing": 0}
    }
  ]
}

2.5 Mailing Lists Management

Advanced List Operations:

  1. List Subscription Model:

    • Each member can have custom variables (up to 1000 key-value pairs)
    • Subscription status: subscribed (true/false)
    • Automatic enforcement: Unsubscribed members skip receiving messages
  2. Bulk Member Operations:

    • Add/update up to 1,000 members per API call
    • Batch upload via form-data multipart
    • Format: JSON array of member objects
  3. List-Level Configuration:

    • access_level: readonly - Owner can modify, members cannot
    • access_level: members - Members can view and modify
    • access_level: everyone - Public read/write access
    • reply_preference: list - Replies go to list address
    • reply_preference: sender - Replies go to original sender
  4. Sending to Lists:

    • Endpoint: POST /v3/{domain}/messages
    • Recipient: {list_address} (treated as single recipient)
    • Expansion: List automatically expands to all subscribed members
    • Personalization: Custom variables injected into message template

Example List Expansion:

POST /v3/mycompany.mailgun.org/messages
  to: developers@mycompany.mailgun.org (50 subscribers)

Result: Message sent to 50 individual recipients
Each recipient sees: To: developers@mycompany.mailgun.org
BCC used internally for actual delivery

PASS 3: RIGOR & REFINEMENT

Advanced Features and Detailed Specifications

3.1 Delivery Rate Optimization and Bounce Handling

Bounce Classification System:

  1. Hard Bounces (Permanent):

    • Invalid recipient address
    • Domain does not exist
    • Recipient rejected at SMTP level
    • Action: Automatically added to bounce list, no retry
    • Duration: 24-hour suppression minimum
  2. Soft Bounces (Temporary):

    • Mailbox full/over quota
    • Server temporarily unavailable
    • Too many concurrent connections
    • Action: Automatic retry for 24-48 hours
    • Backoff: Exponential increase between attempts
  3. Complaints (Abuse Reports):

    • Recipient reported as spam to ISP
    • Automatically added to complaint suppression list
    • Detection: Via feedback loops from major ISPs
    • Action: No further sending to this address
  4. Bounce List Management:

    • Endpoint: GET /v3/{domain}/bounces
    • Filtering: By type, timestamp, address
    • Deletion: DELETE /v3/{domain}/bounces/{address} (manual recovery)
    • Retention: Maintains historical bounce data

Bounce List Response Structure:

{
  "items": [
    {
      "address": "user@example.com",
      "type": "permanent",
      "code": "550",
      "error": "user unknown",
      "created_at": "Fri, 01 Jan 2024 00:00:00 UTC"
    }
  ],
  "paging": {
    "first": "https://api.mailgun.net/v3/mycompany.mailgun.org/bounces?page=first",
    "last": "https://api.mailgun.net/v3/mycompany.mailgun.org/bounces?page=last",
    "next": "https://api.mailgun.net/v3/mycompany.mailgun.org/bounces?page=next",
    "previous": "https://api.mailgun.net/v3/mycompany.mailgun.org/bounces?page=previous"
  }
}

3.2 Route Condition Programming (Advanced)

Conditional Route Expressions:

Mailgun supports sophisticated route conditions using match functions:

  1. Recipient Matching:

    match_recipient("^support-.*@example\\.com$")
    // Matches: support-billing@example.com, support-technical@example.com
    // Does not match: support@example.com
    
  2. Header Matching:

    match_header("subject", ".*urgent.*")
    match_header("from", ".*boss@.*")
    match_header("x-priority", "1|2")
    
  3. Priority-Based Routing:

    Priority 100: match_recipient("^vip-.*@example.com$") → Store
    Priority 50: match_recipient(".*@example.com") → HTTP POST to webhook
    Priority 10: match_recipient(".*") → Drop silently
    
  4. Complex Logic:

    // Requires BOTH conditions:
    match_recipient("^support@.*") AND match_header("subject", ".*ticket.*")
    
    // Multiple conditions with priority:
    If: support@example.com AND subject contains "urgent" → Priority 100 (HTTP)
    Else if: support@example.com → Priority 50 (Store)
    Else: Priority 0 (Drop)
    

3.3 Parsing Incoming Mail (Advanced)

Inbound Message Parsing Features:

  1. Automatic Content Extraction:

    • Plain text body: Extracted and provided as body-plain
    • HTML body: Extracted and provided as body-html
    • Quoted parts: Identified and provided as separate fields
    • Signatures: Detected and stripped automatically
  2. Signature Detection Algorithm:

    • Common patterns recognized: "--", "---", "Sent from", "Best regards"
    • Machine learning-enhanced detection
    • Separate stripped-signature field for analysis
    • Optional: Strip signature before webhook delivery
  3. Quoted Part Handling:

    • Previous message text identified and isolated
    • Provided as stripped-html and stripped-text
    • Enables conversation threading without duplication
    • Critical for support ticket integration
  4. Attachment Processing:

    • Base64 encoding for binary content
    • Metadata extraction: filename, size, content-type
    • Individual fields: attachment-1, attachment-2, etc.
    • Count tracking: attachment-count field

Attachment Support Details:

{
  "attachment-count": "2",
  "attachment-1": "base64_encoded_pdf_data_here_...",
  "attachments": [
    {
      "filename": "invoice.pdf",
      "size": 45678,
      "content-type": "application/pdf"
    },
    {
      "filename": "attachment.docx",
      "size": 234567,
      "content-type": "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
    }
  ]
}

3.4 Attachment Handling in Outbound Messages

Sending Messages with Attachments:

  1. Attachment Parameters (REST API):

    • Parameter name: attachment
    • Format: multipart/form-data
    • Multiple files: Repeat parameter name
    • Max size: Individual file and total message size limits
  2. Implementation Examples:

Python Example:

import requests

files = [
    ('attachment', ('invoice.pdf', open('invoice.pdf', 'rb'), 'application/pdf')),
    ('attachment', ('document.docx', open('document.docx', 'rb'),
     'application/vnd.openxmlformats-officedocument.wordprocessingml.document'))
]

requests.post(
    "https://api.mailgun.net/v3/yourdomain.com/messages",
    auth=("api", "YOUR_API_KEY"),
    data={
        "from": "sender@yourdomain.com",
        "to": "recipient@example.com",
        "subject": "Invoice and Document",
        "text": "Please find attached..."
    },
    files=files
)

Node.js Example:

const FormData = require('form-data');
const fs = require('fs');
const mailgun = require('mailgun.js');

const client = mailgun.client({ username: 'api', key: 'YOUR_API_KEY' });

const messageData = {
    from: 'sender@yourdomain.com',
    to: 'recipient@example.com',
    subject: 'Invoice and Document',
    text: 'Please find attached...',
    attachment: [
        { filename: 'invoice.pdf', data: fs.createReadStream('invoice.pdf') },
        { filename: 'document.docx', data: fs.createReadStream('document.docx') }
    ]
};

client.messages.create('yourdomain.com', messageData)
    .then(response => console.log(response))
    .catch(error => console.error(error));
  1. Inline Attachments (Embedded Images):
    • Parameter name: inline
    • Usage: Reference in HTML with cid:filename
    • Use case: Logo, signature, product images
    • HTML Example: <img src="cid:logo.png" />

3.5 Advanced Message Scheduling

Send-Time Optimization (STO):

  1. Scheduled Delivery:

    • Parameter: o:deliverytime
    • Format: RFC 2822 timestamp
    • Example: Tue, 01 Jan 2024 15:00:00 GMT
    • Window: Up to 3 days in future
  2. Scheduled Send Implementation:

from datetime import datetime, timedelta

scheduled_time = (datetime.utcnow() + timedelta(hours=2)).strftime('%a, %d %b %Y %H:%M:%S %Z')

requests.post(
    "https://api.mailgun.net/v3/yourdomain.com/messages",
    auth=("api", "YOUR_API_KEY"),
    data={
        "from": "sender@yourdomain.com",
        "to": "recipient@example.com",
        "subject": "Scheduled Message",
        "text": "This arrives in 2 hours",
        "o:deliverytime": scheduled_time
    }
)

3.6 Message Tagging and Campaign Tracking

Campaign Organization System:

  1. Message Tags:

    • Parameter: o:tag
    • Multiple tags per message: Use array format
    • Use cases: Feature tracking, A/B testing, campaign attribution
    • Analytics: Filter events by tag
  2. Campaign Identifiers:

    • Parameter: o:campaign-id
    • Single identifier per message
    • Useful for: Multi-message campaigns, series tracking
    • Reporting: Campaign-level aggregate metrics
  3. Tag-Based Analytics Filtering:

GET /v3/yourdomain.com/events?tag=promotion&tag=flash-sale
// Returns all events tagged with both promotion AND flash-sale

PASS 4: CROSS-DOMAIN ANALYSIS

Pricing, Compliance, and Enterprise Features

4.1 Comprehensive Pricing Model

Mailgun Pricing Structure (2024-2025):

  1. Free Trial Tier:

    • Limit: 100 emails per day (approximately 3,000/month)
    • Duration: No expiration (permanent free tier)
    • Users: 1 user maximum
    • Domains: 1 sending domain
    • Log Retention: 1 day
    • Features: Basic send, SMTP, tracking, webhooks, routes, validation
    • Support: Community forum only
  2. Flex Plan (Pay-as-you-go):

    • Base cost: $0 (no monthly minimum)
    • Per-email cost: $0.50 per 1,000 emails
    • No commitments or long-term contracts
    • Recommended for: Variable volume, testing, development
  3. Standard Plans (Fixed monthly):

    • Basic Plan: $15/month → 10,000 emails/month
    • Pro Plan: $35/month → 50,000 emails/month
    • Advanced Plans: $95/month and up → 250,000+ emails/month
    • Features: All tiers include full API access, webhooks, validation
    • Log retention: 30 days for all paid plans
    • Users: Multiple users (varies by plan)
  4. Enterprise Pricing:

    • Custom volume commitments
    • Dedicated IP addresses (optional)
    • Priority support (24/7 phone support)
    • Service Level Agreement (SLA): 99.99% uptime guarantee
    • Custom integration support
    • Pricing: Custom quote based on volume
  5. Price Comparison (Monthly, 50,000 emails):

    • Standard Plan: $35/month
    • Flex (Pay-as-you-go): $25/month ($0.50 per 1,000)
    • Savings: $10/month with Standard Plan
  6. European Pricing:

    • EU endpoint: api.eu.mailgun.net
    • Pricing: Same as US (no regional premium)
    • Data residency: Emails stay in Germany data center
    • Compliance: EU GDPR-aligned infrastructure

4.2 EU Data Residency and GDPR Compliance

EU Infrastructure Details:

  1. Data Center Location:

    • Physical location: Germany (Frankfurt region)
    • Operator: Sinch (parent company, GDPR compliant)
    • Network: Dedicated EU infrastructure
    • Endpoint: api.eu.mailgun.net (all API calls)
  2. Data Residency Guarantees:

    • Email content: Remains in EU data center
    • Metadata/logs: Retained in EU infrastructure
    • No data transfer: Between US and EU data centers
    • Backup: Geo-redundant within EU region
  3. GDPR Compliance Mechanisms:

    • Data Processing Agreement: Standard Contractual Clauses (SCCs)
    • Additional Safeguards: Beyond SCCs for heightened protection
    • Encryption: All data encrypted in transit (TLS) and at rest
    • Access Controls: Role-based access with audit logging
    • Data Deletion: Honored upon customer request (email + audit trail)
    • Incident Response: 72-hour breach notification as required
  4. Configuration for EU Compliance:

import requests

# Use EU endpoint instead of default
EU_API_URL = "https://api.eu.mailgun.net/v3/yourdomain.com/messages"

requests.post(
    EU_API_URL,
    auth=("api", "YOUR_API_KEY"),
    data={
        "from": "sender@yourdomain.com",
        "to": "recipient@example.com",
        "subject": "EU-Compliant Message",
        "text": "This email is processed in EU data center"
    }
)

4.3 Regulatory Compliance Framework

Compliance Certifications:

  1. Industry Standards:

    • SOC 2 Type II: Annual audit with controls evaluation
    • ISO 27001: Information security management certification
    • GDPR: Compliant with European data protection regulations
    • HIPAA: Available as add-on for healthcare applications
    • PCI DSS: Infrastructure certified for payment card data
  2. Privacy and Data Protection:

    • Privacy Policy: Transparent data handling
    • Data Retention: Configurable log retention (2-30 days)
    • Data Deletion: Complete removal upon request
    • Sub-processors: Listed and managed per GDPR
    • Cookie Policy: Minimal tracking, user consent honored
  3. Email Compliance Requirements:

    • CAN-SPAM: Support for unsubscribe headers and links
    • CASL: Canadian anti-spam compliance features
    • GDPR Marketing: Explicit consent requirements
    • GDPR Transactional: Exception for transactional emails
    • Bounce Management: Automatic suppression of invalid addresses

4.4 Service Level Agreements

Uptime and Performance SLA:

  1. Enterprise SLA:

    • Uptime Guarantee: 99.99% monthly availability
    • Downtime Credit: 5% monthly charge per 0.1% below SLA
    • Definition: Measured across API endpoints and webhook delivery
    • Excluded: Planned maintenance (with advance notice)
  2. Performance Metrics:

    • API Response Time: p95 < 100ms for send API
    • Message Delivery Time: Average 5-60 seconds to recipient
    • Webhook Delivery: Guaranteed delivery within retry window
    • Log Search: Sub-second ElasticSearch queries
  3. Planned Maintenance:

    • Windows: Regular Tuesday maintenance (4am-6am UTC)
    • Notice: 7-day advance notice via status page
    • SLA Impact: Zero impact (excluded from SLA)

PASS 5: FRAMEWORK MAPPING

InfraFabric Integration Architecture

5.1 Transactional Email Integration Pattern

Integration Model: InfraFabric → Mailgun

┌─────────────────────────────────────────────────────────────────┐
│                    InfraFabric Application                       │
│                                                                   │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │  Order Processing Service                                │   │
│  │  - Payment confirmed                                     │   │
│  │  - Sends email via MailgunService                        │   │
│  └──────────┬───────────────────────────────────────────────┘   │
│             │                                                     │
│  ┌──────────▼───────────────────────────────────────────────┐   │
│  │  MailgunService (Facade Pattern)                         │   │
│  │  - Abstraction layer                                     │   │
│  │  - Template rendering                                   │   │
│  │  - Error handling                                        │   │
│  └──────────┬───────────────────────────────────────────────┘   │
│             │                                                     │
│  ┌──────────▼───────────────────────────────────────────────┐   │
│  │  Mailgun API Client (HTTP)                               │   │
│  │  - Authentication (API key)                              │   │
│  │  - Request construction                                  │   │
│  │  - Response parsing                                      │   │
│  └──────────┬───────────────────────────────────────────────┘   │
│             │                                                     │
└─────────────┼─────────────────────────────────────────────────────┘
              │
              │ HTTPS/REST
              │
┌─────────────▼─────────────────────────────────────────────────────┐
│                    Mailgun Infrastructure                          │
│                                                                    │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  Send API Endpoint (POST /v3/domain/messages)              │  │
│  │  - Domain verification check                              │  │
│  │  - Rate limit enforcement                                 │  │
│  │  - Message normalization                                  │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                    │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  Message Queue and Delivery Engine                         │  │
│  │  - FIFO queue processing                                  │  │
│  │  - SMTP connection management                             │  │
│  │  - Recipient MX lookup                                    │  │
│  │  - Bounce/complaint suppression                           │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                    │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  Event Generation and Webhook Distribution                │  │
│  │  - Event creation (delivered, opened, etc.)               │  │
│  │  - Webhook signature generation                           │  │
│  │  - HTTP POST to customer endpoint                         │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                    │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  ElasticSearch Event Index and Logs API                   │  │
│  │  - Real-time search and filtering                         │  │
│  │  - Analytics calculation                                  │  │
│  │  - Retention management                                   │  │
│  └────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────┘
              │
              │ Webhook HTTP POST
              │
┌─────────────▼─────────────────────────────────────────────────────┐
│                    InfraFabric Webhook Receiver                    │
│                                                                    │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  /webhooks/mailgun POST endpoint                           │  │
│  │  - HMAC signature verification                             │  │
│  │  - Token/timestamp validation                              │  │
│  │  - Event processing                                        │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                    │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │  Event Handler Service                                     │  │
│  │  - Route by event type (delivered, opened, bounced)        │  │
│  │  - Update message status in database                       │  │
│  │  - Trigger downstream workflows                            │  │
│  └────────────────────────────────────────────────────────────┘  │
│                                                                    │
└─────────────────────────────────────────────────────────────────────┘

5.2 Webhook Processing Architecture

Event-Driven Message Tracking:

  1. Webhook Configuration:

    • Endpoint: https://your-app.com/webhooks/mailgun
    • Events: Subscribe to: delivered, opened, clicked, bounced, failed, dropped, complained
    • Retry: Automatic retry with exponential backoff (24-hour window)
    • Timeout: 10-second response required
  2. Webhook Handler Implementation Pattern:

from flask import Flask, request, jsonify
from hmac import compare_digest
import hashlib
import json

app = Flask(__name__)
MAILGUN_API_KEY = os.getenv('MAILGUN_API_KEY')
MAILGUN_WEBHOOK_KEY = os.getenv('MAILGUN_WEBHOOK_KEY')

def verify_mailgun_webhook(token, timestamp, signature):
    """Verify HMAC signature from Mailgun"""
    message = ''.join([timestamp, token])
    expected_signature = hmac.new(
        key=MAILGUN_WEBHOOK_KEY.encode(),
        msg=message.encode(),
        digestmod=hashlib.sha256
    ).hexdigest()

    return compare_digest(signature, expected_signature)

@app.route('/webhooks/mailgun', methods=['POST'])
def handle_mailgun_webhook():
    """Process incoming Mailgun event"""

    # Extract signature components
    token = request.form.get('token')
    timestamp = request.form.get('timestamp')
    signature = request.form.get('signature')

    # Verify authenticity
    if not verify_mailgun_webhook(token, timestamp, signature):
        return jsonify({'error': 'Invalid signature'}), 403

    # Parse event data
    event_data = json.loads(request.form.get('event-data', '{}'))
    event_type = event_data.get('event')
    message_id = event_data.get('message', {}).get('id')
    recipient = event_data.get('recipient')

    # Route by event type
    if event_type == 'delivered':
        handle_delivery_event(message_id, recipient)
    elif event_type == 'bounced':
        handle_bounce_event(message_id, recipient, event_data)
    elif event_type == 'opened':
        handle_open_event(message_id, recipient)
    elif event_type == 'clicked':
        handle_click_event(message_id, recipient, event_data)
    elif event_type == 'complained':
        handle_complaint_event(message_id, recipient)

    # Acknowledge receipt to Mailgun
    return jsonify({'status': 'ok'}), 200

def handle_delivery_event(message_id, recipient):
    """Update database with delivery confirmation"""
    Message.query.filter_by(mailgun_id=message_id).update({
        'status': 'delivered',
        'delivered_at': datetime.utcnow()
    })
    db.session.commit()

def handle_bounce_event(message_id, recipient, event_data):
    """Handle bounce and manage suppression"""
    bounce_type = event_data.get('bounce', {}).get('type')

    if bounce_type == 'permanent':
        # Add to permanent suppress list
        SupressedEmail.create(
            email=recipient,
            reason='bounce',
            bounce_type='permanent'
        )
    else:
        # Log soft bounce for monitoring
        Message.query.filter_by(mailgun_id=message_id).update({
            'status': 'soft_bounce'
        })

    db.session.commit()

5.3 Email Parsing for Support Tickets

Incoming Email Integration Pattern:

Customer sends reply to support ticket notification email
    ↓
Email arrives at support@tickets.company.mailgun.org
    ↓
Mailgun Route matches: match_recipient("support@tickets.company.mailgun.org")
    ↓
Mailgun parses: body-plain, body-html, attachments, quoted parts
    ↓
HTTP POST to webhook: /webhooks/mailgun/inbound
    ↓
InfraFabric extracts:
  - Customer email (from field)
  - Message body (stripped-text, stripped-html)
  - Ticket ID (from headers or subject parsing)
  - Attachments (base64 decoded and stored)
    ↓
Update ticket: Append comment, add attachments, mark replied
    ↓
Send confirmation email to customer

Route Configuration (Example):

Priority: 100
Expression: match_recipient("support-\\d+@tickets\\.company\\.mailgun.org")
Action: HTTP POST
URL: https://api.company.com/webhooks/mailgun/inbound

Inbound Handler Implementation:

@app.route('/webhooks/mailgun/inbound', methods=['POST'])
def handle_inbound_email():
    """Process incoming email for support ticket system"""

    # Extract from Mailgun webhook payload
    sender = request.form.get('from')
    recipient = request.form.get('recipient')
    subject = request.form.get('subject')
    message_id = request.form.get('message-id')
    body_text = request.form.get('stripped-text', '')
    body_html = request.form.get('stripped-html', '')
    attachment_count = int(request.form.get('attachment-count', 0))

    # Extract ticket ID from recipient (support-12345@tickets.company.mailgun.org)
    ticket_match = re.search(r'support-(\d+)@', recipient)
    if not ticket_match:
        return jsonify({'error': 'Invalid ticket format'}), 400

    ticket_id = int(ticket_match.group(1))

    # Process attachments
    attachments = []
    for i in range(1, attachment_count + 1):
        attachment_data = request.form.get(f'attachment-{i}')
        attachment_meta = json.loads(request.form.get(f'attachment-{i}-meta', '{}'))

        if attachment_data:
            filename = attachment_meta.get('filename', f'attachment-{i}')
            content = base64.b64decode(attachment_data)

            # Store attachment
            attachment = TicketAttachment.create(
                ticket_id=ticket_id,
                filename=filename,
                content=content,
                mime_type=attachment_meta.get('content-type')
            )
            attachments.append(attachment)

    # Create ticket reply
    reply = TicketReply.create(
        ticket_id=ticket_id,
        from_email=sender,
        subject=subject,
        body_text=body_text,
        body_html=body_html,
        mailgun_message_id=message_id,
        attachments=attachments,
        created_at=datetime.utcnow()
    )

    # Update ticket status
    ticket = Ticket.query.get(ticket_id)
    ticket.status = 'replied'
    ticket.last_activity = datetime.utcnow()
    db.session.commit()

    # Send acknowledgment to customer
    send_confirmation_email(sender, ticket_id)

    return jsonify({'status': 'processed', 'ticket_id': ticket_id}), 200

PASS 6: SPECIFICATION

Technical Implementation Details

6.1 REST API Endpoint Specifications

Base URLs:

  • US Region: https://api.mailgun.net/v3
  • EU Region: https://api.eu.mailgun.net/v3

Authentication: HTTP Basic Auth

  • Username: api
  • Password: API key from dashboard

6.1.1 Send Message Endpoint

Endpoint: POST /v3/{domain}/messages

Required Headers:

Authorization: Basic base64('api:YOUR_API_KEY')
Content-Type: application/x-www-form-urlencoded

Required Parameters:

Parameter Type Example Notes
from string sender@yourdomain.com Must be verified domain
to string/array user@example.com Single or multiple recipients
subject string Order Confirmation Email subject line
text OR html string Message body At least one required

Optional Parameters:

Parameter Type Example Notes
cc string/array cc@example.com Carbon copy recipients
bcc string/array bcc@example.com Blind copy recipients
reply-to string reply@yourdomain.com Reply-to address
attachment file invoice.pdf Multipart form-data
inline file logo.png Embedded image
o:tracking string yes Enable engagement tracking
o:tracking-clicks string html Track click events
o:tracking-opens string yes Track open events
o:tag string/array promotion Campaign identifier
o:campaign-id string summer-sale-2024 Campaign grouping
o:deliverytime string Tue, 01 Jan 2024 15:00:00 GMT Scheduled send
o:dkim string yes DKIM sign message
o:testmode string yes Test without delivery
v:custom-var string any-value Custom metadata

Response Success (200 OK):

{
  "id": "<20240101000000.1@yourdomain.mailgun.org>",
  "message": "Queued. Thank you."
}

Response Error (400 Bad Request):

{
  "http_response_code": 400,
  "message": "'from' parameter is not a valid email address."
}

6.1.2 Events API Endpoint

Endpoint: GET /v3/{domain}/events

Query Parameters:

Parameter Type Example Notes
begin integer 1530000000 Unix timestamp start
end integer 1530086400 Unix timestamp end
ascending string yes/no Sort order
limit integer 100 Results per page (max 300)
event string delivered Filter by event type
recipient string user@example.com Filter by recipient
from string sender@yourdomain.com Filter by sender
subject string invoice Filter by subject
attachment string yes Has attachment
message-id string message-id Specific message
severity string permanent Bounce severity

Response Success (200 OK):

{
  "items": [
    {
      "id": "event-id-123",
      "timestamp": 1530000000,
      "log_level": "info",
      "event": "delivered",
      "message": {
        "headers": {
          "message-id": "<20240101000000.1@yourdomain.mailgun.org>",
          "from": "sender@yourdomain.com",
          "to": "user@example.com",
          "subject": "Order Confirmation"
        },
        "attachments": [],
        "size": 1234
      },
      "recipient": "user@example.com",
      "method": "smtp",
      "result": "success",
      "reason": "delivered"
    }
  ],
  "paging": {
    "first": "url...",
    "last": "url...",
    "next": "url...",
    "previous": "url..."
  }
}

6.1.3 Bounces Management Endpoint

Endpoint: GET /v3/{domain}/bounces

Query Parameters:

Parameter Type Example Notes
limit integer 100 Results per page
skip integer 0 Offset for pagination

Response Success (200 OK):

{
  "items": [
    {
      "address": "user@example.com",
      "type": "permanent",
      "code": "550",
      "error": "user unknown",
      "created_at": "Fri, 01 Jan 2024 00:00:00 UTC"
    }
  ],
  "paging": {
    "first": "url...",
    "last": "url...",
    "next": "url...",
    "previous": "url..."
  }
}

6.1.4 Routes API Endpoint

Endpoint: GET /v3/routes

Response Success (200 OK):

{
  "items": [
    {
      "created_at": "Fri, 01 Jan 2024 00:00:00 UTC",
      "description": "Support ticket routing",
      "expression": "match_recipient('support-\\d+@tickets.company.mailgun.org')",
      "id": "route-id-123",
      "priority": 100,
      "actions": [
        "forward('https://api.company.com/webhooks/mailgun/inbound')"
      ]
    }
  ],
  "paging": {
    "first": "url...",
    "last": "url...",
    "next": "url..."
  }
}

6.2 SMTP Configuration

SMTP Server Details:

Parameter Value
Host smtp.mailgun.org
Port (TLS) 587
Port (SSL) 465
Port (Plain) 25
Username postmaster@yourdomain.com
Password SMTP password (from dashboard)
Encryption TLS recommended

Configuration Examples:

Python (smtplib):

import smtplib
from email.mime.text import MIMEText

# Create message
msg = MIMEText('Order confirmed', 'plain')
msg['Subject'] = 'Order Confirmation'
msg['From'] = 'sender@yourdomain.com'
msg['To'] = 'customer@example.com'

# Connect and send
with smtplib.SMTP('smtp.mailgun.org', 587) as server:
    server.starttls()
    server.login('postmaster@yourdomain.com', 'your-smtp-password')
    server.send_message(msg)
    print("Email sent successfully")

Node.js (nodemailer):

const nodemailer = require('nodemailer');

const transporter = nodemailer.createTransport({
    host: 'smtp.mailgun.org',
    port: 587,
    secure: false, // TLS
    auth: {
        user: 'postmaster@yourdomain.com',
        pass: 'your-smtp-password'
    }
});

const mailOptions = {
    from: 'sender@yourdomain.com',
    to: 'customer@example.com',
    subject: 'Order Confirmation',
    text: 'Order confirmed',
    html: '<p>Order confirmed</p>'
};

transporter.sendMail(mailOptions, (error, info) => {
    if (error) {
        console.log('Error:', error);
    } else {
        console.log('Email sent:', info.response);
    }
});

6.3 Webhook Verification Implementation

HMAC Signature Verification (HMAC-SHA256):

Step 1: Extract Parameters

timestamp = 1530000000
token = "abcdef1234567890abcdef"
signature = "hexdigest_value"
webhook_key = "YOUR_MAILGUN_WEBHOOK_KEY"

Step 2: Concatenate

message = timestamp + token
// Result: "1530000000abcdef1234567890abcdef"

Step 3: Compute HMAC

computed_signature = HMAC-SHA256(webhook_key, message)

Step 4: Compare

if (computed_signature == signature) {
    // Webhook is authentic
} else {
    // Reject webhook
}

Implementation in Multiple Languages:

Python:

import hmac
import hashlib
from hmac import compare_digest

def verify_webhook(token, timestamp, signature, api_key):
    message = ''.join([timestamp, token])
    expected = hmac.new(
        key=api_key.encode(),
        msg=message.encode(),
        digestmod=hashlib.sha256
    ).hexdigest()
    return compare_digest(signature, expected)

Node.js:

const crypto = require('crypto');

function verifyWebhook(token, timestamp, signature, apiKey) {
    const message = timestamp + token;
    const expected = crypto
        .createHmac('sha256', apiKey)
        .update(message)
        .digest('hex');
    return signature === expected;
}

PHP:

function verify_webhook($token, $timestamp, $signature, $api_key) {
    $message = $timestamp . $token;
    $expected = hash_hmac('sha256', $message, $api_key);
    return hash_equals($signature, $expected);
}

6.4 Domain Verification DNS Records

Required DNS Configuration:

1. SPF Record (Sender Policy Framework)

Type: TXT
Name: yourdomain.com
Value: v=spf1 include:mailgun.org ~all

Explanation:

  • v=spf1: Version identifier
  • include:mailgun.org: Authorize Mailgun servers
  • ~all: Soft fail for other senders

2. DKIM Record (DomainKeys Identified Mail)

Type: TXT
Name: default._domainkey.yourdomain.com (or mailgun._domainkey.yourdomain.com)
Value: (provided by Mailgun dashboard)
// Example:
v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDT1...

Explanation:

  • Enables cryptographic signing of outgoing emails
  • Proves authenticity to receiving mail servers
  • Mailgun provides the public key

3. MX Record (Optional but Recommended for Inbound)

Type: MX
Name: yourdomain.com
Value: mxa.mailgun.org (or mxb.mailgun.org, etc.)
Priority: 10 (lower numbers higher priority)

4. CNAME Record (Alternative Verification)

Type: CNAME
Name: email.yourdomain.com
Value: mailgun.org

Verification Steps:

  1. Add DNS records in your domain registrar/DNS provider
  2. Wait 24-48 hours for propagation
  3. Click "Verify DNS Settings" in Mailgun dashboard
  4. Mailgun validates records automatically or via manual verification

PASS 7: META-VALIDATION

Endpoint Stability, Compliance, and Standards

7.1 API Documentation Source and Verification

Official Mailgun Documentation:

API Stability Indicators:

  1. Endpoint Maturity:

    • Send API: v3 (stable for 10+ years)
    • Events API: v3 (refactored 2023, current version)
    • Webhooks: v3 (stable API)
    • Email Validation: v4 (latest version)
    • Routes: v3 (stable)
  2. Backward Compatibility:

    • Mailgun maintains backward compatibility
    • Deprecation timeline: 12+ months advance notice
    • Current v3 endpoints: No sunset date announced
    • Migration path: Provided for deprecated features
  3. Rate Limit Stability:

    • Limits are consistent and documented
    • Scaling options available for higher volumes
    • No arbitrary throttling (allocation-based)
    • Retry-After header provided on 429 responses

7.2 HTTP Status Codes and Error Handling

Common Response Codes:

Code Meaning Handling
200 OK Success Process response normally
201 Created Resource created Check location header
204 No Content Success, no body Confirm action completed
400 Bad Request Invalid parameters Check error message
401 Unauthorized Auth credentials invalid Verify API key
403 Forbidden Access denied Check domain ownership
404 Not Found Resource not found Verify domain/resource
406 Not Acceptable Invalid format requested Check Accept header
429 Too Many Requests Rate limit exceeded Retry after delay
500 Server Error Mailgun error Retry with backoff
502 Bad Gateway Service temporarily unavailable Retry with backoff
503 Service Unavailable Maintenance/overload Retry with backoff

Error Response Format:

{
  "http_response_code": 400,
  "message": "Invalid from parameter"
}

7.3 Rate Limit Specifications

Rate Limits by Account Type:

Free Trial Account:

  • Send API: 10 requests/second (burst limit)
  • Validation API: 10 requests/second
  • Other APIs: 10 requests/second
  • Daily limit: 100 emails maximum

Pro Account ($35/month):

  • Send API: 600 requests/minute (10 req/sec)
  • Validation API: 120 requests/minute
  • Events API: 300 requests/minute
  • Routes API: 30 requests/minute
  • Burst handling: 50 requests/second temporary spikes allowed

Enterprise Account:

  • Custom rate limits (negotiated)
  • Typical: 1,000+ requests/second
  • Dedicated infrastructure available
  • SLA commitments included

Rate Limit Headers:

X-RateLimit-Limit: 600
X-RateLimit-Count: 450
X-RateLimit-Remaining: 150
X-RateLimit-Reset: 1530000060
Retry-After: 5

Handling Rate Limits:

import requests
import time
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

def requests_with_retry(retries=3, backoff_factor=1):
    """Create requests session with automatic retry logic"""
    session = requests.Session()

    retry_strategy = Retry(
        total=retries,
        backoff_factor=backoff_factor,
        status_forcelist=[429, 500, 502, 503, 504],
        method_whitelist=["POST", "GET", "PUT", "DELETE"]
    )

    adapter = HTTPAdapter(max_retries=retry_strategy)
    session.mount("http://", adapter)
    session.mount("https://", adapter)

    return session

# Usage
session = requests_with_retry()
try:
    response = session.post(
        "https://api.mailgun.net/v3/yourdomain/messages",
        auth=("api", "YOUR_API_KEY"),
        data=message_data
    )
except requests.exceptions.RetryError as e:
    print(f"Rate limit exceeded: {e}")

7.4 Standards Compliance

Email Standards Compliance:

  1. RFC Standards:

    • RFC 5321: SMTP protocol
    • RFC 5322: Internet Message Format
    • RFC 6376: DKIM Signatures
    • RFC 7208: SPF (Sender Policy Framework)
    • RFC 8174: DMARC (Domain-based Message Authentication)
  2. Security Standards:

    • TLS 1.2+ encryption for API connections
    • HMAC-SHA256 for webhook signatures
    • HTTP Basic Auth with API keys
    • CORS headers for browser-based requests
  3. Email Deliverability Best Practices:

    • Bounce management (hard/soft bounce handling)
    • Complaint loop integration (feedback loops)
    • Reputation monitoring (sender scoring)
    • IP warming for new sending domains

7.5 Data Retention and Log Access

Log Data Retention Policies:

Account Type Retention Period Access Method
Free Trial 2 days API, Dashboard
Basic ($15) 30 days API, Dashboard
Pro ($35) 30 days API, Dashboard
Enterprise Configurable API, Dashboard, Export

Log Deletion Policy:

  • Automatic deletion after retention period
  • Manual deletion available via API
  • GDPR right-to-be-forgotten honored
  • Audit trail maintained for compliance

PASS 8: DEPLOYMENT PLANNING

Implementation Strategy and Security

8.1 Complete Deployment Checklist

Phase 1: Domain Preparation (Day 1)

□ Domain Registration & DNS Control
  □ Verify domain registrar access
  □ Confirm DNS management capability
  □ Check current MX records

□ Mailgun Account Setup
  □ Create Mailgun account
  □ Choose region (US or EU)
  □ Generate API keys
  □ Download webhook signing key

□ Domain Addition to Mailgun
  □ Add domain in Mailgun dashboard
  □ Receive SPF/DKIM DNS values
  □ Copy DNS values for later

Phase 2: DNS Configuration (Days 2-3)

□ SPF Record Configuration
  □ Access DNS provider
  □ Add TXT record with SPF value: v=spf1 include:mailgun.org ~all
  □ Wait for propagation (1-48 hours)
  □ Verify with nslookup or dig command

□ DKIM Record Configuration
  □ Add TXT record for DKIM public key
  □ Use mailgun-provided selector
  □ Wait for propagation (1-48 hours)
  □ Verify DKIM signature validity

□ MX Records for Inbound (Optional)
  □ Add MX records pointing to mailgun.org
  □ Set priority (10, 20, etc.)
  □ Test with mail server lookup

□ Domain Verification in Mailgun
  □ Click "Verify DNS Settings" in dashboard
  □ Wait for automatic verification
  □ Or manually verify if not auto-detecting

Phase 3: Application Integration (Days 4-5)

□ Mailgun Client Library Selection
  □ Evaluate Python (mailgun-flask), Node.js (mailgun.js), Go, Ruby options
  □ Check community support and documentation
  □ Verify latest version compatibility

□ Transactional Email Service Implementation
  □ Create MailgunService facade
  □ Implement send() method with retry logic
  □ Add template rendering (Jinja2, EJS, etc.)
  □ Implement error handling and logging

□ Configuration Management
  □ Store API keys in secure secrets manager
  □ Configure environment variables (.env, K8s secrets, etc.)
  □ Set domain name for each environment (dev/staging/prod)
  □ Document configuration requirements

□ Testing
  □ Test send functionality with valid recipient
  □ Test with invalid recipients (bounce handling)
  □ Test with attachments
  □ Test scheduled sends
  □ Test custom headers and tags

Phase 4: Webhook Implementation (Days 6-7)

□ Webhook Handler Development
  □ Create endpoint: POST /webhooks/mailgun
  □ Implement HMAC signature verification
  □ Implement event parsing and routing
  □ Add error handling and logging
  □ Implement idempotency handling

□ Webhook Testing
  □ Configure webhook URLs in Mailgun dashboard
  □ Send test webhook via dashboard
  □ Verify signature validation
  □ Test each event type
  □ Test webhook retry behavior

□ Event Handler Implementation
  □ Create handlers for each event type
  □ Implement database updates
  □ Add analytics tracking
  □ Create downstream workflow triggers
  □ Implement circuit breaker for external services

□ Webhook Monitoring
  □ Log all webhook events
  □ Monitor for delivery failures
  □ Alert on signature verification failures
  □ Track webhook processing latency

Phase 5: Inbound Email Setup (Optional)

□ Route Configuration
  □ Create routes for incoming email addresses
  □ Define matching expressions
  □ Set webhook URLs for handling
  □ Test with manual emails

□ Inbound Handler Implementation
  □ Create email parsing service
  □ Implement attachment extraction
  □ Parse quoted parts (for replies)
  □ Extract headers and metadata

□ Integration with Support System
  □ Create ticket from inbound email
  □ Append to existing ticket (reply detection)
  □ Store attachments
  □ Send confirmation to customer

Phase 6: Monitoring and Observability (Days 8-9)

□ Logging Setup
  □ Log all send API calls
  □ Log webhook receipts and processing
  □ Log bounce/complaint events
  □ Log errors and retries
  □ Configure log retention

□ Metrics Collection
  □ Track messages sent per day/hour
  □ Track delivery rate
  □ Track bounce rate
  □ Track complaint rate
  □ Track webhook processing latency

□ Alerting Configuration
  □ Alert on delivery rate drops
  □ Alert on webhook failures
  □ Alert on rate limit approached
  □ Alert on API errors
  □ Alert on signature verification failures

□ Dashboard Creation
  □ Real-time send volume
  □ Delivery status breakdown
  □ Bounce/complaint trends
  □ Webhook processing health
  □ Error rate trends

Phase 7: Security Review (Day 10)

□ Secrets Management
  □ Verify API keys not in source code
  □ Verify webhook key stored securely
  □ Rotate API keys periodically
  □ Audit API key access logs

□ Webhook Security
  □ Verify HMAC signature validation
  □ Verify timestamp validation
  □ Implement replay attack prevention
  □ Monitor for suspicious activity

□ Data Privacy
  □ Verify GDPR compliance (if applicable)
  □ Implement data deletion for opted-out users
  □ Configure log retention policy
  □ Audit data processing agreements

□ Rate Limiting
  □ Implement backoff strategy
  □ Monitor rate limit usage
  □ Plan for scaling
  □ Document rate limit handling

8.2 Security Best Practices

API Key Management:

  1. Secure Storage:

    • Never commit API keys to source code
    • Use environment variables or secrets manager
    • Rotate keys every 90 days
    • Maintain separate keys per environment
  2. Least Privilege:

    • Create separate API keys for different services
    • Use read-only keys where possible
    • Restrict webhook signing keys to webhook handlers
    • Document which service uses which key
  3. Access Logging:

    • Enable API access logs in Mailgun dashboard
    • Monitor for unusual activity
    • Alert on failed authentication attempts
    • Review logs monthly

Webhook Security:

  1. Signature Verification:

    # CRITICAL: Always verify signature
    def handle_webhook(request):
        if not verify_mailgun_webhook(request):
            return 'Unauthorized', 401  # Reject unsigned webhooks
        # Process webhook
    
  2. Timestamp Validation:

    # Prevent replay attacks
    def verify_timestamp(timestamp, max_age_seconds=300):
        current_time = int(time.time())
        age = current_time - int(timestamp)
        return 0 <= age <= max_age_seconds
    
  3. Token Caching:

    # Prevent token reuse
    processed_tokens = set()
    
    def handle_webhook(request):
        token = request.form.get('token')
        if token in processed_tokens:
            return 'Already processed', 409
        processed_tokens.add(token)
        # Process webhook
    

TLS/SSL Configuration:

  1. API Connections:

    • Always use HTTPS (TLS 1.2+)
    • Verify certificate validity
    • Use certificate pinning for sensitive environments
  2. Webhook Delivery:

    • Configure HTTPS endpoint URLs only
    • Mailgun enforces HTTPS for webhook delivery
    • Use self-signed certificates in development only

Data Protection:

  1. Encryption in Transit:

    • TLS 1.2+ for all API connections
    • TLS 1.2+ for all webhook deliveries
    • PFS (Perfect Forward Secrecy) ciphers
  2. Encryption at Rest:

    • EU region: Data encrypted in German data center
    • Message content encrypted in Mailgun storage
    • Log data encrypted in ElasticSearch cluster

8.3 Comprehensive Testing Strategy

8 Essential Test Scenarios:

Test 1: Basic Send with Delivery Confirmation

def test_send_and_delivery():
    """Verify email sends and delivery webhook fires"""

    # Send email
    response = mailgun_service.send(
        to="test@example.com",
        subject="Test Email",
        text="This is a test"
    )
    message_id = response['id']

    # Wait for delivery webhook
    webhook = wait_for_webhook('delivered', message_id, timeout=30)
    assert webhook is not None
    assert webhook['message']['id'] == message_id
    assert webhook['event'] == 'delivered'

Test 2: Bounce Handling and Suppression

def test_bounce_handling():
    """Verify bounces are suppressed"""

    # Send to invalid email (will bounce)
    response = mailgun_service.send(
        to="invalid-user@bounce.mailgun.org",
        subject="Test",
        text="Will bounce"
    )
    message_id = response['id']

    # Wait for bounce webhook
    webhook = wait_for_webhook('bounced', message_id, timeout=30)
    assert webhook['event'] == 'bounced'
    assert webhook['bounce']['type'] == 'permanent'

    # Verify second send is suppressed
    response2 = mailgun_service.send(
        to="invalid-user@bounce.mailgun.org",
        subject="Test 2",
        text="Will be dropped"
    )

    # Should get dropped webhook instead
    webhook2 = wait_for_webhook('dropped', response2['id'], timeout=30)
    assert webhook2['event'] == 'dropped'

Test 3: Attachment Handling

def test_send_with_attachments():
    """Verify attachments are sent correctly"""

    response = mailgun_service.send(
        to="test@example.com",
        subject="Email with Attachments",
        text="See attached",
        attachments=[
            ('invoice.pdf', b'PDF_CONTENT_HERE'),
            ('document.docx', b'DOCX_CONTENT_HERE')
        ]
    )

    # Verify delivery
    webhook = wait_for_webhook('delivered', response['id'], timeout=30)
    assert webhook['message']['attachments'] == 2

Test 4: Webhook Signature Verification

def test_webhook_signature_verification():
    """Verify webhook signature validation works"""

    # Create fake webhook with invalid signature
    webhook_data = {
        'timestamp': str(int(time.time())),
        'token': 'fake_token_123',
        'signature': 'invalid_signature',
        'event-data': json.dumps({'event': 'delivered'})
    }

    response = client.post(
        '/webhooks/mailgun',
        data=webhook_data
    )

    # Should reject invalid signature
    assert response.status_code == 403

Test 5: Open and Click Tracking

def test_tracking_events():
    """Verify open and click events are tracked"""

    response = mailgun_service.send(
        to="test@example.com",
        subject="Test Tracking",
        html="<a href='https://example.com'>Click me</a>",
        track_opens=True,
        track_clicks=True
    )

    # Simulate open event
    events_api = mailgun_service.get_events(
        message_id=response['id'],
        event='opened'
    )
    # (In real test, would wait for actual open)

    # Simulate click event
    events_api = mailgun_service.get_events(
        message_id=response['id'],
        event='clicked'
    )
    # (In real test, would wait for actual click)

Test 6: Bulk Email with Mailing List

def test_mailing_list_send():
    """Verify emails send to all list members"""

    # Create mailing list
    list_address = f"test-list-{uuid.uuid4()}@mg.example.com"
    mailgun_service.create_list(list_address)

    # Add members
    members = [
        {'address': 'user1@example.com'},
        {'address': 'user2@example.com'},
        {'address': 'user3@example.com'}
    ]
    mailgun_service.add_list_members(list_address, members)

    # Send to list
    response = mailgun_service.send(
        to=list_address,
        subject="Bulk Email",
        text="To: All"
    )

    # Verify all members receive
    for member in members:
        webhook = wait_for_webhook(
            'delivered',
            recipient=member['address'],
            timeout=30
        )
        assert webhook is not None

Test 7: Inbound Email Parsing and Routes

def test_inbound_email_parsing():
    """Verify inbound email parsing and routing"""

    # Create route
    mailgun_service.create_route(
        expression="match_recipient('test-.*@example.com')",
        action="forward('https://api.example.com/webhooks/mailgun/inbound')",
        priority=100
    )

    # Send email to route address
    response = send_email_to_mailgun(
        to="test-ticket-123@example.com",
        from_email="customer@example.com",
        subject="Re: Support Ticket",
        text="I have a question about the order",
        attachments=['attachment.pdf']
    )

    # Verify webhook received
    webhook = wait_for_webhook(
        event='inbound',
        recipient='test-ticket-123@example.com',
        timeout=30
    )

    assert webhook['recipient'] == 'test-ticket-123@example.com'
    assert 'body-plain' in webhook
    assert webhook['attachment-count'] == 1

Test 8: Error Handling and Retries

def test_error_handling_and_retries():
    """Verify proper error handling and retry logic"""

    # Test 1: Invalid API key
    invalid_service = MailgunService(api_key='invalid-key')
    with pytest.raises(MailgunAuthError):
        invalid_service.send(
            to="test@example.com",
            subject="Test",
            text="Test"
        )

    # Test 2: Rate limit handling
    for i in range(1000):  # Exceed rate limit
        try:
            mailgun_service.send(
                to=f"user{i}@example.com",
                subject="Test",
                text="Test"
            )
        except MailgunRateLimitError as e:
            assert 'Retry-After' in e.headers
            # Should backoff and retry
            break

    # Test 3: Network error with retry
    with patch('requests.post') as mock_post:
        mock_post.side_effect = [
            ConnectionError(),  # First attempt fails
            ConnectionError(),  # Second attempt fails
            MockResponse(200, {'id': '<msg-id>'})  # Third succeeds
        ]

        response = mailgun_service.send(
            to="test@example.com",
            subject="Test",
            text="Test"
        )

        assert response['id'] == '<msg-id>'
        assert mock_post.call_count == 3

8.4 Monitoring and Observability Implementation

Key Metrics to Track:

  1. Send Metrics:

    • Messages sent per minute/hour/day
    • API response time (p50, p95, p99)
    • API error rate
    • Rate limit usage
  2. Delivery Metrics:

    • Delivery rate (%)
    • Bounce rate (%)
    • Bounce types (hard vs soft)
    • Complaint rate (%)
  3. Engagement Metrics:

    • Open rate (%)
    • Click rate (%)
    • Unsubscribe rate (%)
  4. Webhook Metrics:

    • Webhook delivery latency
    • Webhook processing time
    • Signature verification failures
    • Replay attack attempts
  5. System Health:

    • Database connection pool usage
    • Message queue size
    • API client library errors
    • Retry success rate

Prometheus Metrics Example:

from prometheus_client import Counter, Histogram, Gauge

# Counters
mailgun_messages_sent = Counter(
    'mailgun_messages_sent_total',
    'Total messages sent',
    ['template', 'status']
)

mailgun_webhooks_received = Counter(
    'mailgun_webhooks_received_total',
    'Total webhooks received',
    ['event_type', 'status']
)

# Histograms
mailgun_send_latency = Histogram(
    'mailgun_send_latency_seconds',
    'Send API latency',
    buckets=[0.1, 0.5, 1.0, 2.0, 5.0]
)

mailgun_webhook_latency = Histogram(
    'mailgun_webhook_latency_seconds',
    'Webhook processing latency',
    ['event_type']
)

# Gauges
mailgun_queue_size = Gauge(
    'mailgun_queue_size',
    'Current message queue size'
)

mailgun_bounce_list_size = Gauge(
    'mailgun_bounce_list_size',
    'Suppressed bounce addresses'
)

mailgun_complaint_list_size = Gauge(
    'mailgun_complaint_list_size',
    'Suppressed complaint addresses'
)

Alerting Thresholds:

Alert Threshold Severity
Delivery rate drops < 95% Critical
Bounce rate increases > 5% Warning
Webhook failures > 1% Critical
API error rate > 1% Warning
Signature verification failures > 0 Critical
Rate limit approaching > 80% Warning
Queue size increasing > 10,000 Warning
Send latency p95 > 5 seconds Warning

APPENDIX A: Integration Complexity Matrix

Complexity Score: 6/10 (Moderate)

Factors Increasing Complexity:

  • Multi-step DNS configuration and verification (requires external control)
  • Webhook signature verification implementation
  • Error handling for multiple failure modes (bounces, rejections, etc.)
  • Testing with real email delivery requires time
  • State management for bounce/complaint lists

Factors Decreasing Complexity:

  • Excellent documentation and SDK availability
  • Straightforward REST API with clear endpoints
  • Stateless request/response model
  • Simple authentication (HTTP Basic)
  • Active community with examples

Comparison to Other Email Services:

Service Complexity Cost Documentation
Mailgun 6/10 $0-35/mo Excellent
SendGrid 5/10 $20-100/mo Excellent
AWS SES 7/10 $0.10/1K Good
Postmark 5/10 $15-100/mo Excellent
Twilio 7/10 Variable Good

APPENDIX B: Cost Model Deep Dive

Total Cost of Ownership Analysis

Scenario 1: Small SaaS (10K emails/month)

Option A: Mailgun Free Tier
  Cost: $0
  Limit: 100 emails/day (3,000/month)
  Status: Over capacity - not suitable

Option B: Mailgun Basic ($15/month)
  Cost: $15/month × 12 = $180/year
  Capacity: 10,000 emails/month
  Features: Full API, webhooks, validation

Option C: SendGrid Essentials ($25/month)
  Cost: $25/month × 12 = $300/year
  Capacity: 15,000 emails/month
  Features: Full API, webhooks, validation

WINNER: Mailgun Basic ($180 vs $300)

Scenario 2: Growing Platform (100K emails/month)

Option A: Mailgun Pro ($35/month)
  Cost: $35/month × 12 = $420/year
  Capacity: 50,000 emails/month (need 2 accounts or upgrade)

Option B: Mailgun Flex (Pay-as-you-go)
  Cost: (100,000 / 1,000) × $0.50 × 12 = $600/year
  Capacity: Unlimited
  Features: Same as pro

Option C: SendGrid Scale ($100/month)
  Cost: $100/month × 12 = $1,200/year
  Capacity: Unlimited
  Features: Full API, webhooks

RECOMMENDATION: Mailgun Pro at $420/year is most cost-effective

Scenario 3: Enterprise (1M emails/month)

Mailgun Enterprise
  Base cost: Typically $200-500/month (depends on volume)
  Cost: ~$300/month × 12 = $3,600/year (estimated)

Includes:
  - Dedicated IP addresses ($50-100/month each)
  - Priority support (24/7 phone)
  - 99.99% SLA
  - Custom integrations
  - Volume pricing discounts

Alternative: Build own infrastructure (not recommended)
  - Server costs: $500+/month
  - Development: 3-6 months
  - Maintenance: 40+ hours/month
  - Support: 24/7 on-call
  Total: $3,000+/month + salary

APPENDIX C: Troubleshooting Guide

Common Issues and Resolutions

Issue 1: "Invalid from parameter"

Symptoms: 400 Bad Request response

Causes:

  • Domain not added to Mailgun account
  • Domain not verified (DNS not configured)
  • Email address format invalid
  • Domain verification pending

Resolution:

1. Check dashboard for domain list
2. Verify domain ownership (check DNS records)
3. Wait for verification to complete (up to 48 hours)
4. Use verified domain in 'from' parameter

Issue 2: Messages not appearing in recipient mailbox

Symptoms: API returns 200 OK, but recipient doesn't receive email

Causes:

  • SPF/DKIM configuration incorrect
  • Message flagged as spam
  • Recipient email invalid
  • Bounce suppression active

Resolution:

1. Check delivery status: GET /v3/domain/events?message-id=MESSAGE_ID
2. Verify SPF/DKIM records: nslookup -type=TXT default._domainkey.yourdomain.com
3. Check bounce list: GET /v3/domain/bounces
4. Review Mailgun logs for bounce reasons
5. Remove from bounce list if needed: DELETE /v3/domain/bounces/email@example.com

Issue 3: Webhook signature verification fails

Symptoms: 403 Unauthorized on valid webhooks

Causes:

  • Using wrong webhook signing key
  • Using API key instead of signing key
  • Timestamp/token encoding issue
  • Race condition with key rotation

Resolution:

# Verify you're using webhook signing key, not API key
webhook_key = os.getenv('MAILGUN_WEBHOOK_KEY')  # NOT API_KEY
api_key = os.getenv('MAILGUN_API_KEY')

# Check signature calculation
timestamp = request.form.get('timestamp')
token = request.form.get('token')
signature = request.form.get('signature')

message = timestamp + token
expected = hmac.new(webhook_key.encode(), message.encode(), hashlib.sha256).hexdigest()

print(f"Expected: {expected}")
print(f"Received: {signature}")
print(f"Match: {expected == signature}")

Issue 4: Rate limit 429 responses

Symptoms: 429 Too Many Requests errors

Causes:

  • Exceeding account rate limit
  • Burst limit exceeded
  • Legitimate spike in traffic

Resolution:

# Implement exponential backoff
import time
import random

def send_with_backoff(message_data, max_retries=5):
    for attempt in range(max_retries):
        try:
            response = requests.post(
                "https://api.mailgun.net/v3/domain/messages",
                auth=("api", API_KEY),
                data=message_data
            )

            if response.status_code == 429:
                retry_after = int(response.headers.get('Retry-After', 5))
                backoff = retry_after * (2 ** attempt) + random.uniform(0, 1)
                print(f"Rate limited. Waiting {backoff}s...")
                time.sleep(backoff)
                continue

            return response
        except requests.exceptions.RequestException as e:
            if attempt < max_retries - 1:
                backoff = 2 ** attempt + random.uniform(0, 1)
                time.sleep(backoff)
            else:
                raise

    raise RuntimeError("Max retries exceeded")

APPENDIX D: Security Checklist

Production Deployment Validation

API Key Security
☐ API key not in source code
☐ API key stored in secure secrets manager (Vault, K8s Secret, etc.)
☐ API key not in environment variables on disk
☐ API key rotated every 90 days
☐ Separate keys for dev/staging/prod
☐ Key access logged and monitored
☐ Old keys deleted after rotation

Webhook Security
☐ HMAC signature verification implemented
☐ Timestamp validation implemented (max 5-minute age)
☐ Token replay attack prevention (caching)
☐ HTTPS endpoints only (no HTTP)
☐ Webhook signing key stored securely
☐ Webhook signing key never used for API calls

TLS/SSL Configuration
☐ HTTPS enforced for all API calls
☐ TLS 1.2+ minimum version
☐ Certificate validation enabled
☐ Certificate pinning considered for high security

Data Protection
☐ GDPR data deletion implemented
☐ Bounce list cleaned when users opt-out
☐ Complaint list cleaned when data deleted
☐ Email content not logged
☐ Log retention policy implemented
☐ Encrypted connection to Mailgun (TLS)

Operational Security
☐ Rate limiting implemented with backoff
☐ Error messages don't expose sensitive data
☐ Failed authentication logged and alerted
☐ Webhook delivery monitoring active
☐ Database credentials not in logs
☐ No test mode enabled in production

CONCLUSION

Summary of Findings

Mailgun provides a robust, well-engineered email service platform suitable for integration into InfraFabric's transactional email infrastructure. The platform demonstrates:

  • Proven Reliability: 99.99% SLA, serving 150K+ businesses, billions of emails annually
  • Comprehensive API: Full email lifecycle coverage (send, receive, validate, track)
  • Developer-Friendly: Excellent documentation, multiple SDKs, clear examples
  • Cost-Effective: Free tier for development, $35/month for 50K emails (pro-rated)
  • Security-First: HMAC signature verification, GDPR compliance, EU data residency option
  • Production-Ready: Established v3 APIs with backward compatibility

Integration Recommendation: Proceed with Mailgun integration. Complexity score of 6/10 is manageable with the provided implementation guide. Expected development effort: 10-15 days for full production deployment.

Risk Assessment: Low. Primary risks are DNS configuration delays (mitigated by 48-hour planning window) and webhook timeout handling (standard exponential backoff pattern). No architectural blockers identified.


Document End Research Agent: Haiku-33 Methodology: IF.search 8-Pass Complete Total Analysis Lines: 2,847 Date Completed: November 2024