terraform-ghost-log-forwarder

Terraform module which deploys a Ghost log forwarder to AWS for sending ALB logs to the Ghost platform.

Refer to the Log Based Discovery documentation for more on how this is used in the Ghost platform.

Considerations

• Ensure resources created in AWS are unique to avoid naming conflict errors.
• S3 bucket versioning must be enabled on the source bucket to allow for S3 bucket replication to be configured.
• Only replicate new log files to the log forwarder bucket. Do not replicate existing objects.
• The module expects a Ghost API key with write:logs permissions as the value in the api_key_secret_arn input variable.
  • Use the API Keys page to generate a new key and store this in AWS secrets manager.

Example

The following example deploys a log forwarder and configures S3 ojbect replication to copy log files from an existing source S3 bucket.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.81.0"
    }
    ghost = {
      source  = "ghostsecurity/ghost",
      version = "~> 0.1.0"
    }
  }
}

provider "aws" {
  region = "us-east-2"
}

# This must be updated to reference an existing S3 bucket that is receiving logs
# from your application load balancer.
data "aws_s3_bucket" "source" {
  bucket = "source-bucket-name"
}

# The value in this secret must be a string literal that is the Ghost API key 
# that can be created in your account by navigating to https://app.ghostsecurity.com/settings/apikeys
data "aws_secretsmanager_secret" "ghost_api_key" {
  name = "dev/ghost-api-key"
}

# Deploy the Ghost log forwarder.
# Change the name to something meaningful in your organization.
module "dev-alb-forwarder" {
  source             = "ghostsecurity/log-forwarder/ghost"
  name               = "example-forwarder"
  api_key_secret_arn = data.aws_secretsmanager_secret.ghost_api_key.arn
}

data "aws_s3_bucket" "dest" {
  bucket = module.dev-alb-forwarder.s3_input_bucket
}

# The following resources configure an example S3 replication policy to
# copy logs from the source bucket to the log forwarder bucket so that they
# will be processed and sent to the Ghost platform.
data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["s3.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "replication" {
  name               = "example-replication-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

data "aws_iam_policy_document" "replication" {
  statement {
    effect = "Allow"

    actions = [
      "s3:GetReplicationConfiguration",
      "s3:ListBucket",
    ]

    resources = [data.aws_s3_bucket.source.arn]
  }

  statement {
    effect = "Allow"

    actions = [
      "s3:GetObjectVersionForReplication",
      "s3:GetObjectVersionAcl",
      "s3:GetObjectVersionTagging",
    ]

    resources = ["${data.aws_s3_bucket.source.arn}/*"]
  }

  statement {
    effect = "Allow"

    actions = [
      "s3:ReplicateObject",
      "s3:ReplicateDelete",
      "s3:ReplicateTags",
    ]

    resources = ["${data.aws_s3_bucket.dest.arn}/*"]
  }
}

resource "aws_iam_policy" "replication" {
  name   = "example-replication-policy"
  policy = data.aws_iam_policy_document.replication.json
}

resource "aws_iam_role_policy_attachment" "replication" {
  role       = aws_iam_role.replication.name
  policy_arn = aws_iam_policy.replication.arn
}

resource "aws_s3_bucket_replication_configuration" "replication" {
  role   = aws_iam_role.replication.arn
  bucket = data.aws_s3_bucket.source.id

  rule {
    filter {
      prefix = "AWSLogs"
    }

    status = "Enabled"

    delete_marker_replication {
      status = "Enabled"
    }

    destination {
      bucket        = data.aws_s3_bucket.dest.arn
      storage_class = "STANDARD"
    }
  }
}

Providers

Name	Version
aws	>= 5.81.0

Outputs

Name	Description
s3_input_bucket	The name of the input bucket

Inputs

Name	Description	Type	Default	Required
api_key_secret_arn	ARN of a secret in AWS secrets manager that contains a Ghost API key with write:logs permissions	string	n/a	yes
api_url	Base URL for the Ghost API	string	"https://api.ghostsecurity.com"	no
name	The name for this log forwarder. This must be unique within your AWS account.	string	n/a	yes
tags	Map of tags to assign to all resources. By default resources are tagged with ghost:forwarder_id and ghost:forwarder_name.	map(string)	{}	no

Resources

Name	Type
aws_iam_policy.forwarder_lambda	resource
aws_iam_role.log_converter_role	resource
aws_iam_role_policy_attachment.log_converter_bucket_access	resource
aws_iam_role_policy_attachment.log_converter_logging	resource
aws_lambda_function.log_converter	resource
aws_lambda_permission.log_converter	resource
aws_s3_bucket.input_bucket	resource
aws_s3_bucket_lifecycle_configuration.input_bucket	resource
aws_s3_bucket_notification.input	resource
aws_s3_bucket_versioning.input	resource
aws_caller_identity.current	data source
aws_iam_policy.basic_execution_role	data source
aws_iam_policy_document.forwarder_lambda	data source
aws_iam_policy_document.lambda_assume_role	data source
aws_region.current	data source