Aurora point-in-time recovery (switching copilot add-on to restored DB)

[gidsg]

We have an aurora add-on database for our service as described in: https://aws.github.io/copilot-cli/docs/developing/addons/workload/#how-do-i-add-an-s3-bucket-a-ddb-table-or-an-aurora-serverless-cluster

We would like to test the backup and restore process as described in: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-pitr.html

It appears this will create a new database cluster for the restored database.

How may we modify our add-on to point at this restored database (with the associated credentials)?

[KollaAdithya]

Hello @gidsg !

Sorry for the late response.

Here is a slightly modification of the Copilot generated Addon Aurora template to restore from point in time.

The changes in the template are adding properties RestoreToTime and SourceDBClusterIdentifier in the AWS::RDS::DBCluster resource,

Note: If you are using the same logic id and database name for both source db cluster and newly created db cluster will lead to deletion of the SourceDbCluster. To avoid this please change Logical Ids of the DB cluster and Instance.

References: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#cfn-rds-dbcluster-restoretotime

Parameters:
  App:
    Type: String
    Description: Your application's name.
  Env:
    Type: String
    Description: The environment name your service, job, or workflow is being deployed to.
  Name:
    Type: String
    Description: Your workload's name.
  # Customize your Aurora Serverless cluster by setting the default value of the following parameters.
  MyAuroraDBName:
    Type: String
    Description: The name of the initial database to be created in the Aurora Serverless v2 cluster.
    Default: testaurora
    # Cannot have special characters
    # Naming constraints: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.Constraints

  # New parameters for RestoreTime and SourceDBClusterIdentifier.
  # Please specify RestoreToTime between Earliest Restorable Time and Latest Restorable Time of source cluster.
  RestoreTime:
    Type: String
    Description: "Timestamp to which the database should be restored (format: yyyy-MM-ddTHH:mm:ssZ)."
    Default: "2023-11-23T08:21:00Z"
  SourceDBClusterIdentifier:
    Type: String
    Description: "The ARN of the source DB cluster to be restored from."
    Default: "dhoni-qa-lbws1-addonsstack-pqgem-myauroradbcluster-yp6ofnsbs1bv"
# Conditions based on above parmeters.
Conditions:
  UseRestoreTime: !Not [!Equals [!Ref RestoreTime, ""]]
  UseSourceDBClusterIdentifier: !Not [!Equals [!Ref SourceDBClusterIdentifier, ""]]

Mappings:
  MyAuroraEnvScalingConfigurationMap: 
    demo:
      "DBMinCapacity": 0.5 # AllowedValues: from 0.5 through 128
      "DBMaxCapacity": 8   # AllowedValues: from 0.5 through 128
    
    qa:
      "DBMinCapacity": 0.5 # AllowedValues: from 0.5 through 128
      "DBMaxCapacity": 8   # AllowedValues: from 0.5 through 128
    
    rfer:
      "DBMinCapacity": 0.5 # AllowedValues: from 0.5 through 128
      "DBMaxCapacity": 8   # AllowedValues: from 0.5 through 128
    
    test:
      "DBMinCapacity": 0.5 # AllowedValues: from 0.5 through 128
      "DBMaxCapacity": 8   # AllowedValues: from 0.5 through 128
    
    All:
      "DBMinCapacity": 0.5 # AllowedValues: from 0.5 through 128
      "DBMaxCapacity": 8   # AllowedValues: from 0.5 through 128

Resources:
  MyAuroraDBSubnetGroup:
    Type: 'AWS::RDS::DBSubnetGroup'
    Properties:
      DBSubnetGroupDescription: Group of Copilot private subnets for Aurora Serverless v2 cluster.
      SubnetIds:
        !Split [',', { 'Fn::ImportValue': !Sub '${App}-${Env}-PrivateSubnets' }]
  MyAuroraSecurityGroup:
    Metadata:
      'aws:copilot:description': 'A security group for your workload to access the Aurora Serverless v2 cluster MyAurora'
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupDescription: !Sub 'The Security Group for ${Name} to access Aurora Serverless v2 cluster MyAurora.'
      VpcId:
        Fn::ImportValue:
          !Sub '${App}-${Env}-VpcId'
      Tags:
        - Key: Name
          Value: !Sub 'copilot-${App}-${Env}-${Name}-Aurora'
  MyAuroraDBClusterSecurityGroup:
    Metadata:
      'aws:copilot:description': 'A security group for your Aurora Serverless v2 cluster MyAurora'
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: The Security Group for the Aurora Serverless v2 cluster.
      SecurityGroupIngress:
        - ToPort: 5432
          FromPort: 5432
          IpProtocol: tcp
          Description: !Sub 'From the Aurora Security Group of the workload ${Name}.'
          SourceSecurityGroupId: !Ref MyAuroraSecurityGroup
      VpcId:
        Fn::ImportValue:
          !Sub '${App}-${Env}-VpcId'
      Tags:
        - Key: Name
          Value: !Sub 'copilot-${App}-${Env}-${Name}-Aurora'
  MyAuroraAuroraSecret:
    Metadata:
      'aws:copilot:description': 'A Secrets Manager secret to store your DB credentials'
    Type: AWS::SecretsManager::Secret
    Properties:
      Description: !Sub Aurora main user secret for ${AWS::StackName}
      GenerateSecretString:
        SecretStringTemplate: '{"username": "postgres"}'
        GenerateStringKey: "password"
        ExcludePunctuation: true
        IncludeSpace: false
        PasswordLength: 16
  MyAuroraDBClusterParameterGroup:
    Metadata:
      'aws:copilot:description': 'A DB parameter group for engine configuration values'
    Type: 'AWS::RDS::DBClusterParameterGroup'
    Properties:
      Description: !Ref 'AWS::StackName'
      Family: 'aurora-postgresql14'
      Parameters:
        client_encoding: 'UTF8'
  MyAuroraDBCluster:
    Metadata:
      'aws:copilot:description': 'The MyAurora Aurora Serverless v2 database cluster'
    Type: 'AWS::RDS::DBCluster'
    Properties:
      MasterUsername:
        !Join [ "",  [ '{{resolve:secretsmanager:', !Ref MyAuroraAuroraSecret, ":SecretString:username}}" ]]
      MasterUserPassword:
        !Join [ "",  [ '{{resolve:secretsmanager:', !Ref MyAuroraAuroraSecret, ":SecretString:password}}" ]]
      DatabaseName: !Ref MyAuroraDBName
      Engine: 'aurora-postgresql'
      EngineVersion: '14.4'
      DBClusterParameterGroupName: !Ref MyAuroraDBClusterParameterGroup
      DBSubnetGroupName: !Ref MyAuroraDBSubnetGroup
      Port: 5432
      VpcSecurityGroupIds:
        - !Ref MyAuroraDBClusterSecurityGroup
      ServerlessV2ScalingConfiguration:
        # Replace "All" below with "!Ref Env" to set different autoscaling limits per environment.
        MinCapacity: !FindInMap [MyAuroraEnvScalingConfigurationMap, All, DBMinCapacity]
        MaxCapacity: !FindInMap [MyAuroraEnvScalingConfigurationMap, All, DBMaxCapacity]

      # Added RestoreToTime and SourceDBClusterIdentifier to restore a database from a point in time.
      RestoreToTime: !If 
        - "UseRestoreTime"
        - !Ref RestoreTime
        - !Ref "AWS::NoValue"
      SourceDBClusterIdentifier: !If 
        - "UseSourceDBClusterIdentifier"
        - !Ref SourceDBClusterIdentifier
        - !Ref "AWS::NoValue"
      UseLatestRestorableTime: false

  MyAuroraDBWriterInstance:
    Metadata:
      'aws:copilot:description': 'The MyAurora Aurora Serverless v2 writer instance'
    Type: 'AWS::RDS::DBInstance'
    Properties:
      DBClusterIdentifier: !Ref MyAuroraDBCluster
      DBInstanceClass: db.serverless
      Engine: 'aurora-postgresql'
      PromotionTier: 1
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region

  MyAuroraSecretAuroraClusterAttachment:
    Type: AWS::SecretsManager::SecretTargetAttachment
    Properties:
      SecretId: !Ref MyAuroraAuroraSecret
      TargetId: !Ref MyAuroraDBCluster
      TargetType: AWS::RDS::DBCluster
Outputs:
  MyAuroraSecret: # injected as MyAurora_SECRET environment variable by Copilot.
    Description: "The JSON secret that holds the database username and password. Fields are 'host', 'port', 'dbname', 'username', 'password', 'dbClusterIdentifier' and 'engine'"
    Value: !Ref MyAuroraAuroraSecret
  MyAuroraSecurityGroup:
    Description: "The security group to attach to the workload."
    Value: !Ref MyAuroraSecurityGroup