CloudFormation Stack Failures, Rollbacks and Drift

Sign in, save the template and helpers:

cd C:\aws-lab
aws login --profile saat-admin
$env:AWS_PROFILE = 'saat-admin'
$v = Get-Content C:\aws-lab\saat-vpc.json | ConvertFrom-Json
$tpl = @'
AWSTemplateFormatVersion: '2010-09-09'
Description: SaatVedha Lesson 67 stack
Parameters:
  VpcId:
    Type: AWS::EC2::VPC::Id
  RoleDescription:
    Type: String
    Default: Lesson 67 app role
Resources:
  LogBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Delete
  AppParam:
    Type: AWS::SSM::Parameter
    Properties:
      Name: /saat/cfn67/env
      Type: String
      Value: prod
  AppSg:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: saat-cfn67-sg
      GroupDescription: Lesson 67 app tier
      VpcId: !Ref VpcId
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 10.20.0.0/16
  AppRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: saat-cfn67-role
      Description: !Ref RoleDescription
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
Outputs:
  BucketName:
    Value: !Ref LogBucket
'@
$tpl | Set-Content cfn67.yaml
function Wait-Stack([string]$Stack) {
  do { Start-Sleep 10; $s = aws cloudformation describe-stacks --stack-name $Stack --query "Stacks[0].StackStatus" --output text } until ($s -notmatch 'IN_PROGRESS$')
  $s
}
function Get-StackFailure([string]$Stack) {
  $ev = aws cloudformation describe-stack-events --stack-name $Stack --query "StackEvents[?ends_with(ResourceStatus, 'FAILED')].[Timestamp,LogicalResourceId,ResourceStatus,ResourceStatusReason]" --output json | ConvertFrom-Json
  $ev | Sort-Object { $_[0] } | Where-Object { $_[3] -notmatch 'cancelled' } | Select-Object -First 1 | ForEach-Object { "$($_[1]) $($_[2]): $($_[3])" }
}
function Get-Drift([string]$Stack) {
  $id = aws cloudformation detect-stack-drift --stack-name $Stack --query StackDriftDetectionId --output text
  do { Start-Sleep 5; $st = aws cloudformation describe-stack-drift-detection-status --stack-drift-detection-id $id --query "[DetectionStatus,StackDriftStatus]" --output text } until ($st -notmatch 'DETECTION_IN_PROGRESS')
  $st
  aws cloudformation describe-stack-resource-drifts --stack-name $Stack --stack-resource-drift-status-filters MODIFIED DELETED --query "StackResourceDrifts[].[LogicalResourceId,StackResourceDriftStatus,PropertyDifferences[0].PropertyPath,PropertyDifferences[0].DifferenceType]" --output text
}
aws cloudformation validate-template --template-body file://cfn67.yaml --query "Parameters[].ParameterKey" --output text

You should see this:

VpcId   RoleDescription

1Deploy the stack

Paste this whole block:

aws cloudformation create-stack --stack-name saat-cfn67 --template-body file://cfn67.yaml --parameters "ParameterKey=VpcId,ParameterValue=$($v.VpcId)" --capabilities CAPABILITY_NAMED_IAM --tags Key=course,Value=saatvedha-aws | Out-Null
Wait-Stack saat-cfn67
aws cloudformation describe-stack-resources --stack-name saat-cfn67 --query "StackResources[].[LogicalResourceId,ResourceStatus]" --output text

You should see something like this:

CREATE_COMPLETE
AppParam        CREATE_COMPLETE
AppRole CREATE_COMPLETE
AppSg   CREATE_COMPLETE
LogBucket       CREATE_COMPLETE

2Preview a change with a change set

Before changing anything, see what CloudFormation would do:

aws cloudformation create-change-set --stack-name saat-cfn67 --change-set-name desc-v2 --use-previous-template --parameters "ParameterKey=VpcId,UsePreviousValue=true" "ParameterKey=RoleDescription,ParameterValue=Lesson 67 app role v2" --capabilities CAPABILITY_NAMED_IAM | Out-Null
aws cloudformation wait change-set-create-complete --stack-name saat-cfn67 --change-set-name desc-v2
aws cloudformation describe-change-set --stack-name saat-cfn67 --change-set-name desc-v2 --query "Changes[].ResourceChange.[LogicalResourceId,Action,Replacement]" --output text
aws cloudformation delete-change-set --stack-name saat-cfn67 --change-set-name desc-v2

You should see this:

AppRole Modify  False

Only the role changes, in place, with no replacement. Change sets are how you avoid surprises such as a database being replaced because one property changed. You deleted it for now; you’ll make this change later.

3Watch a create fail and roll back

Paste this whole block. The second template refers to a managed policy that doesn’t exist:

$bad = @'
AWSTemplateFormatVersion: '2010-09-09'
Description: SaatVedha Lesson 67 failing stack
Resources:
  BadParam:
    Type: AWS::SSM::Parameter
    Properties:
      Name: /saat/cfn67/bad
      Type: String
      Value: created-first
  BadRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: saat-cfn67-bad-role
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/DoesNotExist
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
'@
$bad | Set-Content cfn67-bad.yaml
aws cloudformation create-stack --stack-name saat-cfn67-bad --template-body file://cfn67-bad.yaml --capabilities CAPABILITY_NAMED_IAM | Out-Null
Wait-Stack saat-cfn67-bad
Get-StackFailure saat-cfn67-bad
aws ssm get-parameter --name /saat/cfn67/bad --query Parameter.Value --output text 2>&1 | Select-Object -First 1
aws cloudformation delete-stack --stack-name saat-cfn67-bad
aws cloudformation wait stack-delete-complete --stack-name saat-cfn67-bad

You should see something like this:

ROLLBACK_COMPLETE
BadRole CREATE_FAILED: Policy arn:aws:iam::aws:policy/DoesNotExist does not exist or is not attachable. (Service: Iam, Status Code: 404, Request ID: ...)
An error occurred (ParameterNotFound) when calling the GetParameter operation:

The parameter was created first, then removed by the rollback, so nothing was left half-built. A stack in ROLLBACK_COMPLETE can’t be updated: you fix the template, delete the stack and create it again.

✅ Checkpoint: saat-cfn67 is CREATE_COMPLETE, the change set showed AppRole Modify with no replacement, and saat-cfn67-bad failed on BadRole, rolled back cleanly and was deleted.