This is my architecture — AWS

AWS SSM Bastion with Best Practices — Replacing Traditional SSH Bastions.

What are Bastions? Bastion hosts (also called “jump servers”) are usually used in cloud environments to access the private resources like Servers, Databases, Caching servers etc etc.

Since our article is tightly coupled with AWS Cloud, so we will be demonstrating the use of Bastions which are highly available + security optimized and cost optimized as well (as shown below); to access EC2s, RDS or Caching Servers (Elasticache) without the need of cumbersome SSH based authentication, does it sounds interesting? Let’s deep dive.

10,000 Feet — Bastion Architecture on AWS

Let’s deploy the Bastion!

We don’t need to write our own module, let’s not re-invent the wheel instead re use with the slightly different approach. Cloudposse community has already build the awesome terraform module which can be readily used. Let’s have a look on how can we use it?

Terraform module can be found here. Let’s have a look on how to use it following all the best practices.

Pre-requisites

• Bastion Instance IAM Role must contain following policy AmazonSSMManagedInstanceCore. More information can be found here.
• It must have the SSM agent installed on the EC2, information on which of the AMIs comes with SSM agent pre installed can be found here.

High Availability of Bastions! Are you serious?

Yes, by using the module — it creates the EC2s under Autoscaling group which means it can scale up and down based on CPU. You can also set the min_size & max_size parameters to make the Bastions available on multiple AZs (Availability Zones) based on subnets (subnet_ids) provided.

Can they follow Cost Optimization model?

Yes, by using the module, you can also use the Spot instances feature with Bastions to make it cost effective, rather then running them On-Demand basis. The config example can be found here. Also you can take the advantage of using the Capacity Rebalancing as Capacity Rebalancing helps you maintain workload availability by proactively augmenting your fleet with a new Spot Instance before a running instance is interrupted by Amazon EC2.

Can we secure the Bastions?

Yes (once more), we all know that it’s a cumbersome process to make the Bastions secure — the motivation to make it secure is because it runs on public subnets and since we normally use the full fledged AMI (may it be Amazon or Ubuntu or Centos) but its full fledged OS which means it comes with some of the services that we don’t want to be available/running on Bastions (following PCI — DSS model). What are the options? Use the traditional method of maintaining the custom AMIs? Yes that’s one of the option but do we have another option? Yes Let’s have a look how?

data "aws_ami" "this" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["bottlerocket-aws-k8s-????-x86_64-*"]
  }
  filter {
    name   = "architecture"
    values = ["x86_64"]
  }
}

locals {
  userdata = <<-USERDATA
  [settings]
  motd = "Beware!, you are monitored!"
  USERDATA
}

module "bastion" {
  source = "cloudposse/ec2-autoscale-group/aws"
  …
  image_id                    = data.aws_ami.this.id
  …
}

As you noticed in above code, we are using the Bottlerocket OS — which comes by default with the SSM agent installed and also reduces the attack surface by removing the un-wanted services from the Bastion Host (as its optimized for just running containers). More information about the Bottlerocket OS can be found here. Isn’t it a unique implementation?

Let’s use this Bastion!

Now we have the setup ready, how can we use it?

First question, whats the benefit of using this Bastion Host? Check the Title again.. We don’t need SSH key based authentication anymore. We can just use the SSM command to create the tunnel to target resources like RDS or EC2s.

aws ssm start-session \
    --target instance-id \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{"host":["mydb.example.us-east-2.rds.amazonaws.com"],"portNumber":["3306"], "localPortNumber":["3306"]}'

Above command simply lets you create the tunnel to remote host including port forwarding. Detailed guide can be found here.

Any Caveats?

Hmm.. interesting part! the only caveat is that the user trying to use this Bastion must be authenticated user of AWS Account whether IAM User or assuming any IAM Role (SSO based) with the following permissions.

  Action = [
          "ssm:StartSession",
        ],

On below resources

"Resource" : [
          "arn:aws:ssm:*:*:document/AWS-StartPortForwardingSessionToRemoteHost",
          "arn:aws:ec2:*:*:instance/*",
        ]

Without it above solution is not useful.