Simplifying SSH Access with AWS Systems Manager: A Step-by-Step Guide

It’s now much easier to manage and reach your Linux computers on Amazon Web Services, all thanks to the strong abilities of AWS Systems Manager.

You don’t have to worry about tricky stuff like managing special keys for connecting or the chance of things not being safe anymore.

In this guide, we’ll help you step by step to prepare and reach your Linux computers using AWS Systems Manager. Plus, we’ll make sure things are safe and smooth by setting up the right permission rules and ways of doing things.

Know Your Linux Version

Before you begin, it’s important to be aware of the specific version of Linux running on your instance. Different versions have varying availability of the Systems Manager Agent (SSM Agent).

The AWS Systems Manager Agent (SSM Agent) comes pre-installed on certain Amazon Machine Images (AMIs) provided by AWS.

If you launch an Amazon Elastic Compute Cloud (Amazon EC2) instance from an AMI with any of the following operating systems, you’ll likely find that the SSM Agent is already installed:

• Amazon Linux Base AMIs from September 2017 onwards
• Amazon Linux 2
• SUSE Linux Enterprise Server (SLES) versions 12 and 15
• Ubuntu Server versions 16.04, 18.04, 20.04, and 22.04

Depending on whether your instance uses one of these specified distributions, you might have a smooth and straightforward experience (a “happy path”), or you could encounter some challenges (A ¨not-so-happy one¨).

Happy Path

1. Create a New Role: Begin by visiting the IAM dashboard. You can find it in the left sidebar. Once there, press the “Create role” button.
2. Choose a Use Case for the Role: Opt for “AWS service” as the trusted entity type. Then select “EC2 Role for AWS Systems Manager” as your use case. Click “Next: Permissions.”

Click on next

Next again, we will only need the AmazonSSMManagedInstanceCore policy.

Click on Create Role.

And that’s it! Once you’ve successfully crafted an IAM role equipped with the necessary policies, head over to the EC2 dashboard.

Remember: Keep in mind that an EC2 instance can have only one IAM role. If you require an alternate role with a distinct set of policies, simply add the SSMManagedInstanceCore policy to that role.

Once you’ve successfully created the IAM role head over to the EC2 dashboard.

• Attach the IAM Role: Locate your instance, then click on the “Actions” button. Choose “Instance Settings,” followed by “Attach/Replace IAM Role.”

• Select the IAM Role: From the dropdown list, choose the IAM role you previously created. Then click on Update IAM Role.

The update may require 1 to 2 minutes to become active. If it doesn’t, consider rebooting the instance.

The Not-So-Happy Path

So if you are reading until this point it means that your instance doesn't have the SSM Agent installed, so you will need to install it.

Networking Considerations

Depending on your security requirements and architecture, you might consider the following options:

• Instance in Private Subnet: If your instance is not in a public subnet and you need to access it securely, you can set up a bastion host in a public subnet. Connect to the bastion host, and then use SSH to connect to your private instance.
• Public Subnet: Alternatively, you can place your instance in a public subnet and give it a public IP. This allows you to directly access the instance using SSH without the need for a bastion host.

Step 1: Open Necessary Ports

Before proceeding with SSM Agent installation, ensure that your instance security group allows incoming SSH (port 22) traffic.

• If your instance is in a public subnet, restrict the source to your IP address for direct SSH access.
• If your instance is in a private subnet and you’re using a bastion host, set the SSH rule source to the bastion host’s IP address.

Step 2: Find Your Private Key

Ensure you have the private key that matches the key pair used when launching the instance.

Step 3: Connect Using SSH

Use the following command to connect:

For Amazon Linux or most instances:

ssh -i /path/to/your/private/key.pem ec2-user@instance-ip-address

For Ubuntu instances:

ssh -i /path/to/your/private/key.pem ubuntu@instance-ip-address

Replace /path/to/your/private/key.pem with the actual path to your private key file. Also, replace instance-ip-address with either the public or private IP address based on your instance's location.

Example for Public Subnet:

ssh -i ~/.ssh/my-key.pem ec2-user@x.x.x.x

Example for Private Subnet with Bastion Host

ssh -i ~/.ssh/my-key.pem ec2-user@bastion-ip-address

Remember to replace my-key.pem, x.x.x.x, and bastion-ip-address with your actual private key filename, instance IP address, and bastion host IP address.

Note: The exact username (ec2-user, ubuntu, etc.) might vary based on the Linux distribution you're using. Make sure to use the correct username for your instance.

Step 4: Installing SSM Agent

Once inside the instance utilize the appropriate package manager based on your Linux distribution to install SSM Agent.

For Amazon Linux (YUM):

1. Execute the following commands:

sudo yum update -y
sudo yum install -y amazon-ssm-agent
sudo systemctl enable amazon-ssm-agent
sudo systemctl start amazon-ssm-agent

For Ubuntu/Debian (APT):

1. Run these commands:

sudo apt-get update
sudo apt-get install -y amazon-ssm-agent
sudo systemctl enable amazon-ssm-agent
sudo systemctl start amazon-ssm-agent

Step 5: Accessing the Instance

After installing SSM Agent and setting up networking, you can access the instance using SSH or AWS Systems Manager Session Manager.

Troubleshooting Mini Guide

You might encounter issues along the way. This guide will help you troubleshoot common errors when using Session Manager.

1. Verify IAM Permissions:

• Ensure the IAM user or role you’re using has the necessary permissions for Systems Manager. Make sure it has AmazonSSMManagedInstanceCore managed policies attached.

2. Check EC2 Instance Status:

• Ensure the target EC2 instance is running and healthy. If it’s in a stopped state, you won’t be able to establish a Session Manager connection.

3. Review Instance Role:

• Verify that the target instance has an IAM role with permissions to use Systems Manager. The role should include policies like AmazonSSMManagedInstanceCore.

5. Check Network Connectivity:

• Session Manager requires network connectivity to AWS services. If you’re in a VPC with no internet access, you may need to create Systems Manager (SSM) VPC Endpoints. Here is a guide: https://medium.com/@canobertin/implementing-aws-ssm-vpc-endpoints-a-step-by-step-guide-4b6fb12b9386

8. Ensure SSM Agent is Installed:

• Verify that the SSM Agent is installed and running on the target instances. This agent is necessary for Session Manager to work correctly.
• sudo systemctl status amazon-ssm-agent
• https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html

9. Update SSM Agent:

• If the SSM Agent is installed but outdated, update it to the latest version. Sometimes, outdated agents can cause compatibility issues.

10. Check Security Groups and NACLs:

While Systems Manager (SSM) doesn’t require any specific inbound (ingress) rules in the security group of the instance, confirm that there are no outbound (egress) policies in the security group or any NACL blocking traffic.

11. Disable Local Firewalls:

Please confirm that there are no local firewalls or external firewalls blocking the traffic.