How to Set Up a Local AWS Cloud Stack with Localstack

2020-07-19 14:28:00 | #sysadmin #aws

Tested On

  • Linux Ubuntu 20.04
  • Windows 10
  • macOS Catalina

Problem: You need to learn about or test your code against AWS cloud services, locally and for free, without having to sign up for AWS.

Solution: We're going to spin up a mock AWS Cloud environment, locally, using localstack, a test/mock framework that emulates the AWS cloud stack.

Setting Up AWS Credentials for AWS CLI

We're going to be using AWS CLI to perform various operations, so make sure to install it first. Installation steps for macOS and Windows are also available but, keep in mind, the rest of this article was written with Linux and Unix platforms in mind.

Add the following block of text to your ~/.aws/credentials file. This defines the AWS access keys for a LocalStack profile that we will use for authentication with LocalStack. Leave foo and bar, as is, and do not replace them.

[localstack]
aws_access_key_id = foo
aws_secret_access_key = bar

Setting Up A Python Virtual Environment for LocalStack

Using Windows 10 PowerShell 2.0+

cd ~
New-Item -ItemType "directory" -Path ".\python-localstack"
cd python-localstack
virtualenv venv
.\venv\Scripts\activate

To verify that the virtual environment is active, make sure (venv) is in the PowerShell command prompt. For example, (venv) PS C:\Users\username\python-localstack>

Using Linux Ubuntu 14.04+ or macOS

To keep all of the LocalStack dependencies in one place, run the following commands on Linux to create the project skeleton and activate the virtual environment.

cd ~
mkdir python-localstack
cd python-localstack
virtualenv -p python3 venv
source venv/bin/activate

If you're on a different operating system, just make sure your skeleton looks like the following example and that you're running in the virtual environment.

▾ python-localstack/
  ▸ venv/

Installing LocalStack

Now that you've activated your virtual environment, run pip3 install localstack==0.11.3, which is the most current version LocalStack at the time of this article.

Running LocalStack

Navigate to the localStack github to view all of the supported APIs for this release. For this lesson, we're going to learn how to set up an SQS queue and send and receive messages. Run LAMBDA_EXECUTOR=local SERVICES=sqs localstack start --host to spin up the environment.

If you receive a ModuleNotFoundError similar to the following, this just means you need to pip3 install the required module(s) in order to run the program.

Traceback (most recent call last):
    File "main.py", line 2, in 
      import boto3
  ModuleNotFoundError: No module named 'boto3'

After installing all of the dependencies, you should receive the following output, indicating that LocalStack is running and ready to receive requests.

Starting edge router (https port 4566)...
Starting mock SQS service on http ports 4566 (recommended) and 4576 (deprecated)...
Starting multi API server process on port 50227
Running on 0.0.0.0:4566 over https (CTRL + C to quit)
Running on 0.0.0.0:4576 over http (CTRL + C to quit)
Running on 0.0.0.0:50227 over http (CTRL + C to quit)
Ready.

Keep LocalStack running in the foreground. From here on out, you'll want to run your shell commands from a separate tab/window.

Setting Up an SQS Queue

aws sqs create-queue --queue-name Queue --endpoint-url http://0.0.0.0:4566 --region us-east-1 --profile localstack

You should see the following output, which indicates that the queue has been created:

{
    "QueueUrl": "http://localhost:4566/000000000000/Queue"
}

Here are some other commands you may find useful:

# List queues
aws sqs list-queues --endpoint-url http://0.0.0.0:4566 --region us-east-1 --profile localstack
{
    "QueueUrls": [
        "http://localhost:4566/000000000000/Queue"
    ]
}

# Get queue url
aws sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/Queue --endpoint-url http://0.0.0.0:4566 --region us-east-1 --profile localstack
{
    "Attributes": {
        "ApproximateNumberOfMessages": "0",
        "ApproximateNumberOfMessagesDelayed": "0",
        "ApproximateNumberOfMessagesNotVisible": "0",
        "CreatedTimestamp": "1595517750.412922",
        "DelaySeconds": "0",
        "LastModifiedTimestamp": "1595517750.412922",
        "MaximumMessageSize": "262144",
        "MessageRetentionPeriod": "345600",
        "QueueArn": "arn:aws:sqs:us-east-1:000000000000:Queue",
        "Policy": "None",
        "RedrivePolicy": "None",
        "ReceiveMessageWaitTimeSeconds": "0",
        "VisibilityTimeout": "30"
    }
}

# Send message to queue
aws sqs send-message --queue-url localhost:4566/queue/Queue --message-body 'My message' --endpoint-url http://0.0.0.0:4566 --region us-east-1 --profile localstack
{
    "MD5OfMessageBody": "be59f66a07d05b2c52a1387d1f62d753",
    "MessageId": "5caac084-9950-5050-6a57-a7c1890d7e0b"
}

# Send message with attributes to queue (skip to the next section for an example file)
aws sqs send-message --queue-url localhost:4566/queue/Queue --message-body 'My message' --message-attributes file://message-attributes.json --endpoint-url http://0.0.0.0:4566 --region us-east-1 --profile localstack
{
    "MD5OfMessageBody": "be59f66a07d05b2c52a1387d1f62d753",
    "MessageId": "cf812603-7f5c-9e69-a051-0e694ce444d7"
}

# Receive message from queue
aws sqs receive-message --queue-url localhost:4566/queue/Queue --endpoint-url http://0.0.0.0:4566 --region us-east-1 --profile localstack

# Delete queue
aws sqs delete-queue --queue-url http://localhost:4566/000000000000/Queue --endpoint-url http://0.0.0.0:4566 --region us-east-1 --profile localstack

Refer to the AWS CLI Command Reference for more commands.

Here is an example of a message-attributes.json file you can pass into the send-message command.

{
  "myattribute": {
    "DataType": "String",
    "StringValue": "Something"
  },
  "myattribute2": {
    "DataType": "Number",
    "StringValue": "12345"
  }
}

For more information regarding DataType, refer to the AWS CLI Command Reference.

Running AWS CLI Commands in Production

In order to use aws cli commands against AWS in production, you simply need to substitute two values. The first is the --endpoint-url. Instead of pointing to LocalStack running on port 4566, point to the production service endpoint listed under the us-east-1 region on this page. For example, sqs.us-east-1.amazonaws.com. Secondly, you need to create a production profile block in your ~/.aws/credentials file like so:

[production]
aws_access_key_id = PRODUCTION_ACCESS_KEY
aws_secret_access_key = PRODUCTION_SECRET_KEY

Substitute with your own keys. With that out of the way, the full command for creating an SQS queue should look like this:

aws sqs create-queue --queue-name Queue --endpoint-url https://sqs.us-east-1.amazonaws.com --region us-east-1 --profile production

Conclusion

That's the end of this tutorial. We hope you found it helpful. Make sure to check out our other tutorials, as well.

If you're interested in programs that carry out your computer tasks for you, take our Automation the Easy Way with Python course. This course teaches CSV and Excel file generation, API requests, website scraping, email delivery, task scheduling, and browser click, mouse, and keyboard automation. Automate your daily tasks, free up time, and get ahead, today.

Want To See More Exercises?

View Exercises View Courses

Comments

You must log in to comment. Don't have an account? Sign up for free.

Subscribe to comments for this post

Want To Receive More Free Content?

Would you like to receive free resources, tailored to help you reach your IT goals? Get started now, by leaving your email address below. We promise not to spam. You can also sign up for a free account and follow us on and engage with the community. You may opt out at any time.



Tell Us About Your Project









Contact Us

Do you have a specific IT problem that needs solving or just have a general IT question? Use the contact form to get in touch with us and an IT professional will be with you, momentarily.

Hire Us

We offer web development, enterprise software development, QA & testing, google analytics, domains and hosting, databases, security, IT consulting, and other IT-related services.

Free IT Tutorials

Head over to our tutorials section to learn all about working with various IT solutions.

Contact