Sending Messages to SQS with Python, Flask, and Boto3

2020-07-20 15:28:00 | #programming #python #sysadmin

In this tutorial, we're going to learn how to set up an API that forwards messages to an SQS queue.

By designing our API to leverage AWS Lambda, it will automatically scale across regions to keep up with requests. And by utilizing an SQS queue, we can deploy workers to consume messages without overloading the database.

Completing the Prerequisites

It is important that you complete the prerequisites above, as this article depends on a custom module, and localstack.

Setting Up a Project Skeleton

Run the following commands on Linux to create the project skeleton and activate the virtual environment:

cd ~
mkdir python-sqs-sender
cd python-sqs-sender
virtualenv -p python3 venv
source venv/bin/activate

If you're on a different operating system, just make sure your skeleton looks like the following example and that you're running in the virtual environment. Also, make sure you've imported the file from this article, which will allow our Python Lambda to log to CloudWatch logs.

▾ python-sqs-sender/
  ▸ venv/

Full Code Example for the SQS Message Sender


import os
import boto3
import awsgi
from flask import Flask, request, jsonify
from flask_cors import CORS
import logger
import urllib3

app = Flask(__name__)
CORS(app, resources={r'/*': {'origins': list(os.environ['ORIGINS'].split(','))}}, supports_credentials=True)

    sqs_client = boto3.client(
        use_ssl=os.environ['USE_SSL'] == '1',
except Exception as e:

def json_status(status_code, message):
    res = jsonify({
        'isBase64Encoded': False,
        'statusCode': status_code,
        'body': message
    res.status = '%s %s' % (status_code, message)
    res.message = message
    return res

@app.route('/', methods=['POST'])
def send_message():
    json_data = request.get_json()
    message = json_data.get('message')
    some_attribute = json_data.get('some_attribute')
    another_attribute = json_data.get('another_attribute')

    if message is None:
        return json_status('400', 'Message is required')

    if some_attribute is None:
        return json_status('400', 'Some attribute is required')

    if another_attribute is None:
        return json_status('400', 'Another attribute is required')

        queue_url = sqs_client.get_queue_url(QueueName=os.environ['SQS_QUEUE_NAME'])['QueueUrl']
                'some_attribute': {
                    'DataType': 'String',
                    'StringValue': some_attribute
                'another_attribute': {
                    'DataType': 'String',
                    'StringValue': another_attribute
        logger.ex('Message sent')
        return json_status(200, 'Message sent')
    except Exception as e:
        return json_status(500, str(e))

if __name__ == '__main__':'', port=5100, debug=True)

def lambda_handler(event, context):
    return awsgi.response(app, event, context)

Explanation of the Code

Lines 1-8: imports all the required modules.

Line 10: instantiates our Flask app and line 11 indicates which domains we will allow requests from through CORS.

Lines 13-23: instantiates our SQS client. If you're not familiar with environment variables, they get explained later on in this tutorial.

Lines 26-34: formats a json response that is compatible with AWS Lambda, ensuring that our reponses make it back through API Gateway and into the browser.

Lines 37-56: defines a route we can send requests to that will invoke the Lambda function.

Lines 59-60: allows us to run our Flask app locally on port 5100 and lines 63-64 are necessary for AWS Lambda to invoke the service through Flask routes.

How to Run the Program

Activate your virtual machine by running source venv/bin/activate from inside the project root folder. Then run python to execute the program.

You should receive a ModuleNotFoundError similar to the following. This just means you need to pip install a few dependencies in order to run the program.

Traceback (most recent call last):
    File "", line 2, in 
      import boto3
  ModuleNotFoundError: No module named 'boto3'

Installing Python Module Dependencies

So if you see the above error about boto3, run pip install boto3. When you try to run python again, it will complain about another missing module so pip install that as well. By the end, you should have run the following commands:

source venv/bin/activate
pip install boto3
pip install aws-wsgi
pip install flask
pip install flask_cors

Now, when you run python, you should see another error:

Traceback (most recent call last):
  File "", line 11, in 
    CORS(app, resources={r'/*': {'origins': list(os.environ['ORIGINS'].split(','))}}, supports_credentials=True)
  File "/usr/lib/python3.8/", line 675, in __getitem__
    raise KeyError(key) from None
KeyError: 'ORIGINS'

Setting Environment Variables

While it can be more convenient to add environment variables in our ~/.bashrc file, we're going to set them in the command we use to run the service, making it easier to debug and resolve issues. Run the following command:


ORIGINS is not so important for testing purposes, because we'll be using curl to send requests. But when we launch to production, we'll be able to pass any number of comma-separated domains that we want to allow requests from. Run the following command, for now:

curl -ivvv localhost:5100 -H "Content-Type: application/json" -d '{"message": "My message", "some_attribute": "Something", "another_attribute": "Another thing"}'

If everything was set up properly, you should have received the following output:

  "body": "Message sent", 
  "isBase64Encoded": false, 
  "statusCode": 200

To verify that the SQS queue received the message from our curl request, run the following command:

aws sqs receive-message --queue-url localhost:4566/queue/Queue --endpoint-url --region us-east-1 --profile localstack
    "Messages": [
            "MessageId": "5caac084-9950-5050-6a57-a7c1890d7e0b",
            "ReceiptHandle": "dygasgldmlswerjwymbkcnrsdwnbimzjtvztyxtxiueeqrirxobfrtreapczopidasykadqasvmuuhetozdofpwokydsmerzrnmbghdmyhlcaeabkgmnyzwifnkndtzohebctpxxjzoeylmijjnpxidpqecoecgiwbdiyekvzddmscgfjwtwjzidq",
            "MD5OfBody": "be59f66a07d05b2c52a1387d1f62d753",
            "Body": "My message",
            "Attributes": {
                "SenderId": "AIDAIT2UOQQY3AUEKVGXU",
                "SentTimestamp": "1595518929857",
                "ApproximateReceiveCount": "1",
                "ApproximateFirstReceiveTimestamp": "1595525992436"

This concludes our tutorial. Future articles will cover how to deploy this Lambda function to both localstack and production environments.


You must log in to comment. Don't have an account? Sign up for free.

Subscribe to Our Newsletter

Would you like to receive free whitepapers and other IT news? Just leave your email address below. You may opt out at any time.

Tell Us About Your Project

Contact Us

Do you have a specific IT problem that needs solving or just have a general IT question? Use the contact form to get in touch with us and an IT professional will be with you, momentarily.

Hire Us

We offer web development, enterprise software development, QA & testing, google analytics, domains and hosting, databases, security, IT consulting, and other IT-related services.

Free IT Tutorials

Head over to our tutorials section to learn all about working with various IT solutions.