Created
May 18, 2016 12:09
-
-
Save Vidimensional/3394c11f2d73b7c29e6cf9509a1d50e0 to your computer and use it in GitHub Desktop.
Copy between different SQS queues.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
import json | |
import time | |
import argparse | |
import boto.sqs | |
from termcolor import cprint | |
parser = argparse.ArgumentParser(description="Migrate messages from SQS queues.") | |
parser.add_argument('-s', '--src', required=True, | |
help='Name of the source queue.') | |
parser.add_argument('-d', '--dst', required=True, | |
help='Name of the destination queue.') | |
parser.add_argument('--region', default='us-east-1', | |
help='The AWS region of the queues (default: \'us-east-1\').') | |
args = parser.parse_args() | |
aws_region = args.region | |
src_queue_name = args.src | |
dst_queue_name = args.dst | |
conn = boto.sqs.connect_to_region(aws_region) | |
src_queue = conn.get_queue(src_queue_name) | |
dst_queue = conn.get_queue(dst_queue_name) | |
while True: | |
messages = src_queue.get_messages(10) | |
for src_message in messages: | |
dst_message = boto.sqs.message.RawMessage() | |
print 'Processing message '+src_message.id | |
msg_body = src_message.get_body() | |
dst_message.set_body(msg_body) | |
dst_queue.write(dst_message) | |
src_queue.delete_message(src_message) | |
if len(messages) <= 0: | |
break |
you saved my life too! :)
A couple of comments for the future:
- the source and destination are just the queue name (no ARN, no URL)
- I've changed to make it work in python2 and python3
- added some minor lines of debug, will send PR
- if you comment line#38, it will just copy
- if you copy a large ammount of messages, make sure to increment the visibility timeout of the queue, or the messages will get back and you will read them again.
Thanks again,
Fede
UPDATE: can't send PR to gist, so here are my changes if help anybody.
very cool, just what I needed and didn't want to write.
Thanks for this gist. Saved me a project launch :)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
because src_queue.delete_message(src_message) this should be called a MoveSQS.py , but thanks for your code , save me a life
and i have to install :
pip install boto cprint termcolor