Created
November 30, 2017 21:14
-
-
Save JenZhao/dc073cf584b846a2ec6c225f97d269d7 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import boto3 | |
def get_s3_object(key, bucket_name, | |
object_component='Body', | |
region_name='us-east-1', | |
encoding='utf-8'): | |
""" | |
Method to read s3 file directly in meomory | |
:param key: key of the file on s3 bucket | |
:param bucket_name: the s3 bucket name | |
:param object_component: S3 object component, Body by default | |
:param region_name: AWS region default is 'us-east-1' | |
:param encoding: Object text encoding to decode. Default utf-8 | |
""" | |
with timer('loading %s from s3' % key): | |
s3 = boto3.resource('s3', region_name) | |
obj = s3.Object(bucket_name, key) | |
result = obj.get()[object_component].read().decode(encoding) | |
return result |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment