Skip to content

Instantly share code, notes, and snippets.

@imasimali

imasimali/Google Drive to S3 Uploader Colab

Created July 4, 2021 10:41
Show Gist options
  • Save imasimali/f6fc0617756e83a6476c307aa4cdce23 to your computer and use it in GitHub Desktop.
Save imasimali/f6fc0617756e83a6476c307aa4cdce23 to your computer and use it in GitHub Desktop.
Download ZIP
Upload files from google drive to S3 directly using Google Colab - I am using Linode S3
Raw
Google Drive to S3 Uploader Colab
## Create a connection to Linode S3 file storage
!pip install boto3
AWS_S3_ENDPOINT_URL = f'https://{REGION_NAME}.linodeobjects.com'
import boto3
s3 = boto3.client('s3',
region_name = 'REGION_NAME',
endpoint_url = AWS_S3_ENDPOINT_URL,
aws_access_key_id = 'XXXXXXXXXXXXXXXXXXXX',
aws_secret_access_key = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX')
response = s3.list_buckets()
## Mounting Google Drive for files to upload to s3
from google.colab import drive
drive.mount("/content/drive")
## Uploading files from google drive to s3 bucket
s3.upload_file(Filename='/content/drive/MyDrive/FILE_NAME', Bucket='BUCKET_NAME', Key='KEY_ASSIGNED')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment