Skip to content

Instantly share code, notes, and snippets.

@achinta

achinta/biquery.sh

Created February 27, 2019 05:05
Show Gist options
  • Save achinta/c236471018b0e492fa1bffe9a9236405 to your computer and use it in GitHub Desktop.
Save achinta/c236471018b0e492fa1bffe9a9236405 to your computer and use it in GitHub Desktop.
Download ZIP
export-data-to-bigquery
Raw
biquery.sh
# export data from database as csv and copy to google storage
gsutil cp abhyasis.csv gs://my-bucket
#create dataset in bigquery (onetime, say 'aims')
bq --location=US mk --dataset [PROJECT_ID]:aims
#create table in bigquery (onetime, say 'abhyasis')
bq mk --table [PROJECT_ID]:aims.abhyasis
#load from google storage to bigquery
bq --location=US load --autodetect --source_format=CSV aims.abhyasis gs://my-bucket/abhyasis.csv
#other useful commands
#show schema
bq show --schema --format=prettyjson unifiedplatform-dev:aims.partners
#delete data
bq query --use_legacy_sql=false "delete from aims.partners_test where 1=1"
#count records
bq query "select count(*) as count from aims.partners"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment