service | oss |
---|---|
Cloud Dataflow | Apache Beam |
Cloud Dataproc | Apache Hadoop, Apache Spark, Apache Pig, Apache Hive |
Cloud Datalab | Jupyter |
Cloud Dataprep | ? |
Data Studio | ? |
BigQuery | - |
- prefer denormalized and pre-joined tables.
- https://aws.amazon.com/jp/cloudwatch/
- a cloud monitoring tool.
- a CDN service.
- a DNS service.
- a load balancer for EC2 instances.
- it has "Connection Draining" function that enables to remove the EC2 instance gracefully.
- its console is not isolated, in EC2 console.
- First create one or more "target group", after that create a load balancer.
- https://aws.amazon.com/jp/codestar/
- a tool, and also a development environment and a deploy service that is constructed with some AWS services.
- https://aws.amazon.com/jp/cloudformation/
- a provisioning tool, like Ansible?
- https://aws.amazon.com/jp/codedeploy/
- an appliction deploy tool.
- a git repository service like GitHub.
- a build interface?
- like Jenkins?
- https://aws.amazon.com/jp/codepipeline/
- a software lifecycle pipeline builder.
- https://aws.amazon.com/jp/opsworks/
- Chef on AWS
- https://aws.amazon.com/jp/kinesis/streams/
- a tool for stream processing.
- one of the use cases is log collection.
- store into S3.
- https://aws.amazon.com/jp/datapipeline/
- this tool transfers data from anything to AWS tools like S3, RDS and so on.
- so it is impossible to use this to transfer data to Google BigQuery.
- if you want to do that, use Google Stackdriver Logging https://cloud.google.com/logging/ instead.
- "Stackdriver Logging は Google がカスタマイズしてパッケージ化した Fluentd エージェントを使っています。"
- https://aws.amazon.com/jp/cli/
- This is what we call CLI.
- Commands like
aws rds describe-db-instances
- Needs IAM account and its API key and secret.
- EC2 + middlewares + deploy tools
- EC2 instance that is used by Elastic Beanstalk is automatically created by the wizard.
- t1.micro instance is choosed.