This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aws lambda add-permission \ | |
--function-name "arn:aws:lambda:<REGION>:<ACCOUNT_ID>:function:<FUNCTION_NAME>:<ALIAS_NAME>" \ | |
--source-arn "arn:aws:execute-api:<REGION>:<ACCOUNT_ID>:<API_GTW_ID>/*/POST/my_endpoint" \ | |
--principal apigateway.amazonaws.com \ | |
--action lambda:InvokeFunction |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import time | |
from aws_lambda_powertools import Logger | |
logger = Logger() | |
@logger.inject_lambda_context | |
def lambda_handler(event, context): | |
logger.info({"action":"invoke_lambda", "payload":{"event":event}}) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
table.update_item( | |
Key={ | |
'scatter_gather_id': scatter_gather_id | |
}, | |
UpdateExpression='SET finished_processes = finished_processes + :val, #updated_at = :updated_at', | |
ExpressionAttributeValues={ | |
':val': value_to_sum, | |
':updated_at': datetime_now | |
}, | |
ExpressionAttributeNames={ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import time | |
from aws_lambda_powertools import Logger | |
from dynamo_operations import update_finished_processes | |
logger = Logger() | |
timestamp = int(time.time()) | |
SG_AGGREGATE_TABLE_NAME = os.getenv('SG_AGGREGATE_TABLE_NAME') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AggregatorFunction: | |
Type: AWS::Serverless::Function | |
Properties: | |
CodeUri: src/ | |
Handler: aggregator_lambda.lambda_handler | |
Runtime: python3.9 | |
Architectures: | |
- x86_64 | |
Policies: | |
- AmazonDynamoDBFullAccess |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from dynamo_operations import update_item_finished | |
... | |
SG_PROCESSES_TABLE_NAME = os.getenv('SG_PROCESSES_TABLE_NAME') | |
... | |
@logger.inject_lambda_context | |
def lambda_handler(event, context): | |
... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@logger.inject_lambda_context | |
def lambda_handler(event, context): | |
logger.info({"action":"invoke_lambda", "payload":{"event":event}}) | |
timestamp = int(time.time()) | |
# read the contries from the athena table | |
query = "select distinct state from covid_nytimes_states" | |
unique_states = wr.athena.read_sql_query(query, database=ATHENA_RAW_DATABASE_NAME) | |
# write in SG_AGGREGATE_TABLE_NAME DynamoDB table how many states should be executed | |
item = { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ScatterFunction: | |
Type: AWS::Serverless::Function | |
Properties: | |
CodeUri: src/ | |
Handler: scatter_lambda.lambda_handler | |
Runtime: python3.9 | |
Architectures: | |
- x86_64 | |
Policies: | |
- AmazonDynamoDBFullAccess |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SGProcessesDBTable: | |
Type: AWS::DynamoDB::Table | |
Properties: | |
TableName: !Ref SGProcessesTableName | |
ProvisionedThroughput: | |
ReadCapacityUnits: 1 | |
WriteCapacityUnits: 1 | |
KeySchema: | |
- | |
AttributeName: "scatter_gather_id" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import json | |
import awswrangler as wr | |
from datetime import datetime | |
from aws_lambda_powertools import Logger | |
logger = Logger() | |
ATHENA_RAW_DATABASE_NAME = os.getenv('ATHENA_RAW_DATABASE_NAME') | |
S3_BUCKET_NAME = os.getenv('S3_BUCKET_NAME') |
NewerOlder