-
-
Save VisionOra/610c6ec249180de9f8e4786e2738f0b9 to your computer and use it in GitHub Desktop.
Deploys the Python Selenium library and Chrome Headless to an AWS Lambda layer. You can specify the region, library version, and runtime. An example Lambda function is given.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from selenium.webdriver import Chrome | |
| from selenium.webdriver.chrome.options import Options | |
| options = Options() | |
| options.binary_location = '/opt/headless-chromium' | |
| options.add_argument('--headless') | |
| options.add_argument('--no-sandbox') | |
| options.add_argument('--start-maximized') | |
| options.add_argument('--start-fullscreen') | |
| options.add_argument('--single-process') | |
| options.add_argument('--disable-dev-shm-usage') | |
| driver = Chrome('/opt/chromedriver', options=options) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| REGION=eu-west-1 | |
| RUNTIME=python3.7 | |
| BUCKET= | |
| SELENIUM_VER=3.141.0 | |
| CHROME_BINARY_VER=v1.0.0-55 # based on Chromium 69.0.3497.81 | |
| CHROMEDRIVER_VER=2.43 # supports Chrome v69-71 | |
| OUT_DIR=/out/build/chrome_headless/python/lib/$RUNTIME/site-packages | |
| docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \ | |
| pip install selenium==$SELENIUM_VER -t $OUT_DIR | |
| cp chrome_headless.py build/chrome_headless/python/chrome_headless.py | |
| pushd build/chrome_headless | |
| DRIVER_URL=https://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VER/chromedriver_linux64.zip | |
| curl -SL $DRIVER_URL >chromedriver.zip | |
| unzip chromedriver.zip | |
| rm chromedriver.zip | |
| # download chrome binary | |
| CHROME_URL=https://github.com/adieuadieu/serverless-chrome/releases/download/$CHROME_BINARY_VER/stable-headless-chromium-amazonlinux-2017-03.zip | |
| curl -SL $CHROME_URL >headless-chromium.zip | |
| unzip headless-chromium.zip | |
| rm headless-chromium.zip | |
| zip -r ../../chrome_headless.zip * | |
| popd | |
| aws s3 cp chrome_headless.zip s3://$BUCKET/chrome_headless.zip | |
| aws lambda publish-layer-version \ | |
| --layer-name ChromeHeadless \ | |
| --region $REGION \ | |
| --content S3Bucket=$BUCKET,S3Key=chrome_headless.zip \ | |
| --compatible-runtimes $RUNTIME | |
| rm -rf build *.zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from chrome_headless import driver | |
| def lambda_handler(event, context): | |
| driver.get(event['url']) | |
| return driver.page_source |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment