Skip to content

Instantly share code, notes, and snippets.

@timosalm
Last active October 4, 2020 23:39
Show Gist options
  • Save timosalm/f602f1d94a94f5d7ee4ce60b678586dd to your computer and use it in GitHub Desktop.
Save timosalm/f602f1d94a94f5d7ee4ce60b678586dd to your computer and use it in GitHub Desktop.
Logo detection with SCDF

Stream definition

Definition for any of the default app starters of Spring Cloud Data Flow

object-detection --tensorflow.model-fetch=detection_scores,detection_classes,detection_boxes 
--tensorflow.object.detection.labels=https://scdf-applications-jars.cfapps.io/label_map.pbtxt 
--tensorflow.mode=header --tensorflow.model=https://scdf-applications-jars.cfapps.io/model.pb

Definition using S3 compatible storage as source and sink

S3 compatible storage is currently not supported with the S3 source and sink available as app starters for Spring Cloud Data Flow. See https://gist.github.com/tsalm-pivotal/b17cb53a936b17d2e9f30b8acd760898 for detailed information.

s3-source-M2 --s3.supplier.remote-dir=/logo-in --s3.supplier.delete-remote-files=true 
--cloud.aws.credentials.accessKey=***  --cloud.aws.credentials.secretKey=*** 
--cloud.aws.region.static=eu-central-1 --cloud.aws.stack.auto=false 
--JBP_CONFIG_SPRING_AUTO_RECONFIGURATION='{enabled: false}' --SPRING_PROFILES_ACTIVE=cloud 
--outputType=application/octet-stream | 
object-detection --tensorflow.model-fetch=detection_scores,detection_classes,detection_boxes 
--tensorflow.object.detection.labels=https://scdf-applications-jars.cfapps.io/label_map.pbtxt 
--tensorflow.mode=header --tensorflow.model=https://scdf-applications-jars.cfapps.io/model.pb 
--JBP_CONFIG_SPRING_AUTO_RECONFIGURATION='{enabled: false}' --SPRING_PROFILES_ACTIVE=cloud 
--outputType=application/octet-stream | 
s3-sink-M2 --s3.consumer.bucket=/logo-out --cloud.aws.credentials.accessKey=*** 
--cloud.aws.credentials.secretKey=*** --cloud.aws.region.static=eu-central-1 
--cloud.aws.stack.auto=false --JBP_CONFIG_SPRING_AUTO_RECONFIGURATION='{enabled: false}' 
--SPRING_PROFILES_ACTIVE=cloud --s3.consumer.key-expression=headers.file_name

IMPORTANT You have to deploy your stream with the following deployment properties to increase the memory of the object detection processor and enable the ENVIRONMENT variables:

deployer.object-detection.memory=8000
deployer.*.cloudfoundry.use-spring-application-json=false
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment