Best Practices
Deployments
Pipelines
Docker N Pipeline
Releases

####################################

Mile High View

####################################

I like Devops .......... I love the way is can empower the organisation, it is the answer to every CIO's nitemare, how can we move towards continouse production of SW, how can we make AGILE real, and how can we break down the walls of production(NOC), the monolite in the organisation. This change can be to hard on production folks I know I was once one of them!!!

####################################

Best Practices

####################################
Source => https://www.cloudbees.com/blog/top-10-best-practices-jenkins-pipeline-plugin

Use pipeline plugin, buildflow, build plugins are obsolete. The pipeline plugin expose a rich Pipeline DSL!!! A real world pipleine
https://blog.choldrim.com/wp-content/uploads/2016/07/realworld-pipeline-flow.png
version control your pipeline, treat it as you would any other code
stages, should be seen a units of work, and logical in nature, code intented for such blocks, should all be contained within these blocks. Typical blocks might be

Scm
- Clone Brach
Dev
- build
QA
- test : integration
- test : functional
- test : load
- test : security
Release
- approval
- Deploy : Staging
- Deploy : Prod
Cleanup
- shared files/clones, jars.
Do work in parrellel, so called shifting your work aka steps to the LEFT (developement) will return feedback (system) back to the team quicker.

parallel 'integration-tests':{
node('mvn-3.3'){ ... }
}, 'functional-tests':{
node('selenium'){ ... }
}, 'security-tests':{
node('selenium'){ ... }
}, 'load-tests':{
node('selenium'){ ... }
}
Approvals steps
- to be called outside of a node!!!
- use timeouts (this will allow piplelines to be aborted)
timeout(time:5, unit:'DAYS') { input message:'Approve deployment?', submitter: 'it-ops' }
Enviroment variables to be changed locally using withEnv not Env, which changes globally, unless of course that is your desired affect.

withEnv(["PATH+MAVEN=${tool 'm3'}/bin"]) { sh "mvn clean verify" }

Caching of data
Sharing of data
There is both long term, and short term needs to consider here.

Short term, would be code you have clone, and might want to make available to a stage or node down the line!!! In such a case use the stash and unstash commands

Eg : souce code tagged as "souce" is later used in a stage/node. stash excludes: 'target/', name: 'source' unstash 'source'
2. Long term, is consider intermidiate binaries you can created from your builds!!! In such case use the archive command.

Notes

Any material work, such as clone repos, needs to be distributed out to a node!!!
the test stages above, might use a deploy function to deploy the jar to the respective test enviroments!!!
The input approval is for prod!!!

####################################

Deployments

####################################

NetFlix

Blue Green

The old was is to replace a faulty production version with a previous stable version....
This was a strategy but resulted in downtime, and maintainance windows!!
The new strategy is an example of a state machine, that transitions from blue2green2blue2green, where blue is the first deployement.
This is were you have two identical enviroments one production, and the other staged.
a stagged enviroment would be one that is been prepped, for production use
Each prod enviroment is identified by a color , Red the other Green.
The intent is to reduce downtime, and risks.
At any one time, only one environment is live
The how to is easy.

you have environments each have different version of the application
One enviroment is label blue which will have the current version of the application.
you can shift traffic between two these identical environments via DNS routing, or load balances (F5, elastic load balance)
A switch in traffic only happens if a fault occurs?

Warnings Souce : https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html

If your app uses a relational database, blue-green deployment can lead to discrepancies between your Green and Blue databases during an update. To maximize data integrity, configure a single database for backward and forward compatibility.
Note: You can adjust the route mapping pattern to display a static maintenance page during a maintenance window for time-consuming tasks, such as migrating a database. In this scenario, the router switches all incoming requests from Blue to Maintenance to Green

Links. AWS view on it are here https://d0.awsstatic.com/whitepapers/AWS_Blue_Green_Deployments.pdf Cloudfoundary view on it
https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html

####################################

Pipelines

####################################
There are different types of pipelines, each addressing a need.

Dynamic and Automatic functionality!!!

Both the Multibranch & Organisational Pipelines offer this. If Jenkins discovers a jenkinsfile in a newly created branch, it will execute a pipleline for that branch, if the branch is a new feature, then that feature will be process according to the instructions layed out by the stages layed out in the pipeline.

The discovery of new branches is determined by jenkins reindexing the SCM, if you use MB, you need to setup the polling interval. Organisation piplines do this implicity.
You can extend the reach of jenkins to trigger pipeline execution on pull/change requests, by using the respective plugin

GitHub :github-branchsource
GitBucket :cloudbees-bitbucket-branch-source

Organisational Scaning

This pipeline is the mother, it will scan your github/gitbuck for repository branches for a jenkinsfile, and execute a MB pipeline on your behalf. This is moving the organisation down automation heaven!!!

####################################

Parrelism N Pipeline

####################################
https://wiki.jenkins.io/display/JENKINS/Parallel+Test+Executor+Plugin https://www.cloudbees.com/blog/parallelism-and-distributed-builds-jenkins

####################################

Docker N Pipeline

####################################

Resouces

Sandbox https://labs.play-with-docker.com/p/ba1f3fb2259g00bsilc0

Docker will impact your organisation, use it and reap the rewards.

Unification

How many times have you heard a dev. say to a tester, your enviroment is not setup right!! more correctly is out of sync with the can dev made two minutes, and commit a change!!! Docker will remove this nosense..

It will uinify their build environments across machines
It will uinify their test environments across machines

Efficent

Enables more efficent deployments!!!

Docker Commands (CLI)

Options: --config string Location of client config files (default "/root/.docker") -D, --debug Enable debug mode -H, --host list Daemon socket(s) to connect to -l, --log-level string Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info") --tls Use TLS; implied by --tlsverify --tlscacert string Trust certs signed only by this CA (default "/root/.docker/ca.pem") --tlscert string Path to TLS certificate file (default "/root/.docker/cert.pem") --tlskey string Path to TLS key file (default "/root/.docker/key.pem") --tlsverify Use TLS and verify the remote -v, --version Print version information and quit

Management Commands:

checkpoint Manage checkpoints
config Manage Docker configs
container Manage containers
image Manage images
network Manage networks
node Manage Swarm nodes
plugin Manage plugins
secret Manage Docker secrets
service Manage services
stack Manage Docker stacks
swarm Manage Swarm
system Manage Docker
trust Manage trust on Docker images (experimental)
volume Manage volumes

attach Attach local standard input, output, and error streams to a running container
build Build an image from a Dockerfile
commit Create a new image from a container's changes
cp Copy files/folders between a container and the local filesystem create Create a new container deploy Deploy a new stack or update an existing stack diff Inspect changes to files or directories on a container's filesystem events Get real time events from the server exec Run a command in a running container export Export a container's filesystem as a tar archive history Show the history of an image images List images import Import the contents from a tarball to create a filesystem image info Display system-wide information inspect Return low-level information on Docker objects kill Kill one or more running containers load Load an image from a tar archive or STDIN login Log in to a Docker registry logout Log out from a Docker registry logs Fetch the logs of a container pause Pause all processes within one or more containers port List port mappings or a specific mapping for the container ps List containers pull Pull an image or a repository from a registry push Push an image or a repository to a registry rename Rename a container restart Restart one or more containers rm Remove one or more containers rmi Remove one or more images run Run a command in a new container save Save one or more images to a tar archive (streamed to STDOUT by default) search Search the Docker Hub for images start Start one or more stopped containers stats Display a live stream of container(s) resource usage statistics stop Stop one or more running containers tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE top Display the running processes of a container unpause Unpause all processes within one or more containers update Update configuration of one or more containers version Show the Docker version information wait Block until one or more containers stop, then print their exit codes

Docker commands

docker.image('busybox').inside('-v /tmp/data:/data') { sh 'ls /' }

https://blog.choldrim.com/2016/07/30/jenkins2-0-%E4%BB%8Eci%E5%88%B0cd/
WORKFLOW SCRIPT EXAMPLE(S)

image (Provides a handle to image )
def image = docker. image(‘maven:3.3.3-jdk8’)
image.inside (Runs steps inside image )
image.inside(‘-v /repo:/repo’) { // some block }
image.pull (Pulls image )
image.pull()
image.push (Push image to registry )
image.push() image.push(“latest”)
image.run (Runs Docker image and returns container )
def container = image.run(“–name my-api -p 8080:8080”) container.stop()
image.withRun (Runs image and auto stops container )
image.withRun {api -> testImg. inside(“–link=${api.id}:api”) { // some block } }
image.tag (Records tag of image )
image.tag(“${tag}”, false)
image.imageName() (Provides image name prefixed with registry info )
sh “docker pull ${image. imageName()}”
container.id (ID of running container )
sh “docker logs ${container.id}”
container.stop (Stops and removes container )
container.stop()
build (Builds Docker image )
docker.build(“cb/ api:${tag}”,”target”)
withServer (Runs block on given Docker server )
docker.withServer(‘tcp://swarm. cloudbees.com:2376’, ‘swarm-certs’) { // some block }
withRegistry (Runs block using specified Docker registry )
docker.withRegistry(‘https:// registry.cloudbees.com/’, ‘docker- registry-login’) { // some block }
withTool (Specifies name of Docker client to use )
docker.withTool(‘toolName’) { // some block }

Docker Compose

https://blog.choldrim.com/2016/07/30/jenkins2-0-%E4%BB%8Eci%E5%88%B0cd/ https://dzone.com/refcardz/continuous-delivery-with-jenkins-workflow?chapter=1 https://dzone.com/storage/assets/413450-rc218-cdw-jenkins-workflow.pdf

####################################

Releases

####################################

Beta
Alpha
Candidate
Commerical
Dark Release

####################################

Anti Patterns

####################################

Continosity!!!

####################################

What are we doing, there is a shift, we are doing things on an earlies possible basis, we are bring in, integrating those last in the Q to front (parrellism) if possible, more over its not been left as an after thought!!!
There is a very important sentiment to this word, and that is modular, flow, stream, that lives on its own, but is meant to be consumed as part of an aggreate, that is more commonly know as a SYSTEM (on the small) or Enterpise (on the large)
Do the integration!!! did is fail, because the other system could not deal with it? maybe your system are too closely coupled, maybe your systems are not interface defined, mybe you deployed the wrong system!!!!

Continous Documentation

build help docs, release notes, release producures, then Contious documentaion is the way forward, you can have your code, MP3, and sent out to you on merge!!! 1.

Continous Compliance

Is your code update with latest patch levels, are you using latest SSL!!!

Continous Security

normally left to last
Tools of choice would be to use a ZAP proxy.
Methodology of choice would be BDD-Security https://www.continuumsecurity.net/bdd-security/

Continous Data

A system under deployement, needs to be supported by user provisioning schema (checked into source control)
A system under deployement, needs to be supported by system configuration schema (checked into source control)
A system under deployement, needs to be supported by database schema (checked into source control)!!!

Continous Environmenting!!!

In todays world, we have choices, we can go to could base solutins, but if you cannot afford that, you are on prem!!! and need

Everything the system sits on!!!

Application Server
DB
OS
Runtimes.
Harding

Continous Provisioning

User Data
Schemas
JSON / XML

Continous Pefromance : normally left to last

Continous Testing : normally left to last

----------------------- Water Shed --------------------------

Continous Deployement : normally left to last

####################################

Continouse Security

####################################

Facts

####################################

Do you know in europe,France is the leader in embracing devops in its companies at 39%
Wages is highest in US than in Europe
The techstack in devops is brigger than any other disciple!!!
To be a Google, an Amazon, or a Netflix, you need to be 200 times faster a getting there code out there

#######################################

Collaboration

#######################################

AGILE : SAFE - Portfolia Managment, Scrum Teams, and Release Trains, and End Of Month Hardening Cycles, but what about Train stations!!!!
Cross Team Colloration - this is organmic to the business!!!
large-scale agile frameworks, things like SAFe and DAD and LeSS and Scrum of Scrums.

#######################################

Large Scale Enterpise

#######################################

If systems are lossley coupled liek is easy, automonous products. A sign of this is where system to system integration testing is needed (poor design)
Here’s where I think it eventually goes. With practices like continuous integration, continuous delivery, with loosely coupled architectures, you end up in a situation where small development teams can deploy value to customers independently, without a lot of communication and integration overhead. Less integration testing and, even before that, less scheduling. When you have say only 10 integration test environments, you have to jockey for position to get use of those scarce resources. And even to get there, you have to talk to 30 different committees, 30 different product groups, because you’re all trying to make your changes at the same time, and you can only go through one at a time.

#######################################

Continuous Delivery

####################################### https://www.youtube.com/watch?v=clZgb8GA6xI

It brings together teams....

Developer

Writes the Code!!!
Tests the Code!!!

Builds the Code if its simple (something we dont here...)

consider a c / c++ code base, no long is it build once, run every where, the build and compile process scrope has broaden.
integrates dependencies, compiles the bundle, and builds the SW that will make up the package.
This is best captured in a continous integration process, where code that is checked in, and tested, and if it breaks the build, the commit is rejected, and therefore not merge into the branch.

Test Eng

Test the Build!!!

Deploy

Installs the Build

Release Engineers

Roles

Build/Release Engineers (something we dont here...)

fully document release produces (technical writer, + developers, and system analysis), this is the process used to release to production
branch management

Create new branches, that will be released to production!!!
Back porting new features, and bug fixes to the correct branches

Build/Release engineers are challanged by the 2 week release cycle!!! This is were the multi-branch jenkins job comes to the rescue!!!

Educate new engineers, in teh workings for code repository

Cloud Engineers, are supporting devops!!!!

The days of the NOC, an alias used for Operations, is on the way out, no longer need you wait on tickets to be approached for a new instances of a VM, due to costs, and resouces

QA

#################################

How To Measure Effeciency

#################################

Customer Satisfaction - responsing to customer expectations, and demands!!!! more over Internal teams been able to response to customer demands, a first stop, might be to look at what teams, and not DevOpsized!!
Where are we now? Do with have a basic devops setup, or a mature one, how do we measure that, what bugets are there to correct the deceit if any.
Is Change control newed

holding up the show - , changes quening up, and holding up deployments!!! Its an an anti pattern, manuall review of code before its merged, a more thoughtful pipleine , with resilences, and faster feedback is better.

Embrace OpenSource

anyone can change anyone elses code
no more dependencies
product teams automonous if you need to do a change to a product, make a pull request, have it reviewed, tested, and merged - bingo!!!

Deploy - On the Small (Product - Standalone WebApp) The CD thens to be more complex than the CI, the CD can be an aggregate of many CIs pipelines!!!!

Deploy - On the Large (Product(Enterprise) - Integrated WebApp)

DevOps revolutionizes developments as the approach allows faster delivery, efficient deployment, and improved reliability.

Concluded

How DevOps practices affect deployment frequency, lead time, change failure rate and MTTR

The influence leadership has on DevOps transformations

How high- and low-performing teams automate differently

The impact of architecture and team structure on IT performance

How DevOps helps organizations reach both their financial and non-financial goals

What we want to be able to do is build a delivery pipeline, that realized a product, that is reflective of the customers expectations, all the way to production, in an effectently and affectively as possilbe. Avoid Pitwalls, what will comprise this vision.

Build : move from incremenation to iterative?
Test : move from ....
Deploy : move from

Returning from points of failure.
Introduction change.
Incoporating team requirements, new tech, new this, remove that....

Benefits

Code is deployed 30 times more frequently than classically
50% fewer failures in new releases!!!
Fast flow,
high deployment rates
great reliability
great security
Less error prone
Enable X Team / Discipline intgration on a process level (the process of building, testing, deployment), and task level (jaring, compilien, load, unit, install, configure)

################# Glossary ################

Backporting (https://en.wikipedia.org/wiki/Backporting)

is the action of taking parts from a newer version of a software system or software component and porting them to an older version of the same software.
It forms part of the maintenance step in a software development process. Eg. it is commonly used for fixing security issues in older versions of the software and also for providing new features to older versions(SP XP, and Vista, so games could work on both platforms)
Java provides tools to support backporting. https://en.wikipedia.org/wiki/Java_backporting_tools

ennisa-ire/00-toc.md

Testing

Build Tools/Build Systems

Project Plugins

Source Code ...where the compiler is to look for the source files!!!

Set the Main in the manifest file!!!

Repository for Java Project : Maven, Ivy, Gradle

Dependencies for Java Project

Deploy To App Server

Deploy To Bintray (Nexsus)

Deploy To Bintray (Artifactory)

F5 in IDE : Compile and Test Into Build Dir, and then Package it!!!

Notes

Summary

Table of Contents

Mile High View

Best Practices

Notes

Deployments

NetFlix

Blue Green

Pipelines

Dynamic and Automatic functionality!!!

Organisational Scaning

Parrelism N Pipeline

Docker N Pipeline

Resouces

Unification

Efficent

Docker Commands (CLI)

Management Commands:

Docker commands

Docker Compose

Releases

Anti Patterns

Continosity!!!

Continous Documentation

Continous Compliance

Continous Security

Continous Data

Continous Environmenting!!!

Continous Provisioning

Continous Pefromance : normally left to last

Continous Testing : normally left to last

Continous Deployement : normally left to last

Continouse Security

Facts

Collaboration

Large Scale Enterpise

Continuous Delivery

Roles

Build/Release Engineers (something we dont here...)

Cloud Engineers, are supporting devops!!!!

QA

How To Measure Effeciency

Benefits

Backporting (https://en.wikipedia.org/wiki/Backporting)