- Sascha Bates
- @sascha_d
- http://brattyreadhead.com/
- Works at Chef
- Cohost, Ship Show podcast
- Started Infracoders Minneapolis meetup
- Themes of DevOps:
- Empathy
- Trust
- Reflection
- INCLUSIVENESS!
- Dan Slimmon
- @danslimmon
- http://danslimmon.com/
- Is DevOps working?
- Depends on what you wanted to get
- Then you can measure what happened vs what you wanted
- Have a theory:
- A set of rules
- That generates predictions
- About the behavior of a system
- Then you can experiment
- Experiments weed out bad theories
- Theories tell us what experiments to try
- Conway's Law
- 1968
- Organizations create systems which mirror the organization's communication structure
- We continually redesign our software, systems, etc.
- So too should we redesign our team structure
- Some organizations have been successful taking the reverse of Conway's law: reorganize your communication structure to let the application structure you want fall out
- Destroy the wall of confusion between Dev & Ops
- Theory is a conversation
- Never done, always evolving
- Geocentricism -> Heliocentricism -> Theory of Gravity -> Landing on the Moon
- Heather and Ross from Target
- Explaining from a Dev and an Ops perspective how DevOps benefited Target
- Lot of technology, 50 years of complexity accrued
- Lots of silos
- Silos within silos
- Remove roadblocks to move forward
- Create a safety zone
- Make it safe to fail
- Ben Hughs
- Etsy security team
- @benjammingh
- People don't come into your servers through the front door
- Easy to protect
- Look in the non-obvious places, like your developers' laptops becoming compromised
- You could cause everything to stop when something suspicious happens
- But too many false alarms
- Better to log everything
- Then you can review after-the-fact and close down whatever happened
- Blameless post-mortem
- Seriously, log everything
- Splunk is popular and full featured
- It's super expensive
- Logstash is an open-source project that covers much of the same ground
- not designed as a Splunk clone/replacement
- send log entries to it for later searching
- also alerting on spikes of things and whatnot
- not as full-featured as Splunk
- Uses elastic search for searching
- Easy to scale horizontally
- master/search/data on one server
- master on one server, multiple search servers, multiple clustered data servers
- Mostly the conversation turned to people comparing and contrasting Splunk/Logstash
- What they had to do in Logstash to replace Splunk functionality
- What is easier vs what is harder between them
- I kind of checked out
- Someone mentioned Sumo Logic as a SaaS alternative to both
- It's becoming clear to me based on this conversation that Docker is most interesting/useful to someone running their own physical hardware, as an easier alternative to running VMs on the hardware.
- Basically, keep all lines of communication open
- Be proactive including people outside your team
- If documentation is poor, make it part of acceptance critiera
- Documentation doesn't have to be amazing, just has to exist (can always improve later)
- As part of a bug fix, improve documentation that lead to misunderstand that created the bug
- As part of your on-call alerting, create a "play book" for each alert that outlines common causes/solutions
- Makes it much easier for new on-call staff to be productive in case of an emergency