michaeljoseph · December 24, 2015 00:09
diff --git a/index.html b/index.html
 <!DOCTYPE html>
 <html>
  <head>
    <title>Title</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <style type="text/css">
      /* Slideshow styles */
    </style>
  </head>
  <body>
    <textarea id="source">

 class: center, middle

 # Application To Platform: how we used Python to scale Yola

 ## Abstract

 This is the story of how [Yola](https://yola.com) engineers transformed a website building application into a robust, distributed platform (for hosting and building websites).

 By:
 - decomposing and augmenting our services layer into multiple Python-driven API services
 - in a series of small low-risk changes (which i’ll discuss in more detail)
 we organically built out the service layer.

 This talk mostly concerns itself with the python API services layer, since i’m the Services Team Lead. There are a number of other interesting talks that I have yet to convince some Yola colleagues to prepare:
 - yodeploy and yoconfigurator
 - yola’s hosting platform
 - release automation and continuous deployment

 I shall attempt to:
 - extract and present some patterns that worked for us
 - discuss the organisational flexibility and engineering workflow that enabled us to refactor the system architecture, and
 - share the medium term roadmap for our system.

 ---

 # History

 ![image]()

 ???

 In the beginning, we had an application. It was called The Sitebuilder.
 And it was good.

 This is still the main value proposition of Yola’s business: I help make a website that lets you make a website.

 So, we had an application, it lived on two servers in 2007 (along with a php website and wordpress blog) and not much else. Except smart people :)

 @nbm: architecture goals, decided that soa was the way to go (it was also a buzzword back in the day, it  was a thing guys). over 2 years we evolved a reasonably robust platform <before>+<after>
 our architect created lovely architecture changelogs and when he left us for facebook (which is becoming a damn theme at yola these days) a successor was not named. Then the fun started :)

 <timeline of synthasite / yola architecture: # instances; # deployable artifacts>

 --- 
 ## Before And After

 ![Alt text](http://raw.github.com/michaeljoseph/pyconza2013/slide-4-before-after.jpg)

 ???
 -- end 2009:
 paymentservice, checkoutservice, domainservice
 sitebuilder
 php website
 wordpress blog

 - oct 2013
 We have around 30 deployable applications, service apis, front-ends and automation/testing repositories.

 [applications, installables]
 - frontends: sb is pretty much the same application, we’ve added helper services around it to absorb the load, we now have a logged in user management site: MyYola and a CMS for brochure-ware (the main website)

 - storage: from single spofs to one level of database redundancy, with automatic backups
 (where are the Yola people in the audience? We should test some restores sometime guys?)
  - s3, no more NFS

 - pure services: userservice, , domainservice, michango, rankmonitor

 - applications: myyola, checkoutservice, yolacom [+ apis]

 ## patterns

 - build out and release new service calls independent of consumuing clients
 - refactor apps
  - re-use standard python service clients to reduce development time
 - switch clients
 - rm old implementation / cleanup / decommission
 - same deal for rotating hardware
   - spin up a fresh box
   - provision
   - test
   - load balance in, load balance out

 ## evolving services with service-clients

 - shittiest python code is (in only a few places, hopefully): python 2.6, django < 1.4, piston, sloppy testing practices, complicated code with production bug fixing barnacles that works

 - new apis: python 2.7, djrf, django
 - small api: flask
 - python service clients: [`demands`](https://github.com/yola/demands). <3 kennethreitz
 - new user facing app: django 1.5, cbvs, south, asset compression, front end dependency toolchain (lessc, bower, grunt)

 ## organisational enablers; engineering workflow

 Being careful not to over engineer in advance <abstraction astronaut> we delayed major technology decision making

 In hindsight it seems obvious: the decentralised, internet-based, just-in-time way of working that has risen to popularity in the wake of companies like github, facebook and valve.

 A flat organisational structure and hiring smart people means that you don’t really need to worry about who does what and when, as long as there is work to be done and some degree of priorisation, smart engineers just get the job done.

 -- decision making and meritocracies with a BDFL

 Github pull requests and code review have transformed the way the team works.
 Many eyeballs make all the bugs shallower and there’s always a paper trail.
 [/organisation]

 ## how we’d approach it today

 pretty much the way i just described it.
 <org + workflow>
 release early and often: keep the changes small and pointy, lowers the cognitive overhead of review, increased confidence in pushing out new code, small surface area of introduced bugs to inspect (they’re shallow anyway because we have many eyeballs)

 ---
 # roadmap

 ![succession]()

 ---
 # roadmap

 ![succession]()

 ???
 - database succession
 - feature flags and user segmenting, better dogfooding

 ---
 # roadmap

 ![monitoring]()

 ???
    - monitoring: we use pingdom, graphite, munin and sentry
 		- stuff mainly just goes in there (unless there’s a problem. for the size of cluster we operate, these tools are generally sufficient)
        - we need a monitoring dashboard (next juicy devops project)

 ---
 # roadmap
    - what was hard: slow, boring, waiting for improvements to compound, almost at the singularity
 ![Black hole sun](https://gist.github.com/michaeljoseph/1353700/raw/c9ed26af986771b51e9b6d1df06257e8fabc8af4/singularity.png)

 ---

 # summary

 ![funny picture]()
 about to summit

 ???

 make some stuff up

    </textarea>
    <script src="http://gnab.github.io/remark/downloads/remark-0.5.5.min.js" type="text/javascript">
    </script>
    <script type="text/javascript">
      var slideshow = remark.create();
    </script>
  </body>
 </html>
	<!DOCTYPE html>
	<html>
	<head>
	<title>Title</title>
	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
	<style type="text/css">
	/* Slideshow styles */
	</style>
	</head>
	<body>
	<textarea id="source">

	class: center, middle

	# Application To Platform: how we used Python to scale Yola

	## Abstract

	This is the story of how [Yola](https://yola.com) engineers transformed a website building application into a robust, distributed platform (for hosting and building websites).

	By:
	- decomposing and augmenting our services layer into multiple Python-driven API services
	- in a series of small low-risk changes (which i’ll discuss in more detail)
	we organically built out the service layer.

	This talk mostly concerns itself with the python API services layer, since i’m the Services Team Lead. There are a number of other interesting talks that I have yet to convince some Yola colleagues to prepare:
	- yodeploy and yoconfigurator
	- yola’s hosting platform
	- release automation and continuous deployment

	I shall attempt to:
	- extract and present some patterns that worked for us
	- discuss the organisational flexibility and engineering workflow that enabled us to refactor the system architecture, and
	- share the medium term roadmap for our system.

	---

	# History

	![image]()

	???

	In the beginning, we had an application. It was called The Sitebuilder.
	And it was good.

	This is still the main value proposition of Yola’s business: I help make a website that lets you make a website.

	So, we had an application, it lived on two servers in 2007 (along with a php website and wordpress blog) and not much else. Except smart people :)

	@nbm: architecture goals, decided that soa was the way to go (it was also a buzzword back in the day, it was a thing guys). over 2 years we evolved a reasonably robust platform <before>+<after>
	our architect created lovely architecture changelogs and when he left us for facebook (which is becoming a damn theme at yola these days) a successor was not named. Then the fun started :)

	<timeline of synthasite / yola architecture: # instances; # deployable artifacts>

	---
	## Before And After

	![Alt text](http://raw.github.com/michaeljoseph/pyconza2013/slide-4-before-after.jpg)

	???
	-- end 2009:
	paymentservice, checkoutservice, domainservice
	sitebuilder
	php website
	wordpress blog

	- oct 2013
	We have around 30 deployable applications, service apis, front-ends and automation/testing repositories.

	[applications, installables]
	- frontends: sb is pretty much the same application, we’ve added helper services around it to absorb the load, we now have a logged in user management site: MyYola and a CMS for brochure-ware (the main website)

	- storage: from single spofs to one level of database redundancy, with automatic backups
	(where are the Yola people in the audience? We should test some restores sometime guys?)
	- s3, no more NFS

	- pure services: userservice, , domainservice, michango, rankmonitor

	- applications: myyola, checkoutservice, yolacom [+ apis]

	## patterns

	- build out and release new service calls independent of consumuing clients
	- refactor apps
	- re-use standard python service clients to reduce development time
	- switch clients
	- rm old implementation / cleanup / decommission
	- same deal for rotating hardware
	- spin up a fresh box
	- provision
	- test
	- load balance in, load balance out

	## evolving services with service-clients

	- shittiest python code is (in only a few places, hopefully): python 2.6, django < 1.4, piston, sloppy testing practices, complicated code with production bug fixing barnacles that works

	- new apis: python 2.7, djrf, django
	- small api: flask
	- python service clients: [`demands`](https://github.com/yola/demands). <3 kennethreitz
	- new user facing app: django 1.5, cbvs, south, asset compression, front end dependency toolchain (lessc, bower, grunt)

	## organisational enablers; engineering workflow

	Being careful not to over engineer in advance <abstraction astronaut> we delayed major technology decision making

	In hindsight it seems obvious: the decentralised, internet-based, just-in-time way of working that has risen to popularity in the wake of companies like github, facebook and valve.

	A flat organisational structure and hiring smart people means that you don’t really need to worry about who does what and when, as long as there is work to be done and some degree of priorisation, smart engineers just get the job done.

	-- decision making and meritocracies with a BDFL

	Github pull requests and code review have transformed the way the team works.
	Many eyeballs make all the bugs shallower and there’s always a paper trail.
	[/organisation]

	## how we’d approach it today

	pretty much the way i just described it.
	<org + workflow>
	release early and often: keep the changes small and pointy, lowers the cognitive overhead of review, increased confidence in pushing out new code, small surface area of introduced bugs to inspect (they’re shallow anyway because we have many eyeballs)

	---
	# roadmap

	![succession]()

	---
	# roadmap

	![succession]()

	???
	- database succession
	- feature flags and user segmenting, better dogfooding

	---
	# roadmap

	![monitoring]()

	???
	- monitoring: we use pingdom, graphite, munin and sentry
	- stuff mainly just goes in there (unless there’s a problem. for the size of cluster we operate, these tools are generally sufficient)
	- we need a monitoring dashboard (next juicy devops project)

	---
	# roadmap
	- what was hard: slow, boring, waiting for improvements to compound, almost at the singularity
	![Black hole sun](https://gist.github.com/michaeljoseph/1353700/raw/c9ed26af986771b51e9b6d1df06257e8fabc8af4/singularity.png)

	---

	# summary

	![funny picture]()
	about to summit

	???

	make some stuff up

	</textarea>
	<script src="http://gnab.github.io/remark/downloads/remark-0.5.5.min.js" type="text/javascript">
	</script>
	<script type="text/javascript">
	var slideshow = remark.create();
	</script>
	</body>
	</html>