- ease of feature development
- avoid coupling with tests
- ease of test development
- avoid breakages
- make breakages easy to fix
- A commit in Leanplum/Leanplum causes FE changes:
- public, visual changes - updating text, changing color, etc.
- private changes - refactoring css classes or html attributes
- Tests in Leanplum/Testing break!
- This is only caught during release testing, which makes them expensive
- Feature changes should update tests in Leanplum/Testing
- 😀 tests fixed immediately, and in same diff
- 😞 currently written by different authors. if FE takes this over, reduced value since writing the same implementation twice (easy to do) will miss bugs
- Support official private interface in FE for tests, e.g. add test-specific CSS classes
- 😀 obvious which elements on a page are(n't) tested
- 😞 test coupling
- 😞 very hard to do for some things, e.g. third-party widgets
- Use automated visual diff framework - https://gist.github.com/cvrebert/adf91e429906a4d746cd
- 😀 catches things that aren't explicitly tested, WYSIWYG
- 😞 potential for minor test flakes if pages don't fully load / anti-aliasing, etc.
- 😞 need someone to own and maintain this service (or farm out to saas, e.g. screenster)
- Setup js model data testing
- 😀 no need to write Selenium setup, declarative action => state mapping
- 😞 misses regressions where state doesn't yield expected visual output, e.g. templates