What is an Agile Tester? Colombo Agile Conf, June 2014
Consultant www.crisp.se
Henrik Kniberg
Father
henrik.kniberg@crisp.se @HenrikKniberg
Agile & Lean coach Author
Agile tester mindset
Henrik Kniberg
00:22
Henrik Kniberg
Henrik Kniberg
Henrik Kniberg
Case study: Game development company Design-ready games
Game backlog
15
8
1m
Concept pres.
4h
6m
Actual work: 3 months Time to market: 25 months
Resource planning
1d
1w
Graphics design
Sound design
1m
3w
Production-ready games
12 Dev
6m
6m
3m (1m+2m)
Test & deploy
3w
Before Design-ready games
Game backlog
15
8
1m
Concept pres.
4h
6m
Production-ready games
Resource planning
1d
1w
Graphics design
Sound design
1m
3w
Actual work: 3 months
12 Dev
6m
6m
3m (1m+2m)
Integr. & deploy
3w
Time to market: 25 months
After Cross-functional game teams
7 times faster
Game team (graphics, sound, dev, test, deploy, etc)
Time to market: 3-4 months
Better games
Less Planning
More fun
Cross-functional teams I’m fast!
We’re slow!
6 months
Joe
Dave
Lisa
Release
3 months
We’re alot faster!
Joe I’m a bit slower
Dave Lisa January
February
Release
March
April
May
June 9
Henrik Kniberg
July
Test phase
Test team
Tester role
Henrik Kniberg
Test phase Req
Henrik Kniberg
Code
Test
We own quality
Programmers
Henrik Kniberg
Testers
Test team
I own quality
Tester
Dev team
Henrik Kniberg
Tester role
Role
Competency
We own quality
Cross-functional development team
QA
Henrik Kniberg
= Quality Assurance = Quality Assistance
Roles DB GUI
Design
Backend Henrik Kniberg
Test
Competencies DB
Design
Backend Henrik Kniberg
GUI
Test
Knows a bit about many things
Cross functional team
Knows a lot about one thing
Doesn’t mean everyone has to know everything Product Backlog
I can do Java, but I’m not so good at it.
Skills Needed to implement Top X backlog items Java
Web
Test Domain CM I’m good at Test!
Lisa Joe Fred Jenny David Erik I won’t even go near a database! Henrik Kniberg
DB
I don’t know CM at all. But I’m willing to learn!
ensure
How do you know that your product works? Who are the stakeholders?
1. Understand the problem
What need do they have, that we want to solve? How will we know when we’ve solved it? How will we know if we’re moving in the right direction?
2. Iterate until you’ve solved it Henrik Kniberg
Minimize the distance to MVP Deliver, measure, adjust continuously
Typical activities
Henrik Kniberg
00:22
Make sure backlog items are testable & valuable As a buyer I want to save my shopping cart so that I can continue shopping later
Henrik Kniberg
How to demo: 1) Enter store 2) Put a book in shopping cart 3) Press ”save cart” 4) Leave store, and enter it again 5) Check that the book is in my cart
Find defects early!
Cost of defect
$
Henrik Kniberg
Age of defect
Sit with the developers Developers
Testers
Henrik Kniberg
Developers & Testers
Meeting room
Shorten the feedback loop Maker
5
# of handoffs
People (# of handoffs) 2 1
4 3 2 1
Time (Feedback delay)
0
minutes hours days weeks months years
Length of feedback cycle
Henrik Kniberg
3
User
Push for Continuous Delivery Automatic Manual Code & commit
Henrik Kniberg
Build
Test & integrate
Deploy to staging
Deploy to prod Manual test
Create a shared vocabulary
Integration test?
Acceptance test?
Henrik Kniberg
Unit Test? Quality?
Technical debt?
Set working agreements for test automation Each user story has at least one black-box acceptance test
AT and UT complement each other
• Acceptance tests: coupled to UI, decoupled from internal design • Unit tests: coupled to internal design, decoupled from UI
Complex code has unit tests
Almost all code is covered by SOME kind of test (AT or UT)
Henrik Kniberg
We know which code isn’t covered and have a good reason for it
Do & teach exploratory testing
!
Henrik Kniberg
?
Done includes ”no added technical debt” Backlog
Henrik Kniberg
Ready for dev
In progress
Ready for production
- No added tech debt
Example: Test automation backlog
Step 1: Decide what needs to be tested • • • • • • • •
Henrik Kniberg
Change skin Security alert Transaction history Block account Add new user Sort query results Deposit cash Validate transfer
Step 2: Classify each test Pay every time Test case
Risk
Pay once
Manual Test Automation Cost Cost
Change skin
low
0.5 hrs
high
Security alert
high
1 hrs
high
Transaction history
med
3 hrs
low
Block account
high
5 hrs
low
Add new user
low
0.5 hrs
low
Sort query results
med
2 hrs
medium
Deposit cash
high
1.5 hrs
low
Validate transfer
high
3 hrs
medium
Henrik Kniberg
Step 3: Sort the list Test case
Risk
Manual Test Automation Cost Cost
Block account
high
5 hrs
low
Validate transfer
high
3 hrs
medium
Transaction history
med 3 hrs
low
Sort query results
med 2 hrs
medium
Deposit cash
high
1.5 hrs
low
Security alert
high
1 hr
high
Add new user
low
0.5 hrs
low
Change skin
low
0.5 hrs
high
Henrik Kniberg
Automate first!
Automate later
Don’t bother automating
Example: Tech backlog Product backlog
Tech backlog
Henrik Kniberg
Reserve X% of team capacity for the tech backlog Product backlog
80%
Tech backlog 20%
Henrik Kniberg
Sprint
Example: Spotify
Henrik Kniberg
00:22
Play Everywhere!
Henrik Kniberg
Like a magical music player in which you’ve bought every song in the world!
>400 people in tech
Stockholm Gothenburg
San Francisco
New York
36
Henrik Kniberg
> 50 squads
Henrik Kniberg
Autonomous Squad
Cross-functional, co-located, self-organizing team
Henrik Kniberg
39
Squads are grouped into Tribes Tribe
Tribe
Henrik Kniberg
Tribe
Tribe
Tribe
Tribe
Each Tribe is a lightweight matrix focused on delivery Vertical = Delivery. Horizontal = knowledge sharing & personal development Tribe Tribe
Chapter
Chapter
Chapter
Guild Chapter
Reality is messy
Henrik Kniberg
Decoupling to enable frequent releases
Client App squad
!#?
Feature squads
Henrik Kniberg
Self-service model Infrastructure squads
Client App squads
Enable & support IOS
Android
Feature squads
Enable & support
Henrik Kniberg
Desktop Enable & support
Web
Release trains & Feature toggles
Henrik Kniberg
Failure Recovery is more important than Failure Avoidance Failure Avoidance
Henrik Kniberg
Failure Recovery
“Limited Blast Radius� via decoupled architecture
Henrik Kniberg
”Limited Blast Radius” via gradual rollout
Henrik Kniberg
Trust > Control 100% control = 0% motion If everything’s under control, you’re going too slow! - Mario Andretti
Henrik Kniberg
Idea/Problem “Radio you can save!”
Narrative & Prototypes & Metrics
“Follow your favorite artist”
A/B stats
Build MVP Deploy
Tweak Analyze data Henrik Kniberg
Example: Big Government Project
Henrik Kniberg
00:22
Team structure - before 3 Development teams
Requirements analyst team
!
Test team
!
#%
#%
!?
!?
52
Henrik Kniberg
Ideas
Features
Next 10 features
Development
System test
User acceptance test
53
Henrik Kniberg
Production
Team swimlanes
Next 10 features
Dev in progress
Ready for sys test
Sys test progress
54
Henrik Kniberg
Team swimlanes
Dev Team 1
Henrik Kniberg
Next 10 features
Dev in progress
Dev Team 2
Ready for sys test
Dev Team 3
55
”Daily cocktail party” 9:15 – 10:15
56
Henrik Kniberg
10:00 – 10:15 Project sync
9:45 – 10:00
Test sync
Henrik Kniberg
9:45 – 10:00
Requirements sync
9:30 – 9:45
9:30 – 9:45
Feature team 1
Feature team 2
Dev sync
9:15 – 9:30
Feature team 3 57
Example: Measuring velocity by counting cards
Count cards
Velocity per week
58
Henrik Kniberg
Example: Release planning using a burnup chart All of these will be done
Total # of delivered features
Week Some of these will be done, but not all
None of these will be done 60 Henrik Kniberg
60
�Oh no, bottleneck in System Test!
FLOW
61
Henrik Kniberg
Tech stories
Next 10 features
Next 5 tech stories
62
Henrik Kniberg
Bottleneck ”Let’s stop building new features”
”... and focus on test automation!”
63
Henrik Kniberg
Everyone doing tech stories
64
Henrik Kniberg
Top 3 recurring bugs
65
Henrik Kniberg
Definition of ”ready for development” Definition of ”ready for system test”
66
Henrik Kniberg
Henrik Kniberg
Henrik Kniberg
67
67
Release
Before: Test at end Week 1
Test Week 2
Week 3
Week 4
Week 5
Week 6
Fix %&@#! Week 7
Week 8
Release
Now: Test continuously Week 1
Week 2
Week 3
Week 4 Test
Test at end:
Week 5
Week 6
Week 7
Week 8
Fix
%&@#!
Test continuously: Test
Fix
Time saved!
68
Henrik Kniberg
Bug fixing process Bug found!
Blocker?
Yes
Write sticky-note, find developer, fix now!
No More important than any of the current top 30?
Yes
Replace one of the other top 30 bugs with this one
No Ignore it
Don’t log it. Fix it NOW!
69
Henrik Kniberg
Three input queues Next 10 features
Next 5 tech stories
Next 5 lower priority bugs
70
Henrik Kniberg
Wrapup
Henrik Kniberg
00:22
What is an Agile Tester? • An agile team member with testing expertise • Helps the team become quality-aware • ... and learn how to deliver better stuff
Henrik Kniberg
Mindset
Henrik Kniberg
Quality Assurance
Quality Assistance
Manual test
Automatic test
Functional test
Exploratory test
Requirements
Customer needs
Late involvement
Early involvement
Long feedback loop
Short feedback loop
Find defects
Prevent defects
Agile is a direction, not a place The important thing isn’t how you work. The important thing is how you improve the way you work!
Henrik Kniberg