What is agile tester ?

What is an Agile Tester?

Colombo Agile Conf, June 2014

Consultant www.crisp.se

Henrik Kniberg

Father

henrik.kniberg@crisp.se @HenrikKniberg

Agile & Lean coach Author

Agile tester mindset

Henrik Kniberg

00:22

Henrik Kniberg

Henrik Kniberg

Henrik Kniberg

Case study: Game development company Design-ready games

Game backlog

15

8

1m

Concept pres.

4h

6m

Actual work: 3 months Time to market: 25 months

Resource planning

1d

1w

Graphics design

Sound design

1m

3w

Production-ready games

12 Dev

6m

6m

3m (1m+2m)

Test & deploy

3w

Before Design-ready games

Game backlog

15

8

1m

Concept pres.

4h

6m

Production-ready games

Resource planning

1d

1w

Graphics design

Sound design

1m

3w

Actual work: 3 months

12 Dev

6m

6m

3m (1m+2m)

Integr. & deploy

3w

Time to market: 25 months

After Cross-functional game teams

7 times faster

Game team (graphics, sound, dev, test, deploy, etc)

Time to market: 3-4 months

Better games

Less Planning

More fun

Cross-functional teams I’m fast!

We’re slow!

6 months

Joe

Dave

Lisa

Release

3 months

We’re alot faster!

Joe I’m a bit slower

Dave Lisa January

February

Release

March

April

May

June 9

Henrik Kniberg

July

Test phase

Test team

Tester role

Henrik Kniberg

Test phase Req

Henrik Kniberg

Code

Test

We own quality

Programmers

Henrik Kniberg

Testers

Test team

I own quality

Tester

Dev team

Henrik Kniberg

Tester role

Role

Competency

We own quality

Cross-functional development team

QA

Henrik Kniberg

= Quality Assurance = Quality Assistance

Roles DB GUI

Design

Backend Henrik Kniberg

Test

Competencies DB

Design

Backend Henrik Kniberg

GUI

Test

Knows a bit about many things

Cross functional team

Knows a lot about one thing

Doesn’t mean everyone has to know everything Product Backlog

I can do Java, but I’m not so good at it.

Skills Needed to implement Top X backlog items Java

Web

Test Domain CM I’m good at Test!

Lisa Joe Fred Jenny David Erik I won’t even go near a database!

Henrik Kniberg

DB

I don’t know CM at all. But I’m willing to learn!

ensure

How do you know that your product works?

Who are the stakeholders?

1. Understand the problem

What need do they have, that we want to solve?

How will we know when we’ve solved it?

How will we know if we’re moving in the right direction?

2. Iterate until you’ve solved it Henrik Kniberg

Minimize the distance to MVP Deliver, measure, adjust continuously

Typical activities

Henrik Kniberg

00:22

Make sure backlog items are testable & valuable As a buyer I want to save my shopping cart so that I can continue shopping later

Henrik Kniberg

How to demo: 1) Enter store 2) Put a book in shopping cart 3) Press ”save cart” 4) Leave store, and enter it again 5) Check that the book is in my cart

Find defects early!

Cost of defect

$

Henrik Kniberg

Age of defect

Sit with the developers Developers

Testers

Henrik Kniberg

Developers & Testers

Meeting room

Shorten the feedback loop Maker

5

# of handoffs

People (# of handoffs) 2 1

4 3 2 1

Time (Feedback delay)

0

minutes hours days weeks months years

Length of feedback cycle

Henrik Kniberg

3

User

Push for Continuous Delivery Automatic Manual Code & commit

Henrik Kniberg

Build

Test & integrate

Deploy to staging

Deploy to prod Manual test

Create a shared vocabulary

Integration test?

Acceptance test?

Henrik Kniberg

Unit Test?

Quality?

Technical debt?

Set working agreements for test automation Each user story has at least one black-box acceptance test

AT and UT complement each other

• Acceptance tests: coupled to UI, decoupled from internal design • Unit tests: coupled to internal design, decoupled from UI

Complex code has unit tests

Almost all code is covered by SOME kind of test (AT or UT)

Henrik Kniberg

We know which code isn’t covered and have a good reason for it

Do & teach exploratory testing

!

Henrik Kniberg

?

Done includes ”no added technical debt” Backlog

Henrik Kniberg

Ready for dev

In progress

Ready for production

- No added tech debt

Example: Test automation backlog

Step 1: Decide what needs to be tested • • • • • • • •

Henrik Kniberg

Change skin Security alert Transaction history Block account Add new user Sort query results Deposit cash Validate transfer

Step 2: Classify each test Pay every time Test case

Risk

Pay once

Manual Test Automation Cost Cost

Change skin

low

0.5 hrs

high

Security alert

high

1 hrs

high

Transaction history

med

3 hrs

low

Block account

high

5 hrs

low

Add new user

low

0.5 hrs

low

Sort query results

med

2 hrs

medium

Deposit cash

high

1.5 hrs

low

Validate transfer

high

3 hrs

medium

Henrik Kniberg

Step 3: Sort the list Test case

Risk

Manual Test Automation Cost Cost

Block account

high

5 hrs

low

Validate transfer

high

3 hrs

medium

Transaction history

med 3 hrs

low

Sort query results

med 2 hrs

medium

Deposit cash

high

1.5 hrs

low

Security alert

high

1 hr

high

Add new user

low

0.5 hrs

low

Change skin

low

0.5 hrs

high

Henrik Kniberg

Automate first!

Automate later

Don’t bother automating

Example: Tech backlog Product backlog

Tech backlog

Henrik Kniberg

Reserve X% of team capacity for the tech backlog Product backlog

80%

Tech backlog 20%

Henrik Kniberg

Sprint

Example: Spotify

Henrik Kniberg

00:22

Play Everywhere!

Henrik Kniberg

Like a magical music player in which you’ve bought every song in the world!

>400 people in tech

Stockholm

Gothenburg

San Francisco

New York

36

Henrik Kniberg

> 50 squads

Henrik Kniberg

Autonomous Squad

Cross-functional, co-located, self-organizing team

Henrik Kniberg

39

Squads are grouped into Tribes Tribe

Tribe

Henrik Kniberg

Tribe

Tribe

Tribe

Tribe

Each Tribe is a lightweight matrix focused on delivery Vertical = Delivery.

Horizontal = knowledge sharing & personal development Tribe Tribe

Chapter

Chapter

Chapter

Guild Chapter

Reality is messy

Henrik Kniberg

Decoupling to enable frequent releases

Client App squad

!#?

Feature squads

Henrik Kniberg

Self-service model Infrastructure squads

Client App squads

Enable & support IOS

Android

Feature squads

Enable & support

Henrik Kniberg

Desktop Enable & support

Web

Release trains & Feature toggles

Henrik Kniberg

Failure Recovery is more important than Failure Avoidance Failure Avoidance

Henrik Kniberg

Failure Recovery

“Limited Blast Radius� via decoupled architecture

Henrik Kniberg

”Limited Blast Radius” via gradual rollout

Henrik Kniberg

Trust > Control 100% control = 0% motion If everything’s under control, you’re going too slow!

- Mario Andretti

Henrik Kniberg

Idea/Problem “Radio you can save!”

Narrative & Prototypes & Metrics

“Follow your favorite artist”

A/B stats

Build MVP Deploy

Tweak Analyze data Henrik Kniberg

Example: Big Government Project

Henrik Kniberg

00:22

Team structure - before 3 Development teams

Requirements analyst team

!

Test team

!

#%

#%

!?

!?

52

Henrik Kniberg

Ideas

Features

Next 10 features

Development

System test

User acceptance test

53

Henrik Kniberg

Production

Team swimlanes

Next 10 features

Dev in progress

Ready for sys test

Sys test progress

54

Henrik Kniberg

Team swimlanes

Dev Team 1

Henrik Kniberg

Next 10 features

Dev in progress

Dev Team 2

Ready for sys test

Dev Team 3

55

”Daily cocktail party” 9:15 – 10:15

56

Henrik Kniberg

10:00 – 10:15 Project sync

9:45 – 10:00

Test sync

Henrik Kniberg

9:45 – 10:00

Requirements sync

9:30 – 9:45

9:30 – 9:45

Feature team 1

Feature team 2

Dev sync

9:15 – 9:30

Feature team 3 57

Example: Measuring velocity by counting cards

Count cards

Velocity per week

58

Henrik Kniberg

Example: Release planning using a burnup chart All of these will be done

Total # of delivered features

Week Some of these will be done, but not all

None of these will be done 60 Henrik Kniberg

60

�Oh no, bottleneck in System Test!

FLOW

61

Henrik Kniberg

Tech stories

Next 10 features

Next 5 tech stories

62

Henrik Kniberg

Bottleneck ”Let’s stop building new features”

”... and focus on test automation!”

63

Henrik Kniberg

Everyone doing tech stories

64

Henrik Kniberg

Top 3 recurring bugs

65

Henrik Kniberg

Definition of ”ready for development” Definition of ”ready for system test”

66

Henrik Kniberg

Henrik Kniberg

Henrik Kniberg

67

67

Release

Before: Test at end Week 1

Test Week 2

Week 3

Week 4

Week 5

Week 6

Fix %&@#!

Week 7

Week 8

Release

Now: Test continuously Week 1

Week 2

Week 3

Week 4 Test

Test at end:

Week 5

Week 6

Week 7

Week 8

Fix

%&@#!

Test continuously: Test

Fix

Time saved!

68

Henrik Kniberg

Bug fixing process Bug found!

Blocker?

Yes

Write sticky-note, find developer, fix now!

No More important than any of the current top 30?

Yes

Replace one of the other top 30 bugs with this one

No Ignore it

Don’t log it.

Fix it NOW!

69

Henrik Kniberg

Three input queues Next 10 features

Next 5 tech stories

Next 5 lower priority bugs

70

Henrik Kniberg

Wrapup

Henrik Kniberg

00:22

What is an Agile Tester?

• An agile team member with testing expertise • Helps the team become quality-aware • ... and learn how to deliver better stuff

Henrik Kniberg

Mindset

Henrik Kniberg

Quality Assurance

Quality Assistance

Manual test

Automatic test

Functional test

Exploratory test

Requirements

Customer needs

Late involvement

Early involvement

Long feedback loop

Short feedback loop

Find defects

Prevent defects

Agile is a direction, not a place The important thing isn’t how you work.

The important thing is how you improve the way you work!

Henrik Kniberg