Testing Insepction by Joan Avila

Increasing Responsiveness and Economy of Software Inspection Marko Komssi marko.komssi@f-secure.com 2004-12-03

Contents Problems with inspection processes and rules Generating a minimal number of inspection rules based on risks and goals Progressively increasing the precision of inspection practices

Traditional Inspection Process1

Entry Kickoff and Meeting Planning

Individual Checking

Logging Meeting

Specification and Process Meetings

Editing

Edit Auditing

Exit

Inspection Process Variables • Amount of inspection rules and phases • Number of inspectors and their roles • Duration of inspection sub-phase and total time spent • Checking rate • Proportion of product under inspection • Experience of inspectors • Rigidity of inspection entry and exit criteria

Inspection Rules A common way to seek errors is to verify the product against inspection rules Defects in a code or a document are violations of a rule CUSTOMER — customer requirements must be real • Does the statement in the specification describe the customer? • Is the need of the customer correctly presented?

Problems with Inspection Rules and Processes A rigorous and stable inspection process • Different company maturity levels and development processes • Alternative quality goals and development phases of projects • A high threshold for using a heavy inspection model

Ineffective inspection rules • Inspection rules may reveal only minor defects • Defects may have high severity but low priority

Effectiveness of Inspection Rules (Requirements)

Effectiveness of Inspection Rules (User Interface)

Not Too Much Precision, Please! Write long and precise specifications only if you know what you are doing • “Odd though it may sound, you will typically do less damage if you write too little rather than too much4.“ • “Even the smallest features pack more calories than you might think5.“

Take compactness as part of a inspection strategy • “Then you will have a short, readable document, which means that people will bother to read it and then ask questions4.“

Include formality in review practices just as much as is needed6

Goal- and Risk-Based Inspection Rules The three most effective rules cover 60–80 % of problems It is economical to inspect issues that: • support the goal of the product • are likely to harm the project in the current development phase

Rule generation can be done by imitating the concepts of Goal-Question-Metric2 (GQM) and Risk-based testing3

Progressive Inspection Progressive inspection (PI) is a concept for improving the traditional inspection process: • Increases the precision of review practices as the project development progresses • Uses the cheapest possible review techniques associated with the quality goals and the risks of the product6 • Weights different stages of inspection

Emphasis in the Beginning of Development

Entry and Planning

Kickoff Meeting

Individual Checking

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

Slim Quality Goals in the Beginning of Development

Entry and Planning

Kickoff Meeting

Individual Checking

Few rules

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

Soft exit criteria 13

Emphasis in the End of Development

Entry Kickoff and Meeting Planning

Individual Checking

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

Increasing Precision in the End of Development

Entry Kickoff and Meeting Planning

Individual Checking

A larger set of rules

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

Rigid exit criteria 15

PI inside XP and EVO In XP7, high-level goals of user stories can be questioned with targeted inspections In EVO8, inspection and its techniques can be tailored based on the needs of the project â&#x20AC;˘ Inspections are started by concentrating in planning and brainstorming â&#x20AC;˘ In each inspection cycle, practices are developed based on comments from the inspection data and the team

Conclusions Traditional inspection practices may be cumbersome Three inspection rules suffice in the early stages of development In PI, inspection practices increase in precision and progress during project development

References 1.

Gilb, T., Graham D., Software Inspection. Addison-Wesley (1993).

Basili, V.R., Rombach, H.D., The TAME Project: Toward ImprovementOriented Software Environments. IEEE Transactions on Software Engineering, 14, 6 (1988), 758–773.

Bach, J., James Bach on Risk-Based Testing. How to Conduct Heuristic Risk Analysis? Software Testing & Quality Engineering, 1, 6 (1999), 23–28.

Cockburn, A., Writing Effective Use Cases. Addison-Wesley (2001).

McConnell, S., Achieving Leaner Software. IEEE Software, 14, 6 (1997), 127–128.

Wiegers, K.E., Peer Reviews in Software: A Practical Guide. AddisonWesley (2001).

Beck, K., Embracing Change with Extreme Programming. Computer, 32, 10 (1999), 70–77.

Gilb, T., Principles of Software Engineering Management. Addison-Wesley (1988).

Europe’s Premier Software Testing Event World Forum Convention Centre, The Hague, Netherlands

“The Future of Software Testing”

Traditional QA Meets Agile Development Dietmar Strasser, Borland, Austria WWW.QUALTECHCONFERENCES.COM

Traditional Testing meets Agile Development Dietmar Strasser Director QA, Lifecycle Quality Management dietmar.strasser@borland.com

Agenda Journey towards an Agile Team Our Environment we live in How do we provide Visibility? Q & A

Journey towards an Agile Team

Starting Point

SilkPerformer Developers

SilkTest Developers

Test Manager Developers

PM ... Product Manager PO ... Product Owner Copyright ÂŠ 2008 Borland Software Corporation.

Confidential

Doc

Adding Product Owner

SilkPerformer Developers

SilkTest Developers

Test Manager Developers

PM ... Product Manager PO ... Product Owner Copyright ÂŠ 2008 Borland Software Corporation.

Confidential

Doc

User Story Workflow – Testers not integrated In Progress QA

Unassigned PO

Drafted

QA Ready

Dev PO

Approved

In Progress Dev

Not Started

In Progress

Confidential

Drop Ready Drop Ready QA

RTM Ready

User Story Workflow - „Small Waterfall“

Testing In Progress

Unassigned

QA •User story •Iteration x+1 •4 weeksQA

Drafted

QA Ready Dev Coding

Approved

PO •User story In Progress •Iteration x Dev

•4 weeks

In Progress Confidential

Drop Ready Drop Ready

Release Testing RTM Ready QA

•All user stories •Last RTMIteration Ready •4 weeks

Lesson Learned

“Ask the Team”

Confidential

Adding Tester Skills

SilkPerformer Developers

Tester

SilkTest Developers

Tester

Test Manager Developers

Tester

PM ... Product Manager PO ... Product Owner Copyright ÂŠ 2008 Borland Software Corporation.

Confidential

Doc

Adding Documentation Skills

SilkPerformer Developers

Tester

Doc

SilkTest Developers

Tester

Doc

Test Manager Developers

Tester

Doc

PM ... Product Manager PO ... Product Owner Copyright ÂŠ 2008 Borland Software Corporation.

Confidential

Transition to Engineering Team

SilkPerformer Engineering Team

SilkTest Engineering Team

Test Manager Engineering Team

PM ... Product Manager PO ... Product Owner Copyright ÂŠ 2008 Borland Software Corporation.

Confidential

Adding QM Coach

QM Coach PM

SilkPerformer Engineering Team

SilkTest Engineering Team

Test Manager Engineering Team

PM ... Product Manager PO ... Product Owner Copyright ÂŠ 2008 Borland Software Corporation.

Confidential

Splitting & Re-Locating Teams

QM Coach PM

SilkPerformer Engineering Team

SilkTest Engineering Team

Test Manager Engineering Team

PM ... Product Manager PO ... Product Owner Copyright ÂŠ 2008 Borland Software Corporation.

Confidential

User Story Workflow - Agile Unassigned PO

In Progress

Drafted

Agile Scrum Team

Finish user story in DONE one 4-weeks Iteration

Approved

DONE

In Progress Confidential

Lesson Learned

“Agile is a journey, not a destination”

Confidential

Distributed Team Environment QM Coach

Product Scrum Team(s)

EQC Resource Pool

Daily

EQC Contact

Quarterly

Confidential

Lesson Learned

“Communicate, communicate, communicate, …”

Confidential

Our Environment we live in

Our Environment we live in Project Management, Reporting Management RBT Environment

Iteration Management

Requirements Management

Test Management

Scrum Team(s)

Product Owner

Scrum Team(s)

Source

Functional/

Management

Performance Testing

Scrum Team(s)

Confidential

xUnit

Scrum Team(s)

Lesson Learned

“People are more important than processes & tools”

Confidential

How do we provide Visibility?

Types of Visibility • Internal Visibility • • • •

Daily Stand-Ups Iteration Review Meetings Regular Updates on Production Systems Project Dashboard

• External Visibility • Regular „Drops“ for customers and field people

Confidential

Project Dashboard

Goal Story Report

Scrum Team Reports Iteration-Related

Confidential

Quality-Related

Getting into Details

User Story Reports

Provide Visibility

(Executive Summary)

Goal Story Report

Confidential

User Story Report

Confidential

Q&A

Email to:dietmar.strasser@borland.com