Software QA FYI - SQAFYI

How Google Tests Software

By: James A. Whittaker

here is one question I get more than any other. Regardless of the country I am visiting or the conference I am attending, this one question never fails to surface. Even Nooglers ask it as soon as they emerge from new employee orientation: How does Google test software?

I am not sure how many times I answered that question, or even how many different versions I gave, but the answer kept evolving the longer I spent at Google and the more I learned about the nuances of our various testing practices. I had it in the back of my mind that I would write a book and when Alberto, who likes to threaten to turn testing books into adult diapers to give them a reason for existence, actually suggested I write such a book I knew it was going to happen.

Still, I waited. My first problem was that I was not the right person to write this book. There were many others who preceded me at Google and I wanted to give them a crack at writing it first. My second problem was that I as Test Director for Chrome and Chrome OS (a position now occupied by one of my former directs) I had insights into only a slice of the Google testing solution. There was so much more to Google testing that I needed to learn.

At Google software testing is part of a centralized organization called Engineering Productivity that spans the developer and tester tool chain, release engineering and testing from the unit level all the way to exploratory testing. There is a great deal of shared tools and test infrastructure for web properties such as search, ads, apps, YouTube and everything else we do on the web. Google has solved many of the problems of speed and scale and this allows us, despite being a large company, to release software at the pace of a startup. As Patrick Copeland pointed out in his preface to this book, much of this magic has its roots in the test team.

At Google software testing is part of a centralized organization called Engineering Productivity.

When Chrome OS released in December 2010 and leadership was successfully passed to one of my directs, I began getting more heavily involved in other products. That was the beginning of this book and I tested the waters by writing the first blog post[1] on How Google Tests Software and the rest is history. Six months later the book was done and I wish I had not waited so long to write it. I learned more about testing at Google in the last six months than I did my entire first two years and Nooglers are now reading this book as part of their orientation.

This isnít the only book about how a big company tests software. I was at Microsoft when Alan Page, BJ Rollison and Ken Johnston wrote How We Test Software at Microsoft and lived first-hand many of the things they wrote about in that book. Microsoft was on top of the testing world. They had elevated test to a place of honor among the software engineering elite. Microsoft testers were the most sought after conference speakers. Their first Director of Test Roger Sherman attracted test-minded talent from all over the globe to Redmond Washington. It was a golden age for software testing.

And they wrote a big book to document it all.

I didnít get to Microsoft early enough to participate in that book, but I got a second chance. I arrived at Google when testing was on the ascent. Engineering Productivity was rocketing from a couple hundred people to the 1200 it has today. The growing pains Pat spoke of in his preface were on their last throes and organization was in its fastest growth spurt ever. The Google Testing blog was drawing hundreds of thousands of page views every month and GTAC[2] had become a staple conference on the industry testing circuit. Patrick was promoted shortly after my arrival and had a dozen or so Directors and Engineering Managers reporting to him. If you were to grant software testing a renaissance, Google was surely its epicenter.

Which means the Google testing story merits a big book too. The problem is, I donít write big books. But then, Google is known for its simple and straightforward approach to software. Perhaps this book is inline with that reputation.

How Google Tests Software contains the core information about what it means to be a Google tester and how we approach the problems of scale, complexity and mass usage. There is information here you wonít find anywhere else but if it is not enough to satisfy your craving for how we test, there is more available on the web. Just Google it!

There is more to this story though than that it must be told and I am finally ready to tell it. The way Google tests software may very well become the way many companies test as more software leaves the confines of the desktop for the freedom of the web. If youíve read the Microsoft book, donít expect to find much in common with this one. Beyond the number of authors, both books have 3, and the fact that each book documents testing practices at a large software company, the approaches to testing couldnít be more different.

Full article...

Other Resource

... to read more articles, visit

How Google Tests Software