Insights

Rethinking QA, Part I

By Daniel Pasco

Black Pixel is known for shipping high quality apps. There are three ingredients that contribute to this, two of which are great design and great development.

Let Me Tell You A Secret

Our best-kept secret is the third ingredient: great QA.

Quality assurance is one of our most powerful secret weapons, and I'm not worried about being copied, because - as with other development concepts - simply going through the motions isn't enough to make a difference. Besides, really good secrets continue to be useful even after being publicly disclosed.

Can you say, with a straight face, that the people on your QA team are regarded as heroes throughout your company? If not, there's probably room for improvement, and it will make a *big* difference, whether you're part of the product team or the company CFO.

To be sure, there are organizations that take QA seriously, but I've only met one or two in 16 years of development that I looked at and thought, "Those folks really get it." If you do, maybe we'll have some interesting notes to compare once you've read through this.

"QA is a waste of air"

My past experiences with QA, with previous employers and clients, had been universally negative, and seemed like a waste of time and money - in many cases they actually hampered our progress significantly. As my coworkers and I started working with larger clients, we made a point of aggressively testing our apps ourselves, and routinely pulled far ahead of any provided QA team's ability to identify issues.

It's hard to believe now, but I used to think a QA team wasn't worth paying for.

Consequently, I avoided putting together a team of our own until about three years ago, when we had a major client ask us to provide dedicated QA in addition to our development and design staff. It was a significant project, and we wanted to deliver the best possible results, so we set out to do it right.

"Let's do this differently"

The first person we brought on board to do QA was Nat Irons, who not only made a great impression on us, but also came with an impeccable reference: John Gruber.

Daniel,

My long-time friend Nat Irons informs me that he's applying for a QA position at Black Pixel. Allow me to vouch for him.

I'm not even sure when I first met him. Could be that he was just a very early (i.e. circa 2002) DF reader. But I think we actually got to know each other on the BBEdit users mailing list, back around 2000 or so. Maybe even earlier?

He's damn smart, a good guy, the sort of person who is always willing to reconsider his position in the face of new evidence, and fastidious. He was a primary sounding board for syntax ideas back when I was creating Markdown, and his input improved the result significantly. He's one of the people I love to use as a sounding board for ideas in general, because I know he'll tell me what he really thinks, *not* what he thinks I want to hear -- despite the fact that he's the sort of deeply empathetic humanist who knows instinctively what it is that someone does *want* to hear.

I can also say that he's either first or second in terms of the total number of typographic/grammatical errors reported to me regarding DF posts. Nat and one other guy report a couple of them per week, every week. Hundreds and hundreds of errors that I've corrected on DF posts over the last 8 years, thanks to Nat, simply because he's my friend, has a sharp eye, and is a perfectionist.

I think you'll find he knows his shit, too.

I can't tell you what to do, but if *I* were hiring a QA engineer and Nat was available, I'd hire him on the spot.

Regards,

--J.G.

This was much more in keeping with what we were looking for. I'm not stupid, I hired Nat on the spot.

John's endorsement was an understatement: it became clear that Nat empathized with the negative experiences we'd had and shared our excitement for creating a more empowered role for QA on our team. He joined Black Pixel as our first QA person, and brought with him an intensity, intelligence, and diligence that has completely changed how we see the profession.

Furthermore, Nat has been a consummate A player. As the leader of our QA team, Nat took our original goals and ran with them- far exceeding anything I could have imagined when we started out.

"I'm not just the president, I'm also a client"

Our product team* values our QA team because they help us to make great products. Our accountant values our QA team because they make us money by helping cement our reputation for delivering rock-solid, robust applications. That reputation leads to better pricing, increased sales, and customer loyalty.

I don't want to just spout empty platitudes: my next few posts will discuss perspectives on QA from the outside, as a developer and CEO. The process we evolved exists at other companies, but is startlingly rare. Things can be different, and probably better than you've ever imagined.

We have plenty of great people here that can talk about the specifics of their craft, and they will. I want to address some of the popular misconceptions and inefficiencies in corporate QA, and how to do things better. I also want to talk about what employees and business owners can do to change things at their own companies.

I hope you like what's ahead.

-Daniel

*Especially our developers. I was asked to emphasize this by them :)

Next: Rethinking QA, Part II

Daniel Pasco

Daniel Pasco

Daniel is the CEO of Black Pixel. He tweets at @dlpasco.