Testing: Value Galore

Bob   Bloom    Feb 24, 2012



Automated testing is a key technology being incorporated into the workings of the Joomla Bug Squad. At the present, this is a somewhatspecialized skill


Specialized, indeed!

There is no glory in testing. There is no glamour in testing. But, there is tremendous value in testing. And extracting that value is what attracts us to testing.

Testing is a major PITA. It’s easier to be casual about it. The problem with casual is that your live site becomes one huge test facility. Live sites cannot be fully simulated in tests, but doing as much testing as possible before the code hits your live site increases your chances of enjoying a smooth running, money-making live site.

As Alan Langford told me, it takes time and effort to set up testing. You have to code so that your code is test-able. The reward is two-fold: quality increases hugely which reduces your operational risk; and, subsequent testing becomes easier as the initial testing set-up is done.

Since testing is not glamourous, is not glorious, is usually thankless, and delays code getting to market, it’s not only glossed over but it’s a skill that only a subset of programmers have. Indeed, a “somewhat specialized skill”.

As of February 2010, a tests folder has been added to the Joomla! SVN download. This folder contains a growing library of unit and system (or functional) tests. The unit tests use PHPUnit and the system tests use PHPUnit and Selenium. The unit tests perform tests primarily on the Joomla! framework. The Selenium system tests actually run Joomla! from a browser and test it as a user would.


Hmmm, does this mean that such testing was not part of Joomla 1.0 & Joomla 1.5? The implication is that the value of such testing is recognized to exceed its costs only recently. Regardless of when this recognition occurred, we should take the hint at Club Commerce!

I cannot emphasize enough that controlling the software development process offers massive RoI. Open source software gives us the right to control the code. But the huge value resides in the development process. That’s the “Moneyball” aspect — the conventional wisdom is that the value is with the code. But the true value is the development process.

With a testing regimen, we can hire a ton of different programmers for a ton of different projects. The programmers code, and then we run their code through our tests. Code that passes is integrated into our codebase.

We need an explosion of features. So, our software development process must allow us to program an explosion of features. Up-front investment in testing allows us to achieve an explosion of testing.

Testing is a road less travelled, but rich in value. The Jenkins Server is mostly about testing.

I had no idea how deeply Alan Langford, a Founding Member, is involved with testing:

Ground work on the implementation of unit testing has been done by Enno Klasing during the Summer of Code 2007 project and has been perfected by Alan Langford from the Joomla! Development Team.


This is going to make your eyes glaze over:http://permalink.gmane.org/gmane.comp.php.pear.devel/43667

Last week, after we recorded our “The Bob Bloom Show” interview, we chatted for a half hour about LaSalleMart and testing. Alan agreed to do a special one-on-one podcast on testing and LaSalleMart, which we’ll do in March. What prompted this post-show interview was something Alan said off-the-cuff at the recent Toronto Joomla meet-up about testing — which had nothing to do with Club Commerce or LaSalleMart.

Testing is something I did in a former life as a Business Analyst. Use-case testing, mostly, where you dream up phony numbers, manually calculate what the computer is supposed to spit out, then see if the system actually outputs those numbers. Well, we can do “use case” testing in automated tests, saving the hassle of foisting it onto end users.

So, we have a valuable testing skill-set here at Club Commerce. As much as I yearn for all the pet projects that are piling on already to be coded, I am going to devote myself to constructing the test environment. If this takes a couple of months, so be it, because the payoff is massive. We are going to be leaders in testing and Continuous Integration, with the payoff being the ability to accomodate an explosion of features — and getting this quality code to your sites so you can make money with these features.

It’s going to be very interesting, and exciting, to put other extensions through our testing before you install them on your site. This is completely in keeping with the idea that GPL software is Raw Material.

I’m rushing this blog post, because I want to squeeze it in before publishing my already-late February Hangout podcast. So a few observations…

In order to bring an explosion of features to your live site, we need a software development process that accomodates a massive amount of programming.

This software development process, by definition, cannot be like the process other extension purveyors use — because those processes have left us yearning for features.

If the road well travelled has left us frustrated, then the road less travelled is where the pot of gold is.

Most people do not like to travel on the roads less travelled.

My Club Commerce is for site owners and consultants who realize they need to travel on the road less travelled. With 140 memberships available, this is not a mass marketed club.

The “Moneyball” attitude leads us to probe where the value really resides. We identify where the value really resides; and, then we become skilled at these things that build value.

Believe it or not, we’ve concluded that coding is over-rated!

The software development process is where the value is. And, the soul of value is Continuous Integration.

Continuous Integration = test + build.

Testing is a “somewhat specialized skill” — relatively few coders are skilled at testing.

“Build” has led us to the shocking conclusion of abandoning the Joomla installer!

The Fallacy of the One Big Zip file!

We have a surprising competency in testing at Club Commerce. The Toronto Joomla Group, of which we participate, has an unusual competency in testing. So, Club Commerce will be a leader in testing; and, in building ( = Continuous Integration).

So, despite an initial up-front investment of time/effort creating a CI environment, the payoff will be an explosion of features for you.

There are a lot of programmers. Hiring programmers should be easy. We will need money to hire programmers to code.

Club Commerce is a funding model to hire programmers!

There’s an article about GitHub.com called “Lord of the Files” (clever!) athttp://m.wired.com/wiredenterprise/2012/02/github/all/1. My favourite quotes: “They weren’t the only developers chafing under that Trusted Gatekeeper model of open source”, and “It’s the startup’s outright hostility toward corporate command-and-control that really sets it apart… yet it’s the most productive software development team he’s ever worked on,”.

My favourite quote from an article about how GitHub.com’s software development,http://scottchacon.com/2011/08/31/github-flow.html is, “we deploy all the time… We don’t really have ‘releases’ because we deploy to production every day – often several times a day”. Yikes! How’s that for “continuous”? No version numbers!