Hm… (forefinger to chin)

A tester that is new to a project, has little knowledge of the system he is testing. Thus testing with a black box approach. More often than not, this tester finds new bugs the experienced tester haven’t seen.

Does this mean that black box testing is a better way to find more bugs that influence the end user, rather than a deep knowledge of the system (that clouds the mind?)

Test new functions in production

I like having other people test stuff for me. I’m a test manager, that’s what I do for a living.

I like even better having the customer test what they have requested by my company for me. That gives an absolute feedback on the quality we have produced.

So far this kind of testing have been the norm in “customer environment”, but as we who have tested all know, there are always something different from one environment to the next(different network, hardware etc etc).

So the most correct place to test new functions is production. Postponing these kind of “acceptance” to after the production setting, gives lots of Test and Quality assurance people the shivers. Why? Their job is usually to know the status of things before decisions makers give the go/nogo for production.

In agile world, most of the releases going to production are small and usually add-ons to already used function. And these new functions nobody are using yet. So the risk of having errors on untested functions are sometimes high, but the impact is extremely low.

I as far as I have experienced, customers requested something, like to be the first to try it.

Risk is higher, but the feedback on bugs are better, more reliable and correct, and time to market is a lot shorter.

Skip all environments, put the customers in the driving seat and test it in production.

To automate or not to automate - that is the question

void do_i_automate()

{

    if ((Time spent testing)*10) < (time spent automating and maintaining the automated tests) {

        DON’T AUTOMATE IT;

    }

else

{

        AUTOMATE IT;

    }

(If you will test it more than 10 times the coming year and the total time spent testing it these 10 times exceeds the time automating and maintaining the automated tests, automate it)

(I’m no programmer)

The beauty of documented exploratory testing

How many times have you ever had to go back and check test logs for when something was tested last time? You might if an error occurs in production, but the only benefit you will get is probably who to blame for forgetting something. Dale Carnegie (author of “How To Win Friends And Influence People, 1936”)didn’t think that was a good idea, and me neither. People who screw up usually know when they have done so and feel pretty bad when they do.

A few years back, I started to use something we called documented exploratory testing as the sole way of testing our services.

It's simply exploratory testing with the addition of notes. The tester writes down some notes during his testing, and saves them to our "test log folder". Any tester is using notes while testing a rather large and complicated service, on paper or other media his or she prefers.  These notes serve a testing log for us, and help us check of the traceability “requirement”. By doing this we can focus on actual testing, and “skip” the paperwork, and that’s a lot more fun.

The struggle to use off shored resources for development and test on a system that is “high maintenance”

Imagine that you are working on a system that has a lot of customers that require day to day maintenance, and this maintenance is manual labour. Your team has development and testing assistance from a team on the other side of the planet, but they are not able to ease the workload of the maintenance tasks. They could on the other hand do a lot of the development/test on the new and fun functions of the system.

In mine point of view this is a loose/loose scenario for your team. Your team get stuck with the maintenance task (which usually is not that fun), and are unable to have time to work on new stuff. The solution is of course; get the offshored team to automate all the maintenance tasks/make tools to automate, while your team start toying around with the new functions. OK. Easier said than done; the offshored team are not allowed to fiddle with production data because of company and country rules and regulations. They could make 0.-version of the automation tools, but they are not able to use them or trouble shoot them when installed in production.

So, Internet, what’s your idea to solve this?