Test Plan and User (Human) Psychology & behavior

Test Plan: As defined in IEEE 829-1998, also known as the 829 Standard for software test documentation, test plan is a management planning document describing the scope, approach, resources, and schedule of intended testing activities. Test plan identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning. 

The above is a textbook definition of a test plan. As the definition says, it describes the scope, approaches of the test that's going to be performed on the target product. This does not describe/ mention anything about one important thing that I learned from my experience, people may not emphasize on this part is - the Test Strategy.  

So... What is Test Strategy then?- another textbook description - The test strategy is defined set of methods and objectives that direct test design and execution. The test strategy describes the overall testing approach for the testing of application under test including stages of testing, completion criteria, and general testing techniques. The test strategy forms the basis for test plans. 

Strategy plays the key role for the planning of "attacking on software". According to its It defines the testing approach. And the approach would be, in my opinion, "Attacking the Software" from every possible angle of a "User".  It not about only test methodologies, its about going beyond normal scenarios, scenarios that user may create accidentally or intentionally. 

Human psychology is an integral part of testing.  A Tester/ QA personnel should approach testing software thinking of him-self as a user- non tech, mid tech, high tech user. And even sometimes like crazy and/or baby.  Let me mention few examples that can relate the aspect of human psychology and different behavior from user point of view:

#1 Lets say I was given to test an old style Cassette player (if people still can remember what is a Cassette player...). My general approach would be to test all the functions according to the conventional test plan (derived from specs). So I would have test cases for each function, Play, Rewind, Forward, Stop, Record etc. The product is designed with buttons that user would be able to easily use it- could be a soft-touch, could be physical button etc. The general expectation (which actually leads toward limitation) of testing this product would be whether all the functions work properly and as expected or not. Usually, we would assume / design out testing thinking of user will be using it with "hand".  Would anyone/ any tester would think about user may actually want to use it using foot finger? – The answer could be 1% people or even less. To me, it’s not about the percent, it’s about whether I thought about it and designed my test cases for the user's like this or not. 

#2 Another example from real life- with the most recent IOS 7 release, one of the bugs that user found is, it allows to call from the lock screen. It takes seven consecutive enters/ click to make it happen. Now who would think about that magic number of tries (and why would even do that in the first place)? This could be a great place to engaged automated testing. This kind of scenario has to be covered up to certain try so that we will have less chance to have it found by the users. 

#3 I was working on one Automated Taller Machine (ATM) project of a bank around the year 2008. They were implementing ATM interface with the Core Banking Software (CBS). ATM was communicating with hardware and interfacing with CBS. I was engaged at the time of UAT sign off. I tried few things that I think customer might do and I was able to create one scenario. It was more of an implementation issue. 

When user sends a command for withdraw money, for example $100 ($20 x 5 bills), machine do all the checking with CBS and then count money and give the money through dispense. When money comes out through dispense, customer takes it or if it stays on dispense for 30 seconds, system takes it back inside. When it goes it inside, customer can claim that money later at bank and the use case ends there. The problem arises when I took one note from dispense and let it go back inside with 4 notes. Now, I'm still eligible to claim the money and bank has two things- one they had 4 notes in the tray and a withdraw transaction of 5 notes. The camera at the ATM counter did not able to give clear view. So, theoretically I'm entitled to get my $100 (if I don't fall into some text written in smaller font in terms and condition). This scenario created an issue and people were figuring out how to resolve it. This can be one of the example how user can do so many things that the typical test plan would not cover such issue. 

  

#4 Another experience would like to share. This time it was a simple login issue for a dial-up internet service. In the early days, around 2004, I used to use one dial-up internet connection from a company. One of my friends wanted to use my internet connection service for a short period of time and I gave him my credentials. So, when he uses it, I can’t use it at the same time. One day, we saw, we both were online! How did happened using one connection? We logged in at the exact same time- concurrent user. 2 persons using one in connection at one price! We didn't do not intentionally, just happened! 

I tried to mention some example in different stages of software- starting from design to implementation phase of the software. From these examples we can see that human psychology and behavior is something that needed to be learned/ designed to handle during testing.   Now the question is how can we handle such human behavior in our test life cycle? I would recommend adding section in the test strategy that will handle the user behavior based on user’s demographics- age, knowledge, gender, country etc. Interview user, get to know about their expectation, observe users point of view for the software. Even though it will vary user to user, at least we would know that we covered this kind of user behavior in our testing.    

Without having test cases like this, we can’t be just relying on the word that we are 80% confident of our testing; testing is done 80% etc. I feel the need of this kind of testing be mentioned in the test plan. We can argue on the priority or the weight-age of this kind of bug/ issues later but it at least gives us more coverage of testing and chances to minimize the bug. 

I believe, our job is to find issues, it's the products stake holder's job whether to decide to fix these kinds of issues or to release it with known issues as it involves resources, time and obviously money!

Off track observation- I have a 2 years old son. I gave my old blackberry and android phone to him. The way he actually operates it, it’s like crazy combination/ sequence. Randomness is his specialty. I tried to follow him and sometimes I wonder, if I were the tester of that product, would I ever think about anything like this in this order?! That's why I mentioned; sometimes we should extend our thinking beyond normal user- go crazy! 

That will be all for today. Currently, I'm in the middle of testing and releasing my product and I should go back from my testing break.....  

I'll have another blog where I'll try to mention at what stage testing can be started in a Software Development Life Cycle.