Another motivating story- finding a bug that was crucial in time

Motivation is a driving force…

Motivation helps us to go forward…

……..

and I can go on and on like these quotes about motivation. The profession i’m on is one of the most less interesting job (to many people). They don’t find enough to interest to work in this line for a long period of time. For a short period of time, it can give more money than the other alternative jobs. So some people choose this line as a temporary job and then once they find their own line of job, they switch over to that line.

Motivating factors are different depending on each person's personality/ mentality. I was reading one article about motivation and found some interesting point about it. I do not know enough about the author Audrey Marlene, liked his point of view regarding this. She mentioned on her motivation article-

“Determine What Drives You - as humans are enticed by one or more of the following: Take a look and see which one(s) you can relate to. Ambition Money Independence- To feel in charge of your own life Security- job security, financial security, peace of mind Status/Power/Prestige Self-esteem/The good feeling you get after getting the task done Opportunity to improve, to grow and become more capable. Recognition and respect from others after a job well done. Making a difference/ the feeling you get inside You are competitive/ you have to feel that you are gaining ground and winning every time. ”. [1]

Like I said each person has different trigger, one or many of these above mentioned factors works. I know what makes me motivated and I can tell it just from her above points.

I can’t help sharing a real story about a bug finding that was crucial in terms of time and impression of the company to the (influential) client and also how I got my reward at the end.

Without mentioning anyone's name, let’s say some company sell different device/ equipment to their clients. After purchase, sales rep goes and does a in-service (describes how it works, what to do etc) session with the client. During the in-service session, sales rep struggled with few issues with the device. This device is driven by software. On the software, you select a patient, then select a protocol and run the test and the device starts running with “patients”. (Patients, not regular healthy human being. So it is pretty important to work safely and more importantly it should work as “expected”.)

So Sales rep selected a patient and then selected a protocol that was created and then running the test. During the test, sales rep observed that it is not working properly. So the sales rep places call with service department to provide an solution so that client can run the machine. It went back and forth with sales rep and service department and they could not find any solution. Service department could not reproduce the issue. After two weeks, with no solution, it was getting more and more frustrating to the sales rep and to the client.

So I got a mail, after two weeks of the incident, from service department that stating the issue and mentioned “Customer is getting quite frustrated with this ”.  The way it was reported by service department was not enough (and to some extent it was misleading).

Their complaint was :

1. “The torque is set for 300 ft-lbs for all sets (10 of them), At some point during testing, the system locks up.  At that point they look at the protocol and it has reset all the torques to 6.”

2. At some point of time, they see out if 10 sets, 4 of them retained the 300 ft-lbs values and rest of them changed to 6.
3. When they are in above situation (#2), system does not behave properly.  

So I started my usual methods of getting info about the issue they were getting.

1. Asked for the details step by step procedure how they created protocol (as they created it with modified values, not with default values)
2. Asked how did they run the test
3. Snapshots of each screen they were going it through
4. If possible video of the whole issue that was happening.

While waiting for the things that I asked, I went to service department to see how did they try to duplicate the issue. First step was to create a custom protocol with a 10 set, 300 ft-lbs torque value. They created the protocol for 10 set with 300 torque value and saved the protocol. Upon retrieval, it also shows the value. So we did not see anything obvious there. Then continued with the second step to run it. It was okay as well. So, it is true that service department could not duplicate the issue. I spent some time trying to duplicate the issue and could not figure out initially. Then to rule out whether it’s a machine dependent issue, I again asked them to create the same thing on another machine. On a second try,  they created the protocol by directly typing the 300 value to the drop down box and then saving it. While creating it, they entered all the value and then found it was supposed to be Exercise type and not test type. So then changed the type and noticed that all the torque value resets to defaults value “6” instead of “300”. Now that’s a good news to me that we at least seen the scenario what sales rep was seeing at client site. As sales rep will be going back to that site again, she wanted a solution before she can go back there. I’m under pressure to find the problem or provide them alternate solution. At that point I tried to give one alternate solution so that they can work with something. So I kept digging.

While doing it repeatedly (entering the value on the torque), at some point of time I exited out from application and came back to retrieve that protocol and found it defaults to value “6” instead of “300”. This is the second time it happened and two different way. Meanwhile I received more info that I requested for and it was more or less same what service department did. As I have seen the first complaint i tried to concentrate on that. Rest of the complaint was based on invalid state, so I gave less importance to them. I know the problem started from protocol screen so i tried few more combination on that screen. One time, when I selected the value of 300 from the drop down and saved the protocol for each of the 10 set and it Worked!

Finally, found the problem. Silly problem but it was crucial at that moment. Basically the problem was, as a drop down box, like all other drop down box of that screen, user supposed to select value from the list and not directly enter the value.  The reason they entered the value was the 300 value was at the bottom of few listed values. It would take some time to select from the end of the list and do it 10 times. User choose shortcut, application did not restrict it and did not support it. When user enter the value application does not recognize the value as listed value and thus defaults back to value “6”. This scenario also explains about their complain of #2.

What is the relation of this finding to the motivation? Well the answer come to the last part of the story. So, finally when I figured out the issue, I relay it to sales rep. Sales rep avoided that scenario and everything went smooth. It’s a minor bug with the software but as long as user does what they supposed to do, we have a interim solution. Client was happy with the solution and so does the sales rep (avoided embarrassing situation at the client site).

Sales rep was so happy with the effort (and kind enough) she wrote a mail to the company president about her experience at the client site and the way we provided her solution under stress. She explained the crucial moment “customer's confidence in us was shrinking! ” and then the way we gave us the solution, she was able to gain it back. She wrote, “The reason for my email is just to compliment everyone for their teamwork and undying effort to get this resolved.” “Upon leaving <Client site> today, their head trainer gave me a hug and has invited me back for further in-service training. I love it! This worked exactly as things should for the customer!  ”.  And then my president thanked me as well in a reply.

I could not help myself quoting from her mail. Those are the words that motivates me doing my work more and more. I’m a tester, do not provide a bug fix, but provides/ offers alternate solution to avoid the scenario and keep going for the time being. As long as I get inspiring words like this, these are my rewards that helps me doing what I do and what considered as boring job to many people. Money is not all we live for, as per Audrey Marlene's article,  I had “Self-esteem/The good feeling you get after getting the task done”, “Recognition and respect from others after a job well done”, “Making a difference/ the feeling you get inside”.

Sources:

[1]. Motivation by Audrey Marlene, MS

De-motivating factors in Software Testing

People does not like reading negative stuff by nature. I'm not that happy as well writing about it.  I'm not the expert guy, nor I have seen all sorts company within this related industry, still, while working in this profession, I came to list few things that actually work as negative input for software testing.

Little background of writing this. Ever since I started this blog, many people around the world tries to access my blog contents. Some tries to read it for pleasure, some tries to find standards/ templates and some tries to motivate themselves in this Software Testing. From my blog statistics, I can get few ideas about those people who are looking over here. Few of the search criteria, actually, made me to write this new topic. For example, one person searched with "I feel useless in Software Testing". I felt little upset/concerned  about this. Why anyone in this field would feel something like this. That means, we have some factors that led us to have feelings like this. If I'm not that capable person in software testing, I would have find some other job. When anyone is searching with a topic like that, that means s/he wants to be in Software Testing and want to be good about it. 

So, going back to the main topic....What is causing us to feel like that? Following would be some of many factors that I was able to identify.

1. Person's Mentality- THIS I felt to be the first most factor for de-motivation. Not all people actually come from same background, nor mentality to accept negative things about themselves/ their work. We software tester are, by profession, bound to find negative things about the software. People who are building software, managing the software and testing the software must all have the mentality to accept negative comments about their creation ("Baby"). Developers must know that we will be working to find issues and not necessarily we will find everything. When we find something, trivial or critical, they should be honoring our task. Software tester works without the inner knowledge of the software. So, instead of criticizing them for not being able to find bigger issue than the one they reported, honor them by fixing their reported issues (as assigned/ prioritized by Manager). We work with the highest risk. We double check Developers creation. So they have a bit relaxed position that someone else will be picking up their mistakes.  QA person does not have that. Whatever he verifies goes to user and if anything happens in the field, QA will have to take the heat. 

QA person should also be sense-able about their task. They will have to use their brain to make things easier for the person who will be fixing this. 

2. Process- Not every company has a structural process that follows the standard/ steps of SDLC. What method of SDLC used entirely depend on the company. At least some process of software development has to be followed. Company who produces software only, may have more proven structured process than the one who uses firmware on devices or a bank who developed/ maintain their banking software. I have mixed experience working on two types of those companies. I have seen a common pattern. Process is always hard to follow. Process make more side work and it may be less flexible. If we all follow the process, there will be less miss-communication. 

My experience tells me that people involved in this process, sometimes tends to work outside the process. And when people see that management is not that strict about it, it becomes more of a practice. For example, I have been assigned many times to test software and not to record them as official process (no entry in bug tracking system, verbal communication, email, word doc list etc.). Why I have been assigned like that in the first place is a different question. My point is, my management allowed developer to have that out of system work. For the sake of the product deadline, I did not argue about this. 

Another experience is, I tested one product, entered all the bugs in bug tracking. Seat together with management and developers. Assigned and Prioritized the bugs. Everything was looking promising. Then after a while, due to shortage of man power, priority of bug fixing became lowest and rhythm is gone. Personally, I like to pursue the bugs that I reported (no matter whether anyone likes it or not). I reported bugs and I want it to be fixed or at least my indication that someone will work on it.  

Not all the companies have complete specs to begin with, which actually leads to all miss-communication. If the gray area is not defined properly, then it makes some useless communication whether identify the reported issue as Bug or Issue or Enhancement. Think out all the possible scenarios or at least when testing team comes with those scenarios make a decision with involving developers. 

3. Environment - people, mentality, process, top management and other relates things makes the environment. When the environment does not understand the significance of testing, it starts to create all sorts of problem. Some examples below:

A quote from management "What is the status of the software?" asked in a meeting. Answer was given discussing the status with developer. Development is near to complete. So the top management getting the idea its complete and they were discussing the release/ marketing plan. I felt like I do not exists! Nobody cares whether that developed software is usable or not? Whether its tested or not! That's what I have seen multiple times in multiple companies.

Another example, another quote from management "Do not worry. You don't have test it to death. Nobody will be killed with this software". I do not test it just the shake of testing. I know and understand deadlines. I test the software so that people has less chance to get hurt. People may not get killed but they can be hurt by the software and those people even may sue the company for that. So, it is DAMMMN important to test the software even no one will be killed. 

4. Industry  - 20% less salary. and Its project basis mostly (3/6/12 months). 

I have seen companies who does not prefer to hire full time tester as according to them we don't have work all the time through out the entire SDLC. Philosophy is when you have some software build, then tester can work. Else he will just be a idle resource. Hence they don't need a full timer.

At the very end, it's more about  money than anything else. I'm sorry to say it this way, but that's the truth. Having the same background of education and experience in software industry, just for the demanded role, other people (especially in development) gets more importance, security and money. 

As a tester, when I have to give same effort and time, if I have to think about other things - job tenure/ security it automatically diverts myself from concentrating on testing. I'm not saying other people does not have to got through this, they does; it happens more frequent to us than others.   

I have not worked for Google or Microsoft or Facebook etc big companies. I do not know how they work. But I'm quite sure that no one from those companies will every search with "I feel useless in QA". Its the small to medium sizes, where most of us work and gets the feeling like this. I would not say i have seen many/ everything... these are some of my experience in this field. 


Okay, too much of negative words. At the very end, I would like to share one story ....
There was a company who build different devices, and usages software/firmware to run those equipment. It was a small company. Did not have that many employees. They had a development team working on.  Some people were testing their product as an ad-hoc basis. After a while they decided to hire a "Tester" as a contractor to see how it goes for 3 months as an experiment. They interviewed and hired a person. That person came from a different part of the world. That tester proved himself and company decided to hire him full time as "Test Engineer" before his contract end. Then after 3 years of that he was promoted to "Sr. Test Engineer". The job that started with a 3 months one did not end up there, it continued. 

Don't feel bad/ down, if you can prove yourself, you will be rewarded. 

Self Motivation in Software Testing

Self-motivation is a key factor of quality the Software Testing. Software testing is monotonous, working with same interface / screen over and over could be a boring at times. One will have to find out a way to avoid that scenario where they will feel bore about it and that would be the end of testing. Once  bored, one will find fewer issues about the software and the quality of the software will be compromised. There are many outside issues/ problems that can cause the frustration for the person who will be testing the software, but just being with one software/ product for a longer period of time, can affect largely finding software issues. So, we will have to find/ apply different techniques (no hard and fast rules) to keep ourselves motivated. Following  I’ll be trying to share my experience which worked for me in many projects to keep myself motivated in software testing. 

Testing Approach- Any software that I’ll be testing, prior to look into the software, I usually try to learn/study the details of the software/ product. Even if it’s written on the Specs, I try to get info from different related people through questionnaires. This gives me broader idea of what was supposed to be built, what was designed, and how it supposed to behave. Then I can double check my testing procedure (Test Plan, Test cases) whether I planned to cover most scenarios. 

Perform a Smoke Testing on the software before starting the testing according to the test cases.  Smoke testing actually gives the idea whether this software is ready for testing or not. If I find issues that Software does not run or crashes now and then, I ask for a rebuild of the project and then a newer version. This saves my time and effort to start the formal lengthy testing procedure. 

Prior starting the formal testing procedure, I start with light things like the User Interface stuff- alignment issues, correct text, correct icons on different states, and clean UI etc. No matter how silly, non-functional, less important issue it would be, User Interface is the first thing that User will look at. That’s the first impression user will get about the software. So, it’s important to have a good, clean, simple, understandable UI and my job is to make sure of it. When I find issues, I write them down (leaving the priority of the issue as blank). 

The next thing I do is another easy task, field validation-valid, invalid inputs, min-max checking of each field, tab orders(if applicable), size etc. Whatever Issue I find, I write them down. Most of the time, these issues gets the low/ lowest priority. 

At this point, if I have a good understanding & access to the developer, I can send them the issues so far I have found. If agreed, they can fix the low priority issues while I will be concentrating for main/ major issue findings. From my experience what I have seen is, at the initial stage after releasing the software for testing, developers have some time that they can use to look into low priority/ UI issues that they will not have the time later when I start to find the major issues. Finding major issues takes time. So for an optimum use of resources, I try to keep the developers busy (if agreed among all concern parties) fix those issues. If not, then I start recording the issues, prioritize them consulting with concerned parties.  

While I have few issues on board, I then start to follow the formal steps.  This may look like informal testing procedure, may go beyond my teams boundaries, but I have found it useful in-terms of optimize use of resources. For the formal steps, it’s quite common among the software industries. So I’m going to describe them here. 

One Important thing I have learned from my experience in software testing, as team member and/or as team lead is....

At no point of time, one should wait for another person to finish some task. For example, testing should not wait for the development to finish. Development and Testing should go side by side. This sometimes becomes a demotivated factor for the Testing Team members. They should be engaged in testing through out the whole SDLC. 

Some Suggestions -  During this process, like any other human being, I also get bored about the software after a while. I try to take different measure to keep myself motivated during this testing process. Few things I do (works for me, may not work for others) and suggest are:- 

• Primary motivation comes from finding and reporting Issues. When I starts to report issues, in those easy areas (mentioned before), the number it-self motivates me. When I see I have like 20 issues already on board, it gives a good feelings and I get motivated enough to increase the number by finding more issues.  I use some numbers as Milestone - 25, 50, 100 number for reporting issues. The more I find issues, the more the number gets increased and I feel like i'm on right track. (I usually try to report quality issues, not just issues to reach milestones). 

• I frequently take breaks out of this testing procedure. Looking at the same screen time after time makes me more familiar with the screen. Once I get too familiarized, I started to feel that everything is as it should be. So, when I take a break, when I come back I can get a fresher look at the screen and sometimes I find issues from that. Btw, do not lose your focus/ concentration with the break.

• When I feel like I’m running out of testing idea’s I go back to my drawing board and review all the things I was supposed to do and what I have done so far. Sometimes I find idea that I didn’t planned for before using the software. I update my document as well when I find things like this. 

• Another thing I do when I run out of ideas, I intentionally plan to talk with developers to get more inner knowledge of the software. Black box testing is basis on without knowing internal knowledge. When I talk with them, little chitchat about how did they develop one particular features, or what conditions they used to plot a graph line etc. helps me to think of different testing ideas. And when there is broader area, more chances to find issues and number will increase. 

For example, software supposed to plot a line graph based on a test result and date. On a date, a test can be performed, or can skipped but recorded. If there are test results for two dates, the line goes straight on the graph. If more than two and test performed on each day, then it connect all three points using curve line. If first day has a skip, then for next two day results will be a straight line, and there will not be any line drawn for first two points. If there is more than 10 points for multiple days, then logic is different then others. In short, there were different logic used for different case, single point, multiple point - 2, 3, 10, 11, 12 etc data set. Before knowing these logic, I was just testing it for few test data. When I came to know these logic, I started to layout different technique to test these scenarios.  

With the ideal world, these things can be handled or tested on the white box testing level, but as an end user tester, I don’t believe what other did. I don’t assume even that this was tested at unit level. Caution: do not get biased developer.

• Sometimes, when I feel that something is not right, may be it's according to the specs, I usually report it as Enhancement/ Observation just to avoid any further objection. And I support my findings with logical arguments from a user point of view. Enhancement is category where I actually can contribute my idea for the improvement of the product. Observation is something that I noticed, may be its working as designed, but something to discuss it further. I may not be the person who will take the final decision about the changes, at least, contributing ideas as Enhancement / Observation would definitely feel happy!

Feel free to test anything and everything. Do not care the limit, do not care what you were not supposed to do it or not. Free your mind. Play the role of different level of a user. Find the things that you feel it can motivate you for testing. Use those techniques.

Automating our testing is okay, helps us not to get monotonous/ bored. It’s the human being that designs the automated testing script. If you feel bored, it will affect you writing and updating scripts. Scripts can only do what you think about. If the thoughts are bounded by the tiredness, then it won’t help whether we use automated testing or manual testing.

Lastly, to keep yourself self motivated in software testing, don't just test a software that is given to you, feel the ownership of the software. Have fun in Testing!

Motivation for Software Testing

Motivation- a text book definition- is a psychological feature that arouses an organism to act towards a desired goal and elicits, controls, and sustains certain goal-directed behaviors. It can be considered a driving force; a psychological one that compels or reinforces an action toward a desired goal.

Software Tester/ Software Quality Assurance Engineer/ Firmware Test Engineer etc all these are not that easy as it sounds. Software Testing can be a boring, repetitive task. Many people may not find it interesting enough to continue their career in this field. Many people would think it as 2nd level job (in terms of opportunity and ~20%-30% less payment from others) in software industry. A job that usually comes in shorter months (3 months, 6 months+ etc). It’s very hard to motivate him-selves in job like this and produce quality work. Testers who are motivated are more likely to strive for higher quality outcomes, achieve objectives and work in a more efficient manner and vice versa.

Think about a simple situation- you are working with a device, for example medical device. Medical devices are critical. They are used on patients, not on ordinary people. When you engaged with such job, and you have tested the firmware/ software of that machine, when it comes to the critical time, you know that machine will work; you know you tested it well. It can even save a life and opposite can happen if you are not self-motivated enough, not being able to understand the importance of your job. 

Or think about another scenario- in August 2005, Malaysia Airlines Jetliner, Flight between Perth, Australia and Kualalampur, Malaysia zoomed 3,000 feet upwards. A defective software program had provided incorrect data about the aircraft’s speed and acceleration, confusing flight computers. For a software bug, think about how many people’s life was at risk!

I have chosen to be a software/ firmware test engineer. To me it’s a challenging one. Every time a code is released for testing from the developers, we get a chance to break it, tear it down into pieces. To me it’s interesting, fun and I enjoy doing testing. It does always make feel good to developer's reaction after finding something; developer never thought about it- that's my primary motivation.

Real life experience
I would like to share one experience of mine how I tried to motivate the QA team, i.e. the whole software development team 

Around 2003, I joined as a member of the QA team which had 4 members. There were 15 developers and 4 testing team members. We had a QA team leader who was an expert of databases. They had all the procedure and tools for the testing team in places. From screening codes at the very early stage of software to recording bugs in Bugzilla. What I had experienced during my tenure was, somehow I felt that testing team member were not motivated enough to proactively do something. I felt like, they treat themselves as weaker than the development team.

So, based on that feeling, even though I was just a member of the testing team, I felt like I had to do something in this regard. So, I initiated few internal meetings with our QA team about the existing procedure/ bugs that we had. After that meeting I found the weaker point. QA team was lagging of one main thing – “Quality”. I looked at few bugs and realized why developers used to treat them so lightly. The quality of the bug reporting was poor, not enough description, not enough logic to support that it was a valid bug. Then decided to shared my experience with them, how to write a good bug and standardize the procedure. I noticed that, developers started to like it. Because it helped them, it helped them to duplicate the issue which saved lot more time.

Observing the good response from the development team, I looked the issue to motivate the QA team to keep them reporting the quality bugs. I started another approach for the QA team and later with the development team. That was something that I started to use 10 years back from now, nothing unique about it. It’s the "Rewards" system. Here, I will try to explain how on each phase Reward system how it worked and not any specific score card system.

I started to publish the statistics of QA team in terms of reporting bugs on the notice board, like the top ten chart. That was intended to motivate the QA team and it did! Not to praise myself, but I was top on that chart :). It actually started to work, when developers acknowledged my bugs. My other QA team members actually started to post more and more bugs after that. It became a sort of competition and following weeks numbers were different then the first one.

Reporting more bugs it-self does not generate good reputation for the testing team. As they were reporting issues, if they did not report legitimate bug, it was getting negative response from developers. So, to balance that issue, I put some weight-age on the bugs. What I did was, for each good bug of P1, which was accepted by the developer/lead, gets 10 points. Then for P2 accepted bugs, 8 points, P3 -6 points, P4 -4 points, P5- 2 points. To ensure that QA team would not report everything for higher score, I had used the negative marking as well. For each bug that gets rejected by developer reduced the -5, -4, -3, -2, -1 points. At the end we had a sum of all the numbers. Based on the number it was ranked. The good thing about negative marking was, when a QA team person reported 10 point bugs, when it gets rejected by the developer, he lost points. So they were careful about reporting it. That's one part of the solution- motivated the QA team.

Then comes the other part. As of then I had more bugs on board, developers need to fix them. It was outside of my scope. So approached to our Project/ Engineering Manager- who manages the whole Software Development team. With his permission, then started to introduce another section on my chart for the development team regarding the bug found/fixes. Followed the same positive and negative point system for them as well. Ranking of developers in terms of most issued bugs. When I published that, now developers were more serious about getting those issues fixed. So it helped in both ways, to report bugs and to had them fixed. After a while when I published the score card again, I had seen major changes on the chart.

It worked! Simple old fashion Reward system worked. It was a whole team effort to make that system actively working. Reward technique for this project worked well. But it may not work well with other project/ infrastructure. We can introduce different techniques like this which will be suitable for the project/ infrastructure. Motivation differ from person to person. Money may be the prime driving factor, but something like this small reward system can also motivate some group of people.

Few common barriers of Software testing can be easily handled in a different way. Software testing is not a repetitive job anymore. We can minimize it by engaging the automated testing. As a testing team member/ manager, we should establish/ follow a process of systematic testing, setting the goals for the testing team members, making sure they are clear about their goals and boundaries. Managers can provide training for the test team members if necessary, can have few motivation sessions with test team members. When necessary, don't feel shy to play the leadership role. Develop a good relationship with development team. That can help you to understand any issue with more detailed information (DO NOT get Biased though- it's important).

Lastly I would like to say, we are test engineers. We test things. Developers have an option to fail and we will catch it. But, if we fail, there will be no 2nd level. It will directly go to the end users. And the cost is real high when it fails on Releases.

Be a passionate test engineer!!!