Shopping mall, EAS system and my eBay app

Do you know what is a RF EAS system works or how it works? I don’t. All I can tell you what is stands for- Electronic Article Surveillance  that uses Radio-frequency. Still not sure what that thing is? :) It’s the thing that you may have seen at the checkout point/ exit point of a shopping mall. Those  big bars that detect whether anybody is passing through without paying for the product. Used as Anti-Shoplifting device. May be the following picture would help visualizing what i’m talking about.

Picture Courtesy: checkpointsystems.com

If you are interested how RF EAS system works- read this article from howthingsworks.com. They have explained the system including different variation of the system.

Ok, now we get a visual idea what i’m talking about. Why i’m talking about it? for that i need to share a story with you all.

I was in a shopping mall, walking around, passing the time while my better half was shopping. I was on my phone with ebay app. Browsing for some cell phone, using the buy it now option. I was walking around, going in and out, looking at the search results on ebay app. You people have seen people texting and walking and fall into fountain or in awkward situations. I was kind of like that. I was paying more attention to my ebay and less where i’m heading to. I nearly bumped into one of those EAS RF bars. I stopped. Still was looking at the screen. Then I saw the most interesting thing. All on a sudden something as added to my cart. I didn't even touch anything. I did not do anything and it was adding/ creating touch effect on my ebay app and adding item to my ebay cart! And, it did not stop there, it even tried to buy the thing. I was so lucky that I did not have the payment method setup on my cell phone. That saved me buying a $250 worth of cell phone that I did not wanted/ submitted.

First I would like to rule out the possible hacking issue. I was not connected to shopping mall wifi and my NFC settings was off. What actually happened, I don't know. All I know, when I was close to one of those bar (possibly the transmitter), it sent some electromagnetic wave (which I was kind of feeling it when I was moving the phone around the bar). I was near to the top portion of the bar. It did not happen at the lower level.

When it happened first time, I did not believe it. Then I tried few other times, it did the same thing. May be it is nothing abnormal to people who has enough knowledge about the whole system, to me, as a layman, all I saw it adding stuff to my cart and tried to buy it. It could be related to cell phone having the capacitive touch screen and that electromagnetic field generated the pulse to mimic the touch behavior.  

I tried few other thing, tried from the lock screen- nothing happened, tried from the home screen- nothing, went to app menu screen and then it started the facebook app. Could have tried few more things and was trying to take a video of this, ended up being questioned by the security guards.

I was wondering about whole thing. How would I ever test scenarios that may happen in real world. There are n number of user scenarios in real life. All I could do think about some of them and test my product. Testing is my passion, so I try to think and share about the events in my QA professional life. I dont want to be just a Tester but a real life Tester.

QA person's feelings

Being a human being, it is very difficult to find or to have a mind set for finding issues out of anything in  life – both in professional and private life. I’m a human, not a bot. It easy to say put the QA hat when at profession and play different role when necessary. It may be easy to do it when you are doing positive work- development, business analyst, project management, test management. But when you are in the field, doing the actual work in QA, then it becomes critical. It needs different motivation to keep you smart enough to find issues.

 

I developed myself in a way that, whenever I see a product I start doing testing or think about all possible negative scenarios that can break the software. It just now happens automatically. Even when they say a user story, on the fly I start thinking about all possible scenarios. And most of the I found that, in the requirement or design level, not all possible scenarios are considered and sometimes when I report this kind of issue, it get stuck in the “it’s not in specs” term.

When I find issues, it excites me. Excites me to talk and report about it. It takes me one step ahead to reach towards my milestone for bugs. Nobody else will understand or care about how a tester feels about finding a bug, going forward towards the milestone. Think about it in real life, if you meet someone and noticed something, if you say that to that person, chances are s/he will react (by human nature). Nobody wants to hear bad things about them. Similarly when you are saying (reporting) something to developer about their child (code), s/he will react just like real life. So, who will then appreciate the work you are doing?

Honestly, I don't know. I got mixed appreciation from different level of people as well as I got opposite from the people I work with. When things turned down, like you brought up an issue, and they pushed it way down in the list, it actually demotivates me to bring up similar issue next time. So what I do and depend is on myself. I motivate myself to find issues, to make the goals for bugs.

Sometimes I get some feedback from the top management. To the top management, if it does not hurt anyone, then don’t hang up with testing, making the software perfect. If this is the mind set of top management, if this is how they think about the user, chances are they will earn very bad reputation in the industry once something happen. They see the testing as the most negligible piece in the software development.

As a tester, I never make any decision whether to release a software or not (especially working to commercial bank and/ or medical device company). I present them the facts, known issues for the stable testable product. Depending on many factors, it’s the top management’s decision whether to release the new product following a patch or not to care what happens afterwards as long as they can capture the market.

May not be an apple to apple comparison, but I would like to say something about the cellphone industry. Apple and Samsung- two rival company. One sues another to dominate the market. Samsung brought the smart watch product September 4 2013. They even brought another version in April 2014 [Source: Wikipedia]. And apple, introduced their watch on September 2014. Near about one year later than Samsung. Apple waited this long, to make it perfect not just to enter the market with new idea. I do not have the factual data with me, but I can guess not that many people bought the Samsung watch. And if Samsung's watch was that good than they did not have to introduce another watch within 6/7 months later. Apple’s products are much more dependable, less breakable than the rival party. So it’s not always about whether you can capture the market, you being the first to introduce something to the market. Ask any user, why they buy apple (other than apple’s crazy fans). I believe it would be the quality of Apple’s product.  

More or less, in general, we can guess the quality when we see the word "China" on a product. No offense, its not like they don't make great product, they does. I was just talking about general products. Think about a blender machine. China made blender are cheap in price. We can buy near about two China blender comparing to a known brand product. If people don't care about quality, they will end up buying blender for every 4 months rather buying one good quality one. Eventually it cost more and you cannot depend on it. Those who makes the comment "As long as it does not hurt people, we can fix it later" should be aware about this common sense example. QA helps to be proactive rather than reactive.

Validation, Verification and Testing

Validation, Verification and Testing - these are different terms that defines different meanings.

Testing desktop/ web application versus testing software/firmware on medical device, I get to follow different methods/ procedure. Even though in broader sense these are included in Software Testing Life Cycle (STLC), but may vary with terms and terminology.  For Medical Device testing, I usually follow Verification, Testing and then Validation procedure. There is no hard and fast rule, but it seems fairly reasonable depending on the SDLC, environment and the product i'm working on. 

Following I'll be trying to give description about those terms.

Verification is a process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. It is concerned about whether software is error-free, well designed, documented, etc. It often does not require executing the code. Rather it involves human based checking of documents and files.  It is not about finding issues with the software.  It popularly defined as "Are we building the system right?".

During our software development, we verify whether the requirement analysis was done correctly, whether the design we are going to implement serve the purpose of the software etc. Verification includes, Code inspection, consistency checking, static analysis. Verification do include testing but also includes reviews and other activities that have nothing to do with testing.

Testing, according to David Vogel, "it is one of several verification activities, intended to confirm that software development output meets its input requirement. " Testing does not cover design issues, requirement issues, documentation issues and many other like this. It is more functional oriented related to the software. Testing mostly done on specific method/ features/ usecases. During the testing, we do not look into the other issues that we do in verification stage.

Validation is process of checking whether it satisfies specified requirement. It can be done during or at the end of the software development process. It involves executing the codes. It can be also defined as "Are we building the right system?". Assuming that software specs were correct, validation ensures product actually meets users's requirements.

Validation is like a big umbrella. Underneath it includes Planning, Evaluation, Risk Analysis, Testing. So, it is involved through out the whole process of software life cycle. It's the complete coverage. [1]

Validation objective is to discover defects in a system  and assess whether or not the system is useful and usable in  operational situation. Validation should establishes  confidence that the software is fit for its purpose and does  what the user really requires.[3]

Relationship- Validation, verification and testing

I could not resist myself quoting from the book that just explains these things so perfectly.  As David A. Vogel described in his book with a Venn Diagram (which he explained in his book), 

"Validation is represented by outer ring of the diagram and wholly includes the verification and testing activities and yet there are validation activities that are neither verification nor testing activities. Verification activities are those which verify that individual design inputs have been properly addressed at each phase of the life cycle. Verification includes some testing. Testing verified that individual requirements have been implemented correctly." [2]

Figure: Venn Diagram for V&V and testing relationship

He also gave an visual representation for the software validation coverage as follows:

Figure: software validation coverage (Photo credit- David  A Vogel)

V&V are different, but we should be using them as combine tools through out the SDLC. According to Dolores R. Wallace and Roger U. Fujii, ”Software V&V is a systems engineering discipline which evaluates the software in a systems context, relative to all system elements of hardware, users, and other software." [1]

Sources:
For more details, anyone is interested, then follow my two sources. Buy the book of David Vogel, you will get more explanation and related topics and references. 

[1] The difference between Verification and Validation
[2] Medical Device Software Verification, Validation, and Compliance, by David A Vogel 
[3] I. Sommerville, , Software engineering, Addison – Wesley, eighth Edition, 2007.

Unannounced changes in code impacts testing

One of the common troublesome thing that I found in my career of software testing is how to handle the changes / change impacts. Incremental testing is a common practice now-a-days (I believe). When we have a build on our hand from developers, we do the testing on it, find issues and report it back to developer. Developer fixes the issue. Now, during this procedure, we expect that developer will not break anything that was working in previous build. Yeah right! We can only EXPECT (no offense to developer). As a human being it’s very much natural to break something while fixing it and that’s why we testers are their fall back that we will check all possibilities. Being human, we also inherit the same nature – may slip to find the issue. So what can be done in this regard? 

Answer would be simple- engage automated testing. Okay, I would accept that. Now can anyone tell me what are the areas/ or which test cases I would run? I would certainly take any answer, other than “Run all test cases” answer. Reason is simple; it may not be reasonable/ suitable/ applicable. 

Let’s try to understand it with a hypothetical example- simple deposit transaction. On a deposit transaction, we deposit the money to a particular bank account. For that we do some checking on the account- account number, name, any other info, any restrictions etc. After that, transaction gets posted to the account. So we tested the possible scenarios of a deposit account. Most of the thing was working fine, except it was not allowing editing the date of birth. So, on a next build, we expect the fix would come around that area. Straight and simple. We tested that particular area and found okay. Later, when we released the build found that it was taking awfully longer time to perform a deposit transaction. So what has happened here? Tester found the issue, developer fixed it, tester verified that particular fix. Tester does not know anything about the code. Does not have access to the code or does not even understand the code. At the end, the client is suffering from a bad build. 

Hmmm. Time to investigate. Backtrack the issue- tester tested that particular issue. Developer fixed that particular issue as well. When developer was in that class/method of fixing the issue, he accidentally commented out one line of code that was necessary / would increase the performance or some code that he felt unnecessary and removed the code (hypothetical scenario).  Tester was not aware of this particular incident.  If we would, we would have been running the performance test cases as well.  

Let’s see, now, we have automated test cases ready for regression testing. If we were to run the automated test cases for such incremental build, would I run the whole process, or just the portion of the deposit and related test cases.  Let’s say run most of the known related test cases. If we do that, at the end the software actually would give us test result as “Pass”. But would miss the point- transaction taking longer time.

The point is, we need to establish a process of clear communication among the testing team and development team about the change/touched areas. My personal opinion to make a matrix that help us to know about the change impacts. 

I would like to propose a matrix with few queries like: 

1. Functionality - What are the possible functionality that can have the impact for the change?
2. Performance -  Any chance to effect the performance of the existing operation?
3. Code Clean up - Does this build contains any cleanup by the developer? 
4. Optimization- Was this done to related to any fix or just a generic code optimization?
5. Bug fix - Was this build was specific to a isolated bug fix?
6. Enhancement - Was there any enhancement done with this build?
7. Common Utility function - Was there any change in the Utility function that was used  as common 
     function?

When developer make a change s/he should mark those parameters that is applicable for that change. May be putting some comments would give testers an idea to narrow down his possible testing areas. This will be the additional information for the tester. Based on the matrix tester then can decide what test cases to run or not. 

I was doing it verbally as I have the setup/ environment like that. Every time I get a new build, after performing all my regular testing, I consult with developer, ask him about the changes he made. I do it nicely so that he shares the information with me without getting offended. Then I check the code from the version control, tries to match up his statement. With my limited knowledge  I try to understand the changes in the code and then decide whether I should run anything more or not. I have been doing this for a long time and it saved my several times. 

This is not a new concept for the software development/testing world. Shared the experiences  that I have been facing for long time. Some companies (for example, Parasoft,  Microsoft Test Manager, Vector Software) already made software/solution that can keep track of this kind of issues. Even though, i'm not sure whether i'll get the answer's about the above mentioned parameters that I suggested for matrix. As long as we maintain a system/ structure that is the main goal. 

It's always better to work as team- development team, testing team. Goal is to Avoid miss communication and perform the task effectively. 

Story - Touchscreen display testing - One interesting finding

Testing software on PC is what most of the people are doing. I have the opportunity to test the program on device level. And the experience with the device level firmware/ software are different from desktop/ web application. Sometimes it's more exiting. Today I would like to share one experience with one device that is directly not related to software/firmware but the implications of it. 

We were working on a treadmill product that has the functionality of running the mill speed, incline / decline the mill position (it has few more advance features, but relative to this story). It is a software that is running on the touchscreen display that has windows 7 embedded operating system. All the buttons and everything is on the screen and it responds to user's touch. We were in the process of choosing touchscreen display that suits on our treadmill. So different people were doing looking/ testing from different angle. Industrial designer more emphasizing on the look and feel, whether it meets industry standard etc. Mechanical designer looking the display whether that will mount on the treadmill perfectly or not. Whether the mount will hold the display on the treadmill when people will be jumping/running on it fast etc. I'm not an expert on those fields so, i'm skipping the details of their process. 

Now for my part. First thing I had  to check whether it meets the optimum configuration with the hardware. Once the hardware meet that requirement, then checked the basic functionality. Then then the important thing is the Performance/Response. Think about it, its a treadmill firm/software. Speed button touch will increase/decrease the speed, stop button touch will stop the mill and so on. For extreme circumstances, when you hit the STOP button, you expect the mill will stop. Similarly when you touch decrease button it will reduce the speed. These functionalities sounds very basic, but if it is used by some patients for Rehab or elderly person, it may become a BIG issue. It's like a car, no matter what situation you would expect that break will stop the car. If the response of the touch is too low or too sensitive then we will have problems as well. 

Touch screen displays are usually of two kinds- Resistive and Capacitive.

"Resistive touchscreen comprises of several layers, out of which the flexible plastic and glass layers are two important electrically resistive layers. The front surface of resistive touchscreen panel is a scratch-resistant plastic with coating of a conductive material (mostly Indium Tin Oxide, ITO), printed underside. The second important layer is either made of glass or hard plastic and is also coated with ITO. Both the layers face each other and are separated with a thin gap in between. An electrical resistance is created between both the layers in such a way that charge runs from top to bottom in one layer and side-to-side in another." [1]

"Capacitive touchscreen also consists of two spaced layers of glass, which are coated with conductor such as Indium Tin Oxide (ITO). Human body is an electrical charge conductor. When a finger touches the glass of the capacitive surface, it changes the local electrostatic field. The system continuously monitors the movement of each tiny capacitor to find out the exact area where the finger had touched the screen." [1]

Both has advantages and disadvantages. Capacitive is highly touch sensitive and doesn't need a stylus and supports multi-touch. Resistive is high resistance to dust and water, low cost. Which type of touch screen to choose? It depends on where, how its going to be used, what are the purposes etc. 

For our case, as we were going to use it on rehab treadmill. As  test basis we used the Capacitive touch screen which has lot more advantages. Capacitive touch screen looked and felt good in terms of look and feel. But it failed in one interesting area - sweat! (Basic science - Salt water is a good conductive media.). 

We put some salt water on the display where we have buttons layed out. We saw that it was generating a touch on the button. And the worst case we saw it, when we put the salt water on the speed button, it was ramping up to the highest speed within few seconds. Imagine what would it do to the person on the mill, and what if it was a patient who will be doing a rehab. So, there ends the story of beautiful capacitive display for treadmill (until they fixed this issue). On the other hand despite of other limitations of Resistive touchscreen display, we are going back to resistive one.  

My intention was not to discuss about how to test touch screen display in general. Rather its about how to test products in practical way that actually can happen to end users based on the nature of the product. As it was going to be used on a product that will be used by patient,  patients safety comes first before starting any other type testing. Testing is very important. Testing from practical scenario is very important. It may save lives.

Source: [1] Resistive vs Capacitive Touchscreen

Tips for writing effective Test Cases

Text book definition of Test Case from IEEE- “Documentation specifying inputs, predicted results, and a set of execution conditions for a test item.” [3]

My understanding about the Test Case is, a set of instructions to perform test on software. In test cases, we write the instructions how to perform a test, what to look for, how/what to expect as result depending on different combination of input. In a Test Case we breakdown the software functions into small units, that can be tested with different input, and compare the produced output with the expected output[3]. In short Test Case Document is about checking with 

“Known Input, Expected Output (according to conditions)”. 

It is the guide for the test engineers who can perform the test just following the instructions. It’s not a rigid document. It’s open subject to changes on software version/ features/ product/ project. Very few things can be as highly important as this Test Case documentation in our Software Development Life Cycle / Software Test Life Cycle. Writing a good Test Case is really important than just writing the Test Cases. This is the document of all the test ideas, scenarios. 

In this write up I'll be trying to point out things that will help to improve writing Test Cases. It's not about any Template or format of Test Case. There are few article that I found very appropriate for this topic. If interested you can read their article as well, all mentioned in the sources.

1. Enthusiastic Mind Set [1]- Before starting to write Test Cases, you will have to have a good enthusiastic mind set. Testing should not be considered as lower level job then the development. Might wonder what would be the relationship between the mind set and Test Case writing? I felt it from my experience, it plays vital role writing test cases. It will open up the boundaries’ that usually we the testers get from different level/ circumstances. It will let you open up your mind to think about "unrealistic" scenarios. When we report those unrealistic scenarios, usually we get demotivating reactions from the concerned people, but when user reports the same we get the heat. So, to keep yourself motivated in testing, mind set, what I felt, is the first thing to consider. 

Clinton Staley [1], PhD, Professor of University of California, wrote in one of his writing about this- 

“Good testers see testing as an art, and they’re right.  They’re very enthusiastic about finding bugs.  An SQA group with good morale will be found on lunch breaks bragging about the clever way they made an application fail this morning and how upset the developers were over the rough treatment given to their baby.”

He also mentioned,

“Good SQA people like breaking code.   They’re creative, even devious, in finding ways to do so.  And that’s good, because the kind of bugs that make it into the testing phase of software development are devious and deep, and they take organization and focused creativity to uncover.” 

2. Study about the Project/ Product's. Gather knowledge about your target software. Study the functional specs, Use cases and Business document. It's equally important  to know the business application of the software. Feel free to Raise the Questions about the functional specs. From my experience what I have seen is, most of the time specs does not cover all the possible scenarios and thus lives some gray area which leads to some unpleasant conversation with developers/ leads.  So, clear it up first. 

3. Organize your Test Case. Like any other document, you need to plan how you will organize your test plan. It can be done Following a Sequence on screen, could be done with some simple part, could be done by functionality. Whatever method you follow it should be easy to follow/ relate with the screen/ functional steps. Order the functionalists, write the test case accordingly. Better not to mix up functional test cases with non-functional test cases.

4. Elaborate the Test Activities. Instead of writing some brief instruction, try to elaborate the steps. For example, "Check the Name field, should work accordingly" does not give clear idea what to do with it. If we can guide it with what test to perform, what test data set to use, would help and save time. Mention methods that you going to perform, like, equivalence classes portioning, boundary value analysis, normal and abnormal conditions etc [3]. 

5. Modularize Test Cases. Instead of repeating the same task several times in different scenarios, modulerize the test cases, i.e. create common Test Cases that can be used as a sub test case in different places. Just like the methods/ functions that developers write to organize their code. If I need to test some common field, I usually create a common test case that will use different data sets based on the type of the field. For example, if I need to test a Name field which is a Text field type, I create a common Test case that will cover different combination on that field - Max/ Min length checking, blank, LTrim, RTrim, CAP sensitivity, valid input, invalid input set etc. 

6. Preparing Test Data [1], [2]. As a part of writing test case, when you are thinking of certain combination, you should start creating the test data set for that scenario and link the test data with the test case/ test case steps. This saves a big time and effort later.

7. Avoid any Assumption. Something that I learned and I give importance to it is not to Assume anything while writing the Test Case. Like I said before, if anything is not clear, raise flag, ask question, know it well. Write the test cases referring to Specs. If circumstances does not favor at all, then at least, mention the limitation of the test cases in clear so that anyone can understand. For example, usually now a days, storage capacity is not a issue for devices/ software applications. We should not just assume it that application will never run of storage space. We should have a Test Case that would check such scenarios no matter whether it happens in real world or not. 

8. Write the Testing Cases first and then execute it. Think about all the possible combinations that you can think of by looking at the UI/ Use Cases/ Specs. There could few things that was missed during writing the Test Case document, like I said before, it's not a rigid document, should be updated periodically, if necessary. If you start performing a test without writing it at the first place, from my experience I can tell you there is a great chance you will forget it. (May not be a Agile Approach, but at least I would not use it as excuse). Writing the ideas on the test case will make it less dependent on particular person. It helps for the Tester as well as the company. 

Tried to write down the ideas that I learned from my experience. I'm sure I haven't seen all of them, but at least, for the time being, by following the above helps me to write an effective Test Case that will be easily understandable to all. Some people may feel in-secured documenting all the techniques that s/he applies for testing, finding issue, which is not unusual. Think about it in this way, if you have everything written down then  you won't have to worry about any steps that you missed or not.  Also remember, our own memory can betray with us at any moment. If you are good at something, no matter how many places you wrote it down, nobody can do things like you do. Don't feel in-secured.

Have Fun Testing!!!!    

Sources: 

[1]. Writing Effective Test Cases, by Clinton Staley, Ph.D., University of California, Santa Barbara

http://users.csc.calpoly.edu/~grade_cstaley/General/TestingAndBender/TestingHowTo.htm 
[2] How to Write Good Test Cases 

http://www.softwaretestingclass.com/how-to-write-good-test-cases/

[3] Test Case Writing (creation)
http://university.utest.com/test-case-writing-creation/