Basics of Software Testing - Part-VII : Software Testing with example

A little background about this - Each person may have their own way of testing or finding issues. They may not necessarily find my method helpful to understand the basics, this is something i'm trying to generalize for the newcomers in software/ firmware testing.  My intention is to make a series out of it, if i can and get enough motivation to write the basic stuffs.

So far we know what is a software, environment, SDLC, STLC, different types of software testing. Now, we will see how to test a software with example.

Lets start a test for the Student loan program or sometimes call calculator. What is does is, it gives the amount of how much you need to pay per month as well as how much it will cost you through out the year. You can also see how making extra payments will affect your pay off date. You can google search it and practice it on any of them.

So we will be doing a functional testing on a project. I usually starts from the Home screen. In our example, above is the home screen.

Testing steps:

1. This is one of the quickest way to find issues is to perform a data validation using the following criteria for the sample project. For positive data it should accept the amount, for negative / invalid data it should handle it and display proper error message.  

2. Make sure the required field are either marked or on submit application displays error messages on required fields .

3. Make sure the entered Loan amount is okay using the validation criterias.

4. Confirm that if for example the loan amount can not take a negative value, it should display error message stating that and should not blow up. User need to know the min and max value. It could be mentioned with the error message or it could be mentioned as a label.

5. For loan term, it can be either year or month. If we enter a year, it will display the corresponding month and if we enter a year it will display the corresponding month. Make sure that it behaves this way.  

6. Confirm that if for example the loan term month can not take decimal value then it should display error message.

7. As label indicates, Interest rate value would be a percent number. That means it will be later divided by 100. Make sure it passes the data validation  criteria.

8. For the Loan start date, depending on how it designed according to the Specs, i would assume it should be forward date. I can’t be sure whether it was designed this way or not, it looks like it allows previous dates/ years.

9. Finally, when all data inputs are okay and values are valid, then clicking on Calculate button will display the value on Monthly Payment.

10. Verify the Calculate button displays the correct value for per month. Verify it using the following formula in excel or manually. We need to make sure the calculation is right. For excel you can write the formula on B6 field as “=PMT(B4/B5, B3*B5, -B2)” to get the monthly payment.  

11. Make sure that, even the Monthly Payment field is a text fields which looks similar to input field it should not allow entering any value on this field. This will be a GUI testing. Check with Specs. 
12. Also notice that the amount field text box is not aligned with all other text box for a prefix of “$”. I would double check with the UI specs for this as well.
13. Check for any default values. On this particular case, it was displaying an example format on each text box. As soon as user starts typing on those fields, they were disappeared which was expected. Confirm that the application calculates based on the amount you entered, not the default amounts.
14. Check the spelling of all the labels.

Data Validation Criteria

Field Name	Criteria
Common/ All	Null / empty / blank space  as  input and thus check for the Required fields. Check for the Min field length. Check for the Max field length. Check the data type and determine the possible invalid inputs. Enter single quote ( ‘  ) to check whether database can handle it. Enter valid input combinations
Loan Amount	Negative amount - It should display error message. Amount with decimal point up to max input. Amount with single point decimal point. Long number up to max input allowed. Enter String instead of numeric value. Enter different characters instead of numeric value. Enter commas with a numeric entry. Try with valid ranges, should accept the data without any complain.
Loan Term	From above snapshot, it looks like it can take real numbers as number of years and numeric for months. Number of years should also allow to accept whole numeric numbers. For loan term in year, enter min and max allowed real number. For loan term in month, enter min and max allowed numeric number. For loan term in month, enter a number with decimal point. It should display error message as decimal number in month is not allowed. Try the negative numbers in both year and month. Should not be allowed. Try with valid ranges, should accept the data without any complain.
Interest Rate	Enter values ranging from min (0) to  max(99). Enter values below and above the min and max correspondently. Enter whole number . Enter number with decimal points.
Loan Start Date	Allows to select Year, Month, Day from the dropdown list. Should not allow to write anything on these fields.

Now where do you get these things? If you are lucky enough to work with a structure company, chances are you will get this document right away. Else, you will have to prepare it by yourself. Most of the times, it is basically a one time document, you just add them according to project and new fields.

So far we have done the functionality testing. Now we need to test few other things for this web application:

1. Make sure this web application is accessible using other browsers.
2. Make sure the loan term month and years auto calculation script works.
3. Make sure web browser session & cookies does not expire while you  are working on it. Testing cookies (sessions) are deleted either when cache is cleared or when they reach their expiry.
4. Make sure every time you visit this page, should always come as blank form.
5. Make sure multiple load is handled correctly.
6. Make sure concurrent entry does not create any issue.
7. Make sure the refresh or F5 works properly. It should update the page correctly. Any changes made on the page and submit button was not pressed, refreshing the screen should discard that entry and refresh the page with original entry.
8. Make sure browser back button and forward button works properly. When the back button is pressed and then the forward button is pressed, make sure it comes with blank page/ session. It should not just keep the previous entries.
9. Make sure web application does not allow to take input using the URL submission. For example, whatever the name of the website and application, try appending the word “?amount=234560” without the quotes. If it accepts the value and changes the value of the amount field then this could possibly be vulnerable to the BOSS people (hackers).

             

http://www.someweb.com/calculators/college-planning/loan-calculator.aspx?amount=234560

10. Make sure it can distinguish between an entry from human vs entry from automated bot. Using the Captcha it protects entry from bots. Otherwise, at any point, any bot can keep entering the values and thus creating load on the site. Other people may not be able to access it at certain point.
11. Check all other links on the web page. It should not display any dead link on the page.

Other observation
If you search for Student loan calculator you will find links. Click on them and you will get a webpage that will display the student loan amortization details. Many of them used to be open like the one i have chosen. Then after getting bombarding by the testing tutorial classes, they put it under user registration. This site should also have done it, otherwise, they are vulnerable to many points like steps 7, 8 if not many more. Also they were not using the Https as well!

If we summaries it then you will see, through out these steps we have performed, 

• Functional Testing,
• Usability Testing,
• GUI Testing,
• Compatibility Testing,
• Load Testing, 
• Performance Testing, 
• Security Testing and 
• some other common web application properties testing.

Few things we did not do here as they were not applicable. For example, Database testing - as no database was involved, Web service connectivity - single page calculates value. This is how you need to correlate the theory of testing and implement and perform it as needed on software.

This is how i would try to test this project. This was a very narrow function example. I would say, a software is consist of such small functions and once we identify the features, we will be able to perform testing on each individual pieces. 

Next, may be i'll try few more examples to give more idea. I'll also try to include at least one example for firmware testing in future. . And I would urge people to excuse my mistakes during writing up this series. I test other works, does not necessiarily means that I can’t make mistakes

Everybody has their own way of doing it and I respect all of them as long as it covers everything.

Basics of Software Testing - Part-VI : Different Types of Testing

A little background about this - Each person may have their own way of testing or finding issues. They may not necessarily find my method helpful to understand the basics, this is something i'm trying to generalize for the newcomers in software/ firmware testing.  My intention is to make a series out of it, if i can and get enough motivation to write the basic stuffs.

This may not be so interesting topic to read, but sometimes it is needed to know some keywords. I’ll try to express them in simple language (if possible avoiding any definition/ jargon).

Testing can be categories in two major parts:

1.0  Black Box Testing

2.0  White Box Testing

[It can be categories in many ways. Depending on what angle you look into it. The way i’m dividing may not necessarily be the only way. ]

1.0 Black Box Testing  - It is nothing related to “Black Box” as the word says, but it’s about testing the software with knowing anything internal and check the output against the Specs. [1]The tester is aware of what the software is supposed to do but is not aware of how it does it.

2.0 White Box Testing - Usually done by the developers. And yes, we are not the only one who test, developers also test their software by writing some test code. Internal knowledge of the software is vastly needed for this kind of testing.

Little bit more description about these two broad categories:

Black Box Testing
Functional Testing Graphical User Interface Testing Internationalization	Equivalence Partitioning Boundary Value	End-to-End  Testing User Acceptance Testing (UAT)	Smoke Sanity System Testing Regression

Functional Testing - Without knowing any internals of the software, you enter input and examine the output according to the Specs. Think about the features of a software. All of those features are functions and thus to test all those feature what we do is the functional testing.

Equivalence Partitioning -  is kind of technique that is used in functional testing. I would say more of technique then a type of test. [1]It is a software testing technique that divides the input data of a software unit into partitions of equivalent data from which test cases can be derived. [4] In equivalence partitioning technique we need to test only one condition from each partition. This is because we are assuming that all the conditions in one partition will be treated in the same way by the software. Similarly, if one of the conditions in a partition does not work, then we assume that none of the conditions in that partition will work so again there is little point in testing any more in that partition.

We all have seen the School grading system. Just want to give one specific example. School grading system says if marks between 80-89 then A-  marks >= 90 then ‘A+’ and so on. Obviously there are many more grades, just want to discuss this.

1. Checking the Range- Marks falls into different conditions/ ranges. If we check one condition, other should also be same. If we check for the conditions that says about ranges then we do not have to test other ranges.
2. We need to check the Lower and Greater condition as well. Anything below 60, anything above 90 condition needs to be checked.
3. Also, we assumed that marks will be positive. What if the marks is negative (someone pissed off the teacher). Invalid partition test needed.  

If we want have more understanding, follow the reference [4] for another example.

Boundary Value -  is another kind of technique that is used in functional testing. It is related to the Equivalence Partition testing. It it testing at the boundaries between partitions. We need to check the boundaries of the input. Here we have both valid boundaries (in the valid partitions) and invalid boundaries (in the invalid partitions). Most of the software has input fields. Let’s say it takes some numeric value for Age. First we have to find out the min and max allowed value for such input. Then we can try three conditions - within the min max range, above the max, below the min value.

Data Validation - It is a technique that ensures the integrity of the data. Data validation is intended to provide certain well-defined guarantees for fitness, accuracy, and consistency for any of various kinds of user input into an application or automated system. When i get the first build, in order to occupy them with some feedback/ issues, first thing I try is to check whether it can handle different kinds of input. Above mentioned two techniques are part of this test.

Graphical User Interface Testing -  It is about testing the User Interface as per the given sample design. Sometimes customer wants something in specific order/ place. This will make sure that everything was placed as it should be. It is easy to go crazy about testing the Graphical User Interface because you can put argument from many angle. Just make sure you don’t carried away with this type of testing. I recommend to be flexible about the minor/ trivial GUI issues. One thing to mention, always to a spell check on labels. It’s a part of this test, if not, I included them now.  

Internationalization Testing - Software may needed to support different international settings. For example, software needs to be compatible with the operating system’s setting so that with a given input with different language/ region/ format it does not blow up. A software that intent to work with German language and region should have the mechanism to handle the commas instead of dot to represent a German floating point number. Germany usages 1.211,2 which is equivalent to 1,211.2 in English. This test could also be termed as Globalization, localization test.

End-to-end Testing - [2]End to end testing is performed by testing team, focus of end to end testing is to test end to end flows e.g. right from order creation till reporting or order creation till item return etc and checking. End to end testing is usually focused mimicking real life scenarios and usage. End to end testing involves testing information flow across applications

User Acceptance Testing - This would be the last part of testing phases. At the end of all the testing, software is given to user/ client to look at it, whether it meets user expected features or not. This is usually a checklist type of test. User sign of the UAT document before releasing / taking the software. This is to check whether it meets their business requirement.

Smoke Testing - Smoking! This about checking the stability of the build given by the developer whether it’s good enough to test or not. Usually you can pick one or few major functions randomly and check whether it worth to test the rest of the build functionalities. If any of the major functionalities fail then it's a Show Stopper and build is rejected to test. It’s like cooking rice, pick a few and get the sense whether rest of the rice is eatable or not.

Sanity Testing -  [3]Sanity testing is a quick evaluation of the software, environment, network, external systems are up & running, software environment as a whole is stable enough to proceed with extensive testing. Sanity tests are narrow and most of the time sanity tests are not documented. [5] After receiving a software build, with minor changes in code, or functionality, Sanity testing is performed to ascertain that the bugs have been fixed and no further issues are introduced due to these changes

System Testing - [2] this includes multiple software testing types that will enable to validate the software as a whole (software, hardware and network) against the requirements for which it was built. Different types of tests (GUI testing, Functional testing, Regression testing, Smoke testing, load testing, stress testing, security testing, stress testing, ad-hoc testing etc.,) are carried out to complete system testing.

Picture Courtesy: Guru99.com

Regression Testing - Do it again and again and again. Regression testing is about doing the testing on the full system again and again. When we start testing and find issues, developers fixes some issues. At that time we just don’t sit idle. We keep going testing other component or features. Once we get the fix we verify those particular fix and then we re-run the whole test on the software. This will ensure that for a particular fix, nothing is broken unexpectedly. This is the hard part for a human being. For the Manual tester, this is the nightmare. That is why, more and more automated testing is getting popular only for this area.

White Box Testing

Unit Testing Component Testing

Unit Testing - I always gets different option when I put this test under the White Box testing. It’s not about whether it belong to white box or not, it is about who is going to perform it. With all respect, developers are the one who should be able to use this kind of testing more effectively than any others. Developers will write test fixture/ suits to test the internal part of the program. They can test a single function or they can test a series of function or class mimicking the inputs using the code.  Some tools are : dB unit testing, HTTPunit testing, Junit Testing, NUnit testing. This is not the traditional “automation testing” but it indeed an automation testing by the developer.

Component Testing - [2]This type of software testing is performed by developers. Component testing is carried out after completing unit testing. Component testing involves testing a group of units as code together as a whole rather than testing individual functions, methods.

Other types of Testing
Integration Testing Security Testing	Load Testing Performance Testing  User based scenario testing	Recovery Testing Conversion Testing Safety testing			Alpha Testing Beta Testing

Integration Testing - [4]Once the individual units or components are tested by developers as working then testing team will run tests that will test the connectivity among these units/component or multiple units/components. There are different approaches for Integration testing namely, Top-down integration testing, Bottom-up integration testing and a combination of these two known as Sand witch testing.

Security Testing - This is the most challenging testing. No matter what we do, they (the BOSS people) finds a way to sneak in. Security consist of many things from Network, Hardware, Database, software etc all component has their own protection mechanism. For software, i would generalize it as check for the common exploit options - like SQL injection, URL encryption for POST, inputs, use of public variables instead of private etc. [2]Security testing basically checks, how good is software’s authorization mechanism, how strong is authentication, how software maintains confidentiality of the data, how does the software maintain integrity of the data.

Load Testing - How much load it can take is the load testing. [4]Load testing is a type of non-functional testing; load testing is done to check the behavior of the software under normal and over peak load conditions. Load testing is usually performed using automated testing tools. Load testing intends to find bottlenecks or issues that prevent software from performing as intended at its peak workloads.

Performance Testing - [2]is a type of software testing and part of performance engineering that is performed to check some of the quality attributes of software like Stability, reliability, availability. Performance testing is carried out by performance engineering team. Unlike Functional testing, Performance testing is done to check non-functional requirements. Performance testing checks how well software works in anticipated and peak workloads. There are different variations or sub types of performance like load testing, stress testing, volume testing, soak testing and configuration testing.

User Based Scenario Testing - Sometimes people call them 1% scenario or Negative scenario testing. I personally think, this is a pretty important testing where we usually miss to understand the user’s possible behavior. I have studied many sites and most of them seems to give important to this type of testing. To learn about more, please read my first posting of my blog, which i’m pretty confidant that you will think about it twice once you read the blog.

Recovery Testing - [1] In software testing, recovery testing is the activity of testing how well an application is able to recover from crashes, hardware failures and other similar problems. Recovery testing is the forced failure of the software in a variety of ways to verify that recovery is properly performed.

Conversion Testing - Conversion testing is to verify that one data format can be converted into another data format so that the converted data format can be used seamlessly by the application under test appropriately.

Safety Testing - Safety first. Software are not always like software. It could drive a device/ machine/ car/ etc. When it gets into such level, and when it interacts with the people, we need to make sure that software does not raise any issue that can harm people. For example, if you are testing a treadmill which has the software control over the speed, we need to make sure no matter what scenarios it may occur, the stop button will always stop the mill gradually. Abrupt stop may actually cause accident, so it should slow down gradually.   

Alpha Testing - Alpha testing is basically by a fresh pair of eye testing. After testing is done, to check by a fresh unbiased pair of eye, it is sent to internal users to test.

Beta Testing - Done by the end users. Possible sites are chosen to try the beta version of the software before releasing it to production. Once the feedback is given by the customer and they find it ok, then the UAT is signed off and software is ready to release.

References:

[1] Wikipedia

[2] http://www.testingexcellence.com/types-of-software-testing-complete-list/

[3] http://www.softwaretestinghelp.com/types-of-software-testing/

[4] http://istqbexamcertification.com/what-is-equivalence-partitioning-in-software-testing/

[5] http://www.guru99.com/smoke-sanity-testing.html