Using Software Testing Checklists to Ensure Software Quality – Sample Checklists Included.

 Below is an example of a sample Test Readiness Review checklist:

Test Readiness Review (TRR) Criteria	Status
All the requirements finalized and analyzed	Done
Test plam created and reviewed	Done
Test cases preparation done
Test case review and sign off
Test data availability
Smoke testing
Sanity testing done?
Team aware of the roles and responsibilities
Team aware of the deliverables expected of them
Team aware of the communication protocol
Team’s access to the application, version controlling  tools, test management
Team’s trained
Technical aspects- server1 refreshed or not?
Defect reporting standards are defined

Now, all you have to do with this list is mark done or not done.

Exit Criteria Checklist:

As the name indicates, this is a check list that aids in the decision making of whether a testing phase/cycle should be stopped or continued.

Since, a defect free product is not possible and we will have to make sure that we test to the best extent possible in the given amount of time – a check list of the below effect is created to track the most important criteria that needs to be met to deem a testing phase satisfactory.

Exit Criteria	Status
100% Test Scripts executed	Done
95% pass rate of Test Scripts
No open Critical and High severity defects
95% of Medium severity defects have been closed
All remaining defects are either cancelled or documented as Change Requests for a future release
All expected and actual results are captured and documented with the test script	Done
All test metrics collected based on reports from the tool
All defects logged in the tool	Done
Test Closure Memo completed and signed off

Points to note:

1. The above two are examples to show case the use of checklists to QA process, but the usage is not limited to these two areas.
2. The items in each list are also indicators to give an idea to the readers about what sort of items can be included and tracked – however, the list can be expanded and/or compacted as needed.

We really hope that the above examples have been successful in bringing forward the potential of checklists to QA and IT processes.

Front end Performance testing tools.

Page Speed

• It provides the web page performance overall score based on the front end performance optimization rules.
• Page speed provides complete detail of the time spent on web page’s different components i.e. Images, CSS and Java Script files etc.
• It lists down all the web page’s bottlenecks along with their details as well as all the possible solutions in order to help the developers to quickly fix those issues.
• Page speed also highlights all the good practices which are already followed by the developers to optimize the web page speed.
• Page speed provides all the above mentioned information for mobile devices browsers as well.
• Link : https://developers.google.com/speed/pagespeed/

Web Page Test

Web page test is a Google sponsored free online service which provides the web page complete performance details on a particular browser when it is accessed from a specific location. Following is the detailed list of web page test’s features:

• It provides web page performance results for modern browsers on different internet connections (including mobile 3G connection) from different geographical locations.
• Web page test also provides advanced test options like multistep transactions, video capturing and content blocking etc.
• It provides results for iPhone and Android devices browsers as well along with the desktop browsers.
• You can also compare different web pages performance results through web page test.
• It also provides web page diagnostics information like waterfall charts and page speed optimization checks along with the performance optimization suggestions.
• Web page test also provides the page statistics on first view and repeated view along with the details of server responses.

Link: http://www.webpagetest.org

BrowserMob – Neustar

Neustar is another free online service (offered by BrowserMob) to test the web page performance from different geographical locations. Following is the list of Neustar’s features:

• Neustar provides web page average load time, its total size and page requests information.
• It provides web page statistics from 4 different geographical locations like California, Washington DC, Singapore and Dublin etc. in order to provide the web page global performance results.
• Neustar also provides the details of each page component share in page load time.

Link: http://www.neustar.biz/lp/web-performance-trial/index.html

Powerful Penetration Testing Tools For Every Penetration Tester

What is Penetration Testing?

It’s the method of testing where the areas of weakness in software systems in terms of security are put to test to determine, if ‘weak-point’ is indeed one, that can be broken into or not.

How is it performed?

Step #1. It starts with a list of Vulnerabilities/potential problem areas that would cause a security breach for the systems.
Step #2. If possible, this list of items has to be ranked in the order of priority/criticality
Step #3. Devise penetration tests that would work (attack your system) from both within the network and outside (externally) to determine if you can access data/network/server/website unauthorized.
Step #4. If the unauthorized access is possible, the system has to be corrected and the series of steps need to be re-run until the problem area is fixed.

1) Metasploit 

This is the most advanced and popular Framework that can be used to for pen-testing. It is based on the concept of ‘exploit’ which is a code that can surpass the security measures and enter a certain system. If entered, it runs a ‘payload’, a code that performs operations on a target machine, thus creating the perfect framework for penetration testing.

It can be used on web applications, networks, servers etc. It has a command-line and a GUI clickable interface, works on Linux, Apple Mac OS X and Microsoft Windows. This is a commercial product, although there might be free limited trials available.

Download link: Metasploit Download

2) Wireshark

This is basically a network protocol analyzer –popular for providing the minutest details about your network protocols, packet information, decryption etc. It can be used on Windows, Linux, OS X, Solaris, FreeBSD, NetBSD, and many other systems. The information that is retrieved via this tool can be viewed through a GUI, or the TTY-mode TShark utility.  You can get your own free version of the tool from here.

Download link: Wireshark download

3) w3af

W3afis a Web Application Attack and Audit Framework.

Some of the features are: fast HTTP requests, integration of web and proxy servers into the code, injecting payloads into various kinds of HTTP requests etc.

It has a command-line interface, works on Linux, Apple Mac OS X and Microsoft Windows.
All versions are free of charge to download.

Download link: w3af download

4) CORE Impact

CORE Impact Pro can be used to test mobile device penetration, network/network devise penetration, password identification and cracking, etc. It has a command-line and a GUI clickable interface, works Microsoft Windows. This is one of the expensive tools in this line and all the information can be found at below page.

Download link: CORE Impact download

5) Back Track

Back Track works only on Linux Machines. The new version is called Kali Linux. This is one of the best tools available for Packet sniffing and injecting. An expertise in TCP/IP protocol and networking are key to succeed using this tool. For information and to download a free copy, visit below page.

Download link: Back Track download

6) Netsparker

Netsparker comes with a robust web application scanner that will identify vulnerabilities, suggest remedial action etc. This tool can also help exploit SQL injection and LFI (local file induction). It has a command-line and GUI interface, works only on Microsoft Windows. This is a commercial product, although there might be free limited trials available at below page.

Download link: Netsparker download

7) Nessus

Nessus also is a scanner and one that needs to be watched out for. It is one of the most robust vulnerability identifier tools available. It specializes in compliance checks, Sensitive data searches, IPs scan, website scanning etc. and aids in finding the ‘weak-spots’. It works on most of the environments.

Download link: Nessus download

8) Burpsuite

Burp suite is also essentially a scanner (with a limited “intruder” tool for attacks), although many security testing specialists swear that pen-testing without this tool is unimaginable. The tool is not free, but very cost effective. Take a look at it on below download page. It mainly works wonders with intercepting proxy, crawling content and functionality, web application scanning etc.  You can use this on Windows, Mac OS X and Linux environments.

Download link: Burp suite download

9) Cain & Abel

If cracking encrypted passwords or network keys is what you need, then Cain& Abel is the tool for you. It uses network sniffing, Dictionary, Brute-Force and Cryptanalysis attacks, cache uncovering and routing protocol analysis methods to achieve this. Check out information about this free to use tool at below page. This is exclusively for Microsoft operating systems.

Download link: Cain & Abel download

10) Zed Attack Proxy (ZAP)

ZAP is a completely free to use, scanner and security vulnerability finder for web applications. ZAP includes Proxy intercepting aspects, variety of scanners, spiders etc. It works on most platforms and the more information can be obtained from below page.

Download link: ZAP download

DevOps and Software Testing

What is DevOps?

DevOps – a combination of Development & Operations – is a software development methodology which looks to integrate all the software development functions from development to operations within the same cycle.

This calls for higher level of coordination within the various stakeholders in the software development process (namely Development, QA & Operations).

Why DevOps?

Although there are subtle differences between Agile & DevOps, those working with Agile will find DevOps a little more familiar to work with (and eventually adopt). While Agile principles are applied successfully on the development & QA iterations, it is a different story altogether (and often a bone of contention) on the operations side. DevOps proposes to rectify this gap.

Now, instead of Continuous Integration, DevOps involves “Continuous Development”, whereby, code written and committed to Version Control, will be built, deployed, tested and installed on the Production environment ready to be consumed by the end-user.

This process helps everyone in the entire chain since environments and processes are standardized. Every action in the chain is automated. It also gives freedom to all stakeholders to concentrate their efforts on designing and coding a high quality deliverable rather than worrying about the various building, operations and QA processes.

It brings down the time-to-live drastically to about 3-4 hours, from the time code is written and committed, to deployment on production for end-user consumption.

In a nutshell, DevOps is an extension of Agile, or, as I like to call it, “Agile on Steroids”.

Changed Role of QA in DevOps

Traditionally, QA would get a build which is deployed on their designated environment and QA would then commence their functional & regression testing. The build would ideally sit with the QA for a couple of days before the QA sign-off on the build. All this changes in DevOps.

QA changes for DevOps:

QA are required to align their efforts in the DevOps cycle.
They have to make sure that all their test cases are automated and achieve near 100% code coverage.
They need to make sure that their environments are standardized and deployment on their QA boxes is automated.
All their pre-testing tasks, cleanups, post-testing tasks, etc. are automated and aligned with the Continuous Integration cycle.
As already mentioned, DevOps requires high level of coordination between various functions of the deliverable chain. What this also means is that boundaries between the various roles of contributors in the chain become porous.

DevOps encourages everyone to contribute across the chain. So, amongst other things, a dev can configure deployments. Deployment engineers can add test cases to the QA repository. QA Engineers can configure their automation test cases into the DevOps chain.

Collectively, everyone in the chain is responsible for the quality and timeliness of the deliverables.

DevOps and Test Automation

To achieve such speed and agility, it is important to automate all the testing processes and configure them to run automatically when the deployment is completed on the QA environment. Specialized automation testing tools and continuous integration tools are used to achieve this integration. This also necessitates building of a mature automation testing framework through which one can quickly script new test cases.

Tips for DevOps Success:

The test cases that are required to be executed for a particular build need to be identified.
The test execution should essentially be lean.
The QA and dev need to sit together and identify the areas affected due to a particular build and execute those related test cases plus a sanity test pass.
You also need to configure specialized code analysis and coverage tools to make sure you achieve near 100% code coverage.
The concept of executing all regression test cases for a test pass is soon becoming obsolete.
Strategy around testing new features need to be formalized and interim builds can be supplied to QA who would in turn create test scripts and run these automation tests on these interim builds till the code becomes stable enough to be deployed on to Production environment.
All the environments required for testing need to be standardized and deployments have to be automated.
Using various automation techniques, QA should be able to fire automation testing runs across various cross-platform (and cross-browser in case of web applications) environments.
Parallel execution of tests helps in reducing time-to-live which is the crux of a successful DevOps implementation.
Exit criteria need to be set for each run so that when the results of the tests are fed back to the chain, a go/no-go decision to Production is taken.
Blocker or Critical bugs found need to be reported and fixed and passed through the same chain of events before the code is deployed on to the Production environment.
Application Monitoring

QA should also be able to detect problems early and report them proactively. To achieve this, they need to set up monitoring on the Production environment to be able to expose bugs before they cause a failure.

Setting up specialized counters like response times, memory & CPU utilization, etc. can provide a lot of insight into the end-user experience.

For example, if the average response time for login is gradually increasing over the various builds, QA should proactively report this issue for optimizing the login code, else future builds might cause end-user frustration due to high response times.

QA can also use a small subset of existing high priority test cases to be executed periodically on production to actively monitor the environment. Bugs like, “This bug appears sometimes” or “Cannot Reproduce” can be caught through this strategy which, in the end makes the application more stable and also gets more satisfied end-users.

Again, these monitors need to be configured to run automatically with rich reporting (like logs & screenshots of failures, etc.).

Conclusion

Waterfall gave way to V-Model which in turn was replaced by Agile as the preferred choice for software development. DevOps is the future. It’s a continuous improvement cycle that software development models undergo from time-to-time. You need to embrace, understand and inculcate it.

You need to master the various automation and continuous integration tools, so that your automation efforts add value to the chain and are lean enough to quickly adapt to changes. You may be working on projects that may involve an alpha, beta and UAT environments before being deployed on the production environment.

New Technique of Covering Test cases Confidentially.

New Technique of Covering Test cases Confidentially.

Let’s take an Decision Table example of below business requirement for a login screen

The first step we do is to name all the branches and leaves with numbers or alphabets as below.

1, 2, 3 are the leaves and a, b, c are the branches.

Test Auto Mation

What makes automated testing so important to these successful companies?

Automated Software Testing Saves Time and Money

Software tests have to be repeated often during development cycles to ensure quality. Every time source code is modified software tests should be repeated. For each release of the software it may be tested on all supported operating systems and hardware configurations. Manually repeating these tests is costly and time consuming. Once created, automated tests can be run over and over again at no additional cost and they are much faster than manual tests. Automated software testing can reduce the time to run repetitive tests from days to hours. A time savings that translates directly into cost savings.

Testing Improves Accuracy

Even the most conscientious tester will make mistakes during monotonous manual testing. Automated tests perform the same steps precisely every time they are executed and never forget to record detailed results.

Increase Test Coverage

Automated software testing can increase the depth and scope of tests to help improve software quality. Lengthy tests that are often avoided during manual testing can be run unattended. They can even be run on multiple computers with different configurations. Automated software testing can look inside an application and see memory contents, data tables, file contents, and internal program states to determine if the product is behaving as expected. Automated software tests can easily execute thousands of different complex test cases during every test run providing coverage that is impossible with manual tests. Testers freed from repetitive manual tests have more time to create new automated software tests and deal with complex features.

Automation Does What Manual Testing Cannot

Even the largest software departments cannot perform a controlled web application test with thousands of users. Automated testing can simulate tens, hundreds or thousands of virtual users interacting with network or web software and applications.

Automated QA Testing Helps Developers and Testers

Shared automated tests can be used by developers to catch problems quickly before sending to QA. Tests can run automatically whenever source code changes are checked in and notify the team or the developer if they fail. Features like these save developers time and increase their confidence.

Team Morale Improves

This is hard to measure but we’ve experienced it first hand, automated software testing can improve team morale. Automating repetitive tasks with automated software testing gives your team time to spend on more challenging and rewarding projects. Team members improve their skill sets and confidence and, in turn, pass those gains on to their organization.

TestComplete is a Powerful and Affordable Automated Software Testing Tool

TestComplete addresses a full range of software testing challenges facing corporate IT departments, product developers, QA engineers, and consultants. TestComplete enhances the software testing process by increasing efficiency, removing complexity and lowering costs

- See more at: http://support.smartbear.com/articles/testcomplete/manager-overview/#sthash.V0w8mfc2.dpuf

QA TIPS

1. Focus on Risk/Change Based Testing – In an Agile environment, the goal is always to get the highest quality product to market in the shortest amount of time. Therefore an Agile tester must be able to assess which areas of the application are changing, how they all fit together and prioritize tests based on areas that pose the greatest risk of failure.

2. Understanding Product Architecture – Put simply, this means understanding exactly how data flows through an application. This allows us to test the impact of sub-system component failures, as well as address potential security vulnerabilities. As a result, when defects are found an Agile tester will be able to help developers fix the issues quickly and thoroughly early on.

3. Understanding Business Objects and Context – Since testing should always align with customer context, an Agile tester must understand how the end user will interact with the product. If you focus on the applications and scenarios that provide value to the end user you will be able to dimensionalize your testing strategy. Meaning, if you can divide a product up based on the product architecture and customer context you will be better equipped to test for potential issues that would negatively impact end users.

The added benefit of dimensionalizing the product from the customer view point is that the tester will be able to tag defects with the affected business objects. That way when you begin to review your test sets, you will be able to recognize areas with higher instances of defects. This will allow you to address the issues in a more focused and effective way.

4. Application Logs - While reporting ‘when feature X breaks we do Y’ provides value, it is not enough in an Agile environment. Agile testers should be leveraging the great amount of information contained in application logs for a number of reasons. First, it provides insight into the underlying architecture that is occurring on the system level, which gives you a better understanding of what the defect is. Second, application logs also allow you to address ‘silent errors’ – i.e. errors that occur without the end user ever even knowing. Lastly, it allows testers to gain credibility and work more closely with the development team, which is an integral component of being Agile.

5. Based Tools – In addition to application logs, browser based tools are incredibly important for Agile testers to be able to quickly troubleshoot defects. Two examples of easy-to-use browser tools are Developer Tools in Google Chrome and Firebug in Firefox. Tools such as these provide value in the form of metrics, errors, analysis, java script consoles and debuggers.

6. Test Document Requirements – Many Agile organizations utilize some variant of SCRUM or Kanban, in which there is an element of work tackled by a team to ensure that all aspects function as intended. In order to do this, the team will need to create test scenarios. The expectance criterion of these test scenarios often becomes a requirement repository over the long term. While some people view the written test scenario as a waste, Brian Rock contends that these test scenarios become very valuable to the Agile tester for future automation, regression testing and product analysis.

7. Automation – Automation is an excellent tool but it goes without saying that it is not the ‘silver bullet’ to fix the testing problem. Frameworks can be expensive to create and even more expensive to maintain. However, Agile testers need to be able to leverage automation for quick, easily repeatable tasks. If the delta of change is low, chances are you should automate it.

8. Exploratory Testing – Exploratory testing (ET) is common sense testing. ET focuses on instantaneous testing and learning and thus should be a primary tool of all Agile testing. Agile testers can leverage knowledge gained through ET on every future product iteration. Additionally, it allows the Agile tester to make variations in testing to quickly ferret out bugs.

9. Testing From the Customer Perspective – In many ways, this goes back to understanding the business objects and how testing should focus the end user – despite the fact that the end user is going to use the product in ways that you never thought feasible, wise or even sane. Testing from the customer perspective goes a step further to state that the fitness of the product for use by the end user is the primary standard of quality. After all, without customers there is no product. Agile testers are in a unique position to face QA challenges from this context.

10. Know That Change is Constant – Perhaps this is a bit cliché, but a static product is a dead product. As an Agile tester you should be able to give a quality assessment of the product at any stage in its life cycle. If you are able to do this, you will be better equipped to handle the ever changing landscape that is Agile.