Using Software Testing Checklists to Ensure Software Quality – Sample Checklists Included.

 Below is an example of a sample Test Readiness Review checklist:

Test Readiness Review (TRR) Criteria	Status
All the requirements finalized and analyzed	Done
Test plam created and reviewed	Done
Test cases preparation done
Test case review and sign off
Test data availability
Smoke testing
Sanity testing done?
Team aware of the roles and responsibilities
Team aware of the deliverables expected of them
Team aware of the communication protocol
Team’s access to the application, version controlling  tools, test management
Team’s trained
Technical aspects- server1 refreshed or not?
Defect reporting standards are defined

Now, all you have to do with this list is mark done or not done.

Exit Criteria Checklist:

As the name indicates, this is a check list that aids in the decision making of whether a testing phase/cycle should be stopped or continued.

Since, a defect free product is not possible and we will have to make sure that we test to the best extent possible in the given amount of time – a check list of the below effect is created to track the most important criteria that needs to be met to deem a testing phase satisfactory.

Exit Criteria	Status
100% Test Scripts executed	Done
95% pass rate of Test Scripts
No open Critical and High severity defects
95% of Medium severity defects have been closed
All remaining defects are either cancelled or documented as Change Requests for a future release
All expected and actual results are captured and documented with the test script	Done
All test metrics collected based on reports from the tool
All defects logged in the tool	Done
Test Closure Memo completed and signed off

Points to note:

1. The above two are examples to show case the use of checklists to QA process, but the usage is not limited to these two areas.
2. The items in each list are also indicators to give an idea to the readers about what sort of items can be included and tracked – however, the list can be expanded and/or compacted as needed.

We really hope that the above examples have been successful in bringing forward the potential of checklists to QA and IT processes.

Front end Performance testing tools.

Page Speed

• It provides the web page performance overall score based on the front end performance optimization rules.
• Page speed provides complete detail of the time spent on web page’s different components i.e. Images, CSS and Java Script files etc.
• It lists down all the web page’s bottlenecks along with their details as well as all the possible solutions in order to help the developers to quickly fix those issues.
• Page speed also highlights all the good practices which are already followed by the developers to optimize the web page speed.
• Page speed provides all the above mentioned information for mobile devices browsers as well.
• Link : https://developers.google.com/speed/pagespeed/

Web Page Test

Web page test is a Google sponsored free online service which provides the web page complete performance details on a particular browser when it is accessed from a specific location. Following is the detailed list of web page test’s features:

• It provides web page performance results for modern browsers on different internet connections (including mobile 3G connection) from different geographical locations.
• Web page test also provides advanced test options like multistep transactions, video capturing and content blocking etc.
• It provides results for iPhone and Android devices browsers as well along with the desktop browsers.
• You can also compare different web pages performance results through web page test.
• It also provides web page diagnostics information like waterfall charts and page speed optimization checks along with the performance optimization suggestions.
• Web page test also provides the page statistics on first view and repeated view along with the details of server responses.

Link: http://www.webpagetest.org

BrowserMob – Neustar

Neustar is another free online service (offered by BrowserMob) to test the web page performance from different geographical locations. Following is the list of Neustar’s features:

• Neustar provides web page average load time, its total size and page requests information.
• It provides web page statistics from 4 different geographical locations like California, Washington DC, Singapore and Dublin etc. in order to provide the web page global performance results.
• Neustar also provides the details of each page component share in page load time.

Link: http://www.neustar.biz/lp/web-performance-trial/index.html

Powerful Penetration Testing Tools For Every Penetration Tester

What is Penetration Testing?

It’s the method of testing where the areas of weakness in software systems in terms of security are put to test to determine, if ‘weak-point’ is indeed one, that can be broken into or not.

How is it performed?

Step #1. It starts with a list of Vulnerabilities/potential problem areas that would cause a security breach for the systems.
Step #2. If possible, this list of items has to be ranked in the order of priority/criticality
Step #3. Devise penetration tests that would work (attack your system) from both within the network and outside (externally) to determine if you can access data/network/server/website unauthorized.
Step #4. If the unauthorized access is possible, the system has to be corrected and the series of steps need to be re-run until the problem area is fixed.

1) Metasploit 

This is the most advanced and popular Framework that can be used to for pen-testing. It is based on the concept of ‘exploit’ which is a code that can surpass the security measures and enter a certain system. If entered, it runs a ‘payload’, a code that performs operations on a target machine, thus creating the perfect framework for penetration testing.

It can be used on web applications, networks, servers etc. It has a command-line and a GUI clickable interface, works on Linux, Apple Mac OS X and Microsoft Windows. This is a commercial product, although there might be free limited trials available.

Download link: Metasploit Download

2) Wireshark

This is basically a network protocol analyzer –popular for providing the minutest details about your network protocols, packet information, decryption etc. It can be used on Windows, Linux, OS X, Solaris, FreeBSD, NetBSD, and many other systems. The information that is retrieved via this tool can be viewed through a GUI, or the TTY-mode TShark utility.  You can get your own free version of the tool from here.

Download link: Wireshark download

3) w3af

W3afis a Web Application Attack and Audit Framework.

Some of the features are: fast HTTP requests, integration of web and proxy servers into the code, injecting payloads into various kinds of HTTP requests etc.

It has a command-line interface, works on Linux, Apple Mac OS X and Microsoft Windows.
All versions are free of charge to download.

Download link: w3af download

4) CORE Impact

CORE Impact Pro can be used to test mobile device penetration, network/network devise penetration, password identification and cracking, etc. It has a command-line and a GUI clickable interface, works Microsoft Windows. This is one of the expensive tools in this line and all the information can be found at below page.

Download link: CORE Impact download

5) Back Track

Back Track works only on Linux Machines. The new version is called Kali Linux. This is one of the best tools available for Packet sniffing and injecting. An expertise in TCP/IP protocol and networking are key to succeed using this tool. For information and to download a free copy, visit below page.

Download link: Back Track download

6) Netsparker

Netsparker comes with a robust web application scanner that will identify vulnerabilities, suggest remedial action etc. This tool can also help exploit SQL injection and LFI (local file induction). It has a command-line and GUI interface, works only on Microsoft Windows. This is a commercial product, although there might be free limited trials available at below page.

Download link: Netsparker download

7) Nessus

Nessus also is a scanner and one that needs to be watched out for. It is one of the most robust vulnerability identifier tools available. It specializes in compliance checks, Sensitive data searches, IPs scan, website scanning etc. and aids in finding the ‘weak-spots’. It works on most of the environments.

Download link: Nessus download

8) Burpsuite

Burp suite is also essentially a scanner (with a limited “intruder” tool for attacks), although many security testing specialists swear that pen-testing without this tool is unimaginable. The tool is not free, but very cost effective. Take a look at it on below download page. It mainly works wonders with intercepting proxy, crawling content and functionality, web application scanning etc.  You can use this on Windows, Mac OS X and Linux environments.

Download link: Burp suite download

9) Cain & Abel

If cracking encrypted passwords or network keys is what you need, then Cain& Abel is the tool for you. It uses network sniffing, Dictionary, Brute-Force and Cryptanalysis attacks, cache uncovering and routing protocol analysis methods to achieve this. Check out information about this free to use tool at below page. This is exclusively for Microsoft operating systems.

Download link: Cain & Abel download

10) Zed Attack Proxy (ZAP)

ZAP is a completely free to use, scanner and security vulnerability finder for web applications. ZAP includes Proxy intercepting aspects, variety of scanners, spiders etc. It works on most platforms and the more information can be obtained from below page.

Download link: ZAP download

DevOps and Software Testing

What is DevOps?

DevOps – a combination of Development & Operations – is a software development methodology which looks to integrate all the software development functions from development to operations within the same cycle.

This calls for higher level of coordination within the various stakeholders in the software development process (namely Development, QA & Operations).

Why DevOps?

Although there are subtle differences between Agile & DevOps, those working with Agile will find DevOps a little more familiar to work with (and eventually adopt). While Agile principles are applied successfully on the development & QA iterations, it is a different story altogether (and often a bone of contention) on the operations side. DevOps proposes to rectify this gap.

Now, instead of Continuous Integration, DevOps involves “Continuous Development”, whereby, code written and committed to Version Control, will be built, deployed, tested and installed on the Production environment ready to be consumed by the end-user.

This process helps everyone in the entire chain since environments and processes are standardized. Every action in the chain is automated. It also gives freedom to all stakeholders to concentrate their efforts on designing and coding a high quality deliverable rather than worrying about the various building, operations and QA processes.

It brings down the time-to-live drastically to about 3-4 hours, from the time code is written and committed, to deployment on production for end-user consumption.

In a nutshell, DevOps is an extension of Agile, or, as I like to call it, “Agile on Steroids”.

Changed Role of QA in DevOps

Traditionally, QA would get a build which is deployed on their designated environment and QA would then commence their functional & regression testing. The build would ideally sit with the QA for a couple of days before the QA sign-off on the build. All this changes in DevOps.

QA changes for DevOps:

QA are required to align their efforts in the DevOps cycle.
They have to make sure that all their test cases are automated and achieve near 100% code coverage.
They need to make sure that their environments are standardized and deployment on their QA boxes is automated.
All their pre-testing tasks, cleanups, post-testing tasks, etc. are automated and aligned with the Continuous Integration cycle.
As already mentioned, DevOps requires high level of coordination between various functions of the deliverable chain. What this also means is that boundaries between the various roles of contributors in the chain become porous.

DevOps encourages everyone to contribute across the chain. So, amongst other things, a dev can configure deployments. Deployment engineers can add test cases to the QA repository. QA Engineers can configure their automation test cases into the DevOps chain.

Collectively, everyone in the chain is responsible for the quality and timeliness of the deliverables.

DevOps and Test Automation

To achieve such speed and agility, it is important to automate all the testing processes and configure them to run automatically when the deployment is completed on the QA environment. Specialized automation testing tools and continuous integration tools are used to achieve this integration. This also necessitates building of a mature automation testing framework through which one can quickly script new test cases.

Tips for DevOps Success:

The test cases that are required to be executed for a particular build need to be identified.
The test execution should essentially be lean.
The QA and dev need to sit together and identify the areas affected due to a particular build and execute those related test cases plus a sanity test pass.
You also need to configure specialized code analysis and coverage tools to make sure you achieve near 100% code coverage.
The concept of executing all regression test cases for a test pass is soon becoming obsolete.
Strategy around testing new features need to be formalized and interim builds can be supplied to QA who would in turn create test scripts and run these automation tests on these interim builds till the code becomes stable enough to be deployed on to Production environment.
All the environments required for testing need to be standardized and deployments have to be automated.
Using various automation techniques, QA should be able to fire automation testing runs across various cross-platform (and cross-browser in case of web applications) environments.
Parallel execution of tests helps in reducing time-to-live which is the crux of a successful DevOps implementation.
Exit criteria need to be set for each run so that when the results of the tests are fed back to the chain, a go/no-go decision to Production is taken.
Blocker or Critical bugs found need to be reported and fixed and passed through the same chain of events before the code is deployed on to the Production environment.
Application Monitoring

QA should also be able to detect problems early and report them proactively. To achieve this, they need to set up monitoring on the Production environment to be able to expose bugs before they cause a failure.

Setting up specialized counters like response times, memory & CPU utilization, etc. can provide a lot of insight into the end-user experience.

For example, if the average response time for login is gradually increasing over the various builds, QA should proactively report this issue for optimizing the login code, else future builds might cause end-user frustration due to high response times.

QA can also use a small subset of existing high priority test cases to be executed periodically on production to actively monitor the environment. Bugs like, “This bug appears sometimes” or “Cannot Reproduce” can be caught through this strategy which, in the end makes the application more stable and also gets more satisfied end-users.

Again, these monitors need to be configured to run automatically with rich reporting (like logs & screenshots of failures, etc.).

Conclusion

Waterfall gave way to V-Model which in turn was replaced by Agile as the preferred choice for software development. DevOps is the future. It’s a continuous improvement cycle that software development models undergo from time-to-time. You need to embrace, understand and inculcate it.

You need to master the various automation and continuous integration tools, so that your automation efforts add value to the chain and are lean enough to quickly adapt to changes. You may be working on projects that may involve an alpha, beta and UAT environments before being deployed on the production environment.