QA process with lack of documentation

The described QA process assumes that development of the entire project is made with all necessary documents such as vision documents, requirements, specifications at different levels. However not all software products are developed using these practices. Often rather small projects are developed without the initial documentation and project features are discussed by development teams on meetings in order to save time required for documents creation.
Since such projects grow, they require more and more QA efforts to maintain the entire project quality. Project development in such cases may become almost chaotic making reverse engineering to create the required documents too expensive and even nearly impossible.
Smartech QA team has experience in handling such projects and has modified QA process for software products that are not “traditional” black-box ones. According to these specific conditions some major QA activities are moved from certain phases to different ones, and the complete set of activities is slightly different.
During the startup phase QA effort are targeted mostly to investigation of the software product, because the only source of information is application itself and its help system for end-user. The following efforts take place during the phase:

1. Acquaintance with product. QA engineers get acquainted with the software product, its features and capabilities and “split” the entire product into major functional sections.
2. Investigation of product modules. All separate modules of the product are investigated, including configuration and data storages.
3. Research of 3-rd party components. All 3-rd party components used in the application (if any) are investigated including their knowledge bases (if available).
4. Initial product testing. Brief manual testing of the software product is made in order to identify the most problematic and potentially troublesome areas of the product. Most serious attention will be paid to these areas in further testing and test plan creation.
5. Definition of testing approaches. Testing approaches and techniques are defined based on software type, project duration, available resources and testing goals. Tools required to perform the necessary testing are identified.
6. Rough resource estimation. Rough QA resource estimation is based on the first experience in software usage and defined testing approaches. This estimation is very approximate and will be corrected during further QA process.

During the main phase of QA process testing itself takes place, QA artifacts are being created, testing repositories are created, automated tests are developed and executed, found defects are detected and reported, fixes verified, QA documents are completed and maintained:

1. Manual ad hoc testing. Test cases and test plans cannot be created for all possible situations in applications. This is why ad hoc testing (also known as random testing) is also necessary in order to identify defects that cannot be found while performing testing using test plans and test cases.
2. Investigation of used technologies. Technologies used in the software product are investigated by QA engineers in order to target testing more precisely to certain features of the product.
3. Creation and maintaining of testing documents. The full required set of QA documents, including low-level documents such as test cases, is created. All QA documents are updated correspondingly when new features are introduced in the product or major changes take place. New test cases are created when defects are identified in order to make sure fixed defects do not appear in the software product during further development.
4. Manual testing according to testing documents. Continuous testing of software based on created QA documents guarantees that all application features are present in the product, are operable and work as expected.
5. Usability testing. Manual testing of software product to make sure the application has user friendly interface that meets common UI standards, application navigation is task oriented and allows execution of common tasks in minimal required user actions.
6. Documentation testing. All product support documentation such as help system, product information on WEB sites and system requirements are tested for consistence with actual software.
7. Security analysis. Security analysis is performed against distributed applications (for instance WEB applications) in order to make sure the applications meet general security requirements. Application should be protected from unauthorized code execution, application data should be protected from unauthorized access, data should be available to authorized users.
8. Automated testing. Automated test scripts are implemented, maintained and executed against the application in order to identify defects that appear in previously operable features of software due to the latest code changes. Automation of such testing helps to save QA resources and to identify such issues at early stage. Integration of automated tests into application build process is common practice.
9. Performance testing. Automated testing of application performance is maid against distributed applications in production-like environments in order to make sure server-side part of application is capable of serving the required number of simultaneous user activities. Also performance testing allows identifying actual number of users the application can serve simultaneously to adjust software system requirements or production configuration.
10. Load/stress testing. Automated load and stress testing scripts are implemented and executed to make sure the application stays stable after continuous load with expected amount of simultaneous users and after short time stress load with more simultaneous users than expected.
11. Monitoring bugs and fixes. Bugs found by QA team, other teams involved in product development as well as customer bugs are monitored by QA team. Bugs that have insufficient information to reproduce them are reproduced by QA team, exact steps and conditions are populated to bug tracking system. Fixed defects are retested and related documents are updated correspondingly.
12. Reporting progress and statistics. At each stage of QA activities reports with the full list of current activities and progress of each activity are created and provided to customer. Detailed reports with currently open defects are provided on demand. Current product quality assessments are provided on demand as well.
13. Recalculation of resource estimation. QA resource estimation is recalculated to be more precise. Calculation is based on QA documents and practical experience how long it takes to perform certain QA activities related to particular software product.

Release phase represents QA activities when software product is about to be deployed to production environment or a new version of software is about to be published for customers.

1. Assessment of testing coverage. Overview of QA activities that have been done, analysis of testing results, assessment of what percentage of the software product has been tested and what additional testing could be performed to guarantee better software quality.
2. Acceptance testing.The final step of testing performed by QA team and representatives of customer team in order to make sure the software product has the desired quality level and matches customer expectations.
3. Review and approval of release notes. Release notes that usually include “What’s new”, “Known issues” and “Bug fixes” sections are made in collaboration with QA team and reviewed by QA team.
4. Transition of QA artifacts.All artifacts created by QA team during testing of the software product are gathered to a single package and delivered to the customer.