Software QA/Testing Technical FAQs

Part:   1  2  3  4  5  6  7  8  9  10  11  12  13  14   15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47 

Which of the following statements about Regression statements are true?
1---Regression testing must consist of a fixed set of tests to create a base line
2---Regression tests should be used to detect defects in new feature
3---Regression testing can be run on every build
4--- Regression testing should be targeted areas of high risk and known code change
5---Regression testing when automated, is highly effective in preventing defects.

Answer1:
1---Regression testing must consist of a fixed set of tests to create a base line
Don't think it is true as a "must" -- it
depends on whether your regression testing style involves repeating identical tests or redoing testing in previously tested areas with similar tests or tests that address the same risks. For example, some people do regression testing with tests whose specific parameters are determined randomly. They broaden the set of values they test while achieving essentially the same testing. Second example--some regression test suites include random stringing together of test cases (they include load testing and duration testing in their regression series, reporting their results as part of the assessment of each build). Depending on your theory of the _point_ of regression testing, these may or may not be entirely valid regression tests.

2---Regression tests should be used to detect defects in new feature
How do you create new regression tests? Should you design new tests as standalone, or should you develop a strategy in which the tests you use for bug-hunting are designed to be reusable as regression tests? If the latter, and I have certainly heard some skilled testers argue that the latter approach worked well in their sistuation, then #2 is sometimes true.

3---Regression testing can be run on every build
This is true, though it might be silly and a big waste of time.

4--- Regression testing should be targeted areas of high risk and known code change
Hmmm, there's a area of computer science called program slicing and one of the objectives of this class of work is to figure out how to restrict the regression test suite to a smaller number of tests, which test only those things that might have been impacted by a change. Bob Glass has criticized the results of some of this work, but if #4 is false, some Ph.D.'s and big research grants should be retracted.

5---Regression testing when automated, is highly effective in preventing defects.
Unit-level automated regression testing is highly effective in preventing defects--read up on test-driven development.

Answer2:
Let me explain why I think 2 & 5 are false
2---Regression tests should be used to detect defects in new feature
Since regression tests only address existing features and functionality, it can't find defects in new features. It can only find where existing features and functionality have been broken by changes.

5---Regression testing when automated, is highly effective in preventing defects.
Since no tests prevent defects, they only find them, it's impossible to prevent defects with a regression test. I will add, however, that if a developer can use an automated regression test to test their own code before submitting it to the code repository (say in the form a series of unit tests coupled to a library, etc.) then you could in some way prevent defects with a regression test.

I also don't like 1- and 4. 1- since a regression test suite grows as the product does. Therefore the tests are not fixed. 4- because a regression test tests the whole application, not just a targeted area. In the past, I have used the concept of test depth (level 1 being the basic regression tests--higher number reflect additional functionality) so you could run a level one regression on the whole program but do level three on the transport layer "because we've updated the library". T

an automated set of tests would be the most likely way to make 3- a possibility. It is unlikely that with daily builds, as many companies run their build process, that anything short of an automated regression test suite would be able to be run daily with any efficacy. if the builds were weekly, then a manual regression test would be likely.

Answer3:
As per the difinition of regression testing and actual workaround if you have to have answer this question then option 3 & 4 is the best choice among all.The reason behind it is :
3---Regression testing can be run on every build It is a normal phenomenon if there is build coming on weekly basis or it is a RC build.Since,there is nothing mention about daily build ,only thing mention is every build so it can be correct.
4---Regression testing should be targeted areas of high risk and known code change This is also true in most of the situation,it is not universally true but in certain condition where there is code change and the related modules are only tested in regression automation rather than whole code.
5 is not true coz in regression we detect the defect not prevent normally.

How do you execute tests?
Execution of tests is completed by following the test documents in a methodical manner. As each test procedure is performed, an entry is recorded in a test execution log to note the execution of the procedure and whether or not the test procedure uncovered any defects. Checkpoint meetings are held throughout the execution phase. Checkpoint meetings are held daily, if required, to address and discuss testing issues, status and activities.
* The output from the execution of test procedures is known as test results. Test results are evaluated by test engineers to determine whether the expected results have been obtained. All discrepancies/anomalies are logged and discussed with the software team lead, hardware test lead, programmers, software engineers and documented for further investigation and resolution. Every company has a different process for logging and reporting bugs/defects uncovered during testing.
* A pass/fail criteria is used to determine the severity of a problem, and results are recorded in a test summary report. The severity of a problem, found during system testing, is defined in accordance to the customer's risk assessment and recorded in their selected tracking tool.
* Proposed fixes are delivered to the testing environment, based on the severity of the problem. Fixes are regression tested and flawless fixes are migrated to a new baseline. Following completion of the test, members of the test team prepare a summary report. The summary report is reviewed by the Project Manager, Software QA Manager and/or Test Team Lead.
* After a particular level of testing has been certified, it is the responsibility of the Configuration Manager to coordinate the migration of the release software components to the next test level, as documented in the Configuration Management Plan. The software is only migrated to the production environment after the Project Manager's formal acceptance.
* The test team reviews test document problems identified during testing, and update documents where appropriate.
Inputs for this process:
* Approved test documents, e.g. Test Plan, Test Cases, Test Procedures.
* Test tools, including automated test tools, if applicable.
* Developed scripts.
* Changes to the design, i.e. Change Request Documents.
* Test data.
* Availability of the test team and project team.
* General and Detailed Design Documents, i.e. Requirements Document, Software Design Document.
* A software that has been migrated to the test environment, i.e. unit tested code, via the Configuration/Build Manager.
* Test Readiness Document.
* Document Updates.
Outputs for this process:
* Log and summary of the test results. Usually this is part of the Test Report. This needs to be approved and signed-off with revised testing deliverables.
* Changes to the code, also known as test fixes.
* Test document problems uncovered as a result of testing. Examples are Requirements document and Design Document problems.
* Reports on software design issues, given to software developers for correction. Examples are bug reports on code issues.
* Formal record of test incidents, usually part of problem tracking.
* Base-lined package, also known as tested source and object code, ready for migration to the next level.

What is a requirements test matrix?
The requirements test matrix is a project management tool for tracking and managing testing efforts, based on requirements, throughout the project's life cycle.
The requirements test matrix is a table, where requirement descriptions are put in the rows of the table, and the descriptions of testing efforts are put in the column headers of the same table.
The requirements test matrix is similar to the requirements traceability matrix, which is a representation of user requirements aligned against system functionality. The requirements traceability matrix ensures that all user requirements are addressed by the system integration team and implemented in the system integration effort.
The requirements test matrix is a representation of user requirements aligned against system testing. Similarly to the requirements traceability matrix, the requirements test matrix ensures that all user requirements are addressed by the system test team and implemented in the system testing effort.

Can you give me a requirements test matrix template?
For a requirements test matrix template, you want to visualize a simple, basic table that you create for cross-referencing purposes.
Step 1: Find out how many requirements you have.
Step 2: Find out how many test cases you have.
Step 3: Based on these numbers, create a basic table. If you have a list of 90 requirements and 360 test cases, you want to create a table of 91 rows and 361 columns.
Step 4: Focus on the the first column of your table. One by one, copy all your 90 requirement numbers, and paste them into rows 2 through 91 of the table.
Step 5: Now switch your attention to the the first row of the table. One by one, copy all your 360 test case numbers, and paste them into columns 2 through 361 of the table.
Step 6: Examine each of your 360 test cases, and, one by one, determine which of the 90 requirements they satisfy. If, for the sake of this example, test case number 64 satisfies requirement number 12, then put a large "X" into cell 13-65 of your table... and then you have it; you have just created a requirements test matrix template that you can use for cross-referencing purposes.

Part:   1  2  3  4  5  6  7  8  9  10  11  12  13  14   15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47