|
Software QA/Testing Technical FAQs
Part:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
When do we prepare a Test Plan?
[Always prepared a Test Plan for every new version or release of the product? ]
For four or five features at once, a single plan is fine. Write new test cases rather than new test plans.
Write test plans for two very different purposes. Sometimes the test plan is a
product; sometimes it's a tool.
What is boundary value analysis?
Boundary value analysis is a technique for test data selection. A test engineer chooses values that lie along data extremes. Boundary values include maximum, minimum, just inside boundaries, just outside boundaries, typical values, and error values. The expectation is that, if a systems works correctly for these extreme or special values, then it will work correctly for all values in between. An effective way to test code is to exercise it at its natural boundaries.
Boundary Value Analysis is a method of testing that complements equivalence partitioning. In this case, data input as well as data output are tested. The rationale behind BVA is that the errors typically occur at the boundaries of the data. The boundaries refer to the upper limit and the lower limit of a range of values or more commonly known as the "edges" of the boundary.
Describe methods to determine if you are testing an application too
much?
Answer1:
While testing, you need to keep in mind following two things always:
-- Percentage of requirements coverage
-- Number of Bugs present + Rate of fall of bugs
-- Firstly, There may be a case where requirement is covered quite adequately but number of bugs do not fall. This indicates over testing.
--- Secondly, There may be a case where those parts of application are also being tested which are not affected by a CHANGE or BUG FIXTURE. This is again a case of over testing.
-- Third is the case as you have suggested, with slight modification, i.e bug has sufficiently dropped off but still testing is being at SAME levels as before.
Methods to determine if an application is being over-tested are--
1. Comparison of 'Rate of Drop in number of Bugs' & 'Effort
Invested in Testing' (With all Requirements been met)
That is, if bug rate is falling (as it generally happens in all
applications), but effort invested in man hours does not fall, this
implies Over testing.
2. Comparison of 'Achievment of bug rate threshold' & 'Effort
Invested in Testing' (With all Requirements been met)
That is, if bug rate has already achieved the agreed-upon value
with business and still the testing efforts are being invested with
no/little reduction.
3. Verifying if the 'Impact Analysis' for 'Change Requests' has
been done properly and being implemented correctly.
That is, to check and verify that the components of AUT which have
got impacted by the new change are being tested only and no other
unrequired component is being tested unneccessarily. If unaffected
components are being tested, this implies Over testing.
Answer2:
If the bug find rate has dropped off considerably, the test group
should shift its testing strategy. One of the key problems with heavy
reliance on regression testing is that the bug find rate drops off even
though there are plenty of bugs not yet found. To find new bugs, you
have to run new tests.
Every test technique is stronger for some types of bugs and weaker for
others. Many test groups use only a few techniques. In our consulting,
James Bach and I repeatedly worked with companies that relied on only
one or two main techniques.
When one technique, any one test technique, yields few bugs, shifting
to new technique(s) is likely to expose new problems.
At some point, you can use a measure that is only partially statistical
-- if your bug find rate is low AND you can't think of any new testing
approaches that look promising, THEN you are at the limit of your
effectiveness and you should ship the product. That still doesn't mean
that the application is overtested. It just means that YOU'RE not going
to find many new bugs.
Answer3:
Best way is to monitor the test defects over the period of time
Refer williams perry book, where he has mentioned the concept of 'under test' and 'over test', in fact the data can be plotted to see the criteria.
Yes one of the criteria is to monitor the defect rate and see if it is almost zero
second method would be using test coverage when it reach 100% (or 100% requirement coverage)
Procedural Software Testing Issues
Software testing in the traditional sense can miss a large number of errors if used alone. That is why processes like Software Inspections and Software Quality Assurance (SQA) have been developed. However, even testing all by itself is very time consuming and very costly. It also ties up resources that could be used otherwise. When combined with inspections and/or SQA or when formalized, it also becomes a project of its own requiring analysis, design and implementation and supportive communications infrastructure. With it interpersonal problems arise and need managing.
On the other hand, when testing is conducted by the developers, it will most likely be very subjective. Another problem is that developers are trained to avoid errors. As a result they may conduct tests that prove the product is working as intended (i.e. proving there are no errors) instead of creating test cases that tend to uncover as many errors as possible.
Part:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
| 