Every software development company focuses on developing quality software. The only way to track the software quality is evaluating it at every stage of its development. It requires some kind of metrics, which is obtained through effective testing methods. Each stage of software testing is effectively monitored for the software QA. 1. Software measurements are used for: 2. Deriving basis for estimates 3. Tracking project progress 4. Determining (relative) complexity 5. Understanding the stage of desired quality 6. Analyzing defects 7. Validating best practices experimentally Here, some software testing metrics are proposed for black box testing that has real world applications. It discusses: Importance of software testing measurement Different techniques/processes for measuring software testing Metrics for analyzing testing Methods for measuring/computing the metrics Advantages of implementing these metrics These metrics helps in understanding the inadequacies in different software QA stages and finding better correcting practices. What is measurement and why it is required? The process of assigning numbers or symbols to attributes of real world entities for describing them according to defined rules is called measurement. For developing quality software, several characteristics like requirements, time and effort, infrastructural cost, requirement testability, system faults, and improvements for more productive resources should be measured. Measuring software testing is required: 1. If the available test cases cover all the system’s aspects 2. For tracking problems 3. For quantifying testing Choose the suitable metrics: Several metrics can measure software-testing process. Here, the following types of metrics are identified: Base metrics: These raw data are collected in a testing effort and applied in formulae used to derive Calculated Metrics. The Test Metrics comprise of the Number of Test Cases Passed, Failed, Under Investigation, Blocked, Re-executed and Test Execution Time. Calculated metrics: They convert the Base Metrics data into useful information. Every test efforts must implement the following Calculated Metrics: % Complete % Defects Corrected % Test Coverage % Rework % Test Cases Passed & Blocked % Test Effectiveness & Efficiency % 1st Run Failures % Failures Defect Discovery Rate Defect Removal Cost Measurements for Software Testing The corresponding software testing process in software development measures each step for ensuring quality product delivery. 1. Software Size: The amount of functionality of an application determines this and is calculated by Function Point Analysis Task Complexity Estimation Methodology 2. Requirements review: Before software development, the Software requirement specifications (SRS) from the client are obtained. It must be: Complete Consistent Correct Structured Ranked Testable Traceable Unambiguous Validate Verified The Review Efficiency is a metric that offers insight on the review quality and testing. Review efficiency=100*Total number of defects found by reviews/Total number of project defects 3. Effectiveness of testing requirements: It is measured by maintaining Requirement Trace-ability matrix and specification of requirements, which should have: SRS Objective, purpose Interfaces Functional Capabilities Performance Levels Data Structures/Elements Safety Reliability Security/Privacy Quality Constraints & limitations Next comes the updating of the crucial requirement trace-ability matrix or RTM, which determines the number and types of tests. While measuring the mapping of test cases, the number and priority of requirement it tests, its execution effort and requirement coverage must be determined. The Requirement compliance factor (RCF) measures the coverage provided by the test cases to one or set of requirement(s). Mathematically, RCFj=∑(Pi*Xi)/(maxXi)*(∑Pi)i=1 Where, j is a set of requirements and (j=1-m); Xi=2, if the test case (say Tj) tests requirements Ri completely, =1, if it tests partially, =0, if otherwise. Effectiveness=RCFj/Ej where Ej=Time required for executing a test case 4. Evaluating estimation accuracy Relative error=(A-E)/A where E is estimate of a value and A is actual value. For a collection of estimates, the mean RE for n projects is __ n RE=1/n∑REi i=1 For a set of n projects, the mean magnitude of RE (MRE) is ___ n MRE=1/n∑MREi i=1 Of a set of n projects, an acceptable level for MRE is less than 0.25. If K is the number of projects whose mean magnitude of relative error is less than or equal to q, then the prediction quality pred(q)=K/n 5. Measurement of Efficiency in testing process In software testing, we must keep tabs on what we had planned and what we have actually achieved for measuring efficiency. Here, the following attributes play major roles: - Cost: The Cost Variance (CV) factor measures the risk associated with cost. CV=100*(AC – PC)/PC, AC=Actual Cost, PC=Planned/Budgeted Cost. Effort: Effort Variance (EV) measures effort. EV=100*(AE – PE)/PE (AE=Actual Effort, PE=Planned Effort) Schedule: Schedule Variance (SV) is important for project scheduling. SV=100*(AD-PD)/PD where AD=Actual duration and PD=Planned duration. Cost of quality: It indicates the total effort expended on prevention, appraisal and rework/failure activities versus all project activities. Prevention Effort=Effort expended on planning, training and defect prevention. Appraisal Effort=Effort expended on quality control activities. Failure effort=Effort expended on rework, idle time etc. COQ=100*(PE + AE + FE)/Total project effort. Product - Size variance: It is the degree of variation between estimated and actual sizes. Size Variance=100*(Actual Software Size–Initial Estimated Software Size)/Initial Estimated Software Size Defect density: It is the total number of defects in software with respect to its size. Defect density=Total number of defects detected/software size Mean Time Between Failures: MTBF is the mean time between two critical system failures or breakdowns. MTBF=Total time of software system operation/Number of critical software system failures. Defects: Defects are measured through: Defect distribution: It indicates the distribution of total project defects. Defect Distribution=100*Total number of defects attributed to the specific phase/Total number of defects. Defect removal effectiveness: Adding the number of defects removed during the phase to the number of defects found later approximates this. Benefits of implementing metrics in software testing: Improves project planning. Understanding the desired quality achieved. Helps in improving the processes followed. Analyzing the associated risks. Improving defect removal efficiency.
ReadyTestGo is a professional (http://www.readytestgo.com/readytestgo.htm) Software Testing Company (http://www.readytestgo.com/services.htm) Outsourcing QA . For more details, please contact marketing@readytestgo.com
