Metrics PLAN

• Measure – quantitative indication of extent, amount, dimension, capacity, or size of some attribute of a product or process. E.g. Number of errors
• Metric – quantitative measure of degree to which a system, component or process possesses a given attribute. “A handle or guess about a given attribute.” E.g. Number of errors found per person hours expended

Why Measure Software ?

• Determine the quality of the current product or process
• Predict qualities of a product/process
• Improve quality of a product/process

Motivation for Metrics

• Estimate the cost & schedule of future projects
• Evaluate the productivity impacts of new tools and techniques
• Establish productivity trends over time
• Improve software quality
• Forecast future staffing needs
• Anticipate and reduce future maintenance needs

Example Metrics

• Defect rates
• Error rates
• Measured by:
  • individual
  • module
  • during development
• Errors should be categorized by origin, type, cost

Metric Classification

• Products
  • Explicit results of software development activities
  • Deliverables, documentation, by products
• Processes
  • Activities related to production of software
• Resources
  • Inputs into the software development activities
  • hardware, knowledge, people

Product vs. Process

• Process Metrics
  • Insights of process paradigm, software engineering tasks, work product, or milestones
  • Lead to long term process improvement
• Product Metrics
  • Assesses the state of the project
  • Track potential risks
  • Uncover problem areas
  • Adjust workflow or tasks
  • Evaluate teams ability to control quality

Types of Measures

• Direct Measures (internal attributes)
  • Cost, effort, LOC, speed, memory
• Indirect Measures (external attributes)
  • Functionality, quality, complexity, efficiency, reliability, maintainability

Size-Oriented Metrics

Size of the software produced

• LOC – Lines Of Code
• KLOC – 1000 Lines Of Code
• SLOC – Statement Lines of Code (ignore whitespace)

Typical Measures:
–Errors/KLOC, Defects/KLOC, Cost/LOC, Documentation Pages/KLOC

LOC Metrics

• Easy to use
• Easy to compute
• Language & programmer dependent

Complexity Metrics

• LOC – a function of complexity
• Language and programmer dependent
• Halstead’s Software Science (entropy measures)
  • n1 – number of distinct operators
  • n2 – number of distinct operands
  • N1 – total number of operators
  • N2 – total number of operands

Measures of Software Quality

• Correctness – degree to which a program operates according to specification
  • Defects/KLOC
  • Defect is a verified lack of conformance to requirements
  • Failures/hours of operation
• Maintainability – degree to which a program is open to change
  • Mean time to change
  • Change request to new version (Analyze, design etc)
  • Cost to correct
• Integrity – degree to which a program is resistant to outside attack
  • Fault tolerance, security & threats
• Usability – easiness to use
  • Training time, skill level necessary to use, Increase in productivity, subjective questionnaire or controlled experiment

Metric tools

• Mc Cabe & Associates ( founded by Tom McCabe, Sr.)
  • The Visual Quality ToolSet
  • The Visual Testing ToolSet
  • The Visual Reengineering ToolSet
• Metrics calculated
  • McCabe Cyclomatic Complexity
  • McCabe Essential Complexity
  • Module Design Complexity
  • Integration Complexity
  • Lines of Code
  • Halstead

Project Metrics are means to measure/track process, performance and achievements based on project data captured over the period of project life cycle.
Quality Assurance Metrics measure process/performance for a project based on QA Activities done for project

• Metrics are driven by Main Process and Sub Process Goals for a project

Key Metrics for DW Testing Projects :
Test Planning Metrics

1. DW Schedule Deviation
2. DW Effort deviation

Test Case Design/Development Metrics

1. DW Test Coverage
2. DW Requirements Coverage

Test Execution Metrics

1. DW Test Case Efficiency
2. DW Defect Detection Rate
3. DW Defect Rejection Ratio
4. DW Defect Slippage Ratio