Software Quality Assurance SEII - Lecture 17

Software Quality AssuranceSEII-Lecture 17Dr. Muzafar KhanAssistant ProfessorDepartment of Computer ScienceCIIT, Islamabad.RecapSoftware reviewsCost impact of software defectsDefect amplification modelReview metrics and their usePreparation effort (Ep), assessment effort (Ep), Rework effort (Er), work product size (WPS), minor errors found (Errminor), major errors found (Errmajor)Formal and informal reviewsReview meeting, review reporting and record keeping, review guidelines2Elements of Software Quality Assurance [1/3]StandardsIEEE, ISO, and other organizationsVolunteer / imposedSQA job is to confirm itReviews and auditsQuality control activityIntended to uncover errors and follow guidelinesTestingKey activityTo find errorsProper planning and execution3Elements of Software Quality Assurance [2/3]Error/defect collection and analysisBetter understanding of errorsChange managementContinuous changesChanges may lead to confusionEducationEducate project teamsSoftware process improvement4Elements of Software Quality Assurance [3/3]Vendor managementShrink-wrapped packages, tailored shell, and contracted softwareQuality guidelines for vendorSecurity managementCyber crimesPrivacy regulationsSafetyDifferent domainsRisk management5SQA TasksPrepare SQA plan for a projectParticipates in the development of the project’s software process descriptionReview software engineering activities to verify compliance with the defined software processAudits designated software work products to verify compliance with those defined as part of the software processEnsures that deviations in software work and work products are documented and handled according to a documented procedureRecords any noncompliance and reports to senior management6Goals, Attributes, and Metrics Requirements qualityDesign qualityCode qualityQuality control effectiveness7Requirements Quality AmbiguityNumber of ambiguous modifiers e.g. many, large etc.CompletenessNumber of TBA, TBDUnderstandabilityNumber of sections/subsectionsVolatilityNumber of changes per requirementTime (by activity) when change is requestedTraceabilityNumber of requirements not traceable to design/codeModel clarityNumber of UML models, descriptive pages per model8Design QualityArchitectural integrityExistence of architectural modelComponent completenessNumber of components that trace to architectural modelComplexity of procedural designInterface complexityAverage number of pick to get to a typical function or contentLayout appropriatenessPatternsNumber of patterns used9Code QualityComplexityCyclomatic complexityMaintainabilityDesign factors e.g. cohesion, coupling etc.UnderstandabilityPercent internal componentsVariable naming conventionsReusabilityPercent reused componentsDocumentationReadability index10Quality Control EffectivenessResource allocationStaff hour percentage per activityCompletion rateActual VS budgeted completion timeReview effectivenessReview metricsTesting effectivenessNumber of errors found and criticalityEffort required to correct an errorOrigin of error11Statistical Quality AssuranceSoftware errors are collected and categorizedTracing underlying cause of each errorUsing the Pareto principle, isolate the 20 percent causesOnce causes are identified, correct the problems“Spend your time focusing on things that really matter, but first be sure that you understand what really matters.”12Possible Causes of Errors [1/2]Incomplete or erroneous specifications (IES)Misinterpretation of customer communication (MCC)Intentional deviation from specifications (IDS)Violation of programming standards (VPS)Error in data representation (EDR)Inconsistent component interface (ICI)13Possible Causes of Errors [2/2]Error in design logic (EDL)Incomplete or erroneous testing (IET)Inaccurate or incomplete documentation (IID)Error in programming language translation of design (PLT)Ambiguous or inconsistent human computer interaction (HCI)Miscellaneous (MIS)14Example15Figure source: Software Engineering: A Practitioner’s Approach, R. S. Pressman, 7th ed., p. 440Six SigmaMotorola in 1980sStatistical analysis of data to measure and improve operational performance by identifying and eliminating defectsSix standard deviations – 3.4 instances (defects) per million occurrences Three core stepsDefine customer requirements, deliverables, and project goalsMeasure the existing process and its outputAnalyze defect metrics and determine the major causesTwo additional stepsImprove and control processSometimes it is called DMAIC16Software Reliability"the probability of failure-free operation of a computer program in a specified environment for a specified time" Example: reliability of 0.999 over eight elapsed processing hoursFailure / nonconformance / annoying / re-work of weeks17Measures of Reliability and AvailabilityPrediction of software reliabilityHardware: failure due to (physical) wear rather than design defectsSoftware: design defectsMean time between failure (MTBF)MTBF = MTTF + MTTRAvailabilityMTTF/(MTTF + MTTR) * 100%18Software SafetyIdentification and assessment of potential hazardsSafety-related requirements can be specifiedList of undesirable eventsThe desired system responsesDifference between software reliability and software safety19SummaryElements of software quality assuranceStandards, reviews and audits, testing, error collection and analysis, change management, education, vendor management, security management, safety, risk managementSQA tasksGoals, attributes, metricsRequirements quality, design quality, code quality, quality control effectivenessStatistical quality assuranceSoftware reliability20