What is Software Quality?

Software quality is a complex topic that encompasses multiple perspectives. At its core, software quality refers to how well a software product conforms to its requirements and meets the needs of its users. It involves both the software product itself as well as the processes used to develop it. From a software product perspective, quality refers to characteristics like reliability, usability, performance, and security. From a process perspective, it refers to using disciplined methods and best practices that are likely to result in a higher quality product.

Overall, software quality arises from the interplay of people, processes, and technology. Software quality assurance engineers strive to build quality into their products by engaging in activities like requirements analysis, design, coding, testing, and maintenance. Processes like inspections, risk management, and configuration management create a framework for quality. And tools like static analysis, test automation, and metrics provide data to assess and improve software quality on an ongoing basis.

The main challenges faced in ensuring software quality are numerous and span the entire software development lifecycle. Common issues include:

Difficulty in clearly defining requirements – If requirements are unclear, incomplete, or ambiguous, it becomes extremely difficult to determine whether the final software product meets expectations. Vague requirements lead to defects down the line.

Maintaining effective communication with stakeholders – Software quality needs and priorities often change over time as stakeholders gain more understanding of the system under development. Constant communication and collaboration with stakeholders are key to ensure the software developers are keeping pace with evolving needs.

Deviations from specifications – If the software substantially deviates from original specifications, this directly undermines overall quality standards. Any deviations should be formally approved through change management processes.

Architecture and design errors – Flaws in the fundamental architecture and design cascade down into the actual software development process and software application. It becomes very expensive to remedy architectural issues late in development.

Coding errors – Bugs introduced into source code during coding and implementation activities that deviate from coding standards further degrade quality. Unit testing and code reviews aim to catch these issues early.

Non-compliance with current processes/procedures – When schedules slip, teams often ignore defined processes and take shortcuts. But straying from disciplined, proven software processes increases risk.

Inadequate work product reviews and tests – Without sufficient peer reviews, testing tool for functional testing, regression testing, QA testing, and other verification activities, defects persist into final product. Rigorous quality control is essential.

Documentation errors – Incorrect, outdated, or altogether missing documentation makes it difficult for a software engineer to track issues, onboard a QA team, and evolve the system.

Back to Top

Software quality refers to the degree to which software conforms to its requirements and meets the needs of its users. It is formally defined as “the capability of a software product to satisfy stated and implied needs when used under specified conditions.” Another definition states that software quality depends on “the degree to which those established requirements accurately represent stakeholder needs, wants, and expectations.” High quality software meets its requirements, which in turn should accurately reflect stakeholder needs. Quality is about aligning the software with both its formal requirements as well as true user needs.

Back to Top

The terminology around software anomalies can be confusing. It’s important to distinguish the subtle differences between errors, defects and failures.

Error – This is a human mistake made by a QA engineer, software quality assurance analyst, tester or other stakeholder. An example is misunderstanding a requirement and coding to the wrong specification.

Defect – A defect is a flaw or imperfection inserted into a software work product due to an error. This could be a bug in the code or issues with other artifacts like requirements. Defects get inserted when errors are made.

Failure – A failure represents the termination of the software’s ability to function as intended. Failures occur when the software executing encounters a defect. Failures are user-facing; the user experiences the software failing in some unintended way.

An error leads to the insertion of a defect, which in turn can lead to observable software failures upon execution. Engineers aim to prevent errors and remove defects before they turn into failures.

Back to Top

Organizational culture and values have an enormous impact on software quality. Cultural elements that positively influence quality include:

• Excellence is valued over releasing quickly. There is pride in craftsmanship.
• Management provides air cover to do things the right way. Technical debt is minimized.
• Ethics and accountability are emphasized over profits or schedules.
• High quality work is rewarded and recognized. Quality is everyone’s responsibility.
• Learning and improvement is promoted, not a blame culture.
• Collaboration is encouraged both internally and with customers.
• Diversity provides different perspectives on quality.
• Customer satisfaction is seen as a key measure of quality.

Conversely, organizations can also have negative cultural influences:

• Lack of leadership commitment to quality.
• Underinvestment in training, tools, and processes.
• Conflicts between engineering and business goals.
• Poor communication and team dynamics.
• Frequent overtime indicates schedule pressure.
• Lack of transparency.

Organizations that lack a top-down commitment to quality often struggle to deliver reliable products, especially over longer periods of time and multiple release cycles.

Back to Top

Software quality management refers to the oversight, control, and coordination of policies, procedures, activities, and people to achieve quality objectives. Key elements include:

• Quality planning – Defining quality objectives, requirements, targets, and planning of quality assurance activities.
• Quality control – Techniques to measure quality characteristics, review work products, and find defects.
• Quality assurance – Processes and audits to ensure compliance with procedures.
• Quality improvement – Defect analysis and process enhancements to improve quality.
• Resources – Infrastructure, tools, training that enable quality processes.
• Standards – Regulations, models, certifications that guide quality work.
• Culture – Values, behaviors that encourage quality mindset.

Quality management spans the entire software lifecycle and involves various roles like project managers, quality engineers, developers, testers, and customers. When done effectively, it provides confidence that software meets critical quality characteristics.

Conversely, organizations can also have negative cultural influences:

• Lack of leadership commitment to quality.
• Underinvestment in training, tools, processes.
• Conflicts between engineering and business goals.
• Poor communication and team dynamics.
• Frequent overtime indicating schedule pressure.
• Lack of transparency.

Organizations that lack a top down commitment to quality often struggle to deliver reliable products, especially over longer periods of time and multiple release cycles.

Back to Top

Software process improvement (SPI) refers to enhancing the processes involved in software development like requirements, design, coding, testing, and project management. The goal is to make these processes more effective, more efficient, and more mature. SPI builds upon the principle that QA processes yield high quality software work products.

Some common SPI approaches include:

• Focus on early defect prevention over late defect detection.
• Use maturity models like CMMI to benchmark and guide improvements.
• Adopt standards like ISO 9001 for process consistency.
• Embrace frameworks like Lean and Six Sigma.
• Automate processes for efficiency and repeatability.
• Measure processes quantitatively to guide enhancements.
• Continually refine processes via PDCA cycles.
• Improve team collaboration and communication flows.
• Upgrade tools and environments.
• Provide training on processes and best practices.
• Incorporate user feedback into processes.

Effective SPI requires a long-term commitment, not just temporary initiatives. Software organizations must build quality processes that stand the test of time.

Back to Top

Software quality measurement provides data to help assess current quality levels and drive improvement initiatives. Common measures include:

• Error density – errors per size of work product (requirements, design, code). Helps find problem areas.
• Defect density – defects per size of software. Used to gauge release readiness.
• Failure rate – mean time between failures. Tracks system reliability.
• Reliability models – estimate of future failures based on defect data. Predicts field quality.
• CoSQ – cost of software quality analysis. Justifies quality spending.
• Escaped defects – defects missed during development. Assesses testing effectiveness.

Measurements are used to monitor trends, compare benchmarks, predict failures, optimize testing, and prioritize improvements. Statistical analysis like Pareto charts helps interpret the data.

Back to Top

Software quality control techniques systematically find issues in work products. Common techniques include:

• Inspections – Formal, disciplined review of requirements, design, code, etc.
• Testing – Executing software to surface failures.
• Static analysis – Analyzing code without executing it.
• Defect tracking – Recording defects and tracking rework.
• Root cause analysis – Finding systematic root causes of defects.

The data from these techniques helps characterize types of defects and their sources. This enables process changes to prevent recurrence in the future. Quality control provides essential feedback for driving continual improvement.

Back to Top

Software quality assurance (SQA) refers to the set of activities that assess and improve processes and work products to provide confidence that software meets quality requirements and business objectives. Key aspects include:

• Evaluating processes and work products against clearly defined standards.
• Ensuring planned quality practices are fully implemented.
• Auditing and reporting on quality issues and risks.
• Working with an independent perspective free from specific project pressures.
• Coordinating quality assurance activities across the organization.
• Verifying that software meets quality objectives.

SQA goes beyond simple testing to provide broad oversight across the entire software lifecycle. It puts practices and controls in place to build quality software.

Back to Top

Software configuration management (SCM) is the discipline of applying administrative and technical direction to:

• Identify and document product characteristics and versions.
• Control and track changes to software artifacts.
• Maintain integrity and consistency across components.
• Support distributed development and parallel work.
• Report and audit change details.
• Verify compliance with specifications.

SCM provides the process backbone for coordinating work between teams and delivering high quality software products.

Back to Top

To perform comprehensive product assurance, software quality characteristics must be specified in requirements. Key characteristics defined in ISO 25010 include:

• Functional suitability – meets needs under conditions.
• Reliability – operates without failure.
• Performance – speed, response, scalability.
• Usability – ease of use.
• Security – protected from threats.
• Maintainability – ability to make changes.
• Portability – ability to work across environments.

Product assurance verifies that the software exhibits these qualities through reviews, testing, static analysis, and other SQA activities conducted through the lifecycle. Traceability between requirements and work products ensures alignment. Independent product assurance provides added rigor for high-risk software.

Back to Top

Software verification and validation (V&V) refers to confirmation that software work products meet specifications and satisfy stakeholder needs across the entire lifecycle. V&V encompasses:

• Reviews – Evaluate work products against standards.
• Testing – Execute software to detect defects.
• Static analysis – Analyze code without executing it.
• Dynamic analysis – Observe executing software.
• Formal verification – Mathematically prove correctness.

V&V provides concrete evidence that requirements are fulfilled. Testing is an essential V&V activity but insufficient alone. V&V is broad, in-depth quality evaluation applied continuously throughout development and maintenance. Independent V&V brings further objectivity for critical software.

Back to Top

The two main classes of software quality tools are:

Static analysis tools and dynamic analysis tools. Static tools analyze software code, documentation, designs without executing the actual code. They operate on the structure and text of software artifacts. Examples include linting, code quality analysis, and architecture evaluation. In contrast, dynamic analysis tools involve some form of software execution or active modeling. They observe running software to collect quality data. Examples include performance testing, security scanning, and fault injection.

The key distinction is that static tools do not execute code while dynamic tools do. But there is overlap – some tools bridge both categories.

Back to Top

Software quality tools serve a variety of valuable purposes including:

• Automating tedious tasks like testing, reviews, inspections. This boosts team efficiency.
• Performing sophisticated analysis like structural code analysis that is difficult manually. Provides deeper insights.
• Visualizing metrics, trends, and quality data. Improves focus for teams and managers.
• Enforcing standards, best practices, architecture rules. Promotes consistency.
• Supporting distributed teams and code reuse. Improves coordination.
• Reducing risks associated with manual processes like human error, subjectivity, and oversight.
• Capturing quality data to guide improvement initiatives. Provides process feedback.
• Integrating quality practices within IDEs to provide developer support.

Overall, quality tools complement human activities with automation, insight, and efficiency to help teams deliver better software.

Back to Top

Failures in software that controls safety-critical systems like medical devices or autonomous vehicles can lead to hazardous or even life-threatening situations. To thoroughly assess potential software hazards before deployment, techniques like Failure Mode and Effects Analysis (FMEA) and Fault Tree Analysis (FTA) are essential. FMEA systematically analyzes how different ways a system can fail could occur and evaluates the effects on the overall system. Meanwhile, Fault Tree Analysis provides a visual model of the many cascading events and component failures that could ultimately lead to a hazard. While these techniques can be performed manually, specialized safety analysis software tools greatly facilitate consistent, high-quality analysis. The tools provide step-by-step guidance to walk engineers through conducting the analysis correctly.

Comprehensive documentation and reporting features enable teams to produce audit-ready artifacts with details of all failure modes or fault conditions explored. The tools also include data management capabilities for tracking the factors involved in various failure scenarios, enabling traceability back to requirements and system design decisions. Some tools even allow collaborative editing of failure models and trees so that knowledge and expertise from multiple team members can be integrated into a living document. The visual mapping features of the tools also lend clarity to the chains of events leading to hazards. By using these sophisticated tools, organizations can promote rigorous, standardized analysis of how system failures arise and their consequences on safety. Such proactive analysis is invaluable for designing effective mitigations and validating that software controlling critical systems works safely across its expected operating conditions.

Back to Top

Software quality is a multifaceted topic encompassing software products, processes, and people. At its essence, it boils down to building software that meets stakeholder needs and conforms to its requirements. To achieve quality software, organizations must focus both on the end product as well as the means used to get there. Software engineers employ processes, methods, tools and a culture of excellence to build high-quality products. They verify quality via well-rounded evaluations spanning the entire lifecycle. And they continuously improve processes and skills to raise the bar on quality.

The specifics of software quality are complex due to the intricate nature of software itself. Quality practices must be tailored to the domain, technology stack, and software risks involved. But universal foundations like solid requirements, peer reviews, controlled changes, and stakeholder involvement provide a baseline all software teams can build upon. With vigilance and the right mindset, software organizations can make quality a competitive advantage – leading to reduced costs, satisfied users, and minimized risks. The result is software that provides outstanding value and meets the highest standards of excellence.

Read more about software quality in the Software Engineer Book of Knowledge (SWEBOK)