Previous	Next
eBook – content table
The life cycle – context	The quantitative techniques – context

What is the context for qualitative techniques in a methodology

Introduction

Qualitative techniques play a crucial role in unlocking valuable insights. While quantitative methods provide numerical data, qualitative techniques offer a deeper understanding of human behaviour, motivations, and emotions.

When it comes to software testing, having a solid methodology in place is essential for ensuring the quality and reliability of your product. In this blog post, we will explore a range of qualitative techniques that can enhance your software testing methodology and help you deliver a truly exceptional product.

The following qualitative techniques are the foundation of the Bona Iqiniso Methodology for software testing.

• The use of a software testing framework
• The use of a quality framework
• The use of a stakeholder framework
• The use of a quality assurance framework
• The use of a process framework
• The use of a (work) product framework
• The use of an assessment framework
• The use of a continuous improvement framework

The use of a software testing framework

Introduction

If we want to implement software testing in an organization, we need to approach it in a structured and organized way.

A framework refers to a basic structure or set of guidelines that provides a standard way of building or organizing something. In software development, a framework is a set of software tools, libraries, and components that are designed to simplify the development of software applications by providing a standardized structure and set of guidelines.

Frameworks provide a foundation for building software applications by providing pre-built functionality that can be reused across different projects. They help to streamline the development process, reduce the amount of repetitive coding, and improve the quality and consistency of the resulting software.

Frameworks can vary in complexity, size, and purpose. Some frameworks are designed for specific programming languages, while others can be used across multiple languages. Common types of software frameworks include web frameworks, application frameworks, and testing frameworks.

Testing frameworks provide a set of tools and components for testing software applications, including automated testing, unit testing, and integration testing.

Using a software testing framework offers several benefits to organizations and software development teams. Here are some key reasons why using a software testing framework is advantageous:

1. Structure and Organization: A testing framework provides a structured and organized approach to software testing. It offers a set of predefined rules, guidelines, and best practices that help testers plan, design, and execute tests systematically. This structure ensures that testing activities are performed consistently across projects and teams, leading to better efficiency and effectiveness in the testing process.
2. Standardization: A testing framework promotes standardization by establishing a common set of testing practices and methodologies. It ensures that testing activities are conducted uniformly, regardless of the individual testers’ preferences or approaches. This consistency in testing practices enhances collaboration, simplifies knowledge sharing, and enables seamless transitions between team members, thereby increasing productivity and reducing errors.
3. Efficiency and Time Savings: By providing a ready-made structure and a collection of reusable test components, templates, checklists, guidelines, and examples, a testing framework accelerates the test creation and execution process. Testers can leverage the material available, saving time and effort in developing tests from scratch. Additionally, a framework often includes automation capabilities, allowing for the automation of repetitive and time-consuming test tasks, further improving efficiency and productivity.
4. Test Coverage and Confidence: A well-designed testing framework facilitates comprehensive test coverage. It helps testers identify different types of tests, such as functional, non-functional, regression tests, and operational tests and ensures that all relevant aspects of the software are thoroughly examined. By providing guidelines and templates for test creation, the framework helps ensure that critical scenarios and edge cases are not overlooked. The increased test coverage instills confidence in the quality of the software, reducing the risk of critical defects slipping into production.
5. Maintainability and Scalability: Testing frameworks promote maintainability and scalability of test suites. With a standardized framework, when changes occur in the application or system being tested, testers can easily update and adapt the test cases accordingly. The modular nature of frameworks allows for easy addition or removal of test cases, making it easier to maintain and extend the test suite as the software evolves. This flexibility ensures that the testing process remains robust and can accommodate changing project requirements and complexities.
6. Collaboration and Communication: Testing frameworks often include features or integrations that facilitate collaboration and communication among team members. Test management tools, reporting capabilities, and integrations with issue-tracking systems enable effective communication of test results, defects, and progress. This promotes collaboration between testers, developers, and other stakeholders, leading to better coordination, faster issue resolution, and overall improved team efficiency.
7. Compliance and Auditability: For organizations operating in regulated industries or those requiring compliance with specific standards, a testing framework can help ensure adherence to relevant regulations. A framework that aligns with industry standards or regulatory requirements provides documented processes, traceability, and evidence of compliance. This makes it easier for organizations to undergo audits, demonstrate compliance to clients, and meet legal obligations.

In summary, a software testing framework brings structure, standardization, efficiency, improved test coverage, maintainability, collaboration, and compliance to the testing process. By adopting and implementing a testing framework, organizations can enhance the overall quality of their software, reduce risks, and improve the efficiency and effectiveness of their testing efforts.

The use of a quality framework

Introduction

Since an organization wants its software to be of a certain quality standard,  software testing needs to be about quality.

Bona Iqiniso uses the quality reference model as provided by IEEE Std 1061™-1998 (R2004)

(Revision of IEEE Std 1061-1992) – “IEEE Standard for a Software Quality Metrics Methodology” as the basis for its own Quality Reference Model.

Bona Iqiniso’s model can also be used to cover ISO/IEC 9126 – “Software engineering — Product quality” and the ISO/IEC 250xx series – “SQUARE – Software QUality REquirements”

Quality Reference Model

Bona Iqiniso implements these standards in such a way that not only product quality is covered, but quality in all its aspects

• Quality: Quality is defined for an organizational unit. If no organizational unit is specified, the “quality” must be seen as the Bona Iqiniso library of quality also called the Bona Iqiniso Quality Reference Model
• Quality category: Bona Iqiniso categorizes quality in 5 categories: Process, Product, People, Project and Price
• Quality level: Bona Iqiniso uses 4 levels of detail in all its models: Context, Concept, Physical and Verification
• Quality requirement: A requirement, for an organizational unit in a specified quality category and for a specific quality level, is specified with the aim to answer certain questions related to a goal. The quality requirements must be seen as the ‘means’ to a goal, as the motivation to achieve a certain goal.
• Quality attribute: According to ISO/IEC 9126, a quality attribute is defined as a characteristic of software that relates to its ability to satisfy specified requirements. Quality attributes are used to describe various aspects of software quality, such as functionality, reliability, usability, efficiency, maintainability, and portability.
• Quality characteristic: According to ISO/IEC 9126, a quality characteristic is defined as a fundamental property of software that contributes to its overall quality. These characteristics are used to categorize and evaluate the various aspects of software quality. It is  the decomposition of a quality attribute or quality characteristic to its technical components.
• Quality sub-characteristic: ISO/IEC 9126 defines quality sub-characteristics as the underlying attributes that contribute to each of the main quality attributes. These sub-characteristics further break down the main quality attributes into more specific characteristics that can be evaluated.
• Quality measure element: A metric used to quantify a quality characteristic or quality sub-characteristic
• Metric: A metric used to quantify a quality characteristic or quality sub-characteristic (also see ‘the quantitative techniques – context‘)

Move next block to concept

There are 4 major quality categories for software product development:

1. Process quality assesses the quality of the processes used to develop and maintain software products. It focuses on the methods, techniques, and activities employed throughout the software development life cycle to ensure that the resulting software meets its intended objectives. This implies the software testing processes as well.
2. Internal quality is the totality of characteristics of the software product from an internal view (IT/software development view). Internal quality is measured and evaluated against the internal quality requirements. Details of software product quality can be improved during code implementation, reviewing and testing (verification), but the fundamental nature of the software product quality represented by internal quality remains unchanged unless redesigned.
3. External Quality is the totality of characteristics of the software product from an external view (business/internal organization view). It is the quality when the software is executed, which is typically measured and evaluated while testing in a simulated environment with simulated data using external metrics (validation). During testing, most faults should be discovered and eliminated. However, some faults may still remain after testing. As it is difficult to correct the software architecture or other fundamental design aspects of the software, the fundamental design usually remains unchanged throughout testing.
4. Quality in Use is the user’s view (clients, both internal to the organization as well as external) of the quality of the software product when it is used in a specific environment and a specific context of use. It measures the extent to which users can achieve their goals in a particular environment, rather than measuring the properties of the software itself (validation).

No matter the kind of quality category, if there is a need for a certain kind of quality, a software testing methodology will need to accomodate for it.

In summary, software testing is there to verify and validate against the ‘business domain’, all the deliveries of the software product following the roadmap.

The software testing framework provides the foundation to implement process quality, to enable internal (IT) quality, external quality (business domain internal to the organization), and finally quality in use (clients internal and/or external to the organization).This means that the software testing framework covers the full product or software testing life cycle.

The use of a Stakeholder Framework

Introduction

Many factors can influence the way software testing is done in an organization, and it is essential to understand and address these stakeholders to ensure successful software testing.

• Stakeholder influencers can be internal to the organization. E.g. the board of directors wants full transparency and traceability where software testing is concerned.

• Stakeholder influencers can be external to the organization. E.g. The European Digital Operational Resilience Act dictates what is expected of financial institutions regarding software testing, business continuity, disaster recovery, risk management, etc.

• Stakeholders can also be participants influencing the input of a software testing process or the outcome of a software testing process. They even may participate in the software testing process.

Bona Iqiniso sees the stakeholder reference model as a reference base to indicate influences. In that perspective, a more correct name would have been the Influencer reference model, but since stakeholder is a more generally accepted term, the name stakeholder reference model has been chosen. Mentioning this means that the reference model also will include (non-human/organizational) influences which are not that obvious, like regulations, assumptions, company values, etc.

Stakeholder Reference Model

• Influencer: All entities that have an influence on the success of the organizational unit.
• Influence: Categorization of influences. E.g. An assumption, a habit, a regulation, a technology, etc. All will influence what we are doing. Influences are impersonal, meaning they can be shared between multiple stakeholders. Every influence is linked t at least 1 stakeholder.
• Stakeholder: an individual, group, or organization that has a vested interest or concern in the activities, decisions, or outcomes of a particular organization. Stakeholders can include various parties such as employees, managers, customers, suppliers, shareholders, investors, government entities, regulatory bodies, and even the general public. Each stakeholder may have different expectations, needs, and influence over the organization’s operations and success. It is important for organizations to identify and engage with stakeholders to understand their perspectives and incorporate their input in decision-making processes.
• Function: A process or operation that is performed routinely to carry out a part of the mission of an organization. They can be divided into core functions and support functions.
• Role: The usual or expected function of an actor or the part somebody or something plays in a particular action or event. An Actor may have a number of roles. The part an individual plays in an organization and the contribution they make through the application of their skills, knowledge, experience, and abilities.
• Actor: A person, organization, or system that has a role that initiates or interacts with activities; for example, a sales representative who travels to visit customers. Actors may be internal or external to an organization. 

The use of a Quality Assurance Framework

Introduction

A quality assurance framework is a structured and systematic approach or set of guidelines that organizations use to establish and maintain quality standards and practices within their operations. It provides a framework for managing and assuring the quality of products, services, processes, or systems throughout their lifecycle.

Here are some key components and characteristics of a quality assurance framework:

1. Policies and Objectives: A quality assurance framework typically includes documented policies and objectives that outline the organization’s commitment to quality and define its overall quality goals.
2. Quality Standards: The framework incorporates established quality standards, guidelines, or best practices that serve as benchmarks for measuring and ensuring the quality of deliverables. These standards can be industry-specific, regulatory, or internally developed.
3. Processes and Procedures: A quality assurance framework outlines the processes and procedures that need to be followed to achieve and maintain quality. It provides guidance on activities such as quality planning, quality control, quality monitoring, and quality improvement.
4. Documentation and Documentation Control: A quality assurance framework emphasizes the importance of documentation, including quality plans, procedures, work instructions, and records. It establishes mechanisms for documenting and controlling these documents to ensure accuracy, version control, and accessibility.
5. Training and Competence: The framework includes provisions for training and developing the skills and competencies required for maintaining quality standards. It ensures that employees have the necessary knowledge and capabilities to perform their roles effectively.
6. Audits and Inspections: A quality assurance framework incorporates mechanisms for conducting audits and inspections to assess compliance with quality standards, identify non-conformances, and implement corrective actions.
7. Continuous Improvement: A quality assurance framework promotes a culture of continuous improvement by establishing processes for monitoring and evaluating quality performance, analyzing data, and implementing corrective and preventive actions.
8. Accountability and Responsibility: The framework defines roles, responsibilities, and accountability for quality management throughout the organization. It ensures that individuals and teams are responsible for their contributions to quality and that there are mechanisms for addressing quality issues or concerns.
9. Communication and Collaboration: A quality assurance framework emphasizes effective communication and collaboration across different functions and stakeholders. It encourages feedback, knowledge sharing, and cross-functional cooperation to enhance quality outcomes.

Overall, a quality assurance framework provides a structured approach to establish, maintain, and continuously improve quality standards and practices within an organization. It helps ensure that quality is embedded in all aspects of operations, leading to consistent, reliable, and high-quality deliverables or outcomes.

Quality assurance reference model

1. The rationale/motivation
   • Mission: Represents the organization. It includes the mission statement, policies, and strategies. Tactics (also called methods) are covered in 7. Practice.
   • Goals: Includes the vision statement and the goals the organization wants to achieve.
   • Requirements: includes business rules, influences, regulations, and requirements.
   • Evaluation: include the measurable objectives.
2. Qualitative techniques
   • Governance: how to manage the methods/practices
   • Improvement: How to introduce the method step-wise in the organization
   • Follow up: to assess how well we are following processes
   • Institutionalization: Rollout of process in the organization following the continuous improvement setup.
   • Success: What are the targets set by the organization and are they reached?
3. Quantitative techniques
   • Metrics: how to measure the evaluation/objectives.
   • Scoring: consolidation of all metrics (dashboard)
4. Methods
   • Practice: structured description of the process
5. Paradigm
   • Knowledge: terminology, dependencies, starting point, process reference, etc.
   • Guidance: checklists, guidelines, examples, standards, regulations, etc.

What about life cycle?

Quality assurance is valid for the complete life cycle. The practices/methods will define which part of the life cycle.

The use of a Process Framework

Introduction

Organizations use a process framework for several reasons:

1. Consistency and Standardization: A process framework provides a set of predefined processes, practices, and guidelines that help establish consistency and standardization across different projects and teams within an organization. It ensures that everyone follows a common approach, reducing variability and increasing the likelihood of producing consistent and predictable outcomes.
2. Efficiency and Productivity: A well-defined process framework outlines the steps, activities, and tasks required to accomplish specific objectives. By providing a structured approach, it helps teams work more efficiently and productively. The framework can include best practices, templates, and tools that streamline processes, eliminate redundancies, and optimize resource allocation.
3. Compliance and Governance: For organizations operating in regulated industries or those with specific compliance requirements, a process framework helps ensure adherence to relevant standards and regulations. It provides a documented approach that can be audited and validated to demonstrate compliance.
4. Scalability and Flexibility: A process framework can be designed to accommodate different project sizes, complexities, and contexts. It allows organizations to scale their processes based on the specific needs of each project while maintaining a consistent underlying structure. This flexibility ensures that the framework remains applicable and adaptable to varying project requirements.

Overall, a process framework serves as a guiding structure that supports organizations in delivering projects effectively, efficiently, and with a focus on quality. It provides a common language, a shared understanding, and a set of practices that enable successful project execution and continuous improvement.

Process Reference Model

• Process groups: refer to a collection of related processes that are grouped together based on their common objectives and characteristics.
• Process categories: refers to a high-level grouping or classification of related processes within a process framework. It provides a way to organize and categorize different processes based on their common characteristics or objectives. In the context of software development or project management, a process category typically represents a broad area or domain of activities that contribute to the overall project or software development lifecycle. Each process category encompasses a set of processes that are grouped together based on their shared purpose, goals, or outcomes.
• Process: Processes require a purpose and outcome. All processes have at least one activity. The processes, with their statements of purpose and outcomes, constitute a Process Reference Model (See also ‘the methods – context‘).
• Activity: the building blocks of a process and represent the discrete steps or actions required to achieve a particular goal or outcome. Here are some key characteristics of activities in a process:
  • Purpose: Each activity serves a specific purpose within the process. It contributes to the overall objectives of the process and helps in achieving the desired outcomes.
  • Sequence: Activities are typically performed in a predefined sequence or order. The order of activities may depend on dependencies, constraints, or logical flow defined within the process.
  • Input and Output: Activities often require input data, information, or resources to be performed. They also generate output in the form of results, deliverables, or artifacts that are used as input for subsequent activities or as final outputs of the process.
  • Responsibility: Each activity is assigned to one or more individuals or roles responsible for its execution. These individuals are accountable for completing the activity according to the defined requirements and quality standards.
  • Duration and Effort: Activities have an estimated duration or effort associated with them, indicating the time and resources required to complete the activity. This helps in planning and scheduling activities within the overall timeline of the process.
  • Control and Monitoring: Activities can be monitored and controlled to track their progress, identify bottlenecks or issues, and ensure that they are executed according to the defined process guidelines.
  • Iteration: In some processes, activities may need to be repeated or iterated multiple times, either within the same process instance or across different iterations of the process. This allows for refinement, improvement, or adjustment based on feedback or changing requirements.
• Task: refers to a specific activity or work unit that needs to be performed to accomplish a particular objective within the process. It is a discrete and well-defined unit of work that contributes to the overall completion of the process. Tasks are typically identified and defined during process design or planning. They represent the specific actions, steps, or activities that individuals or teams need to execute within the broader process flow. Tasks often have dependencies, sequencing requirements, and input-output relationships with other tasks or process elements. Here are a few key characteristics of tasks within a process:
  • Specificity: Tasks are defined in a clear and unambiguous manner, outlining what needs to be done, by whom, and within what timeframe.
  • Action-Oriented: Tasks represent actionable steps that require effort, resources, and skills to be completed. They often involve performing activities, making decisions, producing deliverables, or interacting with other process participants.
  • Granularity: Tasks can vary in terms of their granularity or level of detail. Some tasks may be high-level and encompass a broader scope, while others may be more detailed and focused on specific actions or sub-activities.
  • Interdependencies: Tasks within a process are often interconnected and dependent on each other. The completion of one task may trigger or enable the start or completion of other tasks within the process.
  • Measurability: Tasks may have associated metrics or criteria that can be used to track progress, monitor performance, or evaluate outcomes. Measuring task completion can provide insights into process efficiency, effectiveness, and quality.
  • Assignability: Tasks are typically assigned to individuals or teams responsible for their execution. Assigning tasks helps ensure accountability and facilitates coordination and collaboration among process participants.
  • Iteration: In iterative or agile processes, tasks may be repeated or revisited in multiple cycles or iterations, allowing for continuous improvement and refinement of outputs.
• Note: a form of additional information or commentary that accompanies a particular step, activity, or section within the process documentation. It serves to provide further clarification, guidance, or context for understanding and executing the process. Here are a few key characteristics of notes in the context of a process:
  • Supplementary Information: A note is used to provide supplementary details or explanations that expand upon the main content of the process. It may provide additional background information, examples, or specific instructions that help users better understand or perform a particular step or activity.
  • Clarification or Contextualization: Notes can be used to clarify any ambiguous or potentially confusing aspects of the process. They provide additional context or insights that help users interpret the process steps accurately and apply them effectively.
  • Optional Information: Notes are typically optional and may not be directly related to the essential steps or activities of the process. They are included to enhance understanding or provide further guidance but are not considered mandatory for executing the process.
  • Format and Placement: Notes are often presented in a distinct format, such as being enclosed in a box, highlighted with a different font or colour, or denoted by a specific symbol (e.g., asterisk or superscript). They are usually placed adjacent to or within the relevant step, activity, or section to which they relate.
  • Authoritative Source: Notes should ideally come from a reliable and authoritative source, such as process owners, subject matter experts, or documentation standards. They should be accurate, up-to-date, and aligned with the overall objectives and requirements of the process.

The use of a (work) product Framework

Inroduction

A work product framework, also known as a deliverable framework or artifact framework, is a structured approach for organizing and categorizing the various work products or deliverables produced during a project or within a specific process. It provides a framework or structure that helps define, manage, and track the different types of work products or artifacts that are generated throughout the life cycle.

Here are some key aspects of a work product framework:

1. Categorization: A work product framework categorizes work products based on their type, purpose, or nature. It defines different categories or groups to classify the work products effectively. Examples of categories could include requirements documents, design artifacts, test cases, project plans, user manuals, and so on.
2. Naming Conventions: A work product framework may include standardized naming conventions for work products to ensure consistency and ease of identification. These conventions help in organizing and retrieving work products efficiently and also facilitate communication and collaboration among project stakeholders.
3. Templates and Formats: The framework often provides templates or predefined formats for each category of work product. These templates establish a consistent structure and layout for the work products, ensuring that essential information is captured and presented uniformly.
4. Documentation Guidelines: A work product framework may include guidelines or instructions for creating and documenting work products. It defines the required content, quality criteria, and any specific guidelines or best practices that should be followed while creating or updating the work products.
5. Version Control and Configuration Management: The framework may incorporate version control and configuration management practices for work products. It establishes mechanisms to track changes, maintain different versions, and ensure proper configuration control of the work products over time.
6. Storage and Access: A work product framework typically includes guidelines on where and how work products should be stored, organized, and made accessible to project stakeholders. It may involve the use of document management systems, repositories, or collaborative platforms to manage the work products effectively.
7. Traceability: The framework may emphasize the importance of traceability, linking work products to related artifacts or requirements. It ensures that the work products can be traced back to their origins and establishes traceability throughout the project or development lifecycle.

The purpose of a work product framework is to bring structure and organization to the creation, management, and use of work products. It helps teams and stakeholders understand the types of deliverables expected, promotes consistency and quality in their creation, and facilitates efficient retrieval and utilization of the work products throughout the project or process.

Work product reference model

Bona Iqiniso sees the work product reference model as a framework to hold products to be delivered to clients, supporting material (work products) in the production of the product(s) and all material (generic products) that exist to support the production of products and work products.

• Product category: The purpose of defining product categories within a work product framework is to provide a structured approach for organizing, managing, and understanding the different types of work products generated throughout a project or process. It facilitates effective communication, collaboration, and retrieval of the work products, supporting project stakeholders in achieving project objectives and ensuring the quality and completeness of deliverables.
• Product all: Something produced by human effort, mechanical effort or a natural process. The totality of goods or services that an organizational unit produces.
• Product: All deliverables created throughout a project or process to be delivered to a client or end-user. It provides a structured approach to managing, tracking, and controlling these products, ensuring their completeness, quality, and relevance to the project objectives.
• A product component: refers to a distinct and self-contained part or element that is part of a larger product or deliverable. It represents a subdivision or module within a product, highlighting its individual functionality.
• Work product: refers to any tangible or intangible item that is produced as a result of a process. Work products are the deliverables or outputs generated during the execution of tasks or activities.
• Artifact: refers to a tangible or intangible item, document, or output produced during the course of a project or within a specific process. It is a specific instance or representation of a work product within a given category or classification.
• Product level of Appliance: Bona Iqiniso classification for generic products. Examples are checklists, guidelines, templates, examples, articles, etc.
• Product generic: refers to a type of work product or deliverable that is not specific to a particular project or organization. It is a standardized template or artifact that can be used across multiple projects or processes with minimal customization. Examples of generic products in a work product framework could include project plan templates, requirements document templates, test case templates, status report templates, design document templates, and more. These generic products provide a starting point for creating project-specific instances of these work products while ensuring consistency, adherence to standards, and the incorporation of best practices.

The use of an assessment framework

Introduction

If an organization chooses to continuously improves its way of working, or tries to keep up with new ways of working, or just tries to be ahead of its competition, organizations must continuously evaluate their performance, strategies, and capabilities to stay ahead. An effective assessment process plays a vital role in this. In this post, we will explore the reasons why Bona Iqiniso is having an assessment framework and why it is essential for organizations:

1. Objective Evaluation and Benchmarking: An assessment framework provides a structured way for objectively evaluating an organization’s current state. It establishes a baseline to assess performance, identify strengths and weaknesses, and benchmark against industry standards or best practices. By having clear evaluation criteria, organizations can gain valuable insights into their operations, enabling them to set realistic goals and measure progress accurately.
2. Identifying Improvement Opportunities: A well-designed assessment framework helps identify improvement opportunities. It enables the identification of inefficiencies, bottlenecks, gaps in processes, or areas where resources are not optimally utilized. By pinpointing these opportunities, organizations can prioritize their efforts, allocate resources effectively, and implement targeted improvements that drive tangible results.
3. Facilitating Strategic Decision-Making: An assessment framework provides a data-driven approach to support strategic decision-making. By collecting and analyzing relevant information, organizations gain a comprehensive view of their strengths, weaknesses, opportunities, and threats.
4. Enhancing Performance and Efficiency: An assessment framework serves as a tool to enhance performance and efficiency across the organization. See ‘Identifying Improvement opportunities’.
5. Ensuring Compliance and Risk Management: Compliance with regulations, industry standards, and internal policies is vital for organizations across various sectors. An assessment framework provides a structured approach to assess compliance levels and identify potential risks, gaps, and improvement opportunities.
6. Continuous Learning and Adaptability: An assessment framework promotes a culture of continuous improvement and this can only be done when there is a culture of learning and adaptability within the organization.
7. Stakeholder Alignment and Transparency: An assessment framework facilitates stakeholder alignment and transparency. It provides a structured mechanism to communicate the organization’s current state, progress, and improvement initiatives to internal and external stakeholders. This transparency builds trust, promotes accountability, and fosters collaboration between different stakeholders, including employees, customers, investors, and regulators.

Bona Iqiniso applies an assessment framework to achieve the above goals for software testing. Therefore every method within the Bona Iqiniso methodology must come with guidelines or at least a checklist on how to assess the method.

The assessment reference model

Bona Iqiniso bases its Assessment reference model on ISO 15504, which is a process assessment reference model.

This model can be extended to a quality assessment reference model because a quality attribute should be referenced by a process and therefore makes it possible to assess the quality attribute.

• Assessment rating: The ordinal rating scale defined below shall be used to express the levels of achievement of the process attributes (also expressed as % of achievement)
  • N  – Not achieved: There is little or no evidence – 0 to 15 % achievement
  • P  – Partially achieved: There is some evidence of an approach to, and some achievement – > 15 % to 50 % achievement
  • L – Largely achieved: There is evidence of a systematic approach to, and significant achievement – >50 % to 85% achievement
  • F – Fully achieved: There is evidence of a complete and systematic approach to, and full achievement- > 85 % to 100 % achievement.
• Assessment process attribute: Capability is based upon a set of process attributes (PA). Each attribute defines a particular aspect of process capability. 
• Assessment capability level: Capability is defined on a six-point ordinal scale that enables the capability to be assessed. The scale represents increasing capability in an implemented process
  • Level 0: Incomplete process,
  • Level 1: Performed process,
  • Level 2: Managed process,
  • Level 3: Established process,
  • Level 4: Predictable process,
  • Level 5: Optimizing process.
• Assessment key process area: a KPA represents a specific area of focus or a critical aspect of an organization’s processes that contributes to its overall maturity and capability. For example, some common KPAs in CMMI for Development (CMMI-DEV) include:
  • Requirements Development (RD): Focuses on establishing, managing, and evolving requirements throughout the project lifecycle.
  • Project Planning (PP): Addresses the planning and management of projects, including estimating, scheduling, and resource allocation.
  • Configuration Management (CM): Deals with the identification, control, and tracking of project artifacts and configurations.
  • Measurement and Analysis (MA): Involves establishing and maintaining measurement processes to collect and analyze data for decision-making and process improvement.
  • Verification (VER): Concentrates on evaluating work products and processes to ensure compliance with requirements and standards.
  • Validation (VAL): Focuses on evaluating the final product or service to ensure it meets customer needs and requirements.
• Assessment maturity level: Maturity is defined on a five-point ordinal scale that enables maturity to be assessed. The scale represents the increasing maturity of the implemented process. Where capability focuses on the goals and practices within one process, maturity focuses on a set of key process areas (KPA’s). To achieve a higher maturity rating, an organization must first satisfy all the KPAs of the preceding maturity level
  • Level 1: Initial,
  • Level 2: repeatable,
  • Level 3: defined,
  • Level 4: managed,
  • Level 5: optimizing
• Assessment: Core entity for assessments
• Assessment base: Checklist linked to a capability level
• Assessment observation: Every assessment results in a number of observations. Each observation has an assessment rating and a category.
• Assessment observation category: Assessment observation categories express:
  • S – Strengths,
  • W – Weaknesses,
  • O – Opportunities,
  • T – Threats
• Assessment recommendation: Every observation must be accompanied by a recommendation that relates to the observation category. A strength will have a sustain recommendation, a weakness has an improvement recommendation, an opportunity has a change recommendation and a threat has a solution recommendation.

The use of a continuous improvement framework

Introduction

A continuous improvement framework provides a structured approach for organizations to achieve ongoing enhancement, learning, and growth. A continuous improvement framework brings following benefits:

1. Embracing Change and Innovation: A continuous improvement framework enables organizations to proactively embrace change and stay ahead of the curve.

2. Driving Efficiency and Effectiveness: A continuous improvement framework helps organizations streamline operations, eliminate inefficiencies, and maximize productivity. By analyzing existing processes and practices, organizations can identify bottlenecks, redundant activities, and areas for improvement.

3. Enhancing Customer Satisfaction: A continuous improvement framework enables organizations to better understand customer needs, preferences, and expectations. By actively seeking feedback and monitoring customer experiences, organizations can identify areas where improvements can be made to deliver superior products and services.

4. Learning and Development: A continuous improvement framework needs an organization to be a learning organization. To encourage its employees at all levels to seek out opportunities for improvement, share knowledge, and engage in continuous learning. This not only enhances individual skills and competencies but also promotes a sense of ownership, engagement, and commitment to the organization’s success.

5. Empowering Employees and Boosting Morale: A continuous improvement framework empowers employees by giving them a voice, but not only employes. Any stakeholder inside and utside the organization has a voice and a stake in the organization’s success. This makes clients more loyal to the organization’s products and employees are encouraged to contribute ideas, provide feedback, and actively participate in improvement initiatives, it boosts their morale, job satisfaction, and overall engagement.

6. Ensuring flexibility: A continuous improvement framework also allows to quickly react on market changes, technology changes, and grows a mindset of resilience and flexibility.

Continuous improvement reference model

• Improvement process area: process area is a cluster of related practices in an area that, when implemented collectively, satisfies a set of goals considered important for making improvement in that area.
  • COBIT
  • CMMi defines 22 process areas: Causal Analysis and Resolution (CAR), Configuration Management (CM), Decision Analysis and Resolution (DAR), Integrated Project Management (IPM), Measurement and Analysis (MA), Organizational Process Definition (OPD), Organizational Process Focus (OPF), Organizational Performance Management (OPM), Organizational Process Performance (OPP), Organizational Training (OT), Product Integration (PI), Project Monitoring and Control (PMC), Project Planning (PP), Process and Product Quality Assurance (PPQA), Quantitative Project Management (QPM), Requirements Development (RD), Requirements Management (REQM), Risk Management (RSKM), Supplier Agreement Management (SAM), Technical Solution (TS), Validation (VAL), Verification (VER).
  • TPI Next defines 16 process (key) areas: Stakeholder commitment, Degree of involvement, Test strategy, test Organization, Communication, Reporting, Test process management, Estimating and planning, Metrics, Defect Management, Testware management, Methodology Practice, Tester Professionalism, Test case design, Test tools, Test environments∙ 
• Improvement process area component: components of each process area. These components can be:
  • Required:  essential to achieving process improvement in a given process area.
  • Expected: describe the activities that are important in achieving a required component.
  • Informative: ∙ help model users understand required and expected components 
• Improvement maturity:
• Maturity levels, which belong to a staged representation, apply to an organization’s process improvement achievement across multiple process areas. These levels are a means of predicting the general outcomes of the next project undertaken. 
  • CMMi:
    • Level 1: Initial,
    • Level 2: Managed,
    • Level 3: Defined,
    • Level 4: Quantitatively Managed,
    • Level 5: Optimizing 
  • TPI Next:
    • Level1: Initial,
    • Level 2: Controlled,
    • Level 3: Efficient,
    • Level 4: Optimizing.
• Improvement capability: Capability levels, which belong to a continuous representation,apply to an organization’s process improvement achievement in individual process areas. These levels are a means for incrementally improving the processes corresponding to a given process area.  
  • ∙ CMMi 1.3:
    • Level 0: Incomplete process,
    • Level 1: Performed process,
    • Level 2: Managed process,
    • Level 3:Defined process
• Improvement: Initiative to increase capability and/or maturity in 1 or more process areas.
• Improvement base: Reference base for improvements. The improvement base represents the relationship between maturity and capability and the organization’s expectations of an improvement.
• Improvement goal: An improvement goal is what is considered as required in an improvement process area (component). 
• Improvement goals – generic: Generic goals are called “generic” because the same goal statement applies to multiple process areas. A generic goal describes the characteristics that must be present to institutionalize processes that implement a process area
• Improvement goals – specific: A specific goal describes the unique characteristics that must be present to satisfy a particular process area
• Improvement practice: A practice is the description of an activity that is considered important in achieving the associated goal.
• Improvement practice – generic: Generic Practices are called “generic” because the same practices apply to multiple process areas. The generic practices associated with a generic goal describe the activities that are considered important in achieving the generic goal and contribute to the institutionalization of the processes associated with a process area.
• Improvement practice – specific: A specific practice is the description of an activity that is considered important in achieving the associated specific goal. The specific practices describe the activities that are expected to result in achievement of the specific goals of a process area.
• Improvement information: Improvement Information are artifacts that help to understand improvement practices. This can be example boxes, detailed explanations, or other helpful information. Subpractices, notes, references, goal titles, practice titles, sources,example work products, and generic practice elaborations are informative model components. 

Conclusion

Software testing qualitative techniques are used in the software testing life cycle (inhererent to a methodology), fully covered by a software testing framework that accomodates quality needs coming from internal and external influencers.

Previous	Next
eBook – content table
The life cycle – context	The quantitative techniques – context