[ISTQB-TA]Chap 4

Intro

  - about the quality characteristics which may be evaluated by a Test Analyst.

  - While the Test Analyst may not be responsible for the quality characteristics that require  a more technical approach, it is important that the Test Analyst be aware of the other characteristics 

and understand the overlap areas for testing.

 

There are two quality characteristics that are a part of the TA

  - Functional testing

    Attributes: Accuarcy / Suitability / Interoperability

  - Non-Functional testing

    Attirbutes: Usability / Accessiblity

 

 

 

Accuracy Testing

  - test the application's adherence to the specified or implied requirements and may also include computational accuracy.

  - Accuracy testing can be conducted at any stage in the lifecycle and is targeted at incorrect handling of data or situations.

 

 

Suitability Testing

  - evaluating and validating the appropriateness of a set of functions for its intended specified tasks.

  - This testing can be based on use cases.

  - usually conducted during system testing, but may also be conducted during the later stages of integration testing.

  

 

 

Interoperability Testing

  - tests the degree to which two or more systems or components can exchange information 

and subsequently use the information that has been exchanged.

  - may be limited to a selected representative group of environments.

  - good interoperability characteristics can be integrated with a number of other systems without requiring major changes. 

  - The number of changes and the effort required to perform those changes may be used as a measure of interoperability.

  - Examples:

    Use of industry-wide communications standards, such as XML

    Ability to automatically detect the communications needs of the systems it interacts with and adjust accordingly

  - significant for organizations developing Commercial Off The Shelf (COTS) software

  - performed during component integration and system testing

  - Techniques such as decision tables, state transition diagrams, 

use cases and combinatorial testing are all applicable to interoperability testing.

 

 

Usability Testing

  - testing tests the ease by which users can use or learn to use the system 

to reach a specified goal in a specific context.

  - Usability testing is directed at measuring the following:

 

 

  - Attributes that may be measured include: (Attributes that Usability has)

 

 

  - Usability testing is usually conducted in two steps:

 

 

  - Usability tester skills should include expertise or knowledge in the following areas:

    1. Sociology

    2. Psychology

    3. Conformance to national standards

    4. Ergonomics

 

Conducting Usability Tests

  - Many usability tests may be executed by the Test Analyst as part of other tests, 

for example during functional system test.

  - It is very important to have the verifiable usability specifications defined in the requirements 

as well as to have a set of usability guidelines that are applied to all similar projects.

 

 

Usability Test Specification

  - Principal techniques for usability testing are:

 

 

 

1) Inspecting, evaluating, or reviewing

  - cost effective by finding problems early.

  - Heuristic evaluation (systematic inspection of a user interface design for usability) can be used to find the usability problems in the design so that they can be attended to as part of an iterative design process.

  - Reviews are more effective when the user interface is more visible.

 

2) Dynamically interacting with prototypes

  - When prototypes are developed, the Test Analyst should work with the prototypes and help the developers evolve the prototype by incorporating user feedback into the design.

  - In this way, prototypes can be refined and the user can get a more realistic view of how the finished product will look and feel.

 

3) Verifying and validating the actual implementation

  - test may be developed as usability test scenarios.

  - these test scenarios measure specific usability characteristics,

such as learnability or operability, rather than functional outcomes.

  - Test scenarios for usability may be developed to specifically test syntax and semantics.

    Test Syntax: the structure or grammar of the interface

    Semantics: the meaning and purpose of the interface

  - Black box testings are sometimes used in usability testing(ex_ use case testing)

  - Test scenarios for usability testing also need to include the following:

    1. user instructions

    2. allocation of time for pre- and post-test interviews

    3. feedback and an agreed protocol for conducting the sessions.

       The protocol includes a description of how the test will be carried out, timings, note-taking and

       session logging, and the interview and survey methods to be used.

 

 

4) Conducting surveys and questionnaires

  - may be applied to gather observations and feedback regarding user behavior with the system.

  - Standardized and publicly available surveys such as SUMI (Software Usability Measurement Inventory) and 

WAMMI (Website Analysis and MeasureMent Inventory) permit benchmarking against a database of previous 

usability measurements.

  - In addition, since SUMI provides concrete measurements of usability, this can 

provide a set of completion/acceptance criteria.

 

 

Accessibility Testing

  - consider the accessibility to software for those with particular needs or restrictions for its use.

  - This includes those with disabilities(disabled)

  - should consider the relevant standards such as: 

    the Web Content Accessibility Guidelines

    legislation such as Disability Discrimination Acts (UK, Australia), Section 508 (US).

  - Accessibility, similar to usability, must be considered during the design phases.

  - Testing often occurs during the integration levels and continues through system testing and into the acceptance testing levels.

  - Defects are usually determined when the software fails to meet the designated regulations or standards defined for the software.