A Model Based Approach To Evaluating and Improving Mobile User Experience was the first article of Philip Lew that introduced the context that creates the need for a model-based approach to evaluate mobile user experience. This article presents the concepts of the ISO 25010 model and 2Q2U (Quality, Quality in use, actual Usability and User experience).
Author, Philip Lew, Ph.D., CEO of XBOSoft, http://www.xbosoft.com/
In our previous article in this series, A Model Based Approach To Evaluating and Improving Mobile User Experience, we discussed the need for a model for evaluating and improving Mobile User eXperience (MUX). Before we dive into a model, let’s pose some questions that a model should help us to answer and solve:
- What do “horrible usability” and “better UX” mean?
- What is the relationship between Usability and UX? Are they synonyms?
- Evaluating the success rate of users completing tasks correctly. Is this the same as or directly related to UX?
- If users are highly effective in completing tasks but they mostly get unsatisfied, then does UX score still high?
- Does UX depend on application Usability only or also from other characteristics such as Functional and Information Quality, Security, Reliability, and Efficiency?
- Is UX a quality characteristic of the software itself or of an application in use? And what about Usability?
ISO 25010 [1] provides a good starting point and as put forth in the standard, it is designed to be flexible depending on the needs and context of the organization. First, let’s cover some concepts and definitions of the model that we can apply to our efforts in modeling mobile user experience. ISO 25010 outlines a flexible model with product/system quality –also known as internal and external quality (EQ)-, and system-in-use quality –also referred to as quality in use (QinU). Product quality refers to attributes that can be evaluated with the app in execution state both in testing and in operative stages; while QinU quality refers to what can be evaluated by end users when actually executing tasks in a real context of use. As mentioned earlier in the last article, the mobile application context of use is one of the primary reasons that UX is harder to evaluate and achieve. So, QinU deserves special attention for mobile applications.
ISO 25010 also delineates a relationship whereby product quality ‘influences’ QinU and QinU ‘depends on’ product quality. This makes intuitive for instance if the buttons are placed in certain locations (a product attribute), this will ‘influence’ a user’s productivity and experience if they can’t find the buttons. Furthermore, usability is a product quality characteristic, while Effectiveness, Efficiency and Satisfaction are QinU characteristics (see figure 1).
Figure 1. ISO 25010 quality views
However, Actual Usability and User Experience are missing concepts in ISO 25010 standard. To bridge this gap, in previous research, we used the ISO 25010 as a starting point, and developed 2Q2U (Quality, Quality in use, actual Usability and User experience) v2.0 [2 and 3], which ties together all of these quality concepts by relating product quality characteristics with Actual Usability (part of QinU rather than Product quality) and UX as experienced by the end user. The purpose of the 2Q2U quality framework is to instantiate the quality characteristics to evaluate and conduct a systematic evaluation using the ‘depends’ and ‘influences’ relationships. Relevant features of mobile applications with regard to Usability and UX in the light of 2Q2U v2.0 quality models [4] include:
- Typing/input: which includes search bars, and other data entry fields whereby the users should be assisted as much as possible to reduce errors and the ‘cost’ of typing. This includes such measurable attributes as default values, default value removal and shortcuts.
- Entry widgets such as carousels, drop down boxes and lists. System designers need to prevent the need for typing and reduce error rates by using widgets.
- Sort, search and filter: Special considerations are needed for mobile applications in order to reduce the workload and typing input. In addition, the small screen size makes it easy for the user to lose context, so attributes like typo tolerance, predictive contextual help would be desirable.
Going forward to model these mobile applications’ usability from a product quality perspective, the table below shows a condensed version of 2Q2U with characteristics which can then be decomposed further depending on the domain and product goals.
| Characteristic/Attribute | 2Q2U v2.0 Definition |
| 1 Usability | Degree to which the product or system has attributes that enable it to be understood, learned, operated, error protected, attractive and accessible to the user, when used under specified conditions. |
| 1.1 Understandability | Degree to which users can recognize whether a product or system is appropriate for their needs. Note: Same ISO 25010 definition. |
| 1.1.1 Familiarity | Degree to which the user understand what the application, system’s functions or tasks are about, and their functionality almost instantly, mainly from initial impressions |
| 1.1.2 Control icon recognize-ability | Degree to which the representation of the control icon follows or adheres to an international standard or agreed convention. |
| 1.2 Learnability | Degree to which the product or system enables users to learn its app. |
| 1.2.1 Feedback Suitability | Degree to which mechanisms and information regarding the success, failure or awareness of actions is provided to users to help them interact with the application. |
| 1.2.2 Helpfulness | Degree to which the software product provides help that is easy to find, comprehensive and effective when users need it. |
| 1.3 Operability | Degree to which a product or system has attributes that make it easy to operate and control. Note: Same ISO 25010 definition |
| 1.3.1 Data Entry Ease | Degree to which mechanisms are provided which make entering data as easy and as accurate as possible. |
| 1.3.2 Visibility | Degree to which the application enables ease of operation through controls and text that can be seen and discerned by the user in order to take appropriate actions. |
| 1.3.3 Consistency | Degree to which users can operate the task controls and actions in a consistent and coherent way. |
| 1.4 User Error Protection | Degree to which a product or system protects and prevents users against making errors and provides support to error tolerance. |
| 1.4.1 Error prevention | Degree to which mechanisms are provided to prevent mistakes. |
| 1.4.2 Error recovery | Degree to which the application provides support for error recovery. |
| 1.5 UI Aesthetics | Degree to which the UI enables pleasing and satisfying interaction for the user. Note: Same ISO 25010 definition. |
| 1.5.1. Text color style uniformity | Degree to which text colors are used consistently throughout the UI with the same meaning and purpose. |
| 1.5.2. Aesthetic harmony | Degree to which the UI shows and maintains an aesthetic harmony regarding the usage and combination of colors, texts, images, controls and layouts throughout the whole application. |
Note the hierarchical tree structure of the model whereby one characteristic is defined (and evaluated and measured) by other sub-characteristics. Sub-characteristics can then be decomposed to attributes that can be directly measurable. Now, with a product quality model, in the next article we will define a QinU model which will characterize the user’s experience both from a usability point of view (efficiency and effectiveness at completing tasks) and a UX view that incorporates satisfaction and its many elements.
References
1. ISO/IEC 25010: Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models, (2011)
2. Olsina L., Lew P., Dieser A., and Rivera B.: Updating Quality Models for Evaluating New Generation Web Applications. In Journal of Web Engineering, Special issue: Quality in new generation Web applications, S. Abrahão, C. Cachero, C. Cappiello, M. Matera (Eds.), Rinton Press, USA, 11 (3), pp. 209-246. (2012)
3. Philip Lew, Luis Olsina, Li Zhang: Quality, Quality in Use, Actual Usability and User Experience as Key Drivers for Web Application Evaluation. ICWE 2010: 218-232
4. Lew P., and Olsina L.; Relating User Experience with MobileApp Quality Evaluation and Design. LNCS 8295, Springer, Current Trends in Web Engineering, ICWE Int’l Workshops, Q.Z. Sheng and J. Kjeldskov (Eds.), Aalborg, Denmark, pp. 253–268, (2013)
Background Reading
1. Lew P., Olsina L., Becker P., and Zhang, L.: An Integrated Strategy to Understand and Manage Quality in Use for Web Applications. Requirements Engineering Journal, Springer London, Vol.17, No. 4, pp. 299-330, (2012)
2. ISO/IEC 2501 http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=35733
3. Bevan N.: Extending Quality in Use to provide a Framework for Usability Measurement. LNCS 5619, Springer, HCI Int’l 2009, San Diego, USA, pp. 13-22, (2009)
4. Hassenzahl M.: User Experience: towards an experiential perspective on product quality. In: 20th Int’l Conference of the Assoc. Francophone d’IHM; Vol. 339, pp. 11-15, (2008)
About the author
Philip Lew, Ph.D., is the CEO of XBOSoft, a global software quality services firm that consults with clients on software quality improvement methodologies and provides software testing services. With his passion for quality, he consults with clients on how to evaluate and improve their software through optimized processes and strategies.