2Q2U (Quality, Quality in use, actual Usability and User experience) is a specific model used to evaluate the quality of the User Experience (UX) of a mobile application. This article presents a way to use the model and step through the process of doing a real evaluation on the road to improving a mobile application’s usability and UX.
Author, Philip Lew, Ph.D., CEO of XBOSoft, http://www.xbosoft.com/
In the first article of this series, A Model Based Approach To Evaluating and Improving Mobile User Experience, we discussed the special contexts of mobile software that required a model-based approach to evaluate mobile user experience. In the second article, we reviewed the ISO 25010 model and 2Q2U (Quality, Quality in use, actual Usability and User experience) and then instantiated them for the specific purpose of evaluating the User Experience (UX) of a mobile application.
In this follow up article, we present a way to use the model and step through the process of doing a real evaluation on the road to improving a mobile application’s usability and UX. First, reviewing the relationships in ISO 25010 in Figure 1, we can see that the external quality has an influence on the final effect of the software product, namely the Quality in Use (QinU).
Figure 1. ISO 25010 [1] quality views
In previous research, we used the ISO 25010 as a starting point, and developed 2Q2U (Quality, Quality in use, actual Usability and User experience) v2.0 [2 and 3], to instantiate the quality characteristics to evaluate and conduct a systematic evaluation using the ‘depends’ and ‘influences’ relationships. Now, taking relevant characteristics of the model, we examine them in terms of a requirement tree as shown in Figure 2.
Figure 2. Hierarchical Product Usability Model (External Quality)
Note the hierarchical tree structure of the model whereby one characteristic (i.e. Operability) is defined (and evaluated and measured) by other sub-characteristics (Data Entry Ease, Visibility, Consistency). These particular sub-characteristics are then decomposed to attributes that can be directly measured. This represents a product quality model where we can measure by inspection, a particular mobile application.
In this example, if we want to evaluate the Data Entry Ease for a mobile application, we need to specify a particular task and attributes that we want to measure and evaluate. Let’s look at the Amazon mobile application as an example.
Figure 3. Amazon Mobile Application
Task: Search for Continental Gator Hardshell Urban Bicycle Tire
- Attributes for Visibility could include:
o Contrast: Ability to maintain contrast ratio of >=10 using a specific tool (i.e. http://www.snook.ca/technical/colour_contrast/colour.html)
- Attributes for Consistency could include:
o Font type consistency: All font types are the same on one page (3=complete, 2=partial – all but 2, 1=partial all but 2 or more, 0=mostly or all different font types)
We then evaluate these attributes given the particular task.
Examining the visibility attribute, contrast, we see the Amazon mobile site is black print on a white background. They certainly don’t try to be fancy (so they may score lower on aesthetic attributes), but for being visible, they do well with a contrast ratio of over 20.
Examining the consistency attribute, font type consistency, we see they consistently use the same font, Arial, as well, so they score the maximum on this attribute.
Note that this evaluation is heavily dependent on the task at hand, and the attributes we choose to evaluate. If we chose aesthetics we may have other criteria where Amazon would not fare so well.
Once we measure these product quality characteristics and attributes, we can then determine its impact and influence on the end user’s QinU, for instance, their efficiency in getting something done. Figure 4 shows QinU model for this example.
Figure 4. Hierarchical Quality in Use and UX Evaluation Model
In previous research, we have noted that User Experience includes ‘softer’ characteristics such as the satisfaction attribute [2,3]. Other researchers [5,6] have also noted the hedonic nature of UX in that it should at least be partially evaluated from feelings of trust, and attributes such as sense of community, rather than harder attributes such as performance (time to finish a task). So UX, while hard to define, can be defined roughly in terms of satisfaction and software attributes assigned to the satisfaction characteristic. As always, context plays a critical role in the evaluation of User Experience. This is one of the elements that sets UX for mobile applications apart. Contextual factors for mobile applications vary much more than for desktop applications. Contextual factors could include [4]:
- Activity: What the user is doing at the time of usage has a significant influence on the user’s attention span. For example, if they are driving, then they have a very short attention span, maybe 1 second, versus if they are in the middle of a conversation, perhaps they have an attention span of 3 seconds. On the other hand, if they are at home sitting on the sofa, then the user may have other distractions as well;
- Day/time of day: The day and time can impact what a user is doing, and the level of natural light. Unlike desktop or Web Apps which are typically accessed indoors, the usage of Mobile Apps is particularly sensitive to this contextual factor influencing visibility;
- Location: The location of the user influences many elements. For instance, indoors, outdoors, in a car, in an elevator, train or factory all of which can also be related to the user activity;
- User profile: The increasing complexity of software combined with an aging user demographic has an interesting affect on the usability of Mobile Apps. For aging users, usually their close range vision capability has diminished along with their dexterity. On the other hand, applications have become complex, and therefore function and content simple-ness and understandability is also critical and influenced by the particular user group. Not only are there more aging users, there are also more younger users as children these days begin using computing devices as toddlers;
Going back to Figure 1, we can now modify it to match our current evaluation and get some insight and basis for conducting experiments and evaluations.
Figure 5. Product quality influences User Experience Dependency Model
We have measured visibility in our example above. Based on changes that we make in product quality attributes, what impacts will those changes make in User Experience. Suppose for example, that we chose a color scheme that changed the contrast to 7. Would that result in different user performance?
Using this causal relationship model, by changing the values of attributes in product quality and then measuring the effect of those changes in UX (possible through user logs, survey, or observation), organizations can iteratively understand what their users like or dislike and what makes them more productive or not, and then begin to make systematic improvements as a result of their understanding and knowledge gained. Of course, the trick for mobile is all the contextual factors as well.
Conclusion
The first article of this series, A Model Based Approach To Evaluating and Improving Mobile User Experience, discussed the need for a model-based approach to evaluate mobile user experience. Our second article reviewed the ISO 25010 model and 2Q2U (Quality, Quality in use, actual Usability and User experience) and then instantiated them for the specific purpose of evaluating the User Experience (UX) of a mobile application.
In this article, we have shown:
- A model instantiation for product quality
- A model instantiation for user experience
- Enumerated just a few of the contextual factors that need to be accounted for in designing, evaluating and testing mobile applications.
- Usage of the relationships in ISO 25010, namely ‘depends on’ and ‘influences’ as an approach for iterative improvement for attributes in a mobile application.
Using this structured and iterative approach, it is possible to improve mobile application usability and user experience in a systematic way so that you can finally quantify and improve upon ‘wow factor’ as a description of quality for the end user.
References
- ISO/IEC 25010: Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models, (2011)
- Olsina L., Lew P., Dieser A., and Rivera B.: Updating Quality Models for Evaluating New Generation Web Applications. In Journal of Web Engineering, Special issue: Quality in new generation Web applications, S. Abrahão, C. Cachero, C. Cappiello, M. Matera (Eds.), Rinton Press, USA, 11 (3), pp. 209-246. (2012)
- Philip Lew, Luis Olsina, Li Zhang: Quality, Quality in Use, Actual Usability and User Experience as Key Drivers for Web Application Evaluation. ICWE 2010: 218-232
- Lew P., and Olsina L.; Relating User Experience with MobileApp Quality Evaluation and Design. LNCS 8295, Springer, Current Trends in Web Engineering, ICWE Int’l Workshops, Q.Z. Sheng and J. Kjeldskov (Eds.), Aalborg, Denmark, pp. 253–268, (2013)
- Bevan N.: Extending Quality in Use to provide a Framework for Usability Measurement. LNCS 5619, Springer, HCI Int’l 2009, San Diego, USA, pp. 13-22, (2009)
- Hassenzahl M.: User Experience: towards an experiential perspective on product quality. In: 20th Int’l Conference of the Assoc. Francophone d’IHM; Vol. 339, pp. 11-15, (2008)
About the author
Philip Lew, Ph.D., is the CEO of XBOSoft, a global software quality services firm that consults with clients on software quality improvement methodologies and provides software testing services. With his passion for quality, he consults with clients on how to evaluate and improve their software through optimized processes and strategies.