Design Concepts   
Prototypes
   Usability Goals and benchmark Tests   
Test Materials
   Summary of Test Results   
Group Contributions

Low Fidelity Prototype




DESIGN CONCEPTS


From our initial examination of the problem domain we established that an application that would assist the user in making calls to customer service centers with automated menus should be developed. Our target market is users with varying stages of deafness with a consideration towards Late Deafened Adults. Thus a major concern is to make the menu feel as intuitive as possible, to assist users who have been accustomed to these services. We retain our high level diagram detailed in our initial proposal. In our design sketches we focus on the user interface aspect of the application. To improve upon our initial ideas we used 10-plus-10 method, then developed three of the most promising concepts in parallel. One has a focus towards providing dual input methods i.e. English and sign language. The second has a focus towards a step by step setup of the application which is meant to illustrate the various use cases. The third is focused towards providing the maximum similarity with the standard method (not using the application).

An important thing to keep in mind while looking at the designs is that Android does not allow external applications to interact with the Dialer during an ongoing call. We made designs that would make it easy for the user to switch between the Android dialer and our app during the ongoing call.

 missing image  missing image  missing image
 missing image  missing image  missing image
 missing image  missing image
 missing image
 missing image  missing image  missing image



PROTOTYPES

The design decisions behind SmartOp user interface were tested through multiple low fidelity prototypes. For the user feedback multiple paper prototypes were developed to simulate the mobile application. These were based heavily on the preliminary sketches which had been developed after extensive discussion.

We aimed to test two of the approaches:

i. User is guided step by step through the process to illustrate the various use cases.

ii. The application is as similar to the standard interface as possible to minimize the learning process.

A major difference between the two designs is more than superficial and underlies in their difference in philosophy. The first design guides the user step by step through the process, and demonstrates the use case. By offering the user personalization we aim to establish and build trust with the users. This will ease their minds regarding inputting personal information into the application. The second design aims to provide an experience as similar as possible to the standard interface and that’s why the user has to enter his private information before or during every call. Only the intro message will be stored across different phone calls.

Both designs share the 4 top level stages namely ‘Setup’, ‘Dialer’, ‘Call in progress’. And both designs make it a point to let the user know who is on the other end of the line, i.e. Human operator, Robot Voice, Music, Dialtone. This functionality is currently considered to be part of our final design.

Prototype 1

missing image

Prototype 2

missing image


USABILITY GOALS AND BENCHMARK TASKS

Usability Goals

Since our application is based on real time interaction between the user and an operator over the phone, which can be frustrating even without a new technology in the middle, our main focus was to make sure that it is easy for the user to interact with the application during the real time call session. Therefore that is the area where we focused most of the tests but overall the application has three modes and we wanted to test different aspects of each.

‘Setup’ Mode

Goal Number 1: The user is able to enter initial information easily.
The Setup process should be intuitive and promote confidence in the users to enter their personal information.

‘Dialer’ Mode

Goal Number 2: The dialer layout should be obvious.
The users should realize that they are in the dialer mode without any hesitation and so the dialer should be same as android

Goal Number 3: The desired service's number can be searched quickly.
The user should select the relevant industry and then easily be able to find the enterprise.

‘Call in Place’ Mode

Goal Number 4: The calling interface easily understandable.
The user should understand right away that they need to interact with the app as well as dialer.

Goal Number 5: The different operating modes should be easily distinguishable.
The user should immediately know which call state he is in i.e. ‘Machine Speaking’, ‘Human Speaking’ and ‘On Hold’

Goal Number 6: The saved data should easy to access
The user can find pre-saved information easily and without frustration.

Goal Number 7: Responses should be fast in real time.
The use should be able to enter text while the call is in place easily and without frustration.



Benchmark Tasks

Task Description

Relation To Goals

Set up your profile and cross out the fields about the information you are not willing to provide

Related to goal 1. The profile setup should be intuitive to the user and they are willing to enter some personal information

Place a Call to your Bank

Related to Goal 2 and 3. We wanted to check if the user goes straight to the dial pad to enter the phone number or chooses the industry and search bar to find the bank

Once the call is made, navigate through the menus and reach the department responsible for activating your Credit Card

Related to goal 4. Check how the user interacts with the screen of our app and the android dialer at and if they can easily switch between the two

Once the operator picks up, let them know that you want to activate your credit card

Related to goal 5 and 6. Check if the user uses the initial greeting message and the text input interface to successfully have the initial interaction with the operator

Successfully activate your credit card

Related to goal 6 and 7. Check if the user uses the pre saved information and the text input interface to convey the goal to the operator and successfully achieve it



SUMMARY OF TEST RESULTS

Videos

    

Test Results


Task

Goal

Success?

Avg Time Taken

Comments

Profile Setup

Goal Number 1: The Setup is intuitive and the user is willing to provide private information for storage

Goal 1: 2/2

30 seconds

The initial setup was easy for all users. None of the users indicated that they might not want to store sensitive information.

Place A Call to Your Bank

Goal Number 2: The Dialer is as familiar to the user as the normal Android Dialer

Goal Number 3: The user selects relevant industry from the top first before searching for business or entering phone number

Goal 2: 2/2

Goal 3:1/2

25 seconds

None of them had a problem with interacting with the dialer.

1 of the users did not go to change the industry from the top even though the tutorial mentioned it but went straight to entering the phone number or searching for the business in the search bar

Reach the Department Responsible for activating your credit card

Goal Number 4: Once the call is made make sure the user understands right away that they need to interact with both the app and the normal android dialer to make decisions

Goal 4: 1/2

30 seconds

It was not obvious to one user at first that they had to first ‘minimize’ the top app and they went to press the ‘dialer’ button shown in the background

Have the initial interaction with the operator

Goal Number 5: The user can easily differentiate between the three call states i.e. ‘Machine Speaking’, ‘Human Speaking’ and ‘On Hold’

Goal Number 6: The user can find pre-saved information easily and without frustration

Goal 5: 2/2
Goal 6: 1/2

15 seconds

No problems were faced in distinguishing between the three modes

It was not intuitive for the user to press ‘saved values’ instead of ‘type message’ when asked for personal information.

Successfully convey goal to operator and activate your credit card

Goal Number 7: The user can enter text while the call is in place easily and without frustration

Goal 7: 2/2

20 seconds

Entering information was not a problem for any user.


Summary

The users we tested the application on stated their inexperience with any applications of the sort. They mentioned the recent launch of a new product, the VRS(Video Relay Service) system launched by the Canadian Government. This application uses a middleman who they communicated with using sign language to convey their message to businesses. Considering that most services offered to them in terms of communication usually requires a man in the middle, they have become accustomed to a certain level of external involvement. They stated that due to this reason not storing their information becomes a redundant exercise for them in the circumstance. As a result the first prototype i.e. the one that guides the user step by step was preferred as the second prototype offered no distinct advantage.

The average time taken to complete the whole task was 2 minutes and 30 seconds. Both the prototypes suffered from similar design flaws that the users experienced. For example a key mistake was in not understanding that both the android dialer and the app had to be interacted with simultaneously but independently. This is a design restriction placed on us by the Android Operating System as the only method to send a digital key response is to use the stock loader and its screen. Users had to be informed to play a greeting during first contact with operator instead of typing a message. Most tasks were completed without any mistakes but in placing the call, there was mistakes e.g. not selecting industry from top and not searching for industry but keying in the number. Both users said the App was from 80-100 % in ease of use and that the initial tutorial was enough to guide them through the app. One user mentioned that there were too many windows to switch during run time which caused him confusion. Another user said that there was too many windows and too much information at one time, which confused him. Both of them said there initial expectations were matched. For improvements one user said to have only one window for the 'in call mode' screen with all the options displaying at once and one user said to have a separate window for everything so as to create less confusion.



GROUP CONTRIBUTIONS