how did we achieve this?

See our design process

Scope of the usability test

The data on the website is categorized into five different tools that could be used to obtain community information specific to the state of Indiana. These tools are as follows:

• Topic Profiles
• Community Profiles
• Population Profiles
• Indiana Impact
• Indy Vitals

Community Profiles are used to obtain information based on an area. Once the location is chosen by the user, they have access to a wide range of stats specific to that location. The data one can obtain specific to a location can be broadly categorized into population, health, equity, poverty, environment, crime rate and education.

In  Topic Profiles, the user has access to categorized information. These categories have predetermined data cards that are related to the specific topic. The data the users can view is limited as compared to Community Profile but saves users time to recognize the relevant information.

user research

user types

Expert users

Users that have used the website extensively for their own purposes and are familiar with the working of the website. Potential expert participants were identified with the help of staff at the Polis Center.

Number of participants recruited - 2

Secondary users / stakeholders

UX designers who have worked on the website and understand the usage patterns and its users. These participants included designers who previously worked on SAVI at the Polis Center at IUPUI.

Number of participants recruited - 3

Novice users

Users who could help us understand the potential usability for new users. These users were identified by the team members and selected on the basis of the research that they were conducting or residency in Indianapolis.

Number of participants recruited - 5

Recruitment and interview protocol

The staff at Polis Center helped with identifying the potential participants that the team could reach out to. Sessions were arranged with the participants via email and were conducted both online and in-person as per the availability of the participant. The interview protocol varied for all users depending on the type of user and their professions. The interviews were semi-structured in nature.

Think aloud sessions

Following the interview, expert and novice users were asked to participate in a think aloud exercise to understand the users' thought process and to differentiate between the decision making process for the two kinds of users. The think aloud exercise consisted of the following 3 tasks:

• Task 1 - Find information about the crime rate in hamilton county
• Task 2 - Find population of Marion County
• Task 3 - Find the sources of Information for poverty and crime rate in topic profiles

The 3 tasks were kept simple and basic so that both types of users will be able to complete them by identifying certain keywords from the task statement. This also enabled us to understand the common as well as disparate issues faced by both typed of users.

System usability scale

At the end of each session with the users, they were presented with a System Usability Scale form (See right) where they were asked certain qualitative questions regarding their experience with the website and rate it on a scale of 1 to 5.

Assigning a number to each qualitative metric enabled us to quantify the usability of the website.

quantitative results

• Tasks were considered successfully completed if users were able to identify the required information.
• Prompting was not considered as failure for novice users since certain functionalities were broken at the time of conducting testing
• All Novice users used the same steps for all tasks once they had figured out how to complete Task 1 which implies that other methods of doing the task were not intuitive.

• The Industry Usability Standard is 68%. So the overall usability is slightly lower than the acceptable standard.
• We can conclude that the system is significantly below industry standards for usability of a Novice User.

Heuristic drawbacks

consistency and standards

Different visuals for maps are used in different tools. Since the tools are aimed at a similar user group, it should be kept consistent to avoid confusion.

Map visual for Community Profiles

Map visual for Topic Profiles

recognition over recall

Similar tool description used across multiple tools leads to users having to remember the purpose of the tool in each situation. Topic Profiles and Population profiles have almost identical descriptions leading to users being confused about the utility of the tools.

key findings and design recommendations

issue 1 - identification of the tool

• 2/2 expert users were unaware that some of the tasks that they execute in Community Profiles can be accomplished from Topic Profiles
• 3/5 novice users tried to read the description of the tools to figure out where they need to go based on the task given but were still unable to identify the tool required to complete the task
• 1/2 expert users went on to say that they expect to see population information under Population Profile but they did not find it there

recommended solution

Giving a better description of the tools on the overview page so it is easier to identify for the users.

On the right you can see the before and after descriptions of the tool Topic Profiles that aims to give a better description to the users.

issue 2 - "create profile" confusion

• In the context of this application, Profile refers to a snapshot of information pertaining to the selected geographical area that the user can customize.
• In common internet terms, Profile refers to a user account containing information of the account holder.
• 3/5 novice users assumed profile would mean they would have to sign up to the platform.

recommended solution

Changing the word 'Profile' to 'Dashboard' since that is the term most expert and secondary users used to describe the information deck seen after clicking on the 'Create Profile' button.

issue 3 - unable to find sources despite multiple citations

• Sources for the data shown in Topic Profiles is cited at multiple locations - 'More Info' on Topic Profiles landing page, 'About' icon on individual data cards, 'Sources and Further Reading' link on Topic pages.
• Polis Center, the client for this project, specifically wanted feedback on this section as they got multiple requests to validate the data sources. As a result, they needed to know what would be the best place to link their sources to avoid redundancies in the website.
• 0/2 expert users were unaware that they could view sources on the website itself.
• 1/2 expert users were able to identify 'About' icon as the source of information.
• None of the participants expected to find sources in 'More info' section.
• 1/2 expert users and 2/5 new users were able to identify 'Sources and Further Reading' as the source of information.

recommended solution

From the results of the think aloud, it is clear that the users trust the 'Sources and Further Reading' link as the most trustworthy and definitive point for finding the sources of information. Hence it might be better to keep that as the only CTA for leading the users to the sources of information.

issue 4 - distributed menu options on community profile landing page

• Menu options are represented as floating icons around of the map of Indianapolis. This leads to a lot of mouse tracking all around the page to select options.
• The sizing, color and shadow of the floating options, combined with the lines and colors of the background map makes it difficult to see and identify options.

recommended solution

The menu can be streamlined and placed over the map as a solid menu section instead of a floating overlay. A clear search section, followed by selection aids and drawing tools properly laid out.
The help icon should be moved to the menu bar at the top so that it is available to the users even after they move out of the map section.