CS305: HCI in SW Development Continuing Evaluation: Asking

CS305: HCI in SW Development Continuing Evaluation: Asking

CS305: HCI in SW Development Continuing Evaluation: Asking Experts Inspections and walkthroughs Overview The aims Describe how do heuristic evaluation & walkthroughs. Discuss strengths & limitations of these techniques Old topics in this unit (not covered this year) Discuss the role of interviews & questionnaires in evaluation.

Teach basic questionnaire design. Describe how to collect, analyze & present data. Overview: Evaluation By Experts Several approaches Heuristic evaluation Walkthroughs (several flavors) In general: Inexpensive and quick compared to asking users Discount evaluation Experts may suggest solutions (users probably dont)

Asking experts Experts use their knowledge of users & technology to review software usability Expert critiques (crits) can be formal or informal reports Heuristic evaluation is a review guided by a set of heuristics Walkthroughs involve stepping through a pre-planned scenario noting potential problems Heuristic evaluation

Developed Jacob Nielsen in the early 1990s Based on heuristics distilled from an empirical analysis of 249 usability problems These heuristics have been revised for current technology, e.g., HOMERUN for web Heuristics still needed for mobile devices, wearables, virtual worlds, etc. Design guidelines form a basis for developing heuristics Nielsens general heuristics (Remember these?) Visibility of system status

Match between system and real world User control and freedom Consistency and standards Help users recognize, diagnose, recover from errors Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help and documentation Nielsen HOMERUN Derived from general heuristics More specific for commercial Web sites:

High-quality content Often updated Minimal download time Ease of use

Relevant to users needs Unique to the online medium Netcentric corporate culture Discount evaluation Heuristic evaluation is referred to as discount evaluation when 5 evaluators are used. Empirical evidence suggests that on average 5 evaluators identify 75-80% of usability problems. Note how similar to quick and dirty user studies results

3 stages for doing heuristic evaluation Briefing session to tell experts what to do Evaluation period of 1-2 hours in which: - Each expert works separately - Take one pass to get a feel for the product - Take a second pass to focus on specific features Debriefing session in which experts work together to prioritize problems Advantages and problems Few ethical & practical issues to consider

Can be difficult & expensive to find experts Best experts have knowledge of application domain & users Biggest problems - important problems may get missed - many trivial problems are often identified One study has shown: For each true problem, 1.2 false alarms and 0.6 missed problems Evaluating Websites Heuristics like Nielsens general list less applicable for websites

Nielsens HOMERUN Important: Heuristics can become more useful when they become guidelines for development Analysis of outcomes might lead to particular suggestions. E.g. for one set of published results: (1) layout of pages, (2) arrangement of topics, (3) depth of navigation Preeces Web Heuristics/Guidelines Navigation (1) long pages with wasted space, (2) navigation

support like a site map, (3) good menu structures, (4) standard link colors, (5) consistent look and feel Access (1) avoid complex URLs, (2) long download times Information Design (both content and comprehension) (1) outdated or incomplete info, (2) good graphical design, (3) good use of color, (4) avoid gratuitous graphics and animation, (5) consistency

Topic: Heuristics for What are a good set of heuristics for a cellphones UI? Status status of call should be visible (call, connection, roaming, battery) mode (vibrate etc.) unread text messages, voice mails Navigation one-button for phonebook numbers consistent navigation button for back, etc. Error prevention

prevent accidental button presses in pocket, backpack, purse Efficiency Topic: Heuristics for What are a good set of heuristics for a cellphones UI? Status Navigation Error prevention Efficiency Overview: Walkthroughs Like heuristic evaluation because

Experts are involved Criteria are used to evaluate things Different because: Defining characteristic: they walk through one or more tasks In addition to experts, may involve designers and/or users Cognitive walkthroughs Focus on ease of learning Designer presents an aspect of the design & usage scenarios

One of more experts walk through the design prototype with the scenario Expert is told the assumptions about user population, context of use, task details Experts are guided by 3 questions (on next slide) Disadvantages? time-consuming, laborious, narrow focus (maybe thats OK) The 3 questions Will the correct action be sufficiently evident to the user? Will the user notice that the correct action is available?

Will the user associate and interpret the response from the action correctly? As the experts work through the scenario they note problems Pluralistic walkthrough Variation on the cognitive walkthrough theme Performed by a carefully managed team Experts, users, and developers The panel begins by working separately Goes through a task scenario, using screens from a prototype (perhaps)

Then there is managed discussion that leads to agreed decisions The approach lends itself well to participatory design Disadvantages: larger group to schedule, only look at a few scenarios Key points Expert evaluation: heuristic & walkthroughs

Relatively inexpensive because no users Heuristic evaluation relatively easy to learn May miss key problems & identify false ones Walkthroughs: more task focused, more time and cost Heuristic Categories from ID-Book Visibility of System Status Match between system and real world User control and freedom Consistency and

Standards Error Prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recover from errors Help and documentation Navigation

Structure of Information Physical constraints Extraordinary users Class Exercise: Choose One (A) Heuristic evaluation for cell phone Define a few criteria for cell phones Use categories but make them phone-specific E.g. visibility of status; or, error prevention; or recognition vs. recall Produce your own list of important criteria Be experts and review UI for issues

List issues for one (or more) phones (B) Cognitive walkthrough for one or more department web pages Identify tasks Find requirements for major electives for that major Find rules for major(s) in that department Find faculty who are working in a certain research area

etc. Walk-through these tasks identifying issues based on particular criteria (ease of learning) (B1) Same but do a shopping site etc. End, Spring 2008

Recently Viewed Presentations

  • Diagnostics system for SPL cavities ... - University of Toronto

    Diagnostics system for SPL cavities ... - University of Toronto

    5th SPL collaboration meeting 25 November 2010. Diagnostics system for SPL cavities &the 2nd sound measurement @ Cryolab. Introduction. ... 2 fluid model. normal component + superfluid component. Normal fluid. Bose-Einstein condensate. Entropy, viscocity. With no entropy, no viscosity.
  • Weaving Words - SCILT

    Weaving Words - SCILT

    Weaving Words Liz Lochhead's poem "Kidspoem, Bairnsang" has been described as bilingual. What do you think the term bilingual means? ... Katie Bairdie had a dug... Do you think you could try to write a verse using some Gaelic words?...
  • SHADE IN COFFEE - Coffee Industry Board of Jamaica

    SHADE IN COFFEE - Coffee Industry Board of Jamaica

    deep rooted 2. tall spreading and allows light to filter 3. does not host pest of disease that affects coffee 4. long lived and sturdy 5.soil improving 6. show quick regrowth after pruning thank you for your attention shade in...
  • Les Tsimshian - Authentification

    Les Tsimshian - Authentification

    Par contre, au nord des Kwakwaka'wakw se trouvent des groupes considérés « matrilinéaires » : les Kwakiutl septentrionaux (Haisla, Wuikinuxv et Heiltsuk. Ces trois derniers parlent des langues apparentées aux Kwak'wala, aux Haïdas et aux Tlingits.
  • 20 Semana de Pesquisa/SEMPESq Keynote: Education, Technologies, and

    20 Semana de Pesquisa/SEMPESq Keynote: Education, Technologies, and

    (adapted from UNESCO, 1998) Higher Education Trends Increased: participation (19% to 28%) from 2000 to 2015 student mobility and globalization use of technology inequities within and between countries emphasis on workforce development and professional fields pressure for accountability & responsiveness...
  • 9.1 Assessing a change in scale

    9.1 Assessing a change in scale

    Impact on HR - growth. Compensation policy -different financial motivational techniques need to be used. See each of the phases in Greiner's model of growth for ideas on how the compensation would change. Extensive training -the workforce may need training...
  • Building Civility in Your Workplace Tracye Mayolo, SPHR

    Building Civility in Your Workplace Tracye Mayolo, SPHR

    Google "Workplace Civility?" READ EACH ONE. There is a strong sense among Americans that civility has been lost in the modern workplace. A New York Times opinion survey reported that 53% of respondents felt that the workplace is "more angry"...
  • THE FIRST AMENDMENT 1 Class Activity: Quiz  Which

    THE FIRST AMENDMENT 1 Class Activity: Quiz Which

    It cannot ban printed materials or films merely because they contain alarming or offensive ideas * The press is often called "the fourth branch of government." It helps to keep the other three branches in check. Without a free press,...