Calendar, Topics and Readings

PLEASE SEE END/BOTTOM OF THIS PAGE - EXAM WEEK - FOR FINAL EXAM ANSWER KEY AND GRADING INFORMATION

IAT432, Spring 2013

Week 1

Wednesday, January 9th

Lecture D100
10:30-11:20 am,  SUR2740

Lecture Visuals: Introduction to course: .pdf

  • Review Academic honesty policy
  • Overview of course format
  • Overview of course work
  • Course textbooks
  • Overview of Evaluation
    • Evaluation Frameworks
    • Why evaluate?
    • Analytic vs. Participative evaluation methods
    • Objective vs. subjective evaluation methods
    • Formative vs. summative methods
    • Quantiative vs. qualitiative data

Lab:

  • Intro to Labs
  • How in-class lab preps work
  • How in-class exercises work
  • How team assignments work
  • Discussion - What use is usability?

Reading: Jokela, T. When Good Things Happen to Bad Products, Interactions, ACM Press (2004).

The two course textbooks are on hold in library (Burnaby and Surrey) and available at bookstore.


Week 2

Wednesday, January 16th

Lecture D100
10:30 -  11:20 am, SUR2740

Lecture Visuals: Heuristic Evaluation

  • Analytical Methods
  • History
  • Types
  • Pros and Cons
  • Process of applying

Lab:

Intro to labs:

  • How in-class lab preps work
  • How in-class exercises work
  • How team assignments work
  • Find three potential team members for next week
  • Discussion - What use is usability? (Jokela, 2004). Bring your favorite object to lab and also bring an object that you don't like but use frequently. We will discuss these.

Readings: 

  • Chapter 2: What Is Usability Testing? in Rubin, J. Handbook of Usability Testing, Wiley Publishing (2008).
  • Chapter 3: When Should You Test?, in Rubin, J. Handbook of Usability Testing, Wiley Publishing (2008).

Lab: Review Team Assignment 1 also the  Project Briefing document.  Please note due dates have changed to later because of illness.  We will be adjustinmg for this over the next several weeks. 

Readings for Assignment 1: Nielsen's Ten Usability Heuristics, also AskTog, 2001: How to Deliver a Report Without Getting Lynched.

 

Week 3

Wednesday, January 23rd

Visuals:  (.pdf)

Lecture : Heuristic Evaluation 

  • Analytical methods
  • History
  • Types
  • Pros and cons
  • Process of applying

Readings (general):

  • Chapter 9: Evaluation, in Dix et al., Human-Computer Interaction, Pearson (2004). Note this can be difficult in places - read for main points; you will NOT be tested on its statistical material at this time.
  • Read all five bulleted entries for Useit's Heuristic Evaluations (above)

Team Assignment 1: Heuristic Evaluation (5%)

  • Come to lab with a team of three people (all in the same lab section! all have to be present!)
  • Team formation
  • Introduction to Assignment 1
  • Exercise: apply heuristics and compare evaluators
  • Discussion based on individual lab preparation #1: Naive Usability

Readings for Assignment 1:

Deadlines: Moved to week 4 - see below


Week 4

Wednesday, January 30th

Usability Testing I 

Visuals: (.pdf)

  • What is usability?
  • Usability test plans
  • 3 Methods: silent, think aloud, constructive interaction
  • Usability metrics, data & measurement
  • Test setup & test materials
  • Ethics of working with human subjects

Readings:

  • Part II: The Process for Conducting a Test, in Rubin, J. Handbook of Usability Testing, Wiley Publishing (2008).

Team Assignment 2: Usability Testing (15%):

  • Exercise on think aloud or constructive interaction. Here is some background on the "Thinking Alound" usability method from Jakob Nielsen
  • Introduction to Team Assignment 2

Materials for the usability study exercise:

Deadlines: Studio Prep 1 due in Lab this week. [set one week later due to flu]

Lab: Reading for Assignment 2: http://usability.gov/basics/

 

Week 5

Wednesday,  February 6th

Usability Testing II: Query Methods

Visuals: (.pdf);  also see (.pdf)

  • Interviews
  • Surveys
  • Questionnaires

Readings:

  • Part II: The Process for Conducting a Test, in Rubin, J. Handbook of Usability Testing, Wiley Publishing (2008).

Lab: Team Assignment 2 continued


Reading Week Break (Feb. 12-16)

This is SFU's "Reading Week",  so there is NO CLASS (lecture) nor lab sections. However, please note the readings below and expectations for the NEXT week (#6).

Visuals: This lecture material will be discussed the following week. Meantime, look over next week's lecture material as it will help with your readings.

  • Qualitative data analysis
  • Graphing quantitative data
  • Report Writing

Readings:

  • Part II: The Process for Conducting a Test, in Rubin, J. Handbook of Usability Testing, Wiley Publishing (2008).
  • Survey Research (pg. 212-226), Chapter 10 in Martin, D. Doing Psychology Experiments, Wadsworth (2004).

Note: The second text for this course, Martin, Doing Psychology Experiments, is available on reserve at the SFU Library at Burnaby and Surrey Campuses. It is also available for purchase through on-line booksellers such as Amazon.ca and Indego.ca. The SFU bookstore may have used copies locally. We recommend purchase as a paperbound and used text, which lowers the cost considerably.

There are no Lab assignments this week.


Week 6

Wednesday, February 20th

Usability through Qualitative Data Analyaia (.pdf) For now, read this pdf on your own, major points will come up for discussion in future weeks. 

  • Interviews and Surveys
  • Structure of questions, organbisation and available time

Readings:

  • Chapters 1, 2, 7, and 8 in Martin, D. Doing Psychology Experiments, Wadsworth (2004). These chapters will be referred to again in weeks to come, so you'll see them specified in whole or part in assignments to come later. 

Lab: Team Assignment 2 continued:

  • Discussion based on individual lab preparation #2: Writing useful reports
  • Review of report structure for Usability Testing Assignment 2
  • Have in mind a NEW team of 3 for Assignment 3. Let Dan know. 

Deadline: Due at beginning of lab: Studio Prep 2. Also complete team member evaluations, fill-in, print out and give yours to Dan. 


Week 7

Wednesday, February 27th

Lecture: Continuation of Usability Research

Readings: Continue Martin readings from Week 6

Lab: Assignment 2 is due on Friday, March 1st by 5 pm via email to Dan

Team Assignment 3: Controlled Experiment (10%):

  • Introduction to Assignment 3
  • Have a new team of 3 persons, Final Call!
  • Install and get systems working for Assignment 3. Understand all the materials so you can conduct the assignment next week in lab or prior to lab time.
  • To complete Team Assignment 3, you will need the CLASS data file, too. Note that the data need to be ordered by keyboard type before analysis is done.
  • NOTE: This work is spread over the next 3 weeks, due: Martch 22nd

Sample Test Questions, Part I <pdf>


Week 8

Wednesday, March 6th

Controlled Experiments (.pdf)

  • Hypotheses and Variables: Control, Null and other wonders of logical testing
  • Confounding variables or faulty causation

Readings:

  • Chapters 1, 2, 7, 8, 12 and 13 in Martin, D. Doing Psychology Experiments, Wadsworth (2004).

Lab: Team Assignment 3 continued:

  • Review lab prep exercise on experiment design
  • Complete the controlled study for Assignment 3 in lab and give the TA your data, or bring your data to lab. This will be shared amongst each lab and you will then use this for statistical testing.

Handout: Whole class data sheet distributed on Friday, March 8th by email to you. 

Team DATA SHEETS due by email to Dan by Wednesday,  March 6th, 5 pm. 

Sample Test Questions, Part II <pdf>


 Week 9

Wednesday, March 13th

Lecture: Reliability, Validity: (.pdf) and Statistics (.pdf)

  • Reliability, consistency, repeatability
  • Somne basics on statistics: describing and making inference

Readings:

  • Chapters 7, 8, 12, and 13 in Martin, D. Doing Psychology Experiments, Wadsworth (2004).

Read Reliability & Validity in Martin.

Lab: Workshop on Lab Prep #3, review of data gathering for Assignment 3.

Lecture Reading: Martin, Chapters 7 , 8, 12, 13. (this repeats some of the materials assigned earlier).

Extra* Lecture Readings: Statistics Tutorial and
7 common errors in Statistics. These are short, simple and basic. One of the more painless treatments of these potentially difficult topics. *No actual maths required; not to be on exam.

Sample Test Items, Part III <.pdf>

Example of Experiment for Studio Assignment 3 <.pdf>

Due in Lab (at beginning of lab period, i.e. come with it completed): Individual Studio Assignment 3. 

Due in Lab (at end, mainly done in the Lab) Studio Prep #4: Statistics. We will do this in class and it is due at the end of your lab workshop. You will need the sample data file. 

Heads up: Team Assignment #3 Due NEXT week 


Week 10

Wednesday, March 20th

Lecture: Field Studies, Emotion and Affect

  • Conducting field ("real world") studies
  • Measuring affect, emotion & attitude
  • Quantifying qualitative feelings

Readings:

Team Assignment 4: Field Evaluation - Full Body Interaction (15%)

  • Introduce lab prep exercise on 'enjoyment' (due next week)
  • Overview of assignment 4
  • Conduct an example study of gameplay and collect IMI questionnaire data and cued debrief recall data
  • Dan will ask you to form new teams for Assignment 4.

Lab Readings:

  • Qualitative Designs (pg 14-19) in Martin, D. Doing Psychology Experiments, Wadsworth (2004).

Sample Test Items, Part IV <.pdf>

DUE: Team Assignment 3 by 5 pm, Saturday, March 23rd,* via email to Dan. *This is a changed due date by +24 hrs.

Assignment 3 Evaluation Guide <pdf> 


Week 11

Wednesday, March 27th

Lecture: Interface Aesthetics <pdf>

Course Evaluations (in class)

Labs: Team Assignment 4 Field Evaluation: (15%). Please look over this material during the week. Dan and John will be available on line (email) to respond to your questions in lieu of meeting labs (see below). 

We will distribute a Q&A sheet in anticipation (watch for link here when ready).

Readings:

Bertelsen & Pold, Criticism as an approach to interface aesthetics, NordiCHI '04, ACM press (2004), 23-27. (click on PDF when library page appears)

Bentley et al. Evaluation using cued-recall debrief to elicit information about user's affective experiences, OzCHI, 2005. (Focus on Cued Recall Debrief Method).

Sample Test Items, Part V <PDF>

PLEASE NOTE: Friday, 29th March, and Monday, 1st April, are Easter Weekend holidays. Classes will not meet either day. Thus the IAT432 LABS (only) are CANCELLED FOR THE WEEK, i.e. for Thurs and Fri., Mar. 28-29.


Week 12

Wednesday, April 3rd

Lecture: Interface Aesthetics, con't.

Labs: Continued Team Assignment 4 Field Evaluation (15%). 

  • Cued Recall Debrief introduction
  • Coding exercise

Overview: Team Assignment 4, Cued Recall, Reliability

Inter-rater reliability example (this is a download Excel sheet) for Lab Coding Exercise and Assignment 4

Reading:  

Greenberg & Buxton, Usability Evaluation Considered Harmful. CHI 2008, ACM press (2008), 111-120.

DUE: Studio Prep 5: Enjoyment/Engagement by the beginning of your scheduled lab.

  • Dan will also ask you to choose a partner for Assignment 5.

DUE: Sunday April 7th, at 11:59 pm: Team Assignment 4 by email to Dan. 


Week 13

Wednesday, April 10th - Last Day of Class

Lecture: Usability Evaluation Considered Harmful (Some of the time)  adapted from Greenberg & Buxton (2008) - see week 12.

  • Challenges in usability evaluation
  • Applicability of usability evaluation

Exam discussion and Review

  • Course Assessment

Exam Prep Self-Study: Try the following exercise to see if you can design a controlled experiment on your own: <PDF>. Also see: Self test for last weeks of course <PDF>.

DUE: Friday, April 12th, Assignment 5: Enjoyment/Engagement by 5 pm via email to Dan

Other Activity: We will take time (15 min) for course assessment (lecture) and (separately) for labs. 


Exam Week

The final exam was given as scheduled in the assigned classroom. It was a "closed book" test and is to be your own work. The exam had both short answer and essay-style questions. It comprised 30% of your mark. Sample or practice questions had been distributed and discussed several weeks prior.  

Final Exam Format:

20   Multiple choice (2.5 pts each)   50 pts

10   Short Answers (2 pts each)       20 pts

1     Long essay (choice from 2)      30 pts

The exam was given on Wednesday, April 17th, from 12:00 to 15:00 (Noon to 3 pm). Venue was Rm SUR5280. 

Updated 28 April 2013 @ 9:37

IAT432 Final Exam Answer Key. Look here for the preferred exam answers. These are broad answers designed to cover the many possibilities students wrote. Full credit was received on many questions with one or two clear points made, including some not in this key.

Grades for the term have been submitted. They are the product of both objective (statistical) computation and more subjective assessment, formed by the instructors' evaluation (for example, as "participation") or team evaluation, which your peers have completed in lab groups over the term. 

If you have been tracking your grades as simple percentages, please note that different assignments had different weights in determining a final mark (for example, lab preps counted for 2 percentage points in each case and the final exam for 30%). 

We also converted your "raw" marks to standard scores (aka "normalized" or "z-scores") that take into account the overall distribution of student marks for the term. This is a non-linear scale that weights upward good performance and slightly downward the lowest marks.  In each case, we individually inspected your grade and adjusted it where appropriate. 

In main, you will find mid-value grades about 3 percent lower than a simple linear computation of your mark would yield.  This said, class performance was well above "average" at a B+ grade level.  Talk to us if you have further questions. 

Have a good summer break.

John & Dan

© John Bowes 2013