Posts Tagged ‘claims’

Celebrating Toulmin

March 25, 2013 Leave a comment

Stephen Toulmin's Wikipedia and USC photo

The late Stephen Toulmin would have turned 91 today—and he came pretty close, making it to 87.  And as an inspiration to us all, he remained very active through most of his life, released an updated version of his seminal book The Uses of Argument in 2003.  The book has never been out of print, and its ideas have influenced researchers in areas from rhetoric and communication to computer science and engineering.

At the heart of his argumentation methods is the notion of a claim, a statement that you are seeking to argue is correct.  The subtle but important part of that definition is that a claim is falsifiable, in that one can argue successfully for or against a claim.  And, the “truthiness” of a claim may vary as we learn more things—consider, for example, claims about the age of the universe or the intelligence of dinosaurs. I provide an extended look at how the notion of claims evolved in human-computer interaction in a previous post.  Or, you can read my Making Claims book for the long story about claims in HCI!

But on his birthday, we should celebrate not only his work but his his life. Toulmin was born and raised in England, and he released his seminal book in 1958, when he was still a young researcher.  But when his ideas were not well received in England, he moved to the United States.  He spent time on the faculty at Brandeis, Michigan State, Northwestern, the University of Chicago, and the University of Southern California. In the paperback version of his book, released in 1963, he was defensive of his ideas.  He certainly didn’t rest on his accomplishments though—in many ways his 1992 book Cosmopolis provides a more historically-grounded view of his philosophy (and he comes across as much more comfortable with his ideas).  The updated version of his book came out in 2003, and it, like much of his work of that time, reflected both a more confident and grounded philosophy while embracing his life position as a dissenter.

In many ways it would be hard to emulate his career track, as much of his highly-cited work was books and not papers, reflecting a different era in research.  But his career focus and ability to evolve ideas is worth studying.  And our current era has its own advantages–I can instantly post a blog entry on his birthday to initiate a small celebration and reflection!

Categories: Claims Tags: , ,

Thinking visually, engaging deeply

January 27, 2012 Leave a comment

Imagery provides opportunities to encourage thinking by enabling people to identify key aspects of an image and relate their own expertise to it. A well-chosen image can inspire new ideas, spark memories of prior experiences, highlight potential issues and drawbacks, and provide a point for conversation and debate. Eli Blevis has an interactions article, CHI workshop, and regular course at Indiana University that explores the impacts of digital imagery in HCI and design. In his article, he describes digital imagery as a form of visual thinking, where visual forms are used to create content and make sense of the world.

We turned to imagery as a way to inspire groups of designers to think broadly and engage meaningfully with each other during the design process. We looked for ways that images could serve as a starting point for group design activities, and as a gateway to other design knowledge. Specifically, we are interested in how imagery can be used to enhance claims during early-stage design. Claims, conceptualized by the classic Toulmin (1958) book and introduced to HCI by Carroll and Kellogg (1989), present a design artifact together with observed or hypothesized upsides (+) and downsides (-); e.g., a public display of information (+) can notify large groups of people about things of shared concern, BUT (-) often become unattractive, densely-packed discordances of data. Claims are accessible when compared to much denser knowledge capture mechanisms like papers, patterns, and cases. But it is still a daunting task for designers to look through long lists of textual claims toward finding the right ideas.

Our approach to mitigate this problem is to use imagery as a bridge to each claim. We chose to represent each claim with an image, selected not just because it captured a key aspect of the claim but also because it allowed designers who viewed it to include their own interpretation of the technology and the context.

Information exhibit

Information exhibit image used in design sessions

We have used a set of around 30 image-claim cards in design activities (e.g., brainstorming, storyboarding), using the image cards both in printed and digital form. The benefits of the images-first approach were numerous. It allowed designers to process large numbers of claims quickly, connecting the ideas to their own experiences and expertise toward solving a design problem. It supported collaboration among designers through the shared understanding centered around the images. It encouraged broad speculation down paths not captured by the claims, sometimes resulting in new and different directions. A set of papers led by Wahid at Interact, DIS, and CHI capture the lessons and tradeoffs.

All of this is in keeping with the nature of a claim, whose original intent was as a falsifiable hypothesis (Toulmin, 1958; Carroll & Kellogg, 1989). However, a purely textual claim risks narrowing the associations of the reader to the words in the claim, and thus limiting the design considerations and even alienating designers unfamiliar with the text of a claim. It is through imagery, and specifically through images as the initial shared view in a design session, that designers can make sense of a problem and create meaningful and informed content.

The evolution of claims

January 18, 2012 1 comment

This post seeks to trace the evolution of the claim in human-computer interaction (HCI), from its introduction in the Carroll and Kellogg (1989) paper through the appearance of three books, Carroll’s Making Use (2000), Sutcliffe’s The Domain Theory (2002), and Rosson and Carroll’s Usability Engineering (2002). (A chronological list of key papers is provided at the end of this post.) The definition and role of “claims” shifted significantly during that time period; I’m seeking to identify some of the evolutionary shifts from 1989 to 2002. This list isn’t meant to be complete, but rather it seeks to highlight the most important evolutionary points in the conceptualization of the claim.

Three phases highlight the progress in this evolution:
– Carroll and his colleagues at IBM T.J. Watson in the late 1980s and early 1990s. They were seeking ways to design not just toward creating a single design, but toward crafting a theory-based approach to design to enable designers to build on each others’ work in a meaningful, scientific way. This work continued until Carroll left for Virginia Tech, at which time his focus largely shifted to collaborative computing (save for a few papers that seemed to draw on his IBM work).
– Sutcliffe and Carroll’s collaboration, highlighted by Sutcliffe’s sabbatical time at Virginia Tech. Sutcliffe had been working for many years on knowledge abstraction in software design, and, like Carroll and his group, he was inspired by potential roles for theory in HCI.
– Three summative works led by Carroll, Sutcliffe, and Rosson. Each presented a different view of the role of claims—in the fields of design, engineering, and education, respectively.

Claims were introduced to the field of HCI in Carroll and Kellogg’s “Artifact as theory nexus” paper at CHI 1989. They seemed to base their definition on Toulmin’s 1958 use of the term, in which he established claims as a hypothesis-centered approach to crafting arguments. The Carroll and Kellogg paper seeks to move beyond the narrow focus of cognitive-based theories that were prominent in the 1980s (that focused on low-level phenomena like keystrokes) by introducing the a hermeneutic approach based on psychological claims, the effects on people of both natural and designed artifacts. Claims were the central part of a task-analysis framework, an attempt to position the design and interpretation of HCI artifacts as a central component of HCI research. This approach was intended to bridge the gap from research to innovation—reconciling the “hermeneutics vs theory-based design” conflict in the title. Several examples in the paper showed how developing an understanding of a claim—the artifact and its possible effects—can point out how much we have to learn and can encourage us to draw broader conclusions. Many of these issues, in particular the connection of claims and claims analysis to the task-artifact cycle, is elaborated in a Carroll, Kellogg, Rosson 1991, but the ideas were first presented in the 1989 paper.

A 1992 BIT paper by Carroll, Singley, and Rosson provided the first in-depth view of the tech transfer of UE results (though see the Moran and Carroll 1991 special issue and 1996 book described below). It connected the Scriven view of mediated evaluation to claims upsides and downsides, positioning claims as a contributor in the field of design rationale. In so doing, it expounded upon claims as a way to reuse knowledge, by encouraging designer consideration of specialized vs abstract claims. The expectation was that designers could use claims to “avoid throwing away thoughtful empirical work”. They avoided Grudin’s paradox, stating outright that design rationale (including claims-centric design rationale) was not an automatic mechanism, but requires additional human thought to yield a reusable knowledge unit.

A 1992 TOIS article by Carroll and Rosson opined that HCI should be an action-science “that produces ‘knowledge-in-implementation’ and views design practice as inquiry”. The paper argues that the task-artifact cycle is an action-science because designers must respond to user requirements by building artifacts with upsides and downsides—i.e., claims. This paper distinguishes the scenario/claim roles as such: “Where scenarios are a narrative account, claims are a causal account.” It argues that scenarios provided a situation narrative, but they are too rich, hard to classify, and hard to reuse (arguments brought up again and addressed to varying degrees by Sutcliffe, Chewar, and others). It is the claim that establish the link to action-science by facilitating design analysis, providing a mechanism for generalization and hypothesis, and explicitly recognizing potential tradeoffs.

A 1994 IJHCS paper by Carroll, Mack, Robertson, and Rosson provided a software-centric scenario-based design approach, with Point-of-View (POV) scenarios drawing parallels to object-centric/object-oriented development. This paper represents the most process-based, engineering-focused, and software-generative view of scenario-based design—both until this time and thereafter. Although claims play a fairly minor role in this paper (only appearing in step 4, leveraging the upsides and downsides in analysis and hillclimbing), there seemed to be opportunity for a much larger role: identifying objects, specifying interactions between objects, supporting inheritance, etc. There was also initial discussion of an education focus for POV scenarios, SBD, claims, and such—but it was not elaborated, and the 2002 Rosson and Carroll textbook described a more simplified approach to teaching design. This paper seemed to be hypothesized starting points that were not fully pursued by the authors—rich for mining by Sutcliffe, Chewar, and others in the years to come.

Moran and Carroll’s 1996 Design Rationale book (elaborated from their 1991 special issue of the HCI Journal) is pointed to as a landmark in the field of design rationale. It draws together contributions from Jintae Lee, Allan MacLean, Clayton Lewis, Simon Buckingham Shum, Gary Olson, Gerhard Fischer, Colin Potts, Jeff Conklin, Jonathan Grudin, and many others. Of relevance to the topic of claims is the introduction (by co-editors Tom Moran and Jack Carroll) and a Carroll and Rosson chapter. These chapters exhibit connections in their work to Horst Rittel (wicked problems, IBIS), Francis Bacon (deliberated evaluation), Herb Simon (environment and behavior), and Donald Schön (contexts of experience)—putting forth the most synthesized view of the position of claims within the design community. Some of the psychological themes, particularly those of Simon, are elaborated in Carroll’s 1997 journal paper in Annual Reviews of Psychology.

A 1999 Sutcliffe and Carroll IJHCS paper summarizes the joint efforts of the two authors on the use of claims as a knowledge capture and reuse mechanism. It delved into the possibility of using claims as a reuse mechanism, a concept touched upon in previous work but never described in sufficient detail. The paper introduced a formatting and classification scheme for claims (and scenarios) to enable their reuse, including a process and alternate pathways for claim evolution. Among the augmentations was the first explicit connection to its derivation history and background theory (i.e., where it came from), leading to the first claim map that can reflect parentage, original/evolving context, motivation, evidence, and possibilities for reuse. Also of great importance was the acknowledgement of work left to do: methods for indexing, tool support (hypertext links, structure matching), and the need for buy-in (and stay-in) incentives.

Sutcliffe’s 2000 TOCHI paper seeks to address the irrelevance of HCI in industry, particularly with regard to a theory-based engineering approach. The paper seeks to identify ways to deliver HCI knowledge in a tractable form that is reusable across applications—and, more importantly, across application areas. The paper argues that claims could provide a bridge if reuse scope was improved; specifically, if there were generic versions of claims and artifacts, and if there were mechanisms for matching claims to new application contexts. The bulk of the paper provides a three-step process to accomplish this: steps for creating more generic claims, mechanisms for cross-domain reuse, and approaches to recognize broader implications. Parts of these are elaborated in Sutcliffe’s book (described later) and in the dissertations of Christa Chewar and Shahtab Wahid. Other important products of this work are the notion of claim families, a claims-patterns comparison, and an explicit recognition of the importance of claims as “designer-digestible” knowledge (one of my favorite phrases).

This series of papers culminated with three books that offered very different visions of design, with very different roles for claims. I plan to elaborate on these books in a future post, but here’s a brief summary of each. Carroll’s 2000 Making Use book pulled together his vision for scenario-based design for scientists, with an eye toward the discovery process. Claims are used to augment the scenario-based design process, highlighting key aspects of the design (and leaving the generalization of claims as an exercise for the designer). Sutcliffe’s 2002 The Domain Theory provides a reuse-centric view of software engineering, extending the vision of Rittel and the design rationale literature and approaches. The role of claims is to make concrete Domain Theory’s high level of abstraction (too high, according to critics) by leveraging the high utility (but low flexibility and poor reuse) of claims. Finally, Rosson and Carroll’s 2002 Usability Engineering textbook advocates scenario-based development as a teaching tool, with claims and claims analysis a complementary and guiding technique to scenario development during each stage of design. It presents claims in a simplified, stripped-down manner (for better and worse) meant to be highly accessible for students. These books kicked off a period of scientific application, engineering refinement, and creative design that has continued in the years since they appeared.

Chronological bibliography:
== S. E. Toulmin (1958). The Uses of Argument. Cambridge Press.
== J. M. Carroll and W. A. Kellogg (1989). “Artifact as theory-nexus: Hermeneutics meets theory-based design.” In Proceedings of CHI, pp. 7-14.
== J. M. Carroll, D. A. Singley, M. B. Rosson (1992). “Integrating theory development with design evaluation.” Behaviour and Information Technology 11, pp. 247-255.
== J. M. Carroll, M. B. Rosson (1992). “Getting around the task-artifact cycle: How to make claims and design by scenario.” ACM Transactions on Information Systems 10 2, pp. 181-212.
== J. M. Carroll, Mack, S. R. Robertson, M. B. Rosson (1994). “Binding objects to scenarios of use.” International Journal of Human Computer Systems 41, pp. 243-276.
== J. M. Carroll (1997). “Human-computer interaction: Psychology as a science of design.” Annual Reviews in Psychology 48, pp. 61-83.
== A. G. Sutcliffe and J. M. Carroll (1999). “Designing claims for reuse in interactive systems design.” International Journal of Human-Computer Studies 50, pp. 213-241.
== A. G. Sutcliffe (2000). “On the effective use and reuse of HCI knowledge.” ACM Transactions on Computer-Human Interaction 7 2, pp. 197-221.
== J. M. Carroll (2000). Making Use: Scenario-based Design of Human-Computer Interactions. MIT Press.
== A. S. Sutcliffe (2002). The Domain Theory: Patterns for Knowledge and Software Reuse. Lawrence Erlbaum Associates.
== M. B. Rosson and J. M. Carroll (2002). Usability Engineering: Scenario-Based Development of Human-Computer Interaction. Morgan Kaufman.

Context is everything (for this card set)

December 8, 2011 1 comment

Andreas Larsson and his colleagues at Lund University and Haptimap have created a set of “context cards” titled Dynamic User Experiences. (Big thanks to UC Boulder colleagues Stefan Carmien and Clayton Lewis for getting them for me.)

The cards come in a set of around 30, with a picture and caption on one side (e.g., “at the beach” with a picture of a user sitting at the beach with a mobile device on her lap) and a description of the situation on the other side (e.g., “Spending a day on the beach might expose your mobile device to …”). The cards are available through the workshops that Larsson and his group run, accompanied by a nice 64-page book that describes some activities that can be done with the cards.

context cards

Dynamic User Experience context cards from HaptiMap

Part of the draw for me is that the cards are a lot like the PIC-UP cards and Brain cards we developed in our lab—but with a focus on context instead of impact (for PIC-UP) or technological features (for Brain). For each of the card sets, there’s an assumption that the user population has certain things that they know, but certain things they don’t—or things they need to be reminded of. In the case of context cards, it’s context, encouraging the designer to consider how will the technological device you’re developing work at the beach, or in the dark, or while moving. Seems like a reasonable approach, and one that the HaptiMap group apparently uses in developing mobile products and in running workshops. Looking forward to seeing more from this group.

Claims and patterns in HCI

October 27, 2011 1 comment

I gave a claims-centered talk to a small discussion group last week–including four researchers with over 100 years of combined experience in the field of human-computer interaction. The question about the difference between claims and patterns question came up, with the following distinction reached. Claims are hypothetical, intended to be debated and changed based on the context. They are smaller than patterns, and many (most!) lack the rigor that are found in established patterns libraries—but those traits also make them easier to process and change as well. Patterns purport to be the truth, meant to capture things that have been decided after a great many instantiations and studies and experiments and such. There’s typically a collection of people who work toward maintaining the library, with additions and changes to it occurring rarely and with careful deliberation.

For a discipline like HCI, in which changes in context have great influence over the way a user interface should look and act, it seems that claims often would be the better choice. Does that mean claims are good and patterns are bad? Not at all…but it does mean that great care should be exhibited in choosing which to use for a given design problem. Patterns seem well-suited for domains like web development, in which there’s an assumption that a typical individual working alone at a desktop or laptop machine is seeking to accomplish a task. By “typical”, I mean, e.g., that the person has close to 20/20 vision (perhaps corrected), cognitive skills sufficient to process a fairly complex screen of information, motor skills sufficient to use a mouse and keyboard, and some experience using a web browser. But as soon as those typical traits are violated in your target user population, or as soon as you start designing for noisy or busy or mobile situations, or when you’re seeking to do something very different with your interface, it’s necessary to question the truths—which seems to be a strength of claims.

These lessons were underscored in one of my current projects—designing work support interfaces for young people with moderate to severe cognitive disabilities. So many of the mobile interface claims just don’t hold when designing for people with cognitive disabilities: button sizes have to be bigger (sometimes with only a single “button”), the number of choices have to be limited (to two or at most three!), and single-switch scanning should redundantly be employed to communicate on-screen text. In addition, the experiences have to be tailored differently: repetition in experience and questioning is often more important than reflection, and great care must be taken in the use of appropriate symbol sets. An expert at mobile interface design would almost have to “start over”, throwing away (or, at a minimum, reconsidering) all knowledge about how to design the interface.

A lot of so-called “truth” in interface design goes away when the context changes. That doesn’t mean it’s a bad idea to capture design truths, just that they should be treated with scientific skepticism when encountered in a new design problem. Capabilities of humans can differ depending on user population characteristics, as can the situations in which an interface will be used. My current thought is to use claims in this way: making it clear in design activities that they are meant to be challenged and questioned, not taken at face value. It’s there that I think the greatest value for claims (and the distinguishing value from other knowledge capture approaches) can be realized.

Categories: Claims Tags: , ,

Rationale in SE and rationale in UE–what’s the difference?

September 20, 2011 1 comment

The question of the difference between design rationale in software engineering and usability engineering came up at my UC Boulder talk last week. It’s something that I’ve been thinking about for a while, and more recently as I’ve been reading about design rationale from a software engineering perspective (Burge, McCall, Schneider, etc.).

The big difference seems to be in the certainty embedded in the rationale. Design rationale in software engineering seeks to establish a position of truth, rooted in the functionality of the computer (compared to the nature of physical artifacts, as seen in other fields). The preface and introduction of Burge’s Rationale-based Software Engineering book do a good job of distinguishing design rationale in software engineering from design rationale in other fields.

In contrast, design rationale in usability engineering (and even more so in HCI) seeks to provide pointers toward the right directions–through methods, approaches, or lessons learned. Even a repository of web design patterns like van Duyne’s Design of Sites is lauded not because it preaches HCI truths but because it discusses how to design (e.g., how to promote e-commerce, how to settle on a page layout, how to customize for mobile devices). It includes checklists and is rooted in context, allowing the reader to decide what’s relevant.

This is why I feel claims (or something like them) are the future of design rationale in UE/HCI. They are meant to be hypothetical, subject to change based on changes in context, advances in knowledge, and retargeting of product design. They are more flexible than design rationale methods from software engineering and other fields, reflecting the flexible nature of the field of HCI. But that then makes the creation of claims libraries or other reuse repositories much more difficult–one must balance correctness with hypothesis, permanence with flexibility.

PIC-UP Mobile: For mobile by mobile

September 14, 2011 1 comment

Much of the ongoing research in my group focuses on usability tools for application developers, centered on knowledge transfer and decision-making among teams of designers. At the heart of my approach is the notion of a claim as a unit for knowledge capture, sharing, and negotiation–claims provide a falsifiable, designer-digestible chunk of knowledge that encapsulates an interface feature, its upsides, and its downsides. The small, hypothetical nature of a claim provides designers with opportunities to debate and evolve ideas to meet the needs of novel problems. Recently, Shahtab Wahid led a quality series of papers at Interact, DIS, and CHI (available from my pubs page) that summarizes our progress and describes PIC-UP for those interested in learning more about it.

An emerging focus is on tools for diverse teams, particularly teams of domain experts with little or no expertise in usability engineering. Since it’s tough for many companies to hire a large team of usability professionals to oversee interface development efforts, a suite of tools has promise to assist with the capture, sharing, and deliberation that must happen when addressing the needs of large stakeholder groups. It certainly doesn’t remove the need for a usability expert, but it can help magnify the power of experts, allowing a smaller number of experts to contribute to a larger number of projects.

As a next step, we are working to transition PIC-UP to a tool we call PIC-UP Mobile that will be useful in industrial and educational settings–specifically, to assist in the early-project development of mobile interface designs. We want the tool to be embedded with a small but high-quality set of claims that can be used in design activities. Designers will be able to browse, rate, group, and evolve claims during the design process. Design sessions can create scenarios or storyboards that incorporate key claims, with This will enable a large and diverse set of designers to have meaningful roles in the design of appropriate user interfaces for human experts–particularly domain experts with little or no knowledge about user interface design whose opinions all too often are often ignored.

Based on an established context, PIC-UP Mobile should help to answer questions like these:
– How can an interface best show multimodal information to a user within a unique context?
– How can an interface enable appropriate interaction.
– What are the tradeoffs in choosing one design technique over another?
– Where is more research needed to determine an appropriate interface approach?

There are tons of application domains where this seems well-suited: medicine, education/training, command-and-control, gas-and-oil discovery and processing. All of these domains integrate diverse situations in which stakeholders with differing backgrounds must reach decisions about acceptable (if not optimal!) approaches to addressing a problem. We expect that PIC-UP Mobile will help in reaching these decisions toward defining appropriate user interfaces.