20+ years on from gIBIS and QOC

The Issue-Based Information System (IBIS) approach to capturing and using design rationale is one of the leading design theories that addresses how groups identify, structure, and make decisions during the problem-solving process. IBIS was conceived by Horst Rittel in the 1970s as a way to deal with what he called wicked problems, unique and novel problems with no stopping rule or “right” answer. IBIS was the first of many argumentation-based solutions—spawning or directly influencing instantiations that include PHI, QOC, DRL, gIBIS, and Compendium—with a common trait that outlining the problem space is equivalent to outlining the solution space. This post outlines the evolution of these design approaches, briefly exploring some key questions about when these approaches are (and aren’t) well-suited, particularly for the field of human-computer interaction.

Two writings were central to this historical review, connecting to all the other papers referenced here: Hypermedia Support for Argumentation-Based Rationale: 15 Years on from gIBIS and QOC by Buckingham Shum, Selvin, Sierhuis, Conklin, Haley, and Nuseibeh (the title of which inspired this post’s title); and Rationale Management in Software Engineering: Concepts and Techniques by Dutoit, McCall, Mistrik, and Paech. And a great many more readings were highly influential and enlightening: the classic “Design Rationale” book by Moran and Carroll that captured the 1996 state-of-the-art for the design rationale field, the recent pair of special issues of the Human Technology journal on Creativity and Rationale in Software Design, and ongoing practitioners’ views on wicked problems and IBIS via blog posts and white papers by Paul Culmsee, Kailash Awati, Jeff Conklin, and others I’m surely forgetting.

Rittel is often pointed to as the initiator of design rationale due to a series of papers from the early 1970s to the early 1980s. He laid out an extensive definition for wicked problems, featuring ten distinguishing properties, with Melvin Webber in a 1973 paper Dilemmas in a General Theory of Planning (though Rittel and his colleagues had been discussing the issue and postulating approaches for at least five years prior to the paper). Other Rittel papers suggested that the right approach to address wicked problems was through issues, situationally-dependent questions that are “raised, argued, settled, ‘dodged’, or substituted” during a design session. Rittel’s concept of an issue is core to his Issue-Based Information Systems (IBIS) approach to group design and decision making, in which issues start a questioning process that links each issue with facts, positions, arguments, and other structures through knowledge relationships. The result is a knowledge space that doesn’t solve the issue, but rather creates an environment of “support” and “planning” where people better understand each others’ points of view.

IBIS was refined and simplified in subsequent years, and the IBIS tree-like structure led to many automated tools. Rittel’s student Ray McCall created a Procedural Hierarchy of Issues (PHI) refinement to IBIS that included many of the early tools: PROTOCOL, MIKROPLIS, PHIIDIAS, and JANUS. Other approaches to design rationale management drew inspirations from this early IBIS/PHI work: QOC by McLean, Young, Bellotti, and Moran; and DRL from the work of Potts and Bruns and of Lee. Perhaps the first widely-used tool was gIBIS, a graphical IBIS tool developed and popularized in the 1990s by Jeff Conklin and his collaborators, and its follow-up tool QuestMap. Much of the reflective literature on design rationale groups these techniques together, with Dutoit et al. noting that “there are so few significant differences in the schemas of IBIS, QOC, and DRL” (though there’s an excellent detailing of the differences in that paper). In general, these tools tended to be less intrusive than the original IBIS approach (e.g., less formality, resulting in simpler structures) and more prescriptive outcomes (with specific solutions). This simpler model enabled more immediate value to the participants, for whom value from the tool was imperative for any time investment.

Conklin joined with numerous other researchers and practitioners—Simon Buckingham Shum, Al Selvin, and others—to create the most current and widely-used instantiation of the IBIS ideas in the Compendium dialog mapping tool. I discussed Compendium in a previous tool review post, though I didn’t use it to its fullest capacity—in a collaborative situation in which divergent opinions need to be drawn together toward a common understanding. The big issues that I had with Compendium were with scalability and history: it’s hard to see more than a dozen nodes at once, and support for rolling back to previous views was limited. But it’s much more usable than gIBIS, and it seems to have attracted a fairly sizable following among usability consultants. Features like scalability and history don’t seem to be a focus of the Compendium tool. In fact, it seems that the biggest contribution of Compendium is not in how knowledge is represented (which had been done before) or in how it is manipulated (simplified…or in some cases ignored or deferred to a future version of the tool), but in the social processes around how the tool is used: an expert in knowledge management and the IBIS/Compendium provides real-time guidance during the analysis process, toward helping the participants debate directions moving forward.

In summary, two related trends that I notice in these IBIS-based tools are that (1) the “hard” stuff is left for experts; and (2) the approaches seek more immediate value to designers. Perhaps this is a response to a shift from academia to consultant environments—consultants certainly need to carve out an “expert” role for themselves, and they’d better make sure there’s value to the participants at the end of the day.

Another trend from IBIS to QOC to gIBIS to Compendium is that the approaches seem to be increasingly question-driven—as opposed to issue-driven—with progressively fewer structuring options for the knowledge that is generated. Does the path of simpification and certainty of IBIS tools violate the original wicked problem mandate that problems don’t have solutions, merely different states of being? Or does the simplification actually match the vision of wickedness that Rittel initially posed? I worry that the increasingly tree-like structure of many of these graphs draws designers further away from the initial problem and doesn’t encourage revisiting issues (though I’ll acknowledge again that Rittel’s original IBIS graph (non-tree) structure with its many loops and cycles is far harder to understand).

One big drawback I see with all of these approaches is their inability to deal with the changing truth that occurs in most design efforts and is prominent in the field of human-computer interaction. I think that’s central to my issues with Compendium and other tools—regarding scalability and history—in which problem spaces become more complex over time. Lots of factors—the state of technology, the skill sets of the designers, the knowledge, skills, and acceptance levels of the target user population—change over time, and decisions that were made at any single point may not apply later. Recall two other key feature of wicked problems: that solutions (or problem states) aren’t right or wrong, and that there’s no stopping rule. Popper, as paraphrased in Rittel and Webber’s 1973 paper, suggested that solutions to problems should only be posited as “hypotheses offered for refutation”; otherwise, you can end up pursuing tame solutions to wicked problems.

Finally, we must be careful that we’re not reducing the wickedness of a problem to the creation of a claims map, or the mapping of a dialog, or the removal of storms from brains—in effect, turning the wicked problem into a tame one. Or, if we choose to do that, we must ensure that, when a design team goes back to look at a DR representation, each element in it is appropriately questioned. Sometimes computer tools can hurt in that regard—they help designers violate some tenet of wickedness by providing a “memory” that captures truths that don’t exist, or by encouraging the capture of knowledge at the wrong granularity.