Conceptual Visualization of Decision Making

What I should write is an article using my blend of Information Architecture and new Pre-Simulation formulations to explain to Gatherites how the Web 2.0 experience transforms the writing/reading experience through interactive self-extension and how I planned this into the Gather experience literally from the very first white boarding sessions in Gather's Devonshire Street office almost two years ago. This article is a furtherance of previous articles I have written about Visualization, Multivariate Display of Quantitative Information, and the paradigmatic construction of consensually hallucinated virtual meme-spaces and will become a prototypical example of my self-fulfilling praxis theory. I will also be deconstructing / reconstituting John Walter's notion of 'vernissage' as it relates to author-author collaboration on Gather, but before we get there, it's important to continue building out some of the various footings upon which we will elaborate further the theoretical underpinnings of Pre-Simulation. Visualization of the decision-making process of designing information spaces seems like a decent place to start, given that it also has some ideas appropriate for authors as they grapple with complex plot formulations. So…(my brother thinks that previous paragraph is the semantic equivalent of stating my fondness for ponies, but what does an unemployed student of classical literature know accept how to make coffee; would you like regular or decaf with that Euripides?)

Conceptual Visualization of Decision Making: Introduction

Abstraction

Decision diagrams are an important tool in problem solving, especially when conceiving of a complex virtual environment. They support and enable the cognitive task of making decisions especially in completely abstracted spaces built for the purposes of content production and author-author collaboration. They lessen the load on the decision-maker and author alike and increase the likelihood of an effective relationship between the end user and the space itself. This article will give a brief history and overview of decision-making theory, a description of the decision-making process, and will conclude with an examination of a decision tree. This paper has the following sections: Theory, Praxis, Case Study, and Conclusion. Attached to the right are various figures for reference of actual decision trees used to build out a virtual space.

Theory

Decision-Making Processes

Decision-making is described as a task where a person must select a choice from many options where (s)he has some set of information available to him/her within a given problem space. The timeframe is more than a second and there is no obvious correct choice, although there may be some set of least-bad choices. The person must assess risks associated with each choice. (Wickens, Gordon & Liu, 1998).  There are two types of decision-making processes; the first is intuitive decision making which constitutes quick and automatic decisions (think Gladwell's idea of thin-slicing in his book Blink). The second is referred to as analytical decision making which is more deliberate and controlled. (Wickens, Gordon & Liu, 1998). This article is focused on this type of process, it's different conceptual models, and how visualizations are used to support it.

Early decision-making theory revolved around the idea that any decision could be made using mathematic formulas to yield optimal choices. These theories were often referred to as Normative Decision Models because they demonstrated what a person should do in an ideal setting. Within the Normative Model, choices were given utility values which were inputted into a formula to compute probability and overall value of each option. Within this model, there was no allowance for bias, motivation, preference, distracters, framing effects, or cognitive narrowing. Obviously, researches found that human behavior varied from the computed optimal choice quite often. Researchers moved towards Descriptive Decision Models. One is Simon's satisficing where decision-making shortcuts the extensive consideration of all factors involved in making a Normative decision. Instead, a person would consider options until one that is 'good enough' is accepted. The return on additional effort is negligible and so the process is stopped. (Wickens, Gordon & Liu, 1998)

Wickens (1998) describes several stages of decision-making in an information-processing model. The stages are: Cue Reception and Integration, Hypothesis Generation, Hypothesis Evaluation and Selection, and Generating and Selecting Actions. Critical factors that support or hinder the success of these stages are excessive amounts of information, erroneous information, time constraints, prior knowledge, cognitive tunneling, environmental distracters, fatigue, stress, priming, emotions, motivations, confidence levels, mental models, agendas, beer, etc. (Albers, 1996; Wickens, 1998, Shneiderman, 1984; West et al, 2002; Trumbo, 1998) These opposing forces must be considered and user support must be addressed.

Naturalistic Decision Making focuses on "real world" decisions where the process of making a decision is part of a larger realm of problem solving. (Xiao, Milgram, & Doyle as cited in Wickens, 1998) People must make decisions based on their experiences and dozens of other factors. Real-world problems are not clearly defined, have changing information and environments, involve multiple people, and can be stressful or risky. Problem solving and decision making share many of the same basic cognitive processes. However, problem solving is considered to be much more complex as the solution plan is a set of subroutines and steps whereas a decision is a result of one process. (Wickens, Gordon & Liu, 1998)

Using Visualizations

As in the creative process, visualizations act as external representations of working memory and mental models. (Santanen, Briggs, & Vreede, 1999; Gabora, 2002; Crapo, Waisel, Wallace, & Willemain, 2000; Zhang, 1997) Visualizations support cognitive tasks throughout the decision-making process, from creativity to problem solving to communication and collaboration. Problem-solving is thought to be the last stage of the cycle called judgment. Within this stage, a person analyzes alternative solutions and makes judgments on them on whether to accept or reject options.

Ware (2000) describes the problem solving process as an interactive cycle (and sometimes a recursive process). He describes the same loop used in creativity stages, but at a later point in the process. A user builds a conceptual model of the problem space from the creativity stage where Long Term Memory and visual cues interact in the working memory. A conceptual model is defined as providing an appropriate (accurate, consistent, and complete) representation of a target system. (Wu, Dale & Bethel, 1998) In the case of problem-solving, the system can be represented as a visualization. Whereas visualizations are used in the creativity stage to support idea generation and radiant thinking, in the problem-solving stage, visualizations act as a key component in hypothesis testing. Because the visualizations represent an extension of working memory, it acts as a visual cue to a person as (s)he discovers relationships (and maps patterns) within the data and how it corresponds to their mental model. (Crapo, Waisel, Wallace, & Willemain, 2000) The external documentation of the mental models also assists in communication, analyzing relationships, understanding steps, and providing snapshot overviews. The process continues the cycle through the loop to when the user makes adjustments to display, seeks more information, or makes other alterations to the model. Thus, the process moves from exploratory where the goal state is not known, to a problem-solving state where it is. The visualizations that document this process also move from an unstructured state to a linear Euclidean path. Once this change has been made to an iterative problem solving loop to a linear decision making path, there are several stages that a author/architect must go through to make a decision. Visualizations that support this must be more focused as well. The taxonomy of a decision is Input of Data; Consideration of Alternatives; Calculations or other processes performed; Determination of terminal point; Decisions Made; Validation.  After the alternatives have been considered, confidence levels set, decisions made, there is a feedback step where a user validates the decision. Experts have a tendency to skip this step because of their high confidence level. Designers must understand this and build in acceptable supports for an expert mental model.

Visualizations enable users to accomplish tasks by externalizing possible actions and eventualities, supporting conditional logic, countering bias and tunnel vision, managing attention, providing big picture overviews, minimizing load on short-term memory, increasing retention and retrieval of decision model.

Praxis

Decision Diagrams

Harris describes decision diagrams as "graphic representations of alternative decisions or actions that might be taken, plus potential outcomes resulting from those decisions and actions." (1999, 130) He describes the ability to see options and outcomes before decisions are made as one of the main advantages of using decision diagrams. Decision trees and diagrams are also known as sequential evaluation procedures because they involve discrete functions that must be evaluated sequentially. (Moret, 1982)

Harris goes on to describe the several types of Decision Diagrams and how they are used under different circumstances and for different reasons. Decision diagrams can be based on qualitative or quantitative data. There are Binary Decision Diagrams where each decision point results in a yes or no response. Multi-Variate Decision Diagrams are similar except that there are three or more decisions from each conditional statement. Decision Flow Charts are diagrams that have been combined with Flow Charts to illustrate events and feedback loops within a process. Passive Decision Diagrams are used to estimate probabilities of outcomes based on decisions made by others. Finally, Tree Diagrams can utilize probabilities and monetary values to document paths to desirable (and not so desirable) solutions. They are graphical representations of expected value calculations. (Schuyler, 2001)

Flow Charts

Harris (1999) describes flow charts as a visual that displays interrelated information like sequences, events, decisions, functions, and more. They are used to depict procedures and work flow. (Sevilla, 2002). Another name for them is Decision Tree. Flow charts support cognitive tasks in that they can define abstract and complex processes, improve communication, and document procedures. The symbols used in flow charts have been standardized across function. A Decision Flow Chart is a combination diagram between a decision tree and an information or process flow chart. The visualization describes a series of decisions that are made and their outcomes. In addition, alternatives, supporting information, events, and loops are also displayed.

Use Case

I will be examining a Decision Flow Chart. As a Pre-Simulation writer and an information architect, I use decision flow charts all the time to track choice paths within a simulation of experience (especially for my more difficult work). These charts are critical to writers, but equally so in the world of interaction designers and information architects when building virtual information spaces. As well-formed simulated interactive spaces can have anywhere hundreds of possible use cases or plot formulations, there are usually several paths an author or interaction designer can take. Decision Flow Charts track what decision is made where, what the outcome pages are, what supporting materials are used, what events are triggered, and when alternative pages come into play based on prior behavior.  Getting a visual overview of a simulation would be impossible without such a diagram.

Figure 1 (see attached images) is a decision flow chart that represents a simulation about cognitive skills in a consulting space (my previous life). Within the simulation, there are many types of data. The first are decision pages. These are pages where the learner is given the option of two to four choices. These are reflected as squares with a page number and brief title that reflects the content or decision made on the page. When direct child pages have been created as a result of a choice, the pages are displayed appropriately. However, when a choice leads to a page that has already been created, the resulting page is drawn as a circle with the page number inside of it. Because it is the intention to link as many paths as possible back to the good path, the intention of this process was to reduce clutter and confusing connector lines. Often, as many as 15 choices can lead to the same page.

A second item is an influencer. An influencer is similar to a flag where an object is given a particular state. The labeling convention used in this diagram is to highlight in yellow, Influencer #: state. For example, 1:2 is translated to "influencer 1 has been set to state 2". These influencers can be checked on following pages throughout the length of the simulation and do not have to reside solely with the current scenario. When influencers are checked, it is highlighted in pink. A checkmark with the influence number is written above an arrow to an alternate page. For example, ?4:2 is highlighted in pink above an arrow that moves from page 56 to page 57. If the learner has followed a path and made decisions so that influencer 2 was set to 2 on a previous page, then when the learner gets to page 56, (s)he will be sent to page 57, an alternate page, rather than page 56.

Timed events are represented as orange arrows. Here, the learner does not need to take any action because the page will advance at a pre-determined time.

Blue lines and arrows represent active items. When a learner clicks on an active item with the simulation environment (such as a telephone or computer) the page will advance to either another simulation page, or to an html pop up. These html pop-ups are usually supporting materials such as jobaids, advanced organizers, or performance support pages that aid a learner in making a decision.

Finally, pages that are marked in green signify an end page. Either the learner has reached a catastrophic end where (s)he will be kicked out of the simulation, or (s)he has reached the end of the scenario successfully and is allowed to read summary feedback about choices and performance. (S)he is then allowed to continue on to the next scenario.

When I first started designing complex information spaces, I created this process as a means of tracking a simulation. I was not aware of standardized symbols or cultural implications of color codes. My diagrams were used as a basis for a flow chart function within the tool. (See Figure 2) However, standardization of symbols was not incorporated into the design, nor was the use of color. It is very difficult to follow the decision path within the tool's flowchart, especially when influencers are involved in pages. Instead of showing a visual representation of how the tool works (a path goes to a page where an influencer is checked and then moves on to an alternate page if appropriate), the tool shoes alternate pages in the same level as choices. The tool represents alternate pages beneath the original page as a hexagon. Also, the tool duplicates timed events as choices. For each event, there is a square choice and a slanted square. This is confusing.

I don't believe that incorporating the use of standardized formats will work for this application. Figure 3 shows standardized symbols for the first nine pages. Already at this point, the diagram is too muddled and confusing to be of much use. Instead, I recommend altering the tool-generated flow chart to visualize checking influencers better. Figure 4 shows before and after views of the tool's flowchart generation of this function. Basically, my recommendation is to follow the writer's defined path as that is the path that actually mimics player path. The flowchart that is generated by the tool is simply wrong. Alternate pages are never below original pages. They are always on the same hierarchical level as the original page and should be visualized as such. Also, timed events should not be duplicated. There is only one event, so only one event should be shown. I also have found that calling out special data such as influencers, timed events, and active items help to reduce logic and technical errors. I think that color should be incorporated into the tool flowcharts to reduce cognitive load during testing and production. Please refer to Figure 4 for an example of an improved Decision Flow Chart.

Using standardized processes within an organization is desirable over the ad hoc methods that are now used by many organizations. The Decision Flow Charts do so much to support the task of writing and testing a simulation. They document decision options, information supports, and process paths. By incorporating just a few changes, the system could be standardized across industry, although this is less important for writers, consistently using a standardized set of signifiers reduces cognitive dissonance and load.

Conclusion

Unlike creative thinking where there may be no goal state, decision making in designing information spaces tends to be a linear progression because there is a known goal state (posting this article, for instance). The act of using visualizations counters the tendency of cognitive fixation or tunnel vision. Zhang believes that visualizations not only extend working memory, but they also guide, constrain, and determine cognitive behavior. (1997) Visualizations support cognitive tasks by acting as external representations of working memory.

Artifacts help to not overload system and to make best decisions. Decision diagrams, such as the one included in this article, help to document decisions, processes, events, and information whereby supporting decision making, communication as well as creating a map of the problem space's territory.

--------------------------------------

Note: Will Evans is a software information architect for a risk modeling software company in Boston. Previously he was the information architect responsible for designing the Gather user experience. He has published articles about Information Architecture, User Experience, and Interaction Design. He has taught User Centered Design and Building Usable Enterprise Architectures to both small and large corporate audiences.

He enjoys publishing his musings, ideas, poetry and pre-Simulationist and post-modern critiques of modern culture and aesthetics. He drinks way to much coffee and needs more sleep but is really trying to change that.

References

Albers, M. J. (1996). Decision Making: A Missing Facet of Effective Documentation. ACM SIGDOC. pg. 57-65.

Card, S. K., Mackinlay, J. D. Shneiderman, B. (1999). Information visualization: Using vision to think. San Francisco, Morgan Kaufman.

Crapo, A.W., Waisel, L.B., Wallace, W.A. & Willemain, T.R. (2000). Visualization and the process of modeling: A cognitive-theoretic view. Proceedings of the sixth ACMSIGKDD international conference on Knowledge discovery and data mining, pg. 218-226.

Gabora, L. (2002). Cognitive Mechanisms Underlying the Creative Process. Proceedings of the fourth conference on Creativity & cognition. pg. 126-133.

Harris, R. (1996). Information Graphics. Atlanta, GA: Management Graphics.

Moret, B., M. E. Decision Trees and Diagrams. Computing Surveys, 1982. Volume 14, #4. 593-623.

Santanen, E.l., Briggs, R.O., & Vreede, G. (1999). A cognitive network model of creativity: A renewed focus on brainstorming methodology. Proceeding of the 20th international conference on Information Systems. pg. 489-494.

Sevilla, C. (2002). Information Design Desk Reference. Crisp Learning. Retrieved March 6, 2003 from Books 24x7.com.

Shneiderman, B. (1984). Response Time and Display Rate in Human Performance with Computers. Computing Surveys. 16 (3) 265-285.

Schuyler, J. (2001). Risk and Decision Analysis in Projects. Project Managment Institute. Electronic version retrieved March 6, 2003 from Books24x7.com.

Trumbo, J. (1998) Spatial memory and design: a conceptual approach to the creation of navigable space in multimedia design. Interactions. 5 (4) 26-34.

Ware, Colin. (2000). Information Visualization: Perception for Design. San Francisco: Morgan Kaufmann.

West, R., Murphy, K., Armilio, M., Craik, G., & Struss, D. (2002) Effects of time of day on age differences in working memory. The Journals of Gernotology. 57B (1) P3-P10.

Wickens, C, D., Gordon, S. E., Liu, Y. (1998). An Introduction to Human Factors Engineering. New York: Addison Wesley Longman.

Wu, C., Dale, N, & Bethel, L. (1998). Conceptual Models and Cognitive Learning Styles in Teaching Recursion. ACM SIGCSE Bulletin, Proceedings of the twenty-ninth SIGCSE technical symposium on Computer science education, Volume 30 Issue 1, 292-296.

Zhang, J. (1997). The nature of external representations in problem solving. Cognitive Science 21 (2):179 – 217