Exam essay: Design science
January 2, 2009
I discuss design science: what it is, how you do it, and what characterizes good design science.
Exam essay: Design science
Describe design science: what is it, how do you do it, what is good design science, and how do you evaluate it relative to other traditions. Compare to social science research.
In this essay, I will describe what design science is, describe necessary components of that make up good design science research, and discuss how to evaluate it. I’ll then compare design science to behavioral (social science) research. If the purpose of information systems research is to try understand the issues around information systems in order to enable the application of that knowledge to create technology-based solutions to heretofore unsolved and important business problems (Hevner et.al. 2004), then design science is a research methodology in which the researcher creates an IT artifact in order to both increase our IS understanding and to potentially solve the problem with a working solution instantiated in the appropriate environment. In creating IT artifacts, design science aims to present a solution and show that the solution works, how well it works, and under what circumstances. It does not try to show why it works – that is the domain of behavioral science.
A problem can be defined as the difference between a desired end state and the current state of the system (by system, I mean it in the Benbesat and Zmud sense of a technology used by people via processes embedded in an organizational context and environment). The problem space, then is the set of all possible ways of approaching that problem and includes both failed and successful solutions. In design science, the IT artifact something that either helps define the problem space, search it, or is a solution. It is one of four things: constructs, models, methods and instantiations. Constructs are the vocabulary and ideas that we can use to describe things in the problem space. A model uses constructs to describe the problem space. Methods are procedures and processes which one can use to search the problem space for solutions, and instantiations are technological systems that solve (or try to solve) the problem in question. A design science research project may define any number of artifacts, but importantly, those artifacts must be implementable in a real environment.
The kind of problems that design science addresses are what Hevner et. al. (2004) call wicked problems: they have many interdependent sub-components, or depend on creativity for a solution, or many people working together (teamwork), or are malleable.
A good design science project has, according to Hevner et. al, seven characteristics. It produces an artifact, that artifact provides utility in a business sense (possibly meaning that it increases revenue or decreases costs for the target audience if that audience is a for profit business), the researchers formally evaluate the artifact as to its utility in the environment it is supposed to be helping, that artifact provides a research contribution (it solves a previously unsolved and important business problem, or improves upon an existing solution for a solved problem), the study is scientifically rigorous (but not so rigorous that it ceases to be relevant to practitioners), the study describes the problem space and the search mechanism the researchers used to generate their artifacts, and finally the authors communicate their results to practitioners and IS researchers both.
By rigor, I mean that the researchers base their design firmly in the IS knowledge base, and choose appropriate methods and theories from it when defining and searching their problem space, and in designing their artifacts. The emphasis, however, should be on creating implementable artifacts that solve problems in a real world environment, and so researchers should refrain from overly simplifying the problem space in order to be more rigorous.
By evaluate, I mean that the researcher should test the design appropriately in the context in which it was meant to be used, and using appropriately rigorous metrics. In the case of instantiations, fundamentally the researcher is investigating the utility that the artifact brings: how well does it work, and in what contexts.
In comparison with social science (behavioral) research, design science is a proactive research method: we look for a problem, attempt to provide a solution, and determine whether we were successful, how well we were successful, and in what contexts. Behavioral science is reactive. It looks at existing solutions and attempts to understand why a solution works or does not work. In this way, design science and behavioral science are natural partners, and can work cyclically. A design science study solves a problem, and a later behavioral science study tries to understand why the solution works.
My self-critique:
I had to go back to my notes in order to remember all of Hevner’s seven characteristics of good design science research, so that wasn’t so good. Maybe I can remember it better if I tell it as a story of the research process: We want to make an artifact that provides utility both to the business and to the research community (this is the research contribution). We evaluate our solution in order to determine utility and applicability, and we provide a research contribution by solving a heretofore unsolved yet important business problem (or by providing a better solution to a solved problem) in a scientifically rigorous (but not too rigorous) manner. And we want to communicate our results to both practitioners and scientists. Don’t try to remember the numbers from the paper, remember the story I told above, keeping in mind that design science occurs in a real business environment and yet is science, too, and so we have two sets of criteria and two audiences. That and that design science is concerned with utility, and not understanding.
I had trouble remembering how you evaluate non-instantiation artifacts. It goes like this: you evaluate them by how well they help you to design instantiations, essentially. I would guess that this means: evaluate constructs by how well they allow you to model; evaluate models by how well they allow you to build methods and instantiations; and methods by how well they help you design appropriate instantiations. The evaluations go from the general to the specific (assuming that constructs are the most general and instantiations are the most specific).
I should add that the symbiotic relationship between behavioral science and design science also goes behavioral -> design: behavioral science develops the theories and expands the IS knowledge base that design science uses.
I should talk about how design science has two audiences: practitioners and scientists. And in this way it is a way (like action science) of addressing the rigor-relevance dilemma.
I feel like this was a little short. I probably needed to expand the evaluations bit and the rigor bit. I could also expand the communication bit.