Design science and socio-technical systems theory
January 26, 2009
It seems like evaluations of design science artifacts are done in a particular context over a short time scale. Are such evaluations accurate?
Design science and socio-technical systems theory
I’m re-reading Hevner et. al. (2004), esp. the introduction. I was looking through it for something and I saw diagrams that I didn’t remember — there was Henderson and Venkatraman’s strategic alignment model on p. 79! So I re-read the introduction.
Hevner et. al use the strategic alignment model to remind us that “effective transition of strategy into infrastructure involves extensive design activity on both sides of the figure — organizational design to create an effective organizational infrastructure and information systems design to create an effective information systems infrastructure” (p. 78).
I immediately had this question: does effective use of technology not involve both designing the artifact and also adapting human behavior and organizational processes to make best use of that technology? Design science as described by Hevner et. al. does not seem to deal with this second aspect, and so I wonder if some evaluations might be lacking. They say in the next paragraph on p. 78 that this is outside the scope of this paper, but is it outside the scope of design science?
In Does IT Matter?, Nicolas Carr claimed that IT is a commodity, and since everyone can get the same IT artifacts easily, those artifacts can no longer confer competitive advantage to a firm. This is like saying that every firm extracts the same utility from a particular IT artifact. But this idea was criticized by (among others) Seely-Brown and Hegel (HBR, p. 2) who said “Extracting value from IT requires innovations in business practices. Companies that mechanically insert IT into their businesses without changing their practices for exploiting the new capabilities will only destroy IT’s economic value.” You can’t evaluate IT in isolation. You must investigate it in the context of a real organization and over a long period of time, because each organization will differ in its ability to extract utility from the artifact, possibly due to its absorptive capacity among other things. This is the socio-technical systems theory view of IT in the organization.
One of the criteria I’ve seen for evaluating design science research is this from March and Storey (2008):
“The contributions of new constructs, models, and methods are evaluated with respect to their ability to improve performance in the development and use of information systems. Instantiations or implementations demonstrate the feasibility of utilizing those information technology artifacts for a given task. They are evaluated with respect to their effectiveness and efficiency in the performance of the given task” (March and Storey 2008, p. 726).
It seems like evaluations of design science artifacts are done in a particular context over a short time scale. Are such evaluations accurate? Evaluation of your artifact, particularly instantiations, seems tricky. Is value equivalent to problem solving ability? How much of the problem can be solved by the artifact itself, and how much must be solved by the way that people use the artifact and the way that the organization molds itself to the artifact to arrive at a best fit?
Seems like you almost need to combine design research with action research if you really, honestly, want to see what the value of a design is.
I could have a flaw in my reasoning: are the “value” that Seely-Brown and Hegel mean and the “effectiveness and efficiency” that March and Storey (2008) speak of the same thing? Applegate says that IT creates value by creating options: we use it for one purpose now, but, if we are creative, we may leverage the same artifact for a different purpose later (Applegate 2005, pp. 115-135). Is this what Seely-Brown and Hegel mean when they say “Extracting value from IT requires innovations in business practices” (p. 2) that we actually use the artifact differently than our competitors, or do they mean that the most effective use of an artifact in the manner intended by its designers comes from properly aligning the organization to the artifact?
References
- Applegate, L. M., Austin, R. D., and Mcfarlan, W. F. (2005). Corporate Information Strategy and Management: Text and Cases. McGraw-Hill/Irwin.
- Brown, J. and Hagel, J. (2003). Does IT matter? An HBR debate. Harvard Business Review, pages 2-4.
- Hevner, A., March, S., Park, J., and Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1):75-105.
- March, S. T. and Storey, V. C. (2008). Design science in the information systems discipline: An introduction to the special issue on design science research. MIS Quarterly, 32(4):725-730.
