The Ambiguities of Experience, page 7
Even a casual reading of the history of human civilizations suggests that the production and appreciation of objects that provide aesthetic pleasure are significant aspirations of intelligence. Nietzsche described the historian’s job as “inventing ingenious variations on a probably commonplace theme, in raising the popular melody to a universal symbol and showing what a world of depth, power and beauty exists in it” (1957, 37–38). It is not a bad job description for those who would capture the lessons of experience through stories and models.
4
GENERATING
NOVELTY
Although there are numerous unresolved problems, theories of adaptation deal reasonably well with the efficiencies and surprises associated with the processes by which existing ideas, practices, forms, or products survive and reproduce (Cyert and March 1963; March 1988, chap. 8; 1994; 1999a, chap. 15; Nelson and Winter 1982; Hannan and Freeman 1989; Cohen and Sproull 1996; March, Schulz, and Zhou 2000; Hodgkinson and Starbuck 2008). Theories of adaptation typically deal less well with the exploratory processes by which new ideas, forms, products, or practices are created, made available, and protected from premature elimination (Becker, Knudsen, and March 2006). These latter processes and notions about them constitute the rudiments of a theory of novelty and are the focus of this chapter.
Two preliminary caveats may be worth recording. The first is that a theory of novelty deals primarily with global novelty, not local novelty. It focuses on the generation of attributes or practices that are new to a population of organizations, not just new to the organization to which they spread. Theories of the spread of things are essential to understanding organizational innovation and adaptation, but they are somewhat different from theories of the generation of new ideas. In particular, a primary feature of local innovativeness is the base level of ignorance in the innovator. To a novice lover, all techniques of lovemaking are exciting discoveries, regardless of their familiarity to others. Likewise, an experienced scholar is less inclined to claim originality than is a beginner.
The second caveat is that studies of novelty and creativity are sometimes confused by a tendency to conflate the two. Novelty is deviation from established procedures or knowledge; creativity is novelty that is subsequently judged successful. Novelty is a necessary, but not sufficient, condition for creativity. Thus, many of the individual and social attributes that lead to novelty will also lead to creativity, but the complications of identifying the social and competitive processes by which novelty is confirmed as creativity are ignored by a theory of novelty.
In addition, this chapter treats two assumptions as more or less self-evident. The first is that most novel ideas are bad ones—that is, they will subsequently be judged unsuccessful. Only a small, unpredictable fraction of novel initiatives will turn out to be successful. The second assumption is that when novel ideas are generated, there is no reliable way to anticipate which of them will be successful. The few novel ideas that will prove to be creative are indistinguishable from those that will not until considerable time has passed. These two properties of novelty are critical to understanding the problems with novelty.
ADAPTATION AS AN ENEMY
OF NOVELTY
The proposition that humans conserve beliefs and practices and thus are hostile to novelty is well known to the study of social psychology (Lord, Ross, and Lepper 1979; Nickerson 1998). It is also characteristic of learning from experience. As we observed in chapter 2, new rules, routines, forms, and ideas are disadvantaged by the way the replication of success leads to increasing stability in choice. This is partly because novel things have properties that place them at risk. Their outcome distributions tend to have low means and high variances; they often require practice; their returns are often delayed in time and are often global rather than local. It is also partly because the replication of success tends to find a satisfactory (possibly because of chance) alternative and stick with it, particularly if alternatives require practice in order to realize their potential. The replication of success reduces the likelihood of throwing good money after bad, but it also means that good alternatives with poor initial results will be permanently passed over.
Similarly, as pointed out in chapter 3, learning through the construction of stories and models is implicitly hostile to novelty. Stories and models are created out of familiar elements linked together in familiar ways. The familiarity and flexibility of story and model frames make them endure and thereby provide a barrier to new experience or new frames. New things are not experienced because they are coded into narratives and models that preserve existing frames and thereby, in an important sense, do not occur as episodes from which new lessons can be learned. The proposition that there is nothing new under the sun is a proposition as much about the stories that are told and the models that are created as it is about the reality they purport to understand. The construction of stories and models of experience out of familiar themes assures that the lessons of experience will be easily understood and readily assimilated, but it also inhibits new themes and new interpretations. The novelty that will endure is primarily novelty that can be assimilated into old stories and models, which tends to exclude extremely deviant items (March 1992).
Because of these features of adaptation to experience, one of the more common themes of research on adaptation and of the literatures on technological, organizational, or intellectual change is the notion that novelty is not favored by adaptive processes (Garud, Nayyar, and Shapira 1997; Van de Ven 1999). New, deviant ideas are likely to be bad ones (Simonton 1995, 1999; Sutton 2002), and any general inclination to invent and reproduce new ideas is likely to lead to disaster. Moreover, even good new ideas are likely to lead to unfavorable results in the near temporal and spatial distance and thus are unlikely to endure long enough to exhibit their potential value.
Processes that produce actions having a large number of negative outcomes and only a small number of positive ones, most of which are delayed considerably or realized at a distance from the locus of adaptation, are prime candidates for elimination by adaptive processes, even though their expected value may qualify them for survival (Denrell and March 2001; Denrell 2007). In part, this is because adaptive processes, as we normally observe and model them, are myopic. They are relatively insensitive to outcomes that are separated in time or space from the locus of adaptation (Levinthal and March 1993; Levinthal and Posen 2008). This feature of adaptation underlies problems of delayed gratification (self-control) and distant gratification (altruism). It also underlies problems with novelty.
The tendency toward eliminating variability is strengthened by diffusion from others. An important mechanism for the replication of successful practices, routines, and forms is the spread of rules through a connected population (Strang and Soule 1998). Imitation leads to homogenization of practices across individuals and organizations that are connected in such a way that their attributes are visible (DiMaggio and Powell 1983; Djelic 1998). Diffusion of practices among connected organizations is limited by well-known difficulties of reproducing routines (Czarniawska and Sevón 1996, 2005), but it results in considerable convergence and the reduction of variation, particularly where the likelihood of imitating an attribute is increased by the number of organizations exhibiting the attribute (Singh, Tucker, and House 1986; Carroll and Hannan 1989; Hannan 1998).
In short, exploitation drives out exploration (March and Simon 1958, 185; Holland 1975; March 1999c, chap. 7). It seems likely that a theory of novelty will have to provide an understanding of how processes that generate novelty and heterogeneity can survive in the face of processes of refinement and imitation that seem likely to eliminate novelty and its progenitors if they themselves are to survive (Van de Ven 1999; Nooteboom 2000).
THE NOVELTY PUZZLE
Novelty is vulnerable to adaptive processes. Yet new ideas and changes stemming from them, as well as heterogeneity in routines, practices, and forms, are conspicuous aspects of modern organizational experience. Although students of organizations often portray organizations as resistant to novelty and prone to mimetic homogeneity, a comparison of almost any organization as it functions today with the same organization fifty years ago reveals substantial differences, and contemporaneous heterogeneity among organizations is often considerable (Hoopes and Madsen 2008). Organizations are often difficult to change in any particular desired direction, and the complaints of organizational reformers pursuing a specific reform are, therefore, undoubtedly justified. However, organizational change is frequent (March 1999c, chap. 8), and proposals of organizational change are even more frequent. The babble of proposed change is loud and continuous.
Although not all change involves the introduction of novel ideas, novelty is remarkably common. Deviant behaviors and practices are generated at a significant rate more or less continually. Most novel ideas are quickly and beneficially extinguished, but for some reason, this does not seem to slow the flow of novelty. The ultimate destiny of an evolving human species in an evolving universe may, in some sense, be a novelty-free existence, but adaptive processes seem likely to eliminate the species long before they eliminate novelty. Novelty appears in many respects to be irrepressible.
The failure of adaptive processes to eliminate foolishness, curiosity, deviance, and the processes of novelty is a puzzle. How do the mechanisms of novelty generation—whatever they may be—survive and reproduce (Dosi 1988; Dosi, Marengo, Bassanini, and Valente 1999; Gigerenzer 2000; Olsen 2009)? If most novel practices, products, or ideas diminish the chances of success—as seems almost axiomatic—what sustains mechanisms of novelty?
One might imagine that mechanisms of novelty survive despite their poor short-run and nearby returns by virtue of their more distant adaptive usefulness. One version of such an imagination is the speculation that the human organism is hardwired to foster novelty through a curiosity drive or other similar attribute that has evolved over thousands of generations. Such a speculation is, however, more a proclamation of faith than a certifiable phenomenon, and students of adaptive processes are generally reluctant to assume adaptive magic without understanding the specific way in which it has been accomplished.
It is possible to specify situations in which attributes that are locally suboptimal but more useful in the long run or over a broader perspective survive over time; but it is also possible to specify environments in which attributes essential to long-run survival, such as novelty or diversity, make survival in the short run difficult or impossible, and attributes that permit short-run survival are deleterious in the long run. There is no necessary solution to the problem of meeting both nearby and more distant survival requirements, and the precise mechanisms involved have to be identified.
The problem is complicated by the undoubted role of exploitative knowledge in the development and execution of novel ideas. Many studies of creativity identify the creative tension between processes of imagination and processes of execution, recognizing the extent to which the foolishness of exploration has to be juxtaposed to the discipline of exploitation. Either alone is not enough. It is usually argued, however, that the relation has to evolve in such a way that the ratio of imagination to the discipline of conventional knowledge is high relatively early in any particular project and declines over time. The usual concern is that the ratio tends to decline more rapidly than is desirable and thus that novel ideas are normalized too quickly.
TWO THEORETICAL TRACKS FOR
UNDERSTANDING NOVELTY
Novelty is a conspicuous feature of organization life, so any simple idea that the adaptive efficiencies coded into knowledge and adaptive processes are homogeneously inimical to the generation of novelty or the reproduction of novelty-generating mechanisms is probably incomplete. In attempting to understand the survival of novelty in the face of adaptive mechanisms that seem likely to extinguish it, students of adaptation have developed theories of novelty along two main tracks, each of which offers insight but neither of which has led to a strikingly satisfactory resolution.
Adaptive Combinations
The first track postulates some kind of process by which new elements are produced from combinations of established old ones. The classic example in evolutionary biology is sexual reproduction and the genetic combinations that it produces. The classic example in organizational studies is the transfer of rules, routines, or practices from one location to another and the transformations that they experience as they combine with existing rules, routines, and practices within the receiving organization (Czarniawska and Sevón 1996, 2005; Zbaracki 1998). The search for a theory of the combinatorics of organizational routines has occupied many good people for many years. Progenitors of the ideas are found in economics (Schumpeter 1934), psychology (Hebb 1949), and political science (Deutsch 1963).
The combinatoric tradition has numerous contemporary contributors (Padgett and Ansell 1983; Feldman 1989, 2000; Pentland and Rueter 1994; Romer 1994; Pentland 1995; Cohen and Sproull 1996; Czarniawska and Joerges 1996; Weitzman 1998; Gilboa and Schmeidler 2001; Feldman and Pentland 2003; Nooteboom 2000; Padgett, Lee, and Collier 2003; Gavetti, Levinthal, and Rivkin 2005; Obstfeld 2005; Padgett and McLean 2006; Becker, Knudsen, and March 2006; Gavetti and Warglien 2007; Page 2007; Svejenova, Mazza, and Planellas 2007; Nooteboom and Stam 2008).
The hunt for analogues has wandered from Mendel to linguistics to literary theory to haute cuisine to chemistry to neural networks with inconclusive but instructive results. The classic problems, particularly in theories of organizational novelty, are specifying precisely the basic elements involved, the laws of combination, and the ways in which those laws have evolved. Although elements of such a theory have been sketched in interesting ways, none of these problems has been solved in a way that is totally satisfactory. Nothing analogous to Mendelian laws exists for the theory of combinatoric novelty in organizations.
Adaptive Inefficiency
The second track of theories of novelty proceeds from the implicit assumption that processes of adaptation eliminate sources of error but are inefficient in doing so. Casual observations of human life suggest that existing mechanisms of adaptation are far from perfect. The modern social order is an impressively effective system of social control, but it is persistently subject to the contradictions of deviant actors, not all of whom spend years in prisons or asylums. Markets, particularly financial markets, enforce many elements of efficient adaptation, but they leave ample room for wonder at their inefficiencies. Successful gambling casinos and lotteries are monuments to human capabilities to make investment choices that reduce their prospects for wealth accumulation.
The propensity of managers in organizations to be similarly inattentive both to what is known and to evidence about what might come to be known is legendary (Pfeffer and Sutton 2006; Gary, Dosi, and Lovallo 2008), as is the failure of organizations to adopt practices reputed to have led to successes elsewhere. A common interpretation of such elements of apparent maladaptation emphasizes the relatively slow character of adaptive processes relative to the rate of change of the environments they encounter, but such an explanation suffers from the flexibility in estimating the rates of change that it allows.
At least at first blush, it appears that novelty survives not because it offers selective advantage within specifiable adaptive mechanisms but because those mechanisms are actually quite inefficient instruments of improvement and thus inefficient in eliminating the instruments of novelty. They move slowly, and they make mistakes. They have difficulty achieving unity and coherence. As a result, organizations exhibit a rich mixture of continuity and change (Olsen 2009). For example, in an organizational setting, personnel retention procedures that retain apparently successful managers and dismiss apparently unsuccessful managers seem prima facie to be useful in improving the fit of an organization to its environment; yet they can systematically retain less competent managers and dismiss the more competent, thus both tolerating foolish novelty and reducing the long-run fit (March and March 1977; Levinthal 1991; Denrell and Fang 2007).
Generic Labels
One conventional approach to talking about adaptive inefficiency is to provide a generic label for inexplicable deviations: stupidity? madness? revelation? genius? perversity? irreducible noise? The obvious analogue is the concept of mutation, a label for inexplicable deviations in genetic reproduction. Providing such labels is not an entirely foolish solution. Most natural processes or understandings of them exhibit inexplicable variation, and almost every domain of knowledge has labels for its unexplained variance: personality, power, leadership, culture. Since in social science generally, as in theories of adaptation, the unexplained variance is large, the concepts specified by such labels account for a great deal without explaining it (Lave and March 1975).
Such labeling is a kind of intellectual last resort that can be justified primarily by the extent to which it stimulates scholarship that reduces the use of the label. For example, some years ago, the label of “risk preference” was introduced into choice theory as a generic label for observed nonlinearities in the revealed human utility for money over alternative gambles and thus for unexplained deviations from predicted behavior. This led to considerable effort to unpack the nonlinearities, an effort that has sometimes been confused by the label but that has, on balance, probably been fruitful (March and Shapira 1987, 1992; March 1999c, chap. 15; Denrell 2007, 2008).
THE SURVIVAL OF MECHANISMS
OF NOVELTY
An alternative to providing a catchall label for instances of novelty is to try to identify specific processes that generate unavoidable but predictable mistakes of adaptation. For example, we can imagine a theory of knowledge decay. Knowledge can be assumed to be accumulated through adaptation and stored in libraries, genes, rules, and memories. Knowledge is lost through turnover, forgetting, and misfiling, which assure that at any point there is considerable ignorance. Something that was once known is no longer known. In addition, knowledge is lost through its incomplete accessibility. Something that is known at one place is unknown at another (Jeppesen and Lakhani 2009).
