Friday, February 24, 2017

How organizations adapt

Organizations do things; they depend upon the coordinated efforts of numerous individuals; and they exist in environments that affect their ongoing success or failure. Moreover, organizations are to some extent plastic: the practices and rules that make them up can change over time. Sometimes these changes happen as the result of deliberate design choices by individuals inside or outside the organization; so a manager may alter the rules through which decisions are made about hiring new staff in order to improve the quality of work. And sometimes they happen through gradual processes over time that no one is specifically aware of. The question arises, then, whether organizations evolve toward higher functioning based on the signals from the environments in which they live; or on the contrary, whether organizational change is stochastic, without a gradient of change towards more effective functioning? Do changes within an organization add up over time to improved functioning? What kinds of social mechanisms might bring about such an outcome?

One way of addressing this topic is to consider organizations as mid-level social entities that are potentially capable of adaptation and learning. An organization has identifiable internal processes of functioning as well as a delineated boundary of activity. It has a degree of control over its functioning. And it is situated in an environment that signals differential success/failure through a variety of means (profitability, success in gaining adherents, improvement in market share, number of patents issued, ...). So the environment responds favorably or unfavorably, and change occurs.

Is there anything in this specification of the structure, composition, and environmental location of an organization that suggests the possibility or likelihood of adaptation over time in the direction of improvement of some measure of organizational success? Do institutions and organizations get better as a result of their interactions with their environments and their internal structure and actors?

There are a few possible social mechanisms that would support the possibility of adaptation towards higher functioning. One is the fact that purposive agents are involved in maintaining and changing institutional practices. Those agents are capable of perceiving inefficiencies and potential gains from innovation, and are sometimes in a position to introduce appropriate innovations. This is true at various levels within an organization, from the supervisor of a custodial staff to a vice president for marketing to a CEO. If the incentives presented to these agents are aligned with the important needs of the organization, then we can expect that they will introduce innovations that enhance functioning. So one mechanism through which we might expect that organizations will get better over time is the fact that some agents within an organization have the knowledge and power necessary to enact changes that will improve performance, and they sometimes have an interest in doing so. In other words, there is a degree of intelligent intentionality within an organization that might work in favor of enhancement.

This line of thought should not be over-emphasized, however, because there are competing forces and interests within most organizations. Previous posts have focused on current organizational theory based on the idea of a "strategic action field" of insiders and outsiders who determine the activities of the organization (Fligstein and McAdam, Crozier; link, link). This framework suggests that the structure and functioning of an organization is not wholly determined by a single intelligent actor ("the founder"), but is rather the temporally extended result of interactions among actors in the pursuit of diverse aims. This heterogeneity of purposive actions by actors within an institution means that the direction of change is indeterminate; it is possible that the coalitions that form will bring about positive change, but the reverse is possible as well.

And in fact, many authors and participants have pointed out that it is often enough not the case that the agents' interests are aligned with the priorities and needs of the organization. Jack Knight offers persuasive critique of the idea that organizations and institutions tend to increase in their ability to provide collective benefits in Institutions and Social Conflict. CEOs who have a financial interest in a rapid stock price increase may take steps that worsen functioning for shortterm market gain; supervisors may avoid work-flow innovations because they don't want the headache of an extended change process; vice presidents may deny information to other divisions in order to enhance appreciation of the efforts of their own division. Here is a short description from Knight's book of the way that institutional adjustment occurs as a result of conflict among players of unequal powers:
Individual bargaining is resolved by the commitments of those who enjoy a relative advantage in substantive resources. Through a series of interactions with various members of the group, actors with similar resources establish a pattern of successful action in a particular type of interaction. As others recognize that they are interacting with one of the actors who possess these resources, they adjust their strategies to achieve their best outcome given the anticipated commitments of others. Over time rational actors continue to adjust their strategies until an equilibrium is reached. As this becomes recognized as the socially expected combination of equilibrium strategies, a self-enforcing social institution is established. (Knight, 143)
A very different possible mechanism is unit selection, where more successful innovations or firms survive and less successful innovations and firms fail. This is the premise of the evolutionary theory of the firm (Nelson and Winter, An Evolutionary Theory of Economic Change). In a competitive market, firms with low internal efficiency will have a difficult time competing on price with more efficient firms; so these low-efficiency firms will go out of business occasionally. Here the question of "units of selection" arises: is it firms over which selection operates, or is it lower-level innovations that are the object of selection?

Jeffrey Hobson provides a thoughtful review of this set of theories here, part of what he calls "competence-based theories of the firm". Here is Hobson's diagram of the relationships that exist among several different approaches to study of the firm.

The market mechanism does not work very well as a selection mechanism for some important categories of organizations -- government agencies, legislative systems, or non-profit organizations. This is so, because the criterion of selection is "profitability / efficiency within a competitive market"; and government and non-profit organizations are not importantly subject to the workings of a market.

In short, the answer to the fundamental question here is mixed. There are factors that unquestionably work to enhance effectiveness in an organization. But these factors are weak and defeasible, and the countervailing factors (internal conflict, divided interests of actors, slackness of corporate marketplace) leave open the possibility that institutions change but they do not evolve in a consistent direction. And the glaring dysfunctions that have afflicted many organizations, both corporate and governmental, make this conclusion even more persuasive. Perhaps what demands explanation is the rare case where an organization achieves a high level of effectiveness and consistency in its actions, rather than the many cases that come to mind of dysfunctional organizational activity.

(The examples of organizational dysfunction that come to mind are many -- the failures of nuclear regulation of the civilian nuclear industry (Perrow, The Next Catastrophe: Reducing Our Vulnerabilities to Natural, Industrial, and Terrorist Disasters); the failure of US anti-submarine warfare in World War II (Cohen, Military Misfortunes: The Anatomy of Failure in War); and the failure of chemical companies to ensure safe operations of their plants (Shrivastava, Bhopal: Anatomy of Crisis). Here is an earlier post that addresses some of these examples; link. And here are several earlier posts on the topic of institutional change and organizational behavior; linklink.) 

Thursday, February 23, 2017

Divided ...

Why is part of the American electoral system so susceptible to right-wing populist appeals, often highlighting themes of racism and intergroup hostility? Doug McAdam and Karina Kloos address the causes of the radical swing to the right of the Republican Party in Deeply Divided: Racial Politics and Social Movements in Postwar America. Here is the key issue the book attempts to resolve:
If the general public does not share the extreme partisan views of the political elites and party activists and, more to the point, is increasingly dismayed and disgusted by the resulting polarization and institutional paralysis that have followed from those views, how has the GOP managed to move so far to the right without being punished by the voters? Our answer — already telegraphed above — is that over the past half century social movements have increasingly challenged, and occasionally supplanted, parties as the dominant mobilizing logic and organizing vehicle of American politics. (Kindle location 303-307). 
Not surprisingly given McAdam's long history in the social movements research field, McAdam and Kloos argue that social movements are commonly relevant to electoral and party politics; they suggest that the period of relatively high consensus around the moderate middle (1940s and 1950s) was exceptional precisely because of the absence of powerful social movements during these decades. But during more typical periods, national electoral politics are influenced by both political parties and diffuse social movements; and the dynamics of the latter can have complex effects on the behavior and orientation of the former.

McAdam and Kloos argue that the social movements associated with the 1960s Civil Rights movement and its opposite, the white segregationist movement, put in motion a political dynamic that pushed each party off of its “median voter” platform, with the Republican Party moving increasingly in the direction of white supremacy and preservation of white privilege.
More accurately, it is the story of not one, but two parallel movements, the revitalized civil rights movement of the early 1960s and the powerful segregationist countermovement, that quickly developed in response to the black freedom struggle. (lc 1220)
The dynamics of grassroots social movements are thought to explain how positions that are unpalatable to the broad electorate nonetheless become committed platforms within the parties. (This also seems to explain the GOP preoccupation with “voter fraud” and their efforts at restricting voting rights for people of color.) The primary processes adopted by the parties after the 1968 Democratic convention gave a powerful advantage to highly committed social activists, even if they do not represent the majority of a party's members.

This historical analysis gives an indication of an even more basic political factor in American politics: the polarizing issues that surround race and the struggle for racial equality. The Civil Rights movement of the 1950s and 1960s was a widespread mobilization of large numbers of ordinary citizens in support of equal rights for African Americans in terms of voting, residence, occupation, and education. Leaders like Ralph Abernathy or Julian Bond (or of course, Martin Luther King, Jr.) and organizations like the NAACP and SNCC were effective in their call to action for ordinary people to take visible actions to support greater equality through legal means. This movement had some success in pushing the Democratic Party towards greater advocacy of reforms promoting racial justice. And the political backlash against the Democratic Party following the enactment of civil rights legislation spawned its own grassroots mobilizations of people and associations who objected to these forms of racial progress. And lest we imagine that progressive steps in the struggle for racial justice largely derived from the Democratic Party, the authors remind us that a great deal of the support that civil rights legislation came from liberal Republicans:
The textbook account also errs in typically depicting the Democrats as the movement’s staunch ally. What is missed in this account is the lengths to which all Democratic presidents—at least from Roosevelt to Kennedy—went to placate the white South and accommodate the party’s Dixiecrat wing. (kl 411)
The important point is that as long as the progressive racial views of northern liberal Democrats were held in check and tacit support for Jim Crow remained the guiding—if unofficial—policy of the party, the South remained solidly and reliably in the Democratic column. (lc 1301)
So M&K are right -- issues and interests provide a basis for mobilization within social movements, and social movements in turn influence the evolution of party politics.

But their account suggests a more complicated causal story of the evolution of American electoral politics as well. M&K make the point convincingly that the dynamics of party competition by themselves do not suffice to explain the evolution of US politics to the right, towards a more and more polarized relationship between a divided electorate. They succeed in showing that social movements of varying stripes played a key causal role in shaping party politics themselves. So explaining American electoral politics requires analysis of both parties and movements. But they also inadvertently make another point as well: that there are underlying structural features of American political psychology that explain much of the dynamics of both movements and parties, and these are the facts of racial division and the increasingly steep inequalities of income and wealth that divide Americans. So structural facts about race and class in American society play the most fundamental role in explaining the movements and alliances that have led us to our current situation. Social movements are an important intervening variable, but pervasive features of inequality in American society are even more fundamental.

Or to put the point more simply: we are divided politically because we are divided structurally by inequalities of access, property, opportunity, and outcome; and the mechanisms of electoral politics are mobilized to challenge and defend the systems that maintain these inequalities.

Sunday, February 19, 2017

Designing and managing large technologies

What is involved in designing, implementing, coordinating, and managing the deployment of a large new technology system in a real social, political, and organizational environment? Here I am thinking of projects like the development of the SAGE early warning system, the Affordable Care Act, or the introduction of nuclear power into the civilian power industry.

Tom Hughes described several such projects in Rescuing Prometheus: Four Monumental Projects That Changed the Modern World. Here is how he describes his focus in that book:
Telling the story of this ongoing creation since 1945 carries us into a human-built world far more complex than that populated earlier by heroic inventors such as Thomas Edison and by firms such as the Ford Motor Company. Post-World War II cultural history of technology and science introduces us to system builders and the military-industrial-university complex. Our focus will be on massive research and development projects rather than on the invention and development of individual machines, devices, and processes. In short, we shall be dealing with collective creative endeavors that have produced the communications, information, transportation, and defense systems that structure our world and shape the way we live our lives. (3)
The emphasis here is on size, complexity, and multi-dimensionality. The projects that Hughes describes include the SAGE air defense system, the Atlas ICBM, Boston's Central Artery/Tunnel project, and the development of ARPANET. Here is an encapsulated description of the SAGE process:
The history of the SAGE Project contains a number of features that became commonplace in the development of large-scale technologies. Transdisciplinary committees, summer study groups, mission-oriented laboratories, government agencies, private corporations, and systems-engineering organizations were involved in the creation of SAGE. More than providing an example of system building from heterogeneous technical and organizational components, the project showed the world how a digital computer could function as a real-time information-processing center for a complex command and control system. SAGE demonstrated that computers could be more than arithmetic calculators, that they could function as automated control centers for industrial as well as military processes. (16)
Mega-projects like these require coordinated efforts in multiple areas -- technical and engineering challenges, business and financial issues, regulatory issues, and numerous other areas where innovation, discovery, and implementation are required. In order to be successful, the organization needs to make realistic judgments about questions for which there can be no certainty -- the future development of technology, the needs and preferences of future businesses and consumers, and the pricing structure that will exist for the goods and services of the industry in the future. And because circumstances change over time, the process needs to be able to adapt to important new elements in the planning environment.

There are multiple dimensions of projects like these. There is the problem of establishing the fundamental specifications of the project -- capacity, quality, functionality. There is the problem of coordinating the efforts of a very large team of geographically dispersed scientists and engineers, whose work is deployed across various parts of the problem. There is the problem of fitting the cost and scope of the project into the budgetary envelope that exists for it. And there is the problem of adapting to changing circumstances during the period of development and implementation -- new technology choices, new economic circumstances, significant changes in demand or social need for the product, large shifts in the costs of inputs into the technology. Obstacles in any of these diverse areas can lead to impairment or failure of the project.

Most of the cases mentioned here involve engineering projects sponsored by the government or the military. And the complexities of these cases are instructive. But there are equally complex cases that are implemented in a private corporate environment -- for example, the development of next-generation space vehicles by SpaceX. And the same issues of planning, coordination, and oversight arise in the private sector as well.

The most obvious thing to note in projects like these -- and many other contemporary projects of similar scope -- is that they require large teams of people with widely different areas of expertise and an ability to collaborate across disciplines. So a key part of leadership and management is to solve the problem of securing coordination around an overall plan across the numerous groups; updating plans in face of changing circumstances; and ensuring that the work products of the several groups are compatible with each other. Moreover, there is the perennial challenge of creating arrangements and incentives in the work environment -- laboratory, design office, budget division, logistics planning -- that stimulate the participants to high-level creativity and achievement.

This topic is of interest for practical reasons -- as a society we need to be confident in the effectiveness and responsiveness of the planning and development that goes into large projects like these. But it is also of interest for a deeper reason: the challenge of attributing rational planning and action to a very large and distributed organization at all. When an individual scientist or engineer leads a laboratory focused on a particular set of research problems, it is possible for that individual (with assistance from the program and lab managers hired for the effort) to keep the important scientific and logistical details in mind. It is an individual effort. But the projects described here are sufficiently complex that there is no individual leader who has the whole plan in mind. Instead, the "organizational intentionality" is embodied in the working committees, communications processes, and assessment mechanisms that have been established.

It is interesting to consider how students, both undergraduate and graduate, can come to have a better appreciation of the organizational challenges raised by large projects like these. Almost by definition, study of these problem areas in a traditional university curriculum proceeds from the point of view of a specialized discipline -- accounting, electrical engineering, environmental policy. But the view provided from a discipline is insufficient to give the student a rich understanding of the complexity of the real-world problems associated with projects like these. It is tempting to think that advanced courses for engineering and management students could be devised making extensive use of detailed case studies as well as simulation tools that would allow students to gain a more adequate understanding of what is needed to organize and implement a large new system. And interestingly enough, this is a place where the skills of humanists and social scientists are perhaps even more essential than the expertise of technology and management specialists. Historians and sociologists have a great deal to add to a student's understanding of these complex, messy processes.

(Martin Filler's review in the News York Review of Books of three recent books on the massive project in lower Manhattan to rebuild the World Trade Center illustrates some of the political and organizational challenges that stand in the way of large, complex projects; link.)

Thursday, February 2, 2017

Ideologies, policies, and social complexity

The approach to social and historical research that I favor is one that pays attention to the heterogeneity and contingency of social processes. It advises that social and historical researchers should disaggregate the large patterns they start with and try to identify the multiple underlying mechanisms, causes, motivations, movements, and contingencies that came together to create higher-level outcomes. Social research needs to focus on the micro- or meso-level processes that combined to create the macro world that interests us. The theory of assemblages fits this intellectual standpoint very well, since it emphasizes contingency and heterogeneity all the way down. The diagram above was chosen to give a visual impression of the complexity and interconnectedness of factors and causes that are associated with this approach to the social world.

According to the premises of this approach, we are not well served by imagining that there are simple, largescale forces that drive the outcomes in history. Examples of efforts at overly simplified explanations like these include:
  • Onerous conditions of the Treaty of Versailles caused the collapse of the Weimar Republic.
  • The Chinese Revolution succeeded because of post-Qing exploitation of the peasants.
  • The Industrial Revolution occurred in England because of the vitality of English science.
Instead, each of these large outcomes is the result of a large number of underlying processes, motivations, social movements, and contingencies that defy simple summary. To understand the Mediterranean world over the sweep of time, we need the detailed and granular research of a Fernand Braudel rather than the simplified ideas of Johann Heinrich von Thunen in the economic geography of central place theory.

In situations of this degree of underlying complexity, it is pointless to ask for a simple answer to the question, "what caused outcome X?" So the Great Depression wasn't the outcome of capital's search for profits; it was instead the complex product of interacting forms of private business activity, financial institutions, government action, legislation, war, and multiple other forces that conjoined to create a massive and persistent economic depression.

This approach has solid intellectual and ontological foundations. This is pretty much how the social world works. But this ontological vision about the nature of the social world is hard to reconcile with the large intellectual frameworks on the left and on the right that are used to diagnose our times and sometimes to prescribe solutions to the problems identified.

An ideologue is a thinker who seeks to subsume the sweep of history or current events under an overarching narrative with simple explanatory premises and interpretive schemes. The ideologue wants to portray history as the unfolding of a simple set of forces or drivers -- whether markets, classes, divine purposes, or philosophies. And the ideologue is eager to force the facts into the terms of the narrative, and to erase inconvenient facts that appear to conflict with the narrative.

Consider Lenin, von Hayek, and Ronald Reagan. Each had a simplified mental framework that postulated a set of ideas about how the world worked. For Lenin it was expressed in a few paragraphs about class, the economic structure of capitalism, and the direction of history. For von Hayek it was the idea that free economic activity within idealized markets lead to the best possible outcomes for the whole of society. For Reagan it was a combination of von Hayek and the simplified notions of realpolitik associated with Kennan, Morgenthau, or Kissinger.

There are two problems for these kinds of approaches to understanding the social world. First is the indifference ideologues express to the role of facts and empirical validation in their thinking. This is an epistemic shortcoming. But second, and equally problematic, is their insistence on representing the social world as a fundamentally simple process, with a few driving forces whose impact can be forecast. This is an ontological shortcoming. The social world is not simple, and there are not a small number of dominant forces whose effects overshadow the myriad of other socially relevant processes and events that make up a given situation.

Ideologues are insidious for serious historians, since they denigrate careful efforts to discover how various events actually unfolded, in favor of the demands of a particular interpretation of history. It is not possible to gain adequate or insightful historical knowledge from within the framework of a rigid and dogmatic ideology. But even more harmful are policy makers driven by ideologies. An ideological policy maker is an actor who takes the simplistic assumptions of an ideology and attempts to formulate policy interventions based on those assumptions. Ideology-based policies are harmful, of course, because the world has its own properties independent from our theories, and interventions based on false hypotheses about how the world works are unlikely to bring about their intended results. Policies need to be driven by theories that are fact-based and approximately true. And policy makers and officials need to be rejected when they flout science and fact-based inquiry in favor of pet theories and ideologies.

A hard question that this line of thought poses and that I have not addressed here is whether policies can be formulated at all within the context of a fundamentally heterogeneous and contingent world. It might be argued that policy formation requires fairly simple cause-and-effect relationships in order to justify the idea of an intervention; and complexity makes it unlikely that such relationships exist. I believe policies can be formulated within this ontological framework; but I agree that the case must be made. A few earlier posts are relevant to this topic (link, linklink, link, link).

Wednesday, February 1, 2017

SSHA 2017 Call for Papers

Macrohistorical Dynamics Network
42nd Annual Meeting of the Social Science History Association
Montréal, Québec Canada

 2-5 November 2017
Submission Deadline: 3 March 2017

"Changing Social Connections in Time and Space"

Please consider participation in Macrohistorical Dynamics (MHD) panels of the 42nd annual meeting of the Social Science History Association, November 2-5, 2017 in Montréal For more information on the meeting as well as the call for proposals, please refer to the SSHA website at Here is the SSHA call for proposals (link).

The deadline for paper and/or panel submissions is March 3, 2017.

In recognition of Canada’s policy of official bilingualism, SSHA will accept paper presentations in either English or French for our meeting in Montreal. Sessions may be monolingual English, monolingual French, or bilingual English/French. Session organizers must clearly indicate which language(s) will be spoken at their session, and paper submitters must indicate if their paper will be delivered in French. All paper abstracts must be submitted with an English version, regardless of the language in which the paper will be presented. Please contact the Program Committee co-chair Barry Eidlin ( with any questions regarding conference language policies.

The thematic topic of the annual meeting is “Changing Social Connections in Time and Space” – a theme that works very well with the research interests of many of the scholars involved in the Macrohistorical Dynamics network.

Macrohistorical Dynamics (MHD) is an interdisciplinary social science research field that focuses on problems of large-scale, comparative historical inquiry. Contributors to the field have brought perspective on a wide variety of problem areas, including macro- and historical sociology; comparative histories; world history; world-system analysis; comparative study of civilizations; philosophy of history; and studies of long-term socio-ecological, technological, demographic, cultural, and political trends and transformations.  The Macrohistorical Dynamics network brings a rigorous perspective to bear on questions having to do with “large” history.

The list of MHD panel themes for 2017 is open, and we encourage you to submit proposals for panel themes or individual paper topics.

The MHD network will be able to host at least six panels in 2017 and will also be able to place additional papers through co-sponsorship with other networks (for example, with History/Methods, Politics, Culture, State-Society, Historical Geography, etc.).

SSHA requests that submissions be made by means of its web conference management system. Paper title, brief abstract, and contact information should be submitted on the site, where the general SSHA 2017 call for papers is also available.  (If you haven’t used the system previously you will need to create an account, which is a very simple process.)  The direct link for submissions is now open for submissions (link). 

NOTE: There is an SSHA rule concerning book sessions.  For a book session to proceed, the author (or at least one of multiple authors) MUST be present.  Proposals for book sessions should only be submitted if there is high confidence that the author will be able to travel to Baltimore November 17-20, 2016.

SSHA has set up a mechanism for networks to share papers, so even if you have a solo paper, send the idea along.  It is possible and useful to identify a paper not only by the MHD network, but also by some other co-sponsoring networks--for example, Theory/Methods, Historical Geography, Politics, Culture, Economics, etc.  Co-sponsored panels and papers are encouraged by the SSHA Program Committee as a means of broadening the visibility of the various networks.

Feel free to contact the co-chairs of the Macrohistorical Dynamics network for further information.

Prof. Daniel Little
University of Michigan-Dearborn
Prof. Peter Perdue
Department of History
Yale University

Prof. James Lee
School of Humanities and Social Science
Hong Kong University of Science and Technology