Project management on the edge of chaos

by

Arne Vestergaard and Marianne Smedegaard

Abstract

We have for a number of years taught project managers how to manage people and processes. We have heard stories from many different organisations describing numerous ways to understand and execute projects. It seems that there exists not a single correct way to manage projects, but many possible ways. One thing, though, seems to be characteristic of most of the cases. Top management expects the project manager to have complete control over the project and to have answers to any question anyone might want to ask. In many cases it is also a requirement that the project manager ensures that members of the project group have clear goals and that the development of the project is on schedule.

In this paper we want to explore to what extent the effort to control and direct a project is useful, and to what extent a project can be restricted by tight control. Perhaps there are other concepts for project management that better fit with the complexity and variability that characterise projects in today’s world. Perhaps it is preferable to see projects as a living system rather than strategically and bureaucratically.

A change in perspective of this kind would mean that the project organisation must relinquish the idea of having control over the project all the time and, if they do achieve control, see it as a temporary state of affairs. Some degree of chaos then becomes a necessary (but not exclusive) condition for innovation and for handling complexity and change. In other words, the project manager must place the project on the edge of chaos. In this paper we will try to understand what this means in practice.

Project Culture

We have heard many stories about projects that, despite good prospects for success, did not achieve the result expected of them. In retrospect, it often appears that the poor result was due to failed communication of one kind or another. People misunderstood one another or someone was too late telling the client or the project manager when things were not going according to plan etc.

There are also projects that exceed all expectations. An obvious example is the building of the Øresund Bridge between Copenhagen and Malmö. Finished ahead of schedule and with no snags. According to newspaper reports, the positive result was due to good cooperation and communication. Of course, the project staff was very competent but this was not a decisive factor for the success of the project: competent people also work on projects that are not successful. In fact, many of the people taking part of the Øresund project have probably participated in less successful projects. The point is that it is not always the best team on paper that does best.

The expertise of each member of the project team alone is not decisive for the success of a project, at least not professional expertise. This is illustrated by the following story about the project teams used prior to the launching of Apollo 8. It was found that teams based solely on professional qualifications did not perform as well as expected, whereas teams with average qualifications outperformed all others because they found ways to work that meant that the individual members could use their expertise and their preferred team role. This story illustrates that cooperation patterns and communication patterns can be of as much importance for success as professional competence (Belbin 1993).

Trompenaars (1997) defines culture as a group’s way of solving their problems and dilemmas. Campbell (2000) writes about a “house paradigm” which is a given (project) group’s preferred way of thinking and feeling, and which gradually emerges when people collaborate and discuss to co-ordinate work. Each culture or paradigm favours a particular method of working, and other approaches will appear less attractive. It is in this sense that we will discuss project culture.

A project is normally initiated as a result of somebody’s need, idea or vision. It could be a customer who wants delivery of a product that requires a project team for execution. It could be an internal customer who requests a project that is important for the development of the company. Or it could be a development project, a human resource project, a quality assurance project, IT project, construction project etc. In all cases people are involved from the inception of a project. Projects do not start without people talking to one another. On the contrary, projects generally start because people begin to talk about them.

In the early phase, a project is an idea that binds a number of people together. There is a historically and geographically localised collection of stakeholders who discuss. If the resulting conversations convince the participants that there is a common understanding about what should be done, the discussions continue. By means of the discussions the goal of the project becomes clearer by defining what difference the project will make to the future. The progress of the project is therefore, in addition to defining the content, cost and time needed, a progressive weaving of dreams, hopes, and aspirations for the future amongst the involved parties. The project necessitates co-ordination, and through co-ordination the project’s reality is created.

At the start of a project a project culture is formed which is the basis for spoken and unspoken agreements. Gradually the involved parties influence and intertwine with each other and begin to think and feel in a certain way about the project. A “This is the way we run this project” culture emerges that everyone believes in because it is all they have (Campbell 2000). Anyone suggesting that something could be done or understood differently will often be accused of ignorance, incompetence or illoyal behaviour. Whether or not this culture is useful, no one can tell – it will depend on future circumstances.

During the life of a project, it becomes necessary to re-evaluate the “house paradigm” –sometimes more than once - even though it is unpleasant to give up established procedures and conceptions without having anything to replace them. In these periods, the project leader needs to ensure that the team members can deal with uncertainty and anxiety. Nevertheless, challenging the culture, the “house paradigm” in a project is, in a changing and complex world, an important step to gaining influence.

Control and innovation in a complex and changeable world

According to some authors, we are in the midst of a revolution. The revolution has been initiated by dramatic and rapid changes in technology, institutions and values, and accelerated by IT, globalisation and deregulation (Ridderstråle and Nordstrom 1999). This creates a number of dilemmas for projects, including the dilemma between efficient delivery of what was agreed upon and delivery of what fits the reality of the project at the time when it is done.

Often it results in focusing on one part of the dilemma. Some project managers are seen to be totally focused on reaching the agreed goal, so that they close their eyes and ears to signals from the outside world, that would tell them of new trends or needs. While others are focused on the outside world to an extent, so that they ignore the agreed schedule and don't give feedback in their own organisation about the changes in their planning.

Neither of them are necessarily good or bad project managers, even though they represent two extreme positions. Their success will depend on what type of project they are working on and under which conditions, especially with respect to the complexity, variability and the need for innovation.

Different parties within and without the organisation may have widely differing interests and expectations regarding the project. As a consequence, communication becomes more difficult because each party hears, speaks and acts according to his or her particular interest in the project. Different people will be interested in different phases of the project, and some will be more interested in completely different projects, even though they might have approved goals and milestones. Since projects often have many contacts and thereby many interested parties, project management is a complex job.

We recently came across an example of the above during supervision of a project manager. He is employed in the product development department of a large Danish company. The company produces and supplies a range of high technology products in a very competitive market. The production department in the company produces basically standardised products, but which often need to be modified for customers throughout the world. The development department works independently, but gives input to the production department and receives input from marketing on what needs to be developed. There are thus project groups working on the development of new products and project groups working on customer-based modifications.

The management’s objective is to satisfy the customer, but also to ensure the best possible profit margin. The production team strives to live up to these goals. The development department and our project manager can in the mean time see a number of opportunities for improving the existing products, and want to modify the production accordingly, thereby ensuring customer satisfaction in the long run. But who decides? Although the development department has good arguments on hand and are able to convince the management, they meet with resistance in the production department. The reasons for maintaining the original products are many (costs, time, standardisation), but carry little weight with the development department. They feel that the production department and in particular the supply project manager oppose any change.

One can ask what it is that the production department is afraid of? One can also argue what are they trying to protect? Perhaps they regard change as a criticism of what they have been doing, and do not meet respect from the development people. And perhaps they focus on what the management has set for goals in general, and know that changes during a production will make it difficult if not impossible to meet deadlines and financial goals. So even though the management backs the development department’s suggestions, the supply manager fears for the consequences. The story serves to illustrate how time, economics and change are given completely different priorities by the various interested parties and contribute to increasing the complexity of a project.

A second story illustrates how the actions of the project manager influence the emergence of a project culture. Any project culture manifests certain limitations and possibilities with respect to innovation and response to complexity and change. Some cultures support innovation while in other cultures, innovation and creativity are troublesome.

On a training course for project managers we launch 3 development projects lasting approximately 3 hours. Within that time frame, the 3 project groups are asked to develop, produce and present a video film about project management. The groups are formed randomly. We, as course leaders, play the role of top management. During the 3 hours, we expose the projects to different changes and disturbances from the environment: specifications, staffing and cross-project task forces. The projects will therefore be an exercise or experiment in dealing with changes during a project lifetime.

In the fall of 1999 we got an idea for a change in the set-up. The change consisted of introducing small variations in the character of the initial bias from top management. The initial bias is given after a few minutes and is distributed randomly to the three projects.

The choice of variations in initial bias was inspired by Eskerod (1997) and Chistiansen (1999). Both theorists identify three different perspectives that dominate the literature on project management and project organisation. The 3 variations in our exercise were based on our interpretation of these 3 perspectives.

The first project group was after a few minutes asked to prepare and present for top management a clear goal, objectives and a schedule for project implementation. The second group was asked to identify and present for top management some considerations about ensuring motivation, and about organisation and utilisation of the human resources. The third group was asked to develop and present a shared dream or vision for the fulfilment of the project. The groups had 20 minutes to prepare themselves and, after their presentations, we introduced no further intentional differences between the three groups.

A striking difference emerged in the reactions from the three groups to changes from the surroundings (read: us). We had expected some difference due to the small differences in the initial conditions, but the extent was dramatic. The first group, who had worked on clear goals and schedules, reacted to changes with a great deal of frustration and anger, and their work almost stopped. The second group, who worked with motivational and organisational considerations, reacted with somewhat less frustration and there was a moderate decline in progress. This group, however, handled individual frustrations, doubts and anger differently. The project manager and the members were attentive to the effects of the changes on each individual, and addressing these effects (crisis help) made reorganisation and adaptation to the new situation possible. The third group, who had developed a shared vision or dream, reacted to changes not with anger and resistance, but with increased creativity and inspiration: "Now I have a brilliant idea to how we can reinvent the project".

What can this little story tell us about innovation, creativity and project management approach? After repeating the exercise some number of times, and having discussed it with the participating project managers, our hypotheses are that:

· Small influences in the early stages of a project may lead to great differences in the emerging culture in the project.

· There is a connection between the culture and management approach in the project on the one hand, and the level of innovation, creativity and flexibility in response to change on the other.

· The more focus on clarity in goals and objectives, and in schedules, the greater the tendency to reject, resist or ignore change from outside. Feelings of frustration and loss of purpose seem to hinder the group’s ability to enter a space of creativity and innovation.

· The more focus on shared dreams and on visions concerning the impact of the project on the world, - the greater the tendency to react with creativity and innovation to change. A minimum of restrictions, limitations and rules seems to support the project group’s ability to enter a zone of creativity and innovation.

· The more focus on the "human side", the better the handling of individual dilemmas.

During a project's life, there will be periods with varying relevance for the 3 perspectives. But none of them can be totally ignored. The project manager must therefore accommodate all perspectives and not exclude any of them. Instead the focus should be on determining from moment to moment which perspective will ensure a sound balance between planning, organising and the weaving of dreams and visions. Thus, as a project manager, you should ask yourself:

Which perspectives are you triggering in your interactions with project members?

What perspectives are needed in my project right now?

What actions from you may convoke a change in perspective?

Innovative project management.

Managing innovative projects involves management of change and complexity. Never the less many project managers and consultants call for a firm and controlling approach from the project manager. Classic project wisdom recommends e.g. that the project at an early stage is defined thoroughly and unambiguously, to a degree that excludes different interpretations and understandings of the means and ends of the project. This is often done by setting a clear purpose, objectives and criteria for success. The basic assumption is the better the planning and control, the greater probability for success. Creativity and ambiguity must be limited and kept under control. As consultants, we have witnessed many organisations, working with innovative projects, where this line of thought is forced upon project managers by expectations from steering groups, line managers and project members.

However, the classic project wisdom is increasingly being questioned. Kreiner and Christensen (1996) state that the world in which projects take place is less than perfect and largely unpredictable, in contradiction to the presupposition of classic project management axioms. Uncertainties relating to the environment and the future stress the importance of flexibility and innovation, and thus precise goals and objective along with detailed action plans and control systems rather may block than facilitate a good project result. To be truly innovative projects must therefore, at least some of the time, leave the zone of predictability and adopt a more inquiring, experimental and learning approach. Many writers have in recent years turned to complexity- and chaos theory to find inspiration for new metaphors, house-paradigms and leadership principles for managing projects on the edge of uncertainty (Goldstein 1994, Stacey 1996, Wheatley 1992 and 1997 , Battram 1998). In the following, we list some possible implications for managers of innovative projects:

1. Humans make projects, projects possess non-linear properties. We cannot predict exactly how another human being will react and respond to our actions and communications. At a meeting in the project group, it is only possible to predict and control the first few moments. After that time, what we have prepared in the form of pure information and facts, may be interpreted in an unexpected way by some of the project members with a risk for heated disputes. Small incidents can result in large repercussions – and big interventions (e.g. pep talks) can give nearly no effect or undesired consequences. An analogy between a project and a non-linear or complex adaptive system points to some interesting ideas for dealing with innovation and change.

2. Non-linear systems, and projects, shifts between orderly and more random states. Most project management theory, tools and models tend to regard equilibrium as the natural and desirable state for a project manager. In this state prediction, planning and controlling leads to the defined outcome. However, the equilibrium state is nowadays frequently interrupted by fluctuations in market, competitor, technological, societal or organisational factors. Innovative projects shift between maintaining order and operational efficiency on the one side, and adapting to new situations on the other. And none of the states are considered more "natural" than the other (Goldstein 1994).

3. On the edge of chaos, in the transition between order and disorder, the actions of the project manager can have a “butterfly effect” on the project culture. One example is the very early stage of a project’s lifetime. There is neither total chaos nor established order. As we saw in the video projects, small differences in actions and bias may have a large effect on the emerging culture of the project. During a project, the manager may be most influential through actions on the edge of chaos, reinforcing or interrupting the project culture or "house paradigm". Indirectly, the project manager thereby influences the degree of creativity and innovation.

4. Non-linear systems possess an innate ability to develop new adaptations and innovations through spontaneous self-organisation at the edge of chaos. Innovation in these systems takes place in the transition between orderly states and more disordered, random states. In a project, creativity and innovation are more likely to emerge in the moments where the oversight disintegrates, or confusion diminishes and new patterns of thought are proposed. If creativity and innovation are important, the system of the project must spend some of its time at the “edge of chaos”. If there is too much focus on order, plans and control, new ideas or impulses from outside will be rejected, as we saw in the video project case. If too much confusion is the case, no progress will take place. But, when the state shifts, adaptations or innovation emerge by self-organisation. This process does not need to be initiated or controlled by the project manager. He or she should be curious and patient, - and support the people involved. In highly innovative projects, project managers therefore can use their influence to promote self-organisation and decentralise control. Introducing new tools for project management may stimulate a culture where the project is seen as an information process rather than implementation of activities (Eppinger 2001, Goldratt 1997).

5. Creativity in individuals as well as in groups is dependent on the existence of tensions. On the individual level intrapersonal tensions and in-group tensions in the form of paradoxes, dilemmas between different positions (Stacey, 1996). Innovative projects run into, and have to live with, many dilemmas and paradoxes, e.g. different stakeholders pulling in opposite directions with some advocating for continuity, and some for radically new solutions. Planning and control vs. flexible adaptation is another polarity. Balancing time, budget and quality constraints a third. Many of these dilemmas cannot be solved once and for all, - they have to be lived through. Only by maintaining both sides of the dilemma and keeping on moving is there a chance for finding or, "living into" local, temporary solutions that constitute creativity and innovation. The temptation to try to eliminate or deny the existence of one of the sides must be avoided in innovative projects. Containing different positions, - not consensus or harmony is the agenda - for the innovative project group. One big challenge for the project manager is to help the project group to stand the anxiety that is connected to dilemmas and having to let go some beliefs and "truths". Avoiding anxiety is also avoiding positioning the project at the edge of chaos - and avoiding creativity and innovation.

The innovative project manager.

Let us summarise by giving some practical advice for people managing innovative projects.

1. Shift the perspective between seeing projects as: (1) implementing plans, (2) temporary organisations and (3) the realisation of dreams.

2. Oversight is temporary, being in control an exception.

3. Look for opportunities to influence the future processes and culture of the project.

4. From clear goals to meaning and motivation among project members.

5. From planning to the weaving and holding of different dreams.

6. From implementing plans to constant exploring, experimentation and learning

7. Appreciate and contain dilemmas, paradoxes and differences and don't go for too much consensus about the project.

8. Try to move the project to the edge of chaos, where creativity and innovation can take place.

9. Don't avoid anxiety, - but contain it with patience letting new ideas, connections and actions emerge.

10. Don't try to solve the dilemma between planning and adapting- it has to be lived through by temporary "solutions"

11. Trust and support the self-organising skills in the project group.

Literature.

Battram, A. 1997.: Navigating Complexity. Industrial Society. UK.

Belbin, M. 1993.: Team Roles at Work. Butterworth-Heinemann, Oxford..

Belbin, M. 1996.: Management Teams: Why they succeed or fail. Butterworth-Heinemann..

Campbell, D. 2000: The socially constructed Organisation. Karnac Books, London.

Christensen, S. & Kreiner, K. 1996: Projektledelse i løst koblede systemer. København. Jurist- og Økonomforbundets forlag.

Christiansen, J. K. 1999: Evne til perspektivering - en kerne kompetence. I: Håndbog for Symposiet: "Sådan får vi musik i projektet". s. 325- Foreningen for Dansk Projektledelse.

Eppinger, S.D.: Innovation at the speed of Information. Harvard Business Review, Jan 2001,p. 149-158.

Eskerod, Pernille 1997: Nye perspektiver på fordeling af menneskelige ressourcer i et projektorganiseret multiprojekt-miljø. Handelshøjskole Syd: Det erhvervsøkonomiske Fakultets Skriftsserie.

Goldratt, E.M.1997: The critical Chain. North River Press, M.A. .

Goldstein, J. 1994: The unshackled Organisation. Productivity Press, Portland.

Ridderstråle, J& Nordstrom, K. 2000: Funky Business. ft.com, London.

Stacey, R.D. 1996: Complexity and Creativity in Organisations. Berrett-Koehler. San Francisco.

Trompenaars, F.& Hampden-Turner, C. 1997: Riding the Waves of Culture, London. NB Books.

Wheatley, M. 1992: Leadership and the new Science Barrett-Koehler, San Francisco.

Wheatley, M & Kellner-Rogers, M. 1996: A simpler Way. Barrett-Koehler, San Francisco.