What does it mean to collaborate? I despise people who start out with etymological explanations, but in this case I’m going to take the cheap way out. Collaboration is pretty clearly from the Latin: together (co) world (labor) process. So, collaboration is simply “working together.”
But it carries more meaning than this. Leaving aside the Vichy France connotations, to collaborate generally implies a certain degree of collegiality. If I’m going to collaborate on a paper with someone, the implication is that we are not really boss and subordinate–that we are closer on the hierarchical scale. There is probably someone in charge (when there isn’t, it can be a mess), but that is generally decided on as a practical matter. That is “we need to have someone in charge, so who will it be,” rather than: you are higher up on the totem pole so you are naturally first.
Collaboration is rife with difficulties. Who is in charge–and who decides who is in charge? Who gets credit for the resulting work, and how is that distributed? How is that related to how the work is distributed? What do you do about the fact–and it is a fact–that it is impossible to evenly distribute the work needed to be done on a project? What do you do about the people who do very little work, and yet bask in the glory of the finished project?
You may have noticed that all of the above questions seem to circle around this issue of distribution practices. These are not the only sort of problem, but they seems to be a substantial part of the issue. Moving collaboration to a low-bandwidth, online, distributed form only complicates those issues of responsibility, analysis, and synthesis even more. But first, how do we split up complicated work in such a way that it can come together again.
DIVISION OF LABOR
“Who does what” is not just something that needs to be decided by a small group, but an essential question for advanced societies. No one person could make any of the material products we use every day. The division of labor and free exchange of goods provides us with ways of completing large, complicated projects by distributing to specialists. It can do this in two ways.
First, a master project manager can lay down specifications, and these specifications trickle down to less master project managers, until you get down to very simple specifications. Take, for example, the building of the pyramids. A master architect knows that certain walls need to go to certain places, and assigns the wall to one of his lieutenants, who then assigns the acquisition and moving of one of the stones to a subordinate, who in turn tells one crew to hewn out a big chunk of rock, another to get it to the site, and another to place it. Eventually, you get down to the single individual, with a rope over his shoulder, being told to pull in a certain direction. In this case, the master architect doesn’t need to know how to build a tricky crane, or even how best to tug on a rope, but because she can specify the work in such a way that it is clear to a sub-unit, and this works all the way down, she is in good shape.
It would be wrong to assume that this is somehow the “old-fashioned” way of doing things. There is still a good place for highly specified, hierarchical work. Especially for large software projects, the work lives or dies on the completeness of the specification. Even in some of the work I do, I can lay out so clear a specification, I could contract it out to a company or individual to perform. In fact, I did precisely this for a piece of research I completed recently.
If the specifications are clear, this distribution of work need not incur coordination costs. That is to say, the master architect is generally thought to operate in an existing structure, knowing the capacities of her subordinates. But the most interesting complex structures and systems–think New York City–were not master planned. They were created by the coordinated efforts of the invisible hand. The systems of exchange place a value on created artifacts. Sometimes, those artifacts are created without a very clear specification. Lotus 1-2-3 and the steam engine, while they certainly filled niches, were not specified as such. There was a general hunger for machines that could do certain kinds of things (provide a working space for numbers, create circular motion from heat), but their real impact seems to have come from a bottom-up, rather than a top-down organizational building process. That is, someone “discovered” something you could do with a steam engine or spreadsheet that was not part of its initial specification or even expectation. Dynamite.
How does this social-level division of labor translate into the group? First, understanding the nature of specification is important–how do you communicate what is needed? Second, it is important to understand that working outside, beyond, and beside specification is often a useful thing. Small, collaborative groups are worst when individuals produce something that cannot be integrated into a working whole, but they are best when they are free to have an “aha!” moment, communicate that to the group, and create a greater final product, even if it does not adhere to the initial specification.
Generally, we aren’t interested in building pyramids, airplanes, or operating systems. The division and analysis of such projects are important and informative, but when we think of creative collaboration, the process is necessarily organic. Or, I should say, it still relies on specifications, but those specifications are clear as to the goal without being restrictive as to the process. Moreover, they allow for specification rewriting on the fly. They need to allow for the potential of discovery.
In practice, my class assignments, for almost all of my classes, are more ambiguous than many students would like. This is contrary to a lot of educational theory, which suggests that you have much happier and more productive students if you spell out exactly what you expect to see. I think my resistance to this is largely a part of my own educational experience. I remember too many assignments that felt like tugging a stone up a hill. Sure, I could do it, but it wasn’t very rewarding.
My assignments generally say little about process, and a lot about results. I want something that does X, and will work with other parts of the assignment. The reason for this is simple: if a student figures out a way to do X that is outside my expectations (and ethical!), I consider this an even greater success than if they accomplish X in the same way as I would have done it. In the former case, their creative value is added to the process. In the latter case, it is much more akin to pure labor that is added. I already have it all in my head–the students’ role is merely to comply with that vision.
Threre is no magic formula for collaboration that allows for creativity. It is always a tightrope. If you ask people to “help on a project” without specifying what you expect and when, you are bound to fail. Particularly with a group that is unfamiliar, more specification is better than less. But you have to leave room for adjustments along the way: either indications that a job could be done in a better way, or early indications that the specification overlooked something important. These “indications” really come down to the question of communication.
COMMUNICATION AND COLLABORATION
“We have to communicate in order to work together well. Gee, thanks for the great insight.”
I know, the idea that communication is important to the collaborative process is nothing new. But despite widespread agreement that communication is important, there is a surprising diversity of opinion regarding just what this means. Here, I’m going to divide communication into “high-intensity communication” and “low-intensity communication.”
Low intensity communication is all about a single phrase: “How’s it goin’?” Allow me to illustrate with a conversation with a colleague yesterday.
R: How’s it goin’?
A: Oh, hi! I’m glad I ran into you, I meant to email you, I can’t make the meeting on Monday. I forgot I have to give a talk over at Wesleyan. But I’m definitely still interested.
R: Ok, no problem, it’s a short meeting and I’m planning to record it anyway. I’ll email you the MP3.
A: That would be great, and I’ll get back in touch from there about how I might fit in.
R: Sounds good.
Simple conversation heard in any office, anywhere in the world. It’s true, I probably would have remembered to email him and let him know I wasn’t coming, but this informal conversation did so much more than that email would have.
1. It was early warning. I might not have gotten around to emailing until Monday, allowing for less planning and different expectations by R.
2. It was highly interactive. We solved a small coordination problem in a miniscule amount of time. In more formal and asynchronous media, it’s not uncommon for discussion time to be disproportionately large for what should be very simple problems.
3. We shared common ground.
The last one is perhaps the most important, but let’s touch on the first two. In early microcomputers, we often talked about “interrupts,” a particular kind of semaphore that stopped a current operation and kind of said “Hey!” So, if you hit a Control-C, or the Big Red Break Button on some of the early PCs, it would send a signal directly to the processor that said “Don’t watch that, watch this!” and then had it perform something important. Interrupts were often triggered by peripherals: the mouse has moved, the printer is out of paper, etc. Today, email can be an interrupt, but often we are so flooded by email that we don’t respond immediately. I know that is the case for me.
The meeting in the hall has a whole dance built into it. I’m moving as we are talking, slowly indicating that we only have a few seconds to get this done. Moving apart, glancing furtively at the door down the hall. If it’s important, you can stop me, and say “this demands more of your immediate attention.” But generally, there is a push to get our conversation done quickly, but satisfactorily. We’ve been doing this dance since we were born–arguably for even longer than that.
This provides a kind of “situational awareness” or presence, a tacit understanding of what people are up to, what kinds of demands they have on their time, what their state of being is–do they look stressed, happy, confident, frightened. All of this information can be gathered at a glance.
Second, we go back and forth quickly, sometimes speaking at the same time, often interrupting (in the traditional sense) the conversation. This kind of feedback leads to very efficient communication. If you don’t need to tell me something, I’ll make that clear by my facial expression or by saying “yes, I know.” If something is unclear, I can immediately ask you about it. So, you give up control over that moment in time–you cede at least part of your attention to this exchange–but as a result you can forego a lot of back and forth that might otherwise take hours or days. This can be accomplished on the phone or IM as well, though without a lot of cues that make such interaction particularly effective.
I’ll give an example that happens every semester. Someone in the class can’t do something: access a paper, log on to the blog, etc. Inevitably, they email, and I email back. Early on, I recommend an IM or phone conversation, but they just want to solve it over email. I understand that, it’s a lot easier than finding a time we can both meet. Nonetheless, if we open up a chat window and go through the process together, it always takes less time than trying to ascertain exactly what goes wrong at exactly what point. That sort of interactivity is vital to creating a common understanding of the situation.
But the last piece is perhaps the most important. Study after study has shown that teams, either virtual or traditional, live or die on the level of shared trust. There are certain teams that cultivate a culture of extreme reliability: the military and firefighters, for example. Of course, if you have the choice, you don’t go into a firefight (of either kinds) with people you don’t know. But sometimes, in war and in business, you have to work with people you don’t know. In particular, with creative teams, you want people who do not share the same cultural background, the same expectations, the same experiences. Indeed, this “groupthink” encountered in more structured teams can lead to some pretty dumb behaviors. So, we have another tension: bringing together people of different cultural backgrounds, and at the same time, allowing for trust to emerge.
So as long as we want to get something done, but don’t care too much about it being creative, innovative, or different, we know what we need: likeminded folks with a clear hierarchy and explicit task assignments. Unfortunately, when we want creativity, we need to allow for a bit more “wiggle-room.” The only way we can wiggle, and possibly wobble, but not fall down, is to increase the amount of communication within our teams. So, if you want to be creative, put your people in the same room. No surprise that ad agencies and social software companies tend to forego private offices and cubicles for open space–or at least highly configurable space.
How do you encourage the kind of informal communication that allows for the development of common ground among uncommon people, and for the kinds of tacit, interactive knowledge necessary to make good use of time? You need to plan for unplanning!
I like to have breaks in my classes, not just because people need to go to the bathroom, but because students learn more during breaks than they do in class. Why? No matter how interactive my seminars are (and since I tend toward the loquacious, they aren’t always that interactive), break is a sanctioned period of interactivity. It seems to be designed for “Hey, howsitgoin’?” So, why not just make every moment a break?
A break is not just an open period for open interaction, it is a place that exists that fosters that interaction, a set of expectations that such an interaction is permitted, and a certain time limit that forces barriers to be quickly broken down. For those of you who have read Bey’s Temporary Autonomous Zone, this might sound a bit familiar. But for Bey, the temporary nature of the TAZ is a result of something akin to reverse-entropy, a natural law of power that reasserts hierarchies. I think it is quite the contrary: the energy that allows for autonomous communication comes from the fact that it will soon end. Your exchanges are more open and more honest because they exist within a pre-defined time frame.
Knowing how much and under what conditions to encourage “non-mission-critical” communication is difficult. Within hierarchical communication structures–which remain the standard for corporations–such communication appears to be wasteful. After all, what does chatting about the big game have to do with pulling that rope? But in organizations that thrive on knowledge and creativity, experience has shown that there simply is no formal structure that can make up for informal chat. Chat is a prerequisite for getting good work done.
It is not a replacement, however. You cannot give up on the spec or on the structure entirely. In all things, a balance.
There is no one right way to organize a team, but there are many ways to do it badly. Let’s briefly discuss a couple of strategies that seem to work.
“The Mythical Man Month,” by Brooks, is a classic of project management for software development. The central argument that Brooks makes, as the title suggests, is that adding manpower does not necessarily reduce the amount of time it takes to complete a project, and often, in fact, increases the amount of time needed. This is no shock to anyone who has ever worked in a team and simply given up, deciding that they can do the job faster by just doing it themselves.
Brooks recognizes this impulse and suggests it is the right way to go, in part. He explains an idea proposed by Harlan Mills, who argues that one of the best ways to organize a programming team is by modeling it on the surgical team. So, who is on this team? He lists out several roles. Some of these roles may be taken on by the same person in a small team, or by several people in a large team.
First, there is the SURGEON, the person who creates the specification, constructs it, and writes out the documentation. He does this, because his experience allows him to draw on earlier models and effectively execute a new design. The COPILOT, who Brooks describes as the “alter ego of the surgeon” can do part of the job, but may be less experienced. He is there as a thinker, discussant, evaluator, and in-house critic. He knows the project inside and out, and is a safety-net in case the surgeon is hit by a Mack Truck. He doesn’t have primary responsibility for getting the project done, and doesn’t have primary responsibility for any of the pieces, even though he may step in to cover or assist for a piece that doesn’t get done.
The ADMINISTRATOR is the person to whom the surgeon delegates all of the nitty-gritty work to. This is the person who sets up meetings, who keeps up the schedule, makes arrangements for space, and–in a traditional organization–makes sure tools are purchased, people are hired and paid, and the like. There is a long history of the importance of administrators to the creative process–without someone taking on the vital day-to-day stuff, projects get mired in paperwork, small disputes, and tax-audits.
Even though the surgeon creates the documentation, we all know that writing is about rewriting, and so there is an EDITOR. When Brooks wrote this, the processes of writing were a bit different, but it still the case that a lead project person can provide the “guts” of a piece of documentation, and leave it to a professional editor to organize it, clean it up, polish it, and produce it. Of course, in the context we are working in (as opposed to software production), the product IS the documentation. (In fact, that’s true of many software projects now too!) Nonetheless, someone in charge of the final presentation of the work–communicating the project to the higher-ups and to the public–is still an invaluable role.
A TESTER is needed to make sure the work is actually suitable. In the case of interactive design, this is a person who is in charge of running usability, making sure that the expectations of the designers are on a level with the expectations of some set of users. Traditionally, a system was created, and then it was tested by a group, and then released. Organizations who can’t afford to do it wrong are now implementing user testing early and often: running initial sketches by a user community, and then getting feedback along the way. Having someone in charge of this can be very beneficial, so that the surgeon can keep on creating, while receiving criticism, suggestions, and feedback from the tester and her flock of model users. A tester might also take on the role of accessibility guru, making sure the needs of diverse users are considered.
Brooks goes on to add more people, including a couple of secretaries to handle correspondence, a program clerk to keep up internal records and code, a person to keep the machinery in working order, and someone who was a sort of “standards bearer,” making sure the code was up to snuff.
I’ve played pretty fast and loose with the description Brooks offers, and I encourage you to go back and read the entire book, which while dated provides a vital foundation for understanding project management in IT, but I think that this outline provides for a good set of responsibilities for a small, three or four person team. It’s probably not what you have experienced, which looks more like the pyramid construction group.
The problem with the above approach is that it tends to overload the surgeon, and to a lesser degree, the copilot. It’s worth noting that the work of the administrator and editor are not trivial (even though they are often trivialized by the creative or technical people), but even so, a very disproportionate distribution of work can lead to predictable failures–social failures. For example, the person who professes to be the least experienced will sometimes be placed in what seems like less mission-critical positions: testing or administration, for example. The result is a product that is hit-or-miss, all or nothing, and sometimes late. Moreover, the person doing all the work barely has time to get it done, let alone open up lines of communication with everyone else. She needs someone to act as an interrupt. But if the uber-surgeon isn’t the way to go, maybe something a little more… democratic?
The most common way of doing this is for an administrator to spec out PIECES of the final project, have each person work on a piece, have everything due just before the deadline, and be in charge of stitching these together into a cohesive whole. The problem is that the last part doesn’t always come together. And in an effort to allow for equality over quality, the end-product is uneven as the talents of the members in the team. The whole idea of a team is that the end product represents something better than could be accomplished by the most able member, NOT something that represents the mean or average ability of the team. More importantly, everyone on a team learns more from an outstanding project than they do from a mediocre one.
If you look at how surgical teams really work, the surgeon is certainly not on her own; she doesn’t have enough hands! Delegation is important, but that delegation is only possible with seamless communication between people working on a patient. Lots of people have studied how surgeons actually work, and if one person on the team is not paying attention, or is not able to anticipate the needs of the project, the surgery is less successful.
A more recently popular model of creating software is called “agile development.” To provide some history here, earlier attempts at developing large software systems have followed some idealized models. One was the “waterfall model” in which very careful planning and specifications were finalized before moving on to each successive step, ending in testing and distribution. The “spiral model” suggested that more iterative passes at the software could start with a rough (but not pretty) working model, and assuming that the system met the basic needs, it could be fine tuned in a series of passes afterwards. More recently, there is a lot of buzz over “agile methods” (including approaches like “scrum,” “extreme programming,” and “feature driven development”). Again, I’m not going to remain true to the extensive definitions of these, but I do want to pull some ideas.
Basically, the central feature of each of these approaches is that they are deadline driven. Rather than waiting for the perfect completion of a specification, the idea is to produce early and often, and go back and fix or change what needs to be changed. So, a group might say “we need to show a first version of this in the next week, what needs to happen to have a complete version by then.” After feedback, it is tweaked and improved. Again, writing is all about rewriting, and design is all about redesign.
Extreme Programming (XP) is organized around a set of values: Communication, Simplicity, Feedback, and Courage. The first of these, communication, we’ve already talked a bit about in terms of the importance of informal chatting, but it is also vital that the formal requirements are clear, and continually updated and clarified. While you should start with a set of clear, concrete goals, as the project moves forward, those goals will become increasingly specific. That specificity should exist in the spec as well as in the product itself. In other words, the team should have a certain shared consciousness about what they want to be doing with the project, not just what they actually are doing.
The idea behind XP is to start with the simplest solution and then build on that to something more “featured.” Now, this can be dangerous. I can think of a number of examples (cough… Blackboard! Windows!) that started out on a foundation that was ill-suited to support their ultimate roles. Nonetheless a simple and functional basic design is core to a useful design–particularly when that simple design is created from the outset to be extensible.
Feedback, as with the surgical unit, should come from everyone. The more eyes, the more ideas, the better. And feedback should start happening at the earliest stage possible. This ties back pretty closely with communication. If the work is not easily accessible, if everyone is not aware of what work is underway, and (this is vital), if there isn’t a clear review cycle in which people are showing what they have, this feedback becomes impossible.
Courage is a hard thing to drive for, but it means, in some ways, a courage to think big. Are you designing something for the guy down the street: think big! This could become the most viewed site on the web! It could become a classic! How do you make sure that you are making something that can be a classic, that will work today and in ten years, and evolve with new demands. Courage also demands knowing when to cut, when to throw something away that does not work.
These four values provide a framework, but not a process. As alluded to above, that process is really analysis not of the project in space–that is, not into big chunks that get stitched together–but over time. The specification becomes an outline, a wireframe, a sketch, a barely working model, a fully featured model, a skinned model, a documented model. It is split not over unit so much as over the life cycle or evolution of the project. The customer is part of the team at every stage, as is everyone else on the team. In terms of organizing people and resources, the work is often completed in a “bullpen,” the model that is favored, again, by many deadline-driven creative efforts. This is not a space for hermits who like to go off and do the work at their own pace. The delivery dates are back-to-back, with micro-improvements and feedback at each step.
The two approaches described in this section can be used effectively together. That is, while a surgeon may not tell a surgical nurse to “remove that kidney,” asking her to “clamp this” is perfectly acceptable. That is, if you are on an iterative cycle, in which you produce four or five or eight versions of a product before it is right, deligating little bites both avoids large problems of differences in expectations, and allows for the project to “hang together” as a whole.
VIRTUAL COLLABORATION & SOFTWARE
As I noted above, to “collaborate” is to “work together.” So “virtual collaboration” must be an oxymoron: “to work together apart.” Of course, as we all know, collaborating in person is increasingly the exception, rather than the norm. And we all know how easy it is for these to fail. If we look at the models above, we know why. Being physically proximate allows for a greater degree of trust and respect. It also allows for the kind of iterative coordination necessary for projects to quickly and effectively move forward. How do we make this work in virtual space.
Some have suggested the best way for this to work is a form of “virtual reality,” or some other way of making presence felt among a group. In practice, some of these techniques, like extensive use of always-open video connections, can provide some of the presence and interrupt-driven communication that makes a project work. But there are also some structural things we can do that are not directly reliant on software.
There have been a lot of collaborative systems introduced over the years. You may be familiar with (or use) Lotus Notes, for example, one of the most popular of these systems. But there are hundreds of such systems designed specifically for particular industries. All of them provide a way of representing and sharing complex knowledge products. Some of them dictate particular process as well. The latter of these–those that incorporate a very clear process–have met with limited success. But flexible ways of sharing information have been more successful.
It turns out that some of these more flexible tools–telephone, IM, email, wikis–provide the basis of communication for well-organized projects. Structured project development does not rely on structured information systems. Good management works fine with flexible communication systems, and can take advantage of new developments and is open to those outside the group. It also means that people can quickly start working on the work of the project rather than on learning a new way of communicating. Finally, and most vitally, these flexible channels allow for informal communication and chatting to take place. Nothing can replace the dinner party as an organizational principle.
So, how do you make sure a product works?
Update as close to constantly as possible. Construct virtual hallways. Open up your IM client and keep it on. Chat for no reason other than just to say “hi.” Advertise your successes, even when they seem excruciatingly minor.
2. Proliferate your deadlines
Make a deadline every day. You don’t have to get 12 hours worth of work done between those deadlines: 15 minute or 30 minute tasks are fine. But don’t let those deadlines slip. Be ready to show improvement, even minor improvement, at each step. This allows you to catch problems early on and deal with them.
3. Your “final product” is never final.
Plan on something you and the client will be happy with well before your “final” deadline. This provides plenty of time for fine-tuning. But also, expect to revisit this design and re-use it–either as a whole or in pieces–in future work. Design work is never really “done,” and the more you can look forward to ways it might be used in the future, the more rewarding the process will be in the long term.
So, what do you think? Could what I’ve written be improved? I’ve put a copy of it on the wiki here. If you can improve on it, make it simpler or clearer, even in a small way, please do.