Saturday, November 19, 2011

Thinking Together

"Nestor, gladly will I visit the host of the Trojans over against us, but if another will go with me I shall do so in greater confidence and comfort. When two men are together, one of them may see some opportunity which the other has not caught sight of; if a man is alone he is less full of resource, and his wit is weaker. "  Homer, Illiad 10  (Diomedes about to go out spying on the Trojans)
I have always loved the simplicity in the image above, which Plato paraphrased simply as "when two go together, one sees before the other."  Real collaboration works this way - like Diomedes and Odysseus, we think together, and do more, better, faster than we could alone.  For this to work, we have to be thinking about the same thing.  Rather than just periodically sharing fully working ideas, we need to think out loud, sharing the not-yet-sense that eventually workable ideas come from.

I remember when I was a mathematics grad student I thought I would never be capable of producing the brilliantly incomprehensible work that I saw published in professional journals.  It took me forever to read papers and there appeared to be so many little miracles embedded in them that there had to be some magical wizardry involved.  All of that changed when I started thinking together with some real mathematicians.  Seeing the thought process, the not-quite-sense at the base of the ideas, the stops and starts, the workarounds, let me see that while I would never be Diomedes, I could at least have a little fun as Odysseus.

The key here is to actually externalize and share the thought process underneath ideas in the making, rather than just handing them back and forth as "products."  I am belaboring this point because it bears on a dynamic in the open source world that I find alternatively exciting and troubling - the rise of distributed version control systems (DVCS).  Thanks to Git and GitHub, the process of contributing to open source has become much easier and the number of projects and contributors is exploding.  That is the exciting part.  The thing that I sometimes worry about is that the "forking is a feature" meme that can result from DVCS can take us away from thinking together and more toward "black box" exchange of completed ideas.  The traditional, central VCS, central mailing list model works more or less like this:
  1. Someone has an "itch" (idea for enhancement, bug fix, idea for a change of some kind)
  2. Post to the project development mailing list, talking about the idea
  3. Patch, maybe attached to an issue in the project issue tracker, maybe directly committed
  4. Commit diff from the central repo triggers an email to the list - everyone sees each small change
  5. Idea evolves via discussion and commits, all visible to all following the project
Using DVCS, the process can work differently:
  1. Someone has an itch
  2. Clone the repo, creating a local branch
  3. Experiment with new ideas in the local branch, possibly publishing it so others can see and play with it
  4. Submit a request for the changes in the local branch to be incorporated in the "mainline"
  5. Changes are eventually integrated in a batch
Now there is no reason that the second flow can't involve the full community and be done in a thinking together way in small, reversible steps; but it is also easy for it to just happen in relative isolation and to end with a big batch of "black box" changes integrated in step 5.  If this becomes the norm, then the advantages of OSS communities thinking about the same thing as they evolve the code is significantly diminished.  It's also a lot less fun and interesting to review large, unmotivated pulls than to participate in the development of ideas and work collaboratively on the code expressing them.  But most importantly, if the primary means of communication becomes patch sets developed by individuals or small subcommunities, and the only thinking together that the community does is after the fact review, we lose one of the biggest benefits of open source - the collaborative development of ideas by a diverse community influencing their origin and direction.

The DVCS train has left the station and as I said above tools by themselves don't determine community dynamics.  Its up to those of us who participate in OSS communities to keep engaging in the real collaboration that created the great software and communities-around-code that have made OSS what it is.  So lets take a lesson from Diomedes.  Great warrior that he was, he still thought it best to bring "the wily one" with him when he went out into new territory.


  1. Sorry, but I think that both descriptions of possible workflows don't reflect reality because they describe only one change or "itch" (or topic) in isolation.

    In reality there are dozens of different topics worked on in parallel by different people.

    These topics must flow into the central repository one after the other and topics that have not yet been included needs to be repeatedly rebased against a moving trunk.

    This rebasing is a hell with SVN and a no brainer with Git.

  2. Good point, Thomas, and one of the reasons that DVCS is a big enabler for more contributions. The challenge is how to leverage that power in a way that keeps the community thinking together at least regularly and while ideas are forming.

  3. Legend has it that when the train was first invented its detractors claimed that the human brain would go into epilepsy watching the scenery passing by at 30 km/h.

    Your model of DVCS collaboration ignores the fact that people can and do comment and discuss changes made on a local branch. It seems to me that you have not actually used github in practice, otherwise you would not have posted the above.

  4. You are missing the point, Ceki. I see nothing inherently wrong with DVCS and know well it is possible to have great discussion and community engagement using it. The point is that regardless of the tools, you need to focus on keeping the community thinking together. Note the use of the word "can" above. The post is about community engagement, not tool choice.

  5. You are right. I had not read your post with the attention that it deserves. I apologize.