Diaries of a Core Maintainer #6: A tale of two developers

(Hey, neat! This was mentioned on ZDNet: Perfectionists need not apply.)

Most people in the free software community have probably read Eric S. Raymond's The Cathedral and the Bazaar, which compares and contrasts two free software development models: the Cathedral style (think, "perfectionism") and the Bazaar style (think, "chaos").

While Drupal itself is clearly an open, Bazaar-style project, many individuals in the Drupal community tend to take the perfectionist approach to development. After all, *thousands* of people will be using this code, and likely *hundreds* of developers carefully inspecting its inner-workings. How embarrassing would it be for another developer to stumble across a "no-brainer" bug in your code? Best to sit on things until you know it's really solid before putting it out there in front of people. Right?

Sure, that sounds logical. But in my experience, people who take this approach to development in an open source community, and especially the Drupal community, are at a severe disadvantage to those who embrace the chaos and put their changes out in front of everyone as they're going along, warts and all.

I'll attempt to illustrate this by way of a dramatization comparing the problem-solving approach of two hypothetical Drupal developers: Sloppy Sam, and Perfectionist Pat.

Sloppy Sam

Sloppy Sam is a hobbyist, who has been coding PHP off and on for a few months. One morning, she gets an idea for a new feature for Drupal core. She begins by searching the issue to see if something like it exists already and, finding it does not, posts a new issue with a general description of her idea. She then opens her IRC window and asks in #drupal, "Hey, folks! I had an idea for a new feature for core that [brief summary]. What do you think about this? [link]" Chatty Charlie from #drupal reads the issue and says, "Hey, that sounds pretty cool! You should talk to Fancy Fran about this, because I remember her talking recently about doing something similar."

Since Fran isn't online right now, Sam decides to whip out her editor and slap together some basic code that has the general jist of what she's trying to do, and posts it around lunch time to the issue, marking it "patch (code needs work)." She again asks in #drupal, "Ok, here's the general approach I'm thinking: [link] Does this seem pretty good?" This time, Fancy Fran is online, and replies, "No, actually. you're going to run into problems with caching. Here, I've pastebinned a copy of a module I wrote before that did something similar. This will probably help you."

Delighted, Sam takes the code from Fran and merges it into her own work, and posts another patch at supper time, this time upgrading the issue to "patch (code needs review)." Each time that Sam posts a new follow-up to the issue, it's bumped up to the top of the main issues listing. This causes Reviewer Rodney, who spends lots of time browsing this page, to stumble across the issue. He recognizes Sam's name from other issues he's seen, and since he knows her to be an active contributor, decides to spend some time thoroughly testing the patch and reviewing the code. Rodney leaves detailed comments about code standards violations, missing tests, points out a logic problem in one part of the code, and suggests some clarity to the documentation. He then marks the issue back to "patch (code needs work)."

Sam thanks Rodney for his comments, and makes the necessary adjustments just before having her evening tea. When finished, she sets the issue back to "patch (code needs review)." She then asks in #drupal for a "review swap," which both helps to get her patch reviewed *and* gets her more exposure to other parts of Drupal she might not otherwise have encountered. Chatty Charlie, who is already up to speed from having talked to Sam about her idea earlier, takes her up on it and gives the patch a quick test drive and makes a once-over through the code and then proclaims it "patch (reviewed & tested by the community)," just in time for bed. By the time she wakes up the next morning, it's been committed, and is now part of the latest release of Drupal.

Perfectionist Pat

Perfectionist Pat, a brilliant computer engineer with ten years' experience in software development, envisions a new feature for Drupal core. He begins to plan his improvements as he would any of his other projects: with a piece of paper and pencil, carefully analyzing the problem and perhaps drawing out some flowcharts and diagrams to help him to understand the existing code flow and how his improvements fit in.

He sees that there are a few different ways of approaching the feature, and consults a variety of reference material: books, RFCs, and specification documents. After a few days of evaluation, he is confident that the second of his three choices is the most optimal way to go.

The next day, Pat sits down at his IDE (which is naturally configured to automatically conform to Drupal's coding standards to the letter) and begins carefully crafting and documenting the various functions, hooks, and page callbacks he will need. As he goes through, he also writes automated tests so that he can ensure that every branch of logic is thoroughly tested.

Within another three days, he has the functionality nearly completed. Or so he thought! Upon manual testing, he comes across something he hadn't originally forseen: there is a problem when caching is enabled! Pat smiles smugly, proud of himself for catching this before it ever saw the light of day in the public eye. He re-evaluates his earlier analysis, and begins to refactor his solution to take this new information into account.

Finally, just ten days later, Pat has a patch that he feels is core worthy running on his computer. He takes it through the final paces of SimpleTest and Coder modules, just in case, and then posts a new issue to the issue queue, marked "patch (code needs review)." Now, he waits until someone comes across the issue to give it a review.

Because Pat's patches tend to be fewer and further between than Sam's (since his are always uploaded perfectly the first time), Reviewer Rodney doesn't recognize Pat's name when he clicks on the issue. And since Rodney's is a bit pressed for time today, he moves on to another patch instead.

Pat begins to grow frustrated, and comes into #drupal demanding to know why no one has taken a look at his perfect patch that he toiled over for more than a week. Chatty Charlie takes pity on Pat, and decides to click into the issue. He recognizes it as the very same issue that Sam was working on just yesterday, and since hers has already been committed, Charlie marks Pat's issue as a duplicate. Infuriated, Pat clicks into Sam's issue intending to tell her off and show her how a real programmer solves this problem, only to see that through the community review process, Sam came up with exactly the same patch, only in one tenth of the time, and with far more detailed PHPDoc comments, thanks to Rodney's review.

Observations

Obviously, these are two very extreme, made-up examples that paint both a rosy, best-case scenario and a tarnished, worst-case scenario to make a point. In reality, most people and most situations are somewhere in the middle.

Both approaches outlined here are a reasonable way of "getting things done" in the Drupal community, and both also have their time and place. There are definitely times when it makes a lot of sense to get out a whiteboard and start whipping up flowcharts before jumping into coding. And there is also a limit to the community's patience for how many times you bring them in to look at what is still sloppy, first-attempt code. And despite anyone's best efforts, there are definitely times when you can't get people to bite at the community review process, no matter how much you try, and you have to go it alone.

But, if we analyze Perfectionist Pat's approach, we see that it had some detrimental effects:

1. Pat is out of touch with what the Drupal community is doing. Note how his first instinct is to sit down and solve the problem, not look to see what others in the community were doing. Had he posted an issue when he began to solve the problem, not only would he have "staked out" the fact that he was working on it, but Sam would've found the issue in her search days later, and could've worked with him rather than solving the same problem in parallel. This ended up wasting both of their time.
2. Pat works in isolation. As a result, he is missing out on the biggest benefit of using an open source product: the thousands of other developers who understand the system innately and can help him short-cut his learning curve. By not working with the community, he also misses the opportunity to do things like "review swaps" that can help him get more eyes on his code. By not working in the open issue queue, he also misses out on the opportunity for more eyes to stumble across the work he's doing and offer guidance/reviews.
3. Pat fails to earn "karma" in the community. Even though her initial few attempts ended up not being correct, because Sam had more "touch points" of positive interaction with the community, her name stands out more from the crowd. This has a lot of benefits, including people being more likely to find time in their day to help her out. In fact, Pat's frustration-laden comments in #drupal after he didn't get his patch reviewed probably earned him negative karma in the eyes of some, which will have the exact opposite effect.

One other interesting thing to note is that all of the "famous" Drupal developers that you've likely heard of take Sam's approach. And here you thought they were all so smart. ;)

Conclusion

Regardless if you're a Pat or a Sam, when working within the Drupal community, keep the following points in mind:

1. Resist the urge to isolate yourself until "perfection" is achieved. It can be normal in a "day job" (as well as school projects) to lock yourself in a room for 3 days and come out with a shining example of code. But doing this in an open source community is missing out on what is arguably the biggest benefit of using open source -- other peoples' expertise with the system. You can still achieve the same perfection in your code you're used to; you can just do it much faster by talking to other people as you go along.
2. Be open and transparent. Begin any problem solving session with a search of the queue, and a new issue if one doesn't exist. The sooner the Drupal community catches a glimpse of your work, the sooner someone can jump in and make suggestions. Post your progress to the issue as you go: interim patches, questions, summaries of IRC discussions, etc.
3. Scratch other peoples' itches while they're scratching yours. When asking something from others, be prepared to give something in return. For example, "review swapping" or answering questions in #drupal-support, or clarifying documentation. This will not only help ramp up your "karma" within the community, but can help you gain experience in different parts of Drupal you wouldn't otherwise be exposed to.

Follow these guidelines, and you should have a much easier time with whatever you do in Drupal. :) And hope to see you in the issue queue, all you Pats out there! :D