webchick.net

very helpful lioness

How and why D8 Accelerate is spending your hard-earned cash

Thu, 03/26/2015 - 12:13 -- webchick

Hopefully you've heard the news about the Drupal Association's new D8 Accelerate grants program, and the fundraising drive we have currently going on. If not, the gist is that the Drupal Association has created a central fund, managed by the Drupal core committers, to fund both "bottom-up" community grants for things like targeted sprints or "bug bounties," as well as "top-down" spending driven from the core committers on larger strategic initiatives that help accelerate Drupal 8's release. All D8 Accelerate grants that are provided are tracked centrally at https://assoc.drupal.org/d8accelerate/awarded, including what the money was used for, how much was spent, to whom it went, and a report from the grant recipient(s) that outlines the work that was accomplished.

However, it can be a little hard to parse from that format the larger meaning/context of this work, especially if you don't spend upwards of 30 hours per week in the core queue like most of these folks. :) As Chief Wearer of Many Hats™, I sit on both the Drupal Association board as well as the committee who manages these funds. This puts me in a good position to provide a bit of "behind the scenes" info on how the funding process works, as well as provide some of the larger story of how these funds are benefitting not only Drupal 8 core, but the larger Drupal ecosystem.

Drupal 8 Acceleration

Performance

A flamegraph of what functions take the longest on the user/password page in Drupal 8
Source: https://www.drupal.org/node/2370667

As noted in my post-DrupalCon Bogotá critical issue run-down, performance improvements are a large chunk of the work remaining in Drupal 8. We had deliberately postponed most of this work until post-beta to avoid premature optimization and to allow all the major architectural chunks to be in place. However, we definitely can't release Drupal 8 while it is much slower than Drupal 7.

The D8 Accelerate fund has been instrumental in not only helping to address critical performance regressions, but also in accelerating the development of Drupal 8's next-generation cache system.

Or, to sum it up in a cheesy catch-phrase:

For less than $10,000, we're making Drupal 8 twice as fast! :D

Win!

Unblocking the beta-to-beta upgrade path

The second large focus of D8 Accelerate grants has been around D8 upgrade path blockers. These are extremely strategic, because they unblock a beta-to-beta upgrade path between Drupal 8 releases, which is extremely important for early Drupal 8 adopters.

For example, one large chunk of work D8 Accelerate has funded is in making sure Entity Field API and Views work well together. This is critical for features such as Multilingual, so content shows up in the right languages when expected, and it's necessary to complete this work prior to providing an upgrade path since it would be horrendous to write hook_update_N() functions for some of the necessary changes.

We're also exploring other alternatives to provide a beta-to-beta upgrade path to early adopters sooner, which is viable now that the hardest data-model-changing issues are done.

Security

Drupal drop on a padlock
Source: http://www.codepositive.com/code-positive/about

Obviously, we do not want to release Drupal 8 with known security vulnerabilities, but neither do we want to release an "upgrade path beta" that we encourage early adopters to use with known security vulnerabilities. Hence, we are trying to get any critical security issues taken care of sooner than later.

For example, one chunk of work that D8 Accelerate is funding in this area is tightening security around Drupal 8 entity API. Numerous form validation functions in core contain entity-level validation, to the wild dismay of anyone who's ever tried to implement web services on top of Drupal. The reason that's bad is because if you attempt to save something using just the Entity API (as you will in a REST API scenario where there are no forms), you will end up skipping validation routines and could end up with invalid and/or insecure data entry.

Targeted Sprints to Crush Criticals

Menu link critical sprint at DrupalCamp New Jersey
Source: https://groups.drupal.org/node/456353

As demonstrated at the Ghent critical issue sprint last year, nothing is better for D8's velocity than getting a bunch of awesome contributors in the same place to pound on the critical queue together. D8 Accelerate funded a fantastic Menu link critical sprint at DrupalCamp New Jersey which, in addition to turning this area of criticals from "OMGWTFBBQ" to an actionable plan, resulted directly or indirectly in all of the related critical issues in this area being closed, within a couple of weeks after the sprint.

We aim to do more of these same types of targeted sprints throughout the year, the next one being the DrupalCI sprint in a couple of weeks. More on that in a sec.

Drupal Community acceleration

Testbot modernization

Architectural diagram of new testbot
Source: https://www.previousnext.com.au/blog/architecting-drupalci-drupalcon-ams...

Our beloved Drupal.org testbot has been showing its age. https://qa.drupal.org/ is still on Drupal 6, and largely on life-support these days. Testbot doesn't support testing on multiple PHP versions and databases, both of which are Drupal.org (websites/infra) blockers to a Drupal 8 release.

The DrupalCI: Modernizing Testbot Initiative is being driven by numerous devops-inclined community members from around the world. It aims to rebuild testbot from a collection of Drupal modules to a more standard CI stack (using big fancy words like Jenkins and Docker and Puppet and Travis and Silex) that your average PHP/devops folk can both understand and help maintain.

There's been a lot of work on DrupalCI already, and the upcoming D8 Accelerate sprint on DrupalCI: Modernizing Testbot Initiative will bring all the various contributors together to form DrupalCI Voltron to get an MVP of all the various pieces working together. Actual deployment will happen some time later, and both the new and old testbot will run alongside each other for a good while so any kinks can be worked out while D8 development stays stable.

This particular improvement not only allows Drupal 8 to ship, it also will provide great new functionality to all projects on Drupal.org! The architecture allows for ample room for later expansion as well, so we could start doing things like automated code reviews, performance testing, front-end testing, etc.

People power!

Here are some of the awesome Drupal contributors who've benefited from these funds:

dawehner
Daniel Wehner (dawehner)
Daniel is a major driving force in Drupal core, as well as the person with the most commit mentions in Drupal 8. His work spans not only the Views in Drupal Core initiative, but in all other areas upon which he sets his sights. No issue is safe! :)
larowlan
Lee Rowlands (larowlan)
Lee is another powerhouse core generalist who tries to tackle a #CriticalADay. You can learn more about Lee in this Community Spotlight.
amateescu
Andrei Mateescu (amateescu)
Andrei is a co-maintainer Drupal 8 Core's Entity reference field, Field API/UI, and the transliteration system. He's also contributed to dozens of contributed projects.
plach
Francesco Placella (plach)
Fancesco has been a significant contributor to internationalization functionality since the Drupal 7 days. In Drupal 8 he's a key fixture in the D8 Multilingual Initiative.
fago
Wolfgang Ziegler (fago)
Wolfgang has worked with Drupal since 2005. In addition to his major contributions to Drupal 8's Entity Field API, he also maintains various widely-used contributed modules such as Rules and Profile2.
klausi
Klaus Purer (klausi)
In addition to his work driving Drupal 8's REST functionality, Klaus is also a member of Drupal's Security Team and a driver of automated tools for coding standards checking.
Fabianx
Fabian Franz (Fabianx)
In addition to being a mad performance scientist extraordinaire, Fabian has also been a prominent contributor to the Twig in core initiative.
Jelle_S
Jelle Sebreghts (Jelle_S)
One of the drivers of mobile improvements in Drupal 8, Jelle also maintains dozens of contributed modules through his work at Attiks.

Not just code!

D8 Accelerate funds are being used not only to fund development work, but also to fund patch reviews as well as more "project management"-y tasks like "triaging" a set of issues to find the truly critical ones, research on different approaches, etc. Wherever possible, the core committers explicitly look for opportunities to fund two people, usually a developer and a patch reviewer, in order to maintain the sanctity of Drupal core's peer review process.

I think this is great, because it highlights that it's not just raw PHP that's going to get Drupal 8 out the door; it's a joint effort of many complementary skills coming together.

Show me the money!

Skeptical cat is fraught with skepticism
Source: https://blackincense.files.wordpress.com/2008/10/skeptical-cat-is-fraugh...

Why $250k to accelerate D8?

This is a very reasonable question to ask, particularly in light of the widely-cited statistic of 2,750+ contributors to Drupal 8, and various Drupal companies employing major contributors to Drupal core. Here are a few points:

  1. Drupal 8 will be a truly revolutionary release, not only by providing tons more useful functionality out of the box for site builders and content authors (WYSIWYG, mobile support, Views, configuration management, etc.), but by modernizing the underlying code base to address years of technical debt, and help "future-proof" Drupal for the next 10+ years. Unsurprisingly, this means that the total amount of work that has already gone into D8, and that remains needed to move D8 from a late beta to a release, is larger than it was for earlier versions of Drupal. Most technology maturations follow that pattern.

    Drupal 8's release also unlocks the move to a new release cycle that introduces backwards-compatible feature releases every 6 months. This allows us to "release early, release often," as opposed to "release every 4+ years, coupled with lots and lots of API breaks." ;)

  2. For these reasons, as well as many others, there's significant community benefit for 8.0.0 to be released as soon as possible, both so that sites can be built on it, and so that the 8.1.x branch can be opened for development for everyone with a feature idea itch to scratch. Additionally, many organizations and individuals who would benefit from D8 getting released sooner than later don't have the expertise or time to solve the remaining critical issues. These organizations/people might be willing to contribute money, but don't know who to best send it to or don't want to deal with the administration of contracting directly with individual core contributors. This fund is an opportunity for those organizations/people to make a difference without dealing with that administration.

    Make no mistake: Drupal 8 will get done, with or without this money. The goal of the fund is not about saying that our current awesome core contributor base is incapable of completing the work; it's only a recognition that funding work can make it happen faster.

  3. Why is this so? It's a common misconception that most core developers are paid for their work, either by Drupal companies who employ them, or by their customers. In reality, those directly financially compensated for their contributions to core (and especially to Drupal 8, which is not yet commercially viable for the masses) are a tiny fraction of the overall number of contributors.

    While there are numerous contributors who have already spent literally years contributing to core during their nights and weekends, and as a result have developed the kind of expertise needed to finish some of the remaining hard critical issues, relying on their ongoing availability of free time is not sustainable. These include contributors who work as freelance developers for clients, and it's certainly unfair to expect these people to turn down paid client work in order to have free time to work on core, or to quit being freelancers and become employees of forward-thinking Drupal companies who provide company time for core contribution. One of my favorite aspects of D8 Accelerate is that it is helping to "level the playing field" by making it possible for these people to have time to work on core regardless of their current employment situation.

    In short, don't confuse "has a job" with "is paid to work on Drupal core."

  4. It's also important to emphasize here that injecting funding into the "bug fix slog" phase of major Drupal releases, when all the fun stuff that tends to motivate volunteers is long exhausted, is nothing new. That should come as no surprise, given that there have always been companies with financial interests in having a given version of Drupal ready sooner. For example, in Drupal 6, Acquia funded release manager Gábor Hojtsy full-time to help get that release done. In Drupal 7, in addition to employing core contributors full-time, Examiner.com paid numerous "bug bounties" out to folks to help slay specific critical issues. The difference here is that the DA as a non-profit organization needs to be extremely transparent in anything it's doing with the community's money, so there is greater visibility on things this time around.

If you don't want to donate, that's totally okay. You'll still be able to use Drupal 8 all you want, for free, when it's ready. Donating to this fund is only an opportunity to help make that happen sooner, if that's sufficiently valuable to you.

For a lot more "deep thinking" around these topics, see:

Many thanks to effulgentsia for his extensive help on this part!

How do you decide on how money gets spent?

The core committers have a well-documented process that explains how we decide what to fund. The TL;DR version is we look at criteria like:

  • Is a proposal genuinely a release blocker to Drupal 8, or something that will otherwise directly lead to an accelerated Drupal 8 release? (That's a biggie.)
  • Is a proposal resolving a blocker to other work, especially other release blockers?
  • Is a proposal resolving an "ecosystem" blocker? (For example "D8 upgrade path" issues that block early D8 adoption, blockers to a major portion of contributed modules/themes porting)
  • Is this a place where we can inject funding to take an issue the "last 20%" and get it across the finish line quickly?
  • Is momentum in this area slow, making it unlikely to be fixed "organically" by D8 contributors?
  • Are the people working in this area not directly funded (by an employer or client) to fix it already?
  • Do we have some confidence that funding will lead to a successful outcome?

Proposals that answer "yes" to more of these questions than not are more likely to get funded. And the D8 Accelerate team is constantly on the lookout for things that meet this criteria and proactively reaching out to contributors to help get things started.

In short, we take our responsibility with the community's money very seriously, and have turned down multiple community proposals that were fantastic ideas, but did not fit this criteria. (Where appropriate, we refer folks over to the Community Cultivation Grants instead.)

Also please note that a previous restriction around people asking for funding for their own time has been lifted a month or so back (Thanks, DA!). So if you are a contributor who knows a lot about critical issue #12345, you can request a stipend (initially capped at $500 for five hours) to help push it forward.

If that sounds like you, or you have other creative ideas on how we can get Drupal 8 out faster, apply for a grant today!

Thank you for your support!

I wanted to take the opportunity to give a huge shout-out to the "anchor donors" of the D8 Accelerate campaign:


Appnovation
Drupalize.me
Palantir
Phase2
PreviousNext
Wunderkraut
Acquia

Thanks to their efforts, every dollar you contribute is already matched by the Drupal Association and these anchor donors, doubling your impact. If you'd like, you can make a donation to my fundraising drive (I've set a very ambitious goal of $20,000 since that's 8% of $250,000 — get it? ;)):

Fundraising Websites - Crowdrise

(Note: This is just the amount in my personal fundraiser; the overall fundraiser is at $135K and counting.)

...or, find your favourite Drupal person at https://www.crowdrise.com/d8accelerate/fundraiser and donate to theirs instead, or create your own! :)

Thanks as well to the folks who somehow stumbled across it and donated to my fundraiser already—Andreas Radloff, Douglas Reith, and Ian Dunn. I thought Ian's note was particularly awesome! :D

As a WordPress developer, I think we're all on the same side and wish you the best :)

And finally, thank YOU for any and all support you can provide that will help us make Drupal 8 the most successful release of Drupal yet! :D If you have any other questions, please feel free to ask!

Comments

Submitted by Mohamed (not verified) on

I wanted to donate, but PayPal is not an option. Can you add it please?

Submitted by webchick on

Thanks for the inquiry. I forwarded it onto the DA and they said PayPal is definitely not the preferred donation method, because they lock our account from time to time and are generally expensive for us.

However, if you're in Europe and wants to donate in Euros, or you really, *really* don't want to pay with anything else but PayPal, feel free to use the donate form here: https://assoc.drupal.org/donate

Hope that helps! :)