(Apologies for the atrocious state of the HTML that follows; this content is originally from this Google Doc.)
Updated Feb 15, 2015 with "needs triage" marker, per xjm.
DrupalCon Bogotá just finished up, and critical issue-wise we've managed to stay in the 50s for a few days (down from a high of 150 in the summer of 2013!), so now seems like as good a time as any to write down what's left to ship Drupal 8!
This post will attempt to document all of the remaining 55 criticals (as of this writing), and attempt to offer a somewhat "plain English" (or at least "Drupal English" ;)) description of each, loosely categorized into larger areas in which we could really use extra help. There are over 2,600 contributors to Drupal 8 at this time, please join us!
(Note: These descriptions might not be 100% accurate; this is my best approximation based on the issue summary and last few comments of each issue. If I got the description of your pet issue wrong, please update your issue summary. ;))
Table of contents
Current state of critical issues
Session and User Authentication API
Fix regressions relative to Drupal 7
Thrilling conclusion! (also known as "TL;DR")
Quick vocabulary lesson
Within this list, there are numerous "markers" used to signify that some of the issues in this list are more important to fix ASAP. These are:
- D8 upgrade path: An issue tagged D8 upgrade path (currently, 13) means it blocks a beta-to-beta upgrade path for Drupal 8, generally because they materially impact the data schema or they impact security. Once we resolve all of these blockers, early adopters will no longer need to reinstall Drupal between beta releases, but can just run the update.php script as normal. This is currently our biggest priority.
- Blocker: An issue tagged blocker (currently, 5) means it blocks other issues from being worked on. This is currently our second-biggest priority (or 0th priority in the case an issue blocks a D8 upgrade path issue :D). I've noted these as "sub-bullets" of the issues that are blocking them.
- Postponed: Issues that are marked postponed (currently, 9) are either currently blocked by one of the "Blocker" issues, or we've deliberately chosen to leave off until later.
- >30 days: These patches have a patch more than 30 days old, and/or were last meaningfully commented on >30 days ago. If you're looking for a place to start, re-rolling these is always helpful!
- Needs triage: Anything with a Triaged D8 critical tag (33 as of today) means that at least a couple of Drupal core's branch maintainers have agreed that this issue should indeed block Drupal 8's release. Conversely, those tagged Needs D8 critical triage (11 as of today), and also those issues without either tag, are those that may or may not actually be critical and still need looking over. As a result, before dumping a bunch of time into solving these, they should be evaluated against the priority levels of issues document to see if they can be downgraded.
- No patch: This issue doesn't have a patch yet. Oh the humanity! Want to give it a shot?
Other weird core issue nomenclature:
- "meta" means a discussion/planning issue, with the actual patch action happening in related/child issues.
- "PP-3" means "this issue is postponed on 3 other issues" (PP-1 means 1 other issue; you get the drift).
Current state of critical issues
Sections roughly organized from "scariest" to "least scary" in terms of how likely they are to make Drupal 8 take a longer time to come out.
Security
Because Drupal 8 hasn't shipped yet, it's not following Drupal's standard Security Advisory policy, so there are still outstanding, public security issues (13 as of this writing). We need to resolve most of these prior to providing a Drupal 8 beta-to-beta upgrade path, as this is the time when we signal to early adopters that it's an OK time to start cautiously building real sites on Drupal 8.
Skills needed: Various
Security Parity with Drupal 7
This class of security issue is to ensure that when Drupal 8 ships, it won't have any regressions security-wise relative to Drupal 7.
- Check every Drupal 7 contrib SA that may affect Drupal 8 core modules (D8 upgrade path) In order to ship Drupal 8, we need to ensure that there are no outstanding security advisories for contributed modules that were pushed into Drupal 8 core. nickwaring89 has started a fantabulous spreadsheet for tracking this.
- Port SA-CONTRIB-2013-096 to D8 (D8 upgrade path) Here's one such issue for Entity Reference module. SA-CONTRIB-2013-096 addressed a relatively esoteric remote access bypass bug, and the patch needs to be forward-ported to Drupal 8.
- Port SA-CONTRIB-2015-039 to D8 (D8 upgrade path) SA-CONTRIB-2015-039 addressed two issues in Views module, a redirect and default permissions for disabled views. The first was fixed in D8, but access checks are still missing from a few views for the second.
- SA-CORE-2014-002 forward port only checks internal cache (D8 upgrade path) Oopsie. Missed a spot. :P SA-CORE-2014-002 was a moderately critical Form API issue, where anonymous users' form entries on cached forms could potentially leak to other anonymous users. It was partially fixed, but not for reverse-proxies.
- Entity/field access and node grants not taken into account with core cache contexts We need to figure out and document what the API looks like for field/entity access modules that interact with the new render cache in Drupal 8.
Session and User Authentication API
Because of various intricate dependencies, the authentication part of Drupal 8 isn't yet converted to object-oriented code, and prevents us from further optimizing bootstrap. This set of issues fixes various problems with this part of the code, and ensures these important security APIs are complete and ready to ship.
- [meta] Finalize Session and User Authentication API (Blocker, Needs triage) The main tracking issue for work in this area.
- Remove dependency of current_user on request and authentication manager (Needs triage) Aims to solve a circular dependency when implementing alternative authentication schemes, and move authentication to only happening once per request, closing a potential security hole.
- Session for an authenticated user can only be set by Cookie AuthenticationProvider (>30 days, No patch, Needs triage) Currently, alternative authentication providers, such as HTTP basic authentication, do not play nicely with the default login form, because Cookie trumps all.
- [meta] Security audit the Authentication component (Postponed, Needs triage) Since the Authentication component is new to Drupal 8, this issue proposes performing a security audit on it once it's complete, and prior to a release candidate.
REST
- REST user updates bypass tightened user account change validation (D8 upgrade path) Since Drupal 7, when you edit your user account, you have to provide the existing password when you want to change the password or e-mail. This security feature is currently by-passed by REST user updates as you can change the password or e-mail without providing the password.
- External caches mix up response formats on URLs where content negotiation is in use (>30 days) Drupal 8's request processing system is currently based on content negotiation (which allows you to serve multiple versions of a document at the same URI based on what headers are sent e.g. Accept: text/html or Accept: application/json). This is generally considered the "right way" to do REST. However, various external caches and CDNs have trouble with this mechanism, and can mix them up and can send random formats back. The issue proposes changing from content negotiation to separate, distinct paths such as /node/1.json.
New security improvements
These issues affect new security improvements we want to make over and above what Drupal 7 does.
- [meta] Document or remove every SafeMarkup::set() call One of the big security improvements in Drupal 8 is the introduction of Twig's autoescape feature, which ensures that all output to the browser is escaped by default. However, this is quite a big change that requires all of the code that was previously escaping content to stop doing that, else it gets double-escaped (so you start seeing < and " and whatnot in the UI). We originally introduced the ability to manually mark markup safe with SafeMarkup::set(), but the recommended approach is actually to use Twig everywhere, so this issue is to ensure that all remaining instances of the manual way are fixed, or at least documented to explain why they're using the non-recommended method.
- Passing in #markup to drupal_render is problematic (>30 days, Needs triage) Another issue in the Twig autoescape space, we need to ensure that markup set by the "#markup" in e.g. form definitions is properly escaped.
- Limit PDO MySQL to executing single statements if PHP supports it (Needs triage) Remember SA-CORE-2014-005? Yeah, so do we. ;) This issue is to make sure that if another SQL injection vulnerability is ever found again, the damage it can do is more limited by eliminating the ability for MySQL to execute multiple queries per PDO statement.
Performance
Tied with security, 13 of the remaining issues are tagged Performance. While it may seem odd/scary to have this be a big chunk of the work left, it's a common practice to avoid premature optimization, and instead focus on optimization once all of the foundations are in place.
Skills needed: Profiling, caching, optimization, render API
Profiling
Here are a sub-set of issues where we need performance profiling to determine what gives us the biggest bang for our effort.
- [Meta] Make drupal install and run within reasonable php memory limits so we can reset the memory requirements to lower levels Due to a variety of issues, including the YAML parsing slowness mentioned above, Drupal 8 currently requires 64M of memory to install, which will only go up as contrib modules are added. The goal is to reduce that significantly, more towards Drupal 7's numbers. The issue contains a number of profiling results and sub-issues that help.
- Profile to determine which services should be lazy Drupal 8 exposes a number of Services (which contain re-usable functionality and allow for pluggability/replacement). Normally, all services that are dependencies of other services are loaded on page load. However, we recently introduced the ability to mark individual services as "lazy"—meaning, to only load them on-demand. This issue is to determine which services are currently loading on every request, yet unneeded for most, so we can mark them as such.
- Profile/rationalise cache tags Drupal 8's caching API introduces the notion of cache tags, allowing for much more focused and targeted cache clears for much better performance. This issue involves investigating our usage of cache tags in D8 and seeing how they could be optimized/improved.
Fix regressions relative to Drupal 7
- [meta] Resolve known performance regressions in Drupal 8 This is the main tracking issue in this space. During the 8.x cycle we've introduced several known performance regressions compared to Drupal 7 (sometimes to make progress on features/functionality, other times because we introduced changes that we hoped would buy us better scalability down the line), which we need to resolve before release so that Drupal 8 isn't slower than Drupal 7. The performance team meets weekly and tracks their progress in a detailed spreadsheet.
- YAML parsing is very slow, cache it with APCu in Drupal\Core\Config\FileStorage::read (Blocker) Installation in Drupal 8 is not as quick as it otherwise would be due to the slowness of parsing YAML files, sometimes more than once. This issue proposes to add a caching layer to speed things up, and also help eliminate noise found in profiling.
- Convert menu CSRF tokens to use #post_render_cache (Blocker, >30 days) Drupal employs robust Cross-Site Request Forgery protection which involves appending a user-specific token on forms and links. However, this is both a bit overkill (in most systems there is just a single CSRF token per request) and also prevents caching of CSRF-protected forms/links.
- [PP-1] Cache localized, access filtered, URL resolved, (and rendered?) menu trees (Postponed, >30 days) An impressive performance improvement for the new D8 toolbar, as well as menu blocks.
- Add cache wrapper to the UrlGenerator In Drupal 8, the url() function has been replaced by the UrlGenerator class instead. This issue is proposing to add caching to make it able to not re-do work once it's already generated a given URL on the page.
- Optimize the route rebuilding process to rebuild on write Rebuilding the list of routes is expensive, and can result in race conditions (this also affects Drupal 7). This issue proposes to move menu rebuilding to write-only requests, which are expected to be expensive anyway.
- Cache-enabled forms generate cached form data for every user on every request (No patch) There's currently a bug exposed by Views—Views exposed filter form causes enormous form state cache entries—but also visible in other forms that employ caching, which results in the form cache ballooning out of control. Needs to be fixed.
- BlockContentBlock ignores cache contexts required by the block_content entity This is a bug fix (critical because there could be access control implications if a custom block has access-controlled fields on it) that ensures that a block and its associated block content both share the same list of cache contexts (e.g. language, roles, etc.).
Entity Field API
Tracked under the Entity Field API tag (currently 6 issues).
Skills needed: Entity/Field API, Form API, Schema API
- Schema for newly defined entity types is never created (D8 upgrade path) When you first install a module that defines an entity type (for example, Comment), its database tables are correctly generated. However, if an entity definition is later added by a developer to an already-installed module, the related database schema won't get created, nor will it be detected in update.php as an out-of-date update to run.
- FileFormatterBase should extend EntityReferenceFormatterBase (D8 upgrade path) Entity Reference fields define a EntityReferenceFormatterBase class, which contains logic about which entities to display in the lookup, including non-existing entities and autocreated entities. File field's FileFormatterBase class currently duplicates that logic, except it misses some parts, including access checking, which makes this a security issue. The issue proposes to simply make File field's base class a sub-class of Entity Reference's, removing the need of "sort of but not quite the same" code around key infrastructure.
- FieldTypePluginManager cannot instantiate FieldType plugins, good thing TypedDataManager can instantiate just about anything Currently, you get a fatal error if you attempt to use Drupal 8's Plugin API to create a new instance of a field type. The current code in core is avoiding this problem by going roundabout via the Typed Data API instead. This issue's critical because these are two of the most central APIs in Drupal 8, and they should work as expected.
- [META] Untie content entity validation from form validation Despite all the work to modernize Drupal 8 into a first-class REST server, there still remain places where validation is within form validation functions, rather as part of the proper entity validation API, which means REST requests (or other types of workflows that bypass form submissions) are missing validation routines. This meta issue tracks progress of moving the logic to its proper place.
- Entity forms skip validation of fields that are edited without widgets (>30 days) If a field can be edited with a form element that is not a Field API widget, we do not validate its value at the field-level (i.e., check it against the field's constraints). Fixing this issue requires ensuring that all entity forms only use widgets for editing field values.
- Entity forms skip validation of fields that are not in the EntityFormDisplay (No patch, >30 days) Drupal 8 has a new feature called "form modes" (basically analogous to "view modes" in Drupal 7, except allowing you to set up multiple forms for a given entity instead). Currently, we're only validating fields that are displayed on a given form mode, even though those fields might have validation constraints on other fields that are not displayed. Critical because it could present a security issue.
Views
Views issues are generally tracked with the VDC tag. There are currently 6 criticals at this point which touch on Views (some already covered in earlier sections).
- Views base fields need to use same rendering as Field UI fields, for formatting, access checking, and translation consistency (D8 upgrade path, Blocker) This is a critical blocker to multilingual functionality; right now, Views mixes up languages when a node title (base field) and body (field UI field) are in the same view. However, it's also the cause of various other inconsistencies, like the inability to select formatters and formatter options on base fields. This issue proposes treating base entity fields the same as Field UI fields in Views.
- [PP-1] Base entity fields using 'standard' plugin added via EntityViewsData to not respect field level access (D8 upgrade path, Postponed) Basically, a subset of the same problem. It's postponed because the above issue may end up solving it.
- Views should set cache tags on its render arrays, and bubble the output's cache tags to the cache items written to the Views output cache This one is critical because it could result in Views showing stale content due to not correctly associating the cache tags of content displayed inside a view with the view itself.
- Views exposed filter form causes enormous form state cache entries Because serialized views are ginormous, the size of the form cache grows exponentially on repeated load of a view with an exposed filter. This issue aims to reduce what is cached by views to stop this from happening.
Configuration system
The configuration system is remarkably close to being shippable! Only 4 critical issues left. We're now working on finalizing the niggly bits around edge cases that involve configuration that depends on other configuration.
Skills needed: Configuration system, Entity Field API, Views
- [meta-3] CMI path to release: The main tracking issue for CMI-related issues.
- Don't install a module when its default configuration has unmet dependencies (D8 upgrade path) Seems like a good idea. :P Basically handles the situation where a module provides some default configuration (say, a default View), which references a dependency on some other module (say, an Entity Reference field). You want to ensure that the module's default configuration can't be installed unless all the various dependencies it needs are there.
- Determine which config entities can be fixed and which will be deleted when a dependency is removed (Blocker) When we uninstall a module we list which other configuration will be "affected" by the uninstallation. This issue proposes to add new functionality to the configuration system to work out what is going to happen when a specified dependency (or set of dependencies in the case of multiple modules) is going to be removed.
- [PP-1] Delete dependent config entities that don't implement onDependencyRemoval() when a config entity is deleted (D8 upgrade path) In the case where dependent configuration is part of the main configuration (for example, fields on a node type) we want to ensure clean-up is done when the main configuration is deleted.
"Fix it, or else"
This subset of issues are things that are part of core currently, and we would really like to keep, but are willing to make some hard choices in the event they are among the last remaining criticals blocking release. The "postponed" among this list means "postponed until we're down to only a handful of criticals left." If these issues end up remaining in the list, we will move their functionality to contrib, and hope to add it back to core in a later point release if it gets fixed up.
Skills required: Various, but mainly low-level infrastructure and non-MySQL database skills.
- [meta] Drupal.org (websites/infra) blockers to a Drupal 8 release (Blocker) This issue contains a "grab bag" of Drupal.org blockers that prevent an optimal Drupal 8 release, including things like semantic versioning support, testing support for multiple PHP/database versions, and support for Composer-based installations. If this issue is one of the last remaining criticals, we might choose to ship Drupal 8 anyway, and jettison one or more features in the process, such as…
- [Meta] Make Drupal 8 work with PostgreSQL (Needs triage) The meta/planning issue for fixing PostgreSQL (both in terms of functionality and in terms of failing tests). bzrudi71 is predominantly leading the charge here and making steady progress, but more hands would be greatly appreciated.
- PostgreSQL constraints do not get renamed by db_rename_table() One of the sub-issues of the above, critical because it causes failing tests.
- [policy, no patch] Move PostgreSQL driver support into contrib (Postponed, Needs triage) If efforts to fix PostgreSQL fails or we don't get testbot support for PostgreSQL in time, it's off to contrib-land (where, sadly, it is even less likely to survive regressions).
- [meta] Database tests fail on SQLite (>30 days) Same deal as PostgreSQL but for SQLite. Unlike PostgreSQL though, this one doesn't have anyone leading the charge at this time, and it's also a lot harder to punt this to contrib, since we use it for various things such as testbot. Help wanted!
- Remove the UI for installing/updating modules from update module if it is not fixed in time for release (Postponed) One major security improvement of Drupal 7 was providing the ability to install/update modules and themes directly from the browser. However, the feature has atrophied in Drupal 8 due to lack of test coverage and lack of active use/maintenance, and now the functionality is broken. And while the feature's very useful, it's not useful enough to further delay Drupal 8's release if it's one of the last critical issues left. joelpittet is making a valiant effort to try and save this feature in Install a module user interface does not install modules (or themes), but the issue would definitely benefit from other helping hands, particularly for extra testing/patch reviews.
General house-keeping
These are all basic things we need to keep on top of between now and release, to ensure that when we're down to only a handful of criticals, we're ready to ship a release candidate. The good news is, these are also all generally really easy patches to make, and often also to test.
Skills needed: Basic patch rolling / reviewing / testing skills. (good for newbies!)
- [meta] Ensure vendor (PHP) libraries are on latest stable release Basically, exactly what it says. :) Making sure that all of the external libraries referenced in core's composer.json file are up to the latest stable releases.
- Upgrade validator integration for Symfony versions 2.5+ This one is called out specially because doing this brings us inline with Symfony 3, which is important for future-proofing Drupal 8.
- [meta] Various asset (JavaScript) libraries have to be updated to a stable release prior to 8.0.0 (Blocker) Same deal, but for JavaScript libraries, which are generally located in the http://cgit.drupalcode.org/drupal/tree/core/assets/vendor folder.
- [meta] Ship minified versions of external JavaScript libraries (Postponed) Basically, in the Gilded Mobile Age™ we want to ensure that we're sending as little over the wire as possible, so scrunching various JS libraries down to the smallest possible file size needs to be the default. Separate issue from above because it needs to happen for both updated and existing JS libraries. Postponed because there'll be less work to do once all of the out-of-date JS libraries are updated and minified at the same time.
- [META-12] Review 'revisit before release candidate' tag There are a number of issues that for one reason or another (for example, because we made a decision in order to unblock progress but weren't completely sure if it'd be the right one N months/years later when D8 shipped) we've tagged revisit before release candidate. We need to make sure this list is down to zero in order to ship Drupal 8.
- [meta] Provide a beta to beta upgrade path (D8 upgrade path, Postponed) A policy issue that documents what holds up a beta-to-beta upgrade path, and what happens after we ship an "upgrade path beta." Postponed until all other critical D8 upgrade path issues are fixed.
- [policy, no patch] 8.0.0 release candidates, release, patch versions, 8.1.x A policy discussion about what happens once we reach zero critical issues. Needs to be figured out before that happens. :)
Other
I couldn't figure out a nice heading for these, so here's the rest.
- Remove _system_path from $request->attributes (Needs triage) Symfony provides a $request object, which has an "attributes" property for the purpose of storing various contextual bits. But the problem with $request->attributes->get('_MAGIC_KEY') is that the values are undocumented, there's no IDE autocompletion, and it's not clear which are internal vs. public properties, so we have an issue at [meta] Stop using $request->attributes->get(MAGIC_KEY) as a public API. to try and stop doing that.
However, _system_path in particular is used a ton, since it's very common to want to know the path of the current request. The patch exposes a "CurrentPath" service instead, which eliminates all of those issues. - Potential data loss: concurrent node edits leak through preview (Needs triage) Because the temp store that Drupal 8's new node preview system employs uses an entity's ID as the key, rather than something uniquely identifiable to a user, if two users are editing the same node and hit preview at the same time, one of them is going to lose data due to a race condition.
- Ajax file uploads fail on IE 9 Pretty much exactly what it says on the tin. :P
Thrilling conclusion! (also known as "TL;DR")
Well, not so thrilling, but at least a conclusion. :)
- Anywhere you see a blocker issue, attack it with fire. Those are holding other criticals up.
- The biggest area of focus right now is D8 upgrade path blockers. Many of them are security issues.
- Another big area is Performance, both fixing existing regressions, and profiling to determine where our biggest wins are.
- Views and Entity Field API are tied in third place for number of remaining criticals. Let's have a race, shall we? ;)
- The configuration system is looking pretty good, but still has a handful of sticky issues left.
- There are a series of important features we'll lose if they're not fixed up in time.
- If you're looking for something somewhat easy/mundane, help yourself to one of the general house-keeping issues.
- Don't forget about the other miscellaneous issues I was too tired to categorize.
- You're safer fixing a Triaged D8 critical issue, since those are all guaranteed to block release, while others may be downgraded in the future.
Sorry this post was so long (and probably has its share of inaccuracies) but I hope it will be helpful to some. It's basically what I needed to get back up to speed after taking a few months off of Drupal 8, so figured I'd document my way to understanding.
Now, let's get 'er done! :D
Comments
Thanks for simplfiying this for the rest of us, Angie
I often find myself wondering to myself "hmm... I wonder how Drupal 8 is going".
I know you are all working hard and I enjoy monitoring the progress. Having it in convenient list form just simplifies the task to be done to use mere observers.
Keep it up and get 'er done :)
Regards,
Peter Macinkovic
Digital Strategy Consultant / Apex Action