webchick.net

very helpful lioness

An update on Drupal 8.6 pre-feature freeze

Thu, 06/28/2018 - 13:52 -- webchick

Greetings, folks! As we head into feature freeze for Drupal 8.6 (the week of July 18), here's a run-down of the various initiatives, and a hit-list of what they're trying to accomplish in the next two weeks. Patch reviews, testing, design, docs, and many more skills are very welcomed!

A couple of caveats here:

1) This is my own personal best understanding of where this stuff is all at, based on reading issue comments, attending meetings, overhearing things from other people who attended meetings, catching the odd Slack snippet of conversation, carrier piegon, etc. And therefore may not be 100% accurate, or even 80% accurate — there's a lot going on! (please clarify in the comments if you see any errors/omissions)
2) Just because something is listed here, there is absolutely no guarantee that it gets reviewed + (truly) RTBCed + committed in time for feature freeze and makes it into 8.6. As you can see, there are lots of issues in the list below, and we're all doing our best to stay on top of them. Worst-case, there's always 8.7. :)
3) This post gets into nitty-gritty "technical audience" details; if you're interested in a more broad overview of initiatives and their aims for 8.6 and beyond, there's the strategic initiatives overview on Drupal.org. I was also recently on a Lullbabot podcast to that effect.

OK, here we go! These are listed in alphabetical order.

Admin UI & JavaScript Modernization

This initiative has some lofty goals indeed, to redesign Drupal's admin experience, and modernize the underlying JavaScript code in Drupal to meet modern standards/best practices. While there's a ton of work actively going on in these areas right now, most of the fruit won't bear until 8.7 or later. If you're planning/able to go, come join the sprint next week at Drupal Developer Days Lisbon!

For 8.6, one of the big accomplishments of this initiative was introducing Nightwatch.js testing framework to core, which allows us to test JavaScript code with (wait for it)... JavaScript (what a concept!). This will be critical in ensuring that the React-ified components work as expected, and our existing JavaScript-rich functionality continues to work solidly as we expand on dynamic functionality in the UI.

Here are the issues this team has surfaced as important for 8.6:

Make Nightwatch testing more generally useful

  • Add login/logout commands to nightwatch [#2973879]
  • Create nightwatch command to install modules [#2974619 ]

Fix long-standing issues in the JavaScript system

Seriously, check out the five-digit node IDs on these bad boys! :P

  • ajax.js insert command sometimes wraps content in a div, potentially producing invalid HTML and other bugs [#736066] - Fixed!
  • Provide a common API for displaying JavaScript messages [#77245]

Bring JS code up to modern standards

  • Use Prettier for formatting core JavaScript [#2978964]

API-First

This team's 8.6 goals are two-fold: 1) stabilizing and filling gaps in the existing REST API, and 2) attempting to add JSON API to core.

TONS of work has been going on in the JSON API contributed module queue to fix a number of outstanding issues to make it core-worthy. So even if this module doesn't make it in time for 8.6, the entire ecosystem will benefit throughout 8.6's lifecycle by using a much more robust and well-tested contributed module. Additionally, a long-standing gap of file upload support has been added. Huzzah!

For the remainder of 8.6, the team would like to focus on the following:

Unblockers to API-First in general

  • Add DateTimeNormalizer+TimestampNormalizer, deprecate TimestampItemNormalizer: @DataType-level normalizers are reusable by JSON API [#2926508]
  • @DataType=map cannot be normalized, affects @FieldType=link, @FieldType=map [#2895532]

Unblockers to REST

  • EntityResource should add _entity_access requirement to REST routes [#2869426]
  • PATCHing entities validates the entire entity, also unmodified fields, so unmodified fields can throw validation errors [#2821077]

Unblockers to JSON API

These are all issues in the JSON API contrib module, which help unblock "Add experimental JSON API module [#2843147]" for core.

UPDATE July 3, 2018: The known blockers are all out of the way, so now there's a proposed core patch to review in that issue! Git to it! :D (See what I did there? ;))

  • [PP-1] Work around core's ill-designed @FieldType-level TimestampItemNormalizer normalization until #2926508 lands [#2929932] - Fixed!
  • JSON API indicates it supports POST/PATCH/DELETE of config entity types, but that's impossible [#2887313] - Fixed!
  • [>=8.5] Remove JSON API's "file URL" field work-around now that Drupal core 8.5 fixed it [#2926463] - Fixed!
  • Needs Issue: Module name conflict between contrib/core (what happens when we bring a same-named contrib module to core that sites are actively using?)

Automatic Updates / Composer in Core

These two initiatives overlap in that we're aiming to build the automatic update functionality around improving core's underlying Composer support.

The Composer team has compiled an excellent plan of attack for how to provide Composer support without jeopardizing the site builder experience. Most of that work will take place in 8.7.

UPDATE July 3, 2018: The team has proposed some of the concrete tasks needed under here for possible inclusion in 8.6. Needs patches, which then need reviews. Let's git er done! (I'm so, soooo sorry. ;))

However, one of the pre-requisites for Composer to work well, is adding semantic versioning support for contrib. Support for this would also be tremendously helpful to contrib module authors and site builders, regardless if they use Composer to manage their dependencies or not.

Composer-ify Core!

  • Add composer build support to core [#2982674] (plan issue)
  • Add composer-ready kickstart component to Drupal core. [#2982680]
  • Add a composer scaffolding plugin to core [#2982684]
  • Add a composer core-strict plugin to core [#2983089]

Unblockers to semver for contrib

  • Core version key in module's .info.yml doesn't respect core semantic versioning [#2313917]
  • Module version dependency in .info.yml is ineffective for patch releases [#2641658]

Configuration Management 2.0

This team spent most of the 8.6 cycle forming, brainstorming a list of blockers to configuration awesomeness, and prioritizing those efforts. The hope is for a roadmap to get published after the sprint next week at Drupal Developer Days Lisbon.

One major win in 8.6 is the ability to Allow a site-specific profile to be installed from existing config, which is part of the aim to Allow a site to be installed from existing configuration (basically, moving the capabilities of the Config Installer module into core.)

Unblockers of install from existing configuration

  • Allow a site-specific profile to be installed from existing config [#2788777] - Fixed!
  • Install a site from config if the config directory is set in settings.php [#2980670]

Documentation

The Documentation initiative has a lot on the go right now, from designing a top-level landing page for the new docs system, to taking a holistic look at the existing docs and how to refactor the IA around them, and finally creating a repository around "quick start" guides. None of these have a particular deadline around 8.6, because they're happening independently of core.

On the core side, there's work being done on a new experimental module for overhauling the in-app help system and this work has an 8.6 deadline.

New topic-based core help system

  • Refactor using a plugin system [#2961552] - Fixed!
  • Add experimental module for Help Topics [#2920309]

Extended Security Support

For the plan around this initiative to happen, we need to make several adjustments to core's Update Status module, which currently makes several hard-coded assumptions about the last minor release of Drupal expiring immediately once a new minor release is available.

Update Status Improvements

  • If the next minor version of core has a security release, status still says "Security update required!" even if the site is on an equivalent, secure release already [#2804155]
  • Status report should indicate next minor release date (needs issue)
  • (other issues TBD)

Layout

The Layout team has been hard at work improving upon the experimental Layout Builder functionality that was added to 8.5. The main goal of the team for 8.6 is to gather real-world testing feedback from end users, which they are accomplishing by adding Layout Builder to a new branch of the Lightning distribution. Doing this has uncovered a few holes in the implementation relative to what's possible in contrib right now, and filling those gaps is the focus of the remaining 8.6 time for the team.

Layout Builder gaps

  • Allow the inline creation of non-reusable Custom Blocks in the layout builder [#2957425]
  • Allow Custom blocks to be set as non-reusable adding access restriction based on where it was used. [#2976334]
  • Add a validation constraint to check if an entity has a field [#2976356]
  • Determine if Layout Builder should replace entity_view_display for all Entity Types [#2936358]
  • No ability to control "extra fields" with Layout Builder [#2953656]

Integration with other subsysytems/modules

  • [PP-1] LayoutBuilderEntityViewDisplay::getRuntimeSections() does not delegate to plugins [#2976148]
  • Add EntityContextDefinition for the 80% use case [#2932462]
  • [meta] Decide how Layout Builder should function with Content Moderation and Workspaces modules [#2973382]
  • Layout Builder does not respect translations [#2946333]
  • Track Layout override revisions on entities which support revisioning [#2937199]

Media

Media has made tremendous strides in 8.6, including remote video support and a newly designed media library.

Next, we need to integrate that media library into the node form, and ideally allow people to add from there as well in a more streamlined fashion.

Blockers to media awesomeness

  • Create a field widget for the Media library module [#2962525]
  • View inside core modal doesn't refresh correctly when reopened [#2861860]
  • (needs issue) Mark Media Library as beta
  • [PP-1] Allow media to be uploaded with the Media Library field widget [#2938116]
  • Any AJAX call disregards machine name verification when AJAX is used and leads to a fatal error [#2557299]
  • Provide a media library display for "Remote video" [#2982279]

Migrate

The goal of this initiative for 8.6 is to stabilize the migration system which means marking the experimental Migrate Drupal + Migrate UI modules stable. This was also the goal for 8.5. What's making it tricky is multilingual migrations, which are themselves tricky because there are a multitude of ways one might have set up multilingual functionality prior to it being included in core in Drupal 8, which introduces lots of edge cases around making IDs line up and whatnot.

The team is taking a two-pronged approach here:

1) Attempt to close all of the remaining i18n-related issues.
2) Worst-case, split off multilingual migrations to an experimental module, so that the rest of the system that works for 80%+ of sites can be marked stable.

Make Migrate Stable

  • [policy, no patch] Mark Migrate Drupal as stable [#2905736]
  • [policy, no patch] Mark Migrate Drupal UI as stable [#2905491]
  • [META] Multilingual migrations meta issue [#2208401]
  • Experimental migrate_drupal_multilingual module [#2953360]

Out-of-the-Box

The Umami profile was committed (albeit marked hidden) in 8.5, and major efforts have been going on to remove all of the "beta blockers" preventing it from being visible in the UI. The last of these—Install profile in settings.php and mismatch check makes re-installs of Drupal hard [#2975328]—just landed earlier this week!

From here to 8.6, the team is working on stability and accessibility improvements.

Umami awesomesauceness

  • Un-hide Umami in 8.5 to vastly improve Drupal's evaluator experience [#2957464]
  • Umami missing some Media "plumbing" found in Standard profile [#2939594] - Fixed!
  • Improve Umami demo's support for managing field display settings [#2980029]
  • Improve Umami Demo's header layout and responsive behaviour [#2980528]

Workflow

Last, but certainly not least, is the Workflow initiative, which aims to add the Workspace contributed module to core in 8.6 to facilitate content staging and full-site previews. The module was already committed to 8.6 awhile back, but must be brought up to "beta" level stability to remain in the tagged + shipped release.

Because Workspaces can only stage content that's revisionable, there's also a parallel effort to add revision-ability to more types of data in Drupal core.

UPDATE July 3, 2018: There was a meeting between Product Managers + Framework Managers that arrived at a more fully flusehed-out list of beta blockers for Workspaces, which are now reflected here. If you see a place you can help, please jump in!

Blockers to Workspaces Stability

  • WI: Workspace module roadmap [#2732071]
  • Add workspace UI in top dialog [#2949991]
  • Remove the automatic entity update system [#2976035]
  • Provide the ability to wrap the entire page with a border when opening off-canvas in the top position [#2976385]
  • Refactor workspace content replication plugin system to three services [#2958752]
  • Prevent changes that would leak into the Live workspace [#2975334]
  • Content Moderation and Workspace don't work together [#2971699]
  • Add a ContentRepository config entity so we can store fully configured repository handlers, which can be reused for the upstream values of a workspace [#2931067]
  • Rename to "workspaces" [#2916780] - not a "beta blocker," but beta deadline.
  • Expose cacheability metadata in WorkspaceCacheContext [#2934354] - Fixed!

MOAR revisionable thingies

  • Convert taxonomy terms to be revisionable [#2880149]
  • Convert custom menu links to be revisionable [#2880152]
  • Convert comments to be revisionable [#2880154]

Anything else?

Whew! That's QUITE a lot. Are there any issues out there that we're missing that you feel are mission-critical to get into Drupal 8.6? Feel free to suggest them, with the caveat that the longer the list is, the more distributed the community's and core committers' focus is.

Thanks for reading!

Comments

Submitted by webchick on

Hm. The patch there seems 2+ years old, and not passing tests, so unlikely to get into 8.6 in the next two weeks. However, it'd be a great feature, and a very worthy issue for someone to work on for 8.7!

Submitted by Jay (not verified) on

Thank you for the write up, many interesting and valuable developments going on. I personally perceive that none of these initiatives are as important as rules, which, to me, is an essential module for a cms and I fail to comprehend why it is not added to core - apart from that most of the community seems to disagree with me and thus have other priorities.
Tldr, I am missing rules on the list of being added to 8.6 because too few hands on it.

Submitted by webchick on

Funding Rules will help expedite its development: https://d8rules.org/

As you point out, it's not yet ready in contrib, however, so proposing it for core at this point would be premature.

Submitted by webchick on

Thanks, any context around those links? Like cool new capabilities they would unlock, or similar? We need to carefully balance very limited committer availability over the next 2 weeks around those initiatives/issues that have the highest impact.

Submitted by Mark Carver (not verified) on

Theme System Modernization Initiative

Idk... maybe finally giving a shit about the actual theme system instead of continually treating it like a back burner item?

Sorry, not trying to sound bitter... but just fed up with the consistent deprioritization and minor nitpicks that push these issues back all the time.

Here's an updated and formatted list:

Extension System, Part III
Finishing up what was started in 8.6.x... would be really odd and complete f'up if we didn't finish this in a single major release:

  • [#2659940] - ThemeExtensionList and ThemeEngineExtensionList
  • [#2941757] - Properly register installed extension namespaces in container
  • [#474684] - Allow themes to declare dependencies on modules

#context (ironically)

  • [#2511548] - Add a "context" array variable to all theme hooks and "#context" array property to all elements to provide optional contextual data
  • [#2923634] - Use hook_theme_suggestions in views