A plan for media management in Drupal 8

Today, when you install Drupal 8.2, the out-of-the-box media handling is very basic. For example, you can upload and insert images in posts using a WYSIWYG editor, but there is no way to reuse files across posts, there is no built-in media manager, no support for "remote media" such as YouTube videos or tweets, etc. While all of these media features can be added using contributed modules, it is not ideal.

This was validated by my "State of Drupal 2016 survey" which 2,900 people participated in; the top two requested features for the content creator persona are richer image and media integration and digital asset management (see slide 44 of my DrupalCon New Orleans presentation).

This led me to propose a "media initiative" for Drupal 8 at DrupalCon New Orleans. Since then a dedicated group of people worked on a plan for the Drupal 8 media initiative. I'm happy to share that we now have good alignment for that initiative. We want to provide extensible base functionality for media handling in core that supports the reuse of media assets, media browsing, and remote media, and that can be cleanly extended by contributed modules for various additional functionality and integrations. That is a mouthful so in this blog post, I'll discuss the problem we're trying to solve and how we hope to address that in Drupal 8.

Problem statement

While Drupal core provides basic media capabilities, contributed modules have to be used to meet the media management requirements of most websites. These contributed modules are powerful — look at Drupal's massive adoption in the media and entertainment market — but they are also not without some challenges.

First, it is hard for end-users to figure out what combination of modules to use. Even after the right modules are selected, the installation and configuration of various modules can be daunting. Fortunately, there are a number of Drupal distributions that select and configure various contributed modules to offer better out-of-the-box experience for media handling. Acquia maintains the Lightning distribution as a general purpose set of components including media best practices. Hubert Burda Media built the Thunder distribution and offers publishers strong media management capabilities. MD Systems created the NP8 distribution for news publishers which also bundles strong media features. While I'm a big believer in Drupal distributions, the vast majority of Drupal sites are not built with one of these distributions. Incorporating some of these media best practices in core would make them available to all end-users.

Second, the current situation is not ideal for module developers either. Competing solutions and architectures exist for how to store media data and how to display a library of the available media assets. The lack of standardization means that developers who build and maintain media-related modules must decide which of the competing approaches to integrate with, or spend time and effort integrating with all of them.

The current plan

In a way, Drupal's media management today is comparable to the state of multilingual in Drupal 7; it took 22 or more contributed modules to make Drupal 7 truly multilingual and some of those provided conflicting solutions. Multilingual in Drupal 7 was challenging for both end-users and developers. We fixed that in Drupal 8 by adding a base layer of services in Drupal 8 core, while contributed modules still cover the more complex scenarios. That is exactly what we hope to do with media in a future version of Drupal 8.

The plan for the Drupal 8 media initiative is to provide extensible base functionality for media handling in core that supports the reuse of media assets, media browsing, and remote media, and that can be cleanly extended by contributed modules for various additional functionality and integrations.

In order to do so, we're introducing a media entity type which supports plugins for various media types. We're currently aiming to support images and YouTube videos in core, while contributed modules will continue to provide more, like audio, Facebook, Twitter, etc. To facilitate media reuse, WYSIWYG image embedding will be rebuilt using media entities and a media library will be included to allow selecting from pre-existing media.

We consider this functionality to be the minimum viable product for media in Drupal 8 core. The objective is to provide a simple media solution to make Drupal 8 easy to use out of the box for basic use cases. This would help users of sites large and small.

Media library prototype
A work-in-progress prototype of the proposed media library.

Expected timeline and call for help

We believe this could be achieved in a relatively short time — to be included in Drupal 8.3 or Drupal 8.4 as experimental modules. To help make this happen, we are looking for organizations to help fund two dedicated code sprints. The existing contributors are doing an amazing job but dedicated in-person sprints would go a long way to make the plans actually happen. If you are willing to help fund this project, let me know! Looking to help with the implementation itself? The media team meets at 2pm UTC every Wednesday. I also recommend you follow @drupalmedia for updates.

I tried to make a list of all people and organizations to thank for their work on the media initiative but couldn't. The Drupal 8 initiative borrows heavily from years of hard work and learnings on media related modules from many people and organizations. In addition, there are many people actively working on various aspects of the Drupal 8 media initiative. Special thanks to everyone who has contributed now and in the past. Also thank you to Gábor Hojtsy, Alex Bronstein and Janez Urevc for their contributions to this blog post.

Comments

Andreas (not verified):

Thanks for the update, Dries! I am following the discussion around the Media Initiative on d.o and I have one question: Will the Media entity you mention correspond to the Media entity we now have through contrib? In other words, do I have to worry about the Media library I am building up right now in Lightning, because Media handling is improved and eventually changed in core in half a year or a year? Do I have to worry about compatibility?

I would be happy to hear your opinion on this one.

Best,
Andreas

Rich Allen (not verified):

I second this question. I'd love some information on a migration path, and if there will be a preferred contribu module which will make the migration easier.

Andreas (not verified):

Thanks Gábor for pointing to the relevant issues. For non-developers it's sometimes hard to understand which will be the consequences of decisions made on a technical layer. That's why I posted my question here.

While not all functionality will move into core, it will most likely continue living in contrib. That sounds as if it will be safe to build on Media Entities now, if there is no heavy customization involved.

Adam Balsam (not verified):

Hi Andreas. We (the maintainers of Lightning) are committed to an update path from our current media entities to whatever ends up in core - and we're working with the those involved in the media initiative.

Tangentially related, the same holds true for migrating to core's Content Moderation from our current implementation of Workbench Moderation once that becomes stable.

Paul Driver (not verified):

Like others, I am getting ready to start using Drupal 8 and with media being a client expectation it is helpful to know that this is coming along soon. Thank you clarifying that Media Entity module is the way to go in the meantime.

Alain (not verified):

Thanks for the article. We are struggling to implement a robust and complete file management for large public websites at the moment. You are now calling for a sprint effort, but are the requirements already clear and complete? I'd be happy to help for this...

Thanks

Gábor Hojtsy (not verified):

The plan for 8.3 is posted at https://www.drupal.org/node/2825215, and the sub-issues being used for planning the exact details. The high level goals are agreed on as documented in this post as well, the details are still being figured out. We are not using a waterfall type approach where we would have detailed requirements first before doing any implementation, approaching this in an agile way instead.

Gregg Short (not verified):

What about including PDF files in the base core library handling? There are a lot of PDF files that need wrangling. Just make these easy ...

Dries:

The current plan is to handle PDF files through a contributed module. For the MVP we're focused on image support as that is the most important use case. It's good to add YouTube support to core as part of the MVP, as it is probably the number two use case _and_ it's healthy to have one remote media example. Let's revisit adding support for PDF files after we completed the MVP.

knibals (not verified):

Hi Dries! Why YouTube support to core? Why not Dailymotion? Or some other provider?

I mean, should we tie Drupal to a particular company? Be it Google. I would respectfully suggest being as abstract as possible for Media management (PDF, Images, Audio and other "local" documents). And leave remote media management pluggagle, but in contrib modules.

Thx for listening!
POF

Karl Kaufmann (not verified):

Thanks for bringing this to the forefront. Like Rich Allen and others above post, I make heavy use of Media in Drupal 7, and am very interested in the migration path.

I'd love to help in any way I can, but most of my strengths are in design and front-end. Since development time is very valuable, how about doing something similar to what was done with the Rules module--crowdfunding development?

Not all developers, and organizations who employ them may have the resources to give toward community projects, so crowdfunding could help to move it along.

AFowle (not verified):

Embedding a single image in a node is one very important use. It could be done by uploading inline or selecting from a media library.

There are scores of requests all over the web for a gallery feature in Drupal which you do not seem to be addressing. It needs a bit more dressing up than browsing a media library, but I would have thought there was much in common in the implementation. Would it be possible to implement a simple library from the outset?

I like the implementation of D6 with the gallery2 project - both sadly unsupported now, so adding urgency to my request. D7 seems to have come and gone without a decent gallery solution that is easy to use. The D6/G2 combination offered:

  1. Hierarchical galleries (I want one per member, with sub-galleries)
  2. Flexible permissions
  3. Bulk uploading
  4. Space quotas per user
  5. Ease of use for non technical end users (bit of a pig for webnaster to configure sometimes though)

I don't have the skills to contribute code to something like that, but I would add to a chipin or similar.

Gábor Hojtsy (not verified):

There are several of those things that would indeed be good to implement out of the box, whether its Facebook or Twitter post embedding or audio file support or galleries. What we are looking at is how we can improve the core system significantly with some of the common things that *all* of these will need anyway first. Once we delivered that successfully, we should definitely look at next steps and priorities. If we are trying to solve everything at once, that would likely not result in any progress for Drupal users for much longer, so we chose to make smaller steps available to users sooner and then continue to work on it based on feedback and further priorities.