With less than 10 weeks to go before the Drupal 7 code freeze, an update on the current state of "Fields in core" is in order. After all, the fields system will be one of the most significant new features in Drupal 7, and something we've been dreaming about since 2004. If you're familiar with Drupal's Content Construction Kit (CCK), you can think of "Fields in core" as "CCK in core, except better". If you're not familiar with CCK, "Fields in core" allows you to define custom content types like reviews, blog posts, press releases, articles, events, products, ... whatever you can think of.

Current state

In my DrupalCon Boston keynote presentation two years ago, I laid down the challenge to put fields in core and make them first class citizens. We had talked and brainstormed about it a lot by then, and many people felt like it was the logical next step for Drupal. We began implementation at the "Fields in core" code sprint at Acquia in December 2008. Thanks to the hard work of Yves Chedemois, Karen Stevenson, Barry Jaspan, Moshe Weitzman, David Strauss, Florian Loretan, David Rothstein and Károly Négyesi, an initial version of the Field API was committed to the Drupal 7 development branch. This version of the Field API was a significantly refactored CCK, made more generic and with an improved API. Probably the biggest win of that refactoring effort is that as of Drupal 7, it will be possible to attach fields to nodes, users and taxonomy terms, and by extension, to any other entity type that wants them.

Since that initial version was committed in February 2009, a lot of effort has gone into streamlining the Field API. Yves Chedemois and Barry Jaspan took the lead in converting node bodies and node teasers to regular fields instead of treating them as special cases (oh my!), Nat Catchpole focused on improving performance of the Field API, and various people in the community have helped with a steady stream of incremental improvements, including improving the field validation and error reporting to decouple it from the Form API.

Future work

Despite some of the early awesomeness of the Field API, a lot of work is left to be done. While there is more to streamline, it feels like the Field API has arrived at that "good state" where it feels right to start building on top of it. It is that feeling that triggered me to write this post, and to invite people to participate. And with less than three months away before code freeze, it is time to get more people involved as we build on top of the foundations that are now in place. The most important items that we still need to work on include:

Field UI

The only thing we have in core at the time of this writing is the Field API. The Field API is exactly that: a programming interface. I'd love to see a Field UI in core to enable end users to add fields to content types, extend user profiles, and more. Unfortunately, it is not just a straight port of the CCK UI as the Field API has a number of important new features. The CCK UI in Drupal 6 is pretty dull, so hopefully some good UI suggestions will emerge as part of the D7UX effort, or maybe by leveraging the form builder module? Having a good UI in core is a strategic necessity because it enables more people to test, benchmark and experiment with what we have built so far -- it lowers the barriers for more developers to get involved and could speed up our progress significantly.

User profiles

I'd love to see us rewrite the profile module to take advantage of the Field API. We demonstrated that it is possible to attach fields to users but that doesn't necessarily give you the functionality that comes with profile module; we need to be able to attach fields to the registration form, support a number of different views, and provide an upgrade path from the old profile module to a new, Field API aware profile module. Our goal, of course, is to leverage the new Field API so we can bring the power of dozens of CCK field types to user profiles.

Performance

We need more effort in optimizing performance. The Field API introduces additional abstractions and generalizations and, as is often the case, increased power and flexibility has a performance cost. As bigger and bigger sites adopt Drupal, performance is not something we can compromise on. We should investigate caching strategies, and explore how field data is stored in the local SQL database. One promising approach suggested by David Strauss is to implement materialized views at the application level instead of relying on database support; this approach has the additional advantage of potentially speeding up Drupal sub-systems other than the Field API. Another interesting solution may be the per-bundle storage model prototyped by Barry Jaspan; instead of the current per-field-only storage approach, this solution stores data per-bundle to reduce the number of table joins.

More field types in core

I'd like to see us add a few more field types to core; especially ones that enable basic file and image handling. And date fields, of course. Users expect this kind of functionality out of the box.

Other improvements

Other than the things mentioned above, there are lots of other things that could leverage the Field API: taxonomy terms as fields so we can categorize users, terms and vocabularies as field-able entities, translatable fields, extending RDFa support to the Field API and more. All of these are great, but in the overall scheme of things, probably less important than the other items. Let's try to prioritize and focus our efforts.

Summary

A lot of good progress has been made already, but a lot of work is left to do and we must accelerate our efforts. Now that we have the foundation in place, we can actually start to accomplish some of this work in parallel. We need more people to step up and address some of these big to-do list items. If you want to become a recognized Drupal expert, adding your mark on the Field API would certainly buy you street cred. If you want to help, a good starting point would be the Field API documentation and the Fields in core issues.

See you in the issue queue?

Comments

janusman (not verified):

Profile.module should definitely use Field API. We have a (rather confusing?) collection of methods and contributed modules to make user's profiles work in a meaningful way to the kinds of social sites a lot of us are building.

If Drupal wants to make its adoption as a "social publishing" platform then it's a priority to make the social part, which IMO is tightly bound to user profiles (discovering, viewing, etc), work fabulously out of the box.

sun (not verified):

Nice wrap-up!

However, I don't agree with Translatable fields being peripheral. For a lot of Drupal users, proper handling of translations in Drupal is much more important than to have a UI in core (instead of in contrib). The current node-based translation approach is flawed and a system of exceptions.

Roy (not verified):

Just discussed this with Bojhan while planning the UX sprint for end of this month. The objective for the sprint is to get patches in for a lot of the know smaller issues. But.

There's room for a few big topics and we were looking at formbuilder/field UI as one of the most important ones.

We'll have a really strong team sitting together there. A great opportunity to get some real work done. If there's more people who want to jump (live or virtually) during that weekend and work on formbuilder/field UI, let me know and we'll focus the efforts.

yched (not verified):

Getting some focus on this in the UX sprint would be a great thing.

Note that a g.d.o issue started about Field UI: https://groups.drupal.org/node/23282

I'm working on a UI-oriented summary of the Field API concepts and features that I'll post over there to set a basis for discussion.

Dominik Lukes (not verified):

I just got off the phone with someone with whom we'll need to exchange a lot of structured data so the question of RDF is one that is likely to be more and more important. It's probably fine in contrib but having it in core would make Drupal a boon for small data providers who can't afford expensive custom implementations.

Dries:

I'd still really like to get more RDF(a) support in Drupal 7 core, and given enough people to step up and help, it will happen.

Anonymous (not verified):

What about the support for international(multilingual) sites? Atm it is very experimental with lots of different modules involved.

John H (not verified):

"CCK in core" is not an appealing prospect unless it's a configurable option that won't effect performance in the slightest when not utilized. Contrary to most developers, my standard suite of third-party modules does NOT include CCK or Views. These are added only if the project warrants their use, which is often the case but not always. I am admittedly a cycles weenie and strive to achieve page build times of less than 100ms. Unfortunately this kind of performance can typically only be achieved by PHP frameworks, which Drupal in purist terms is not. It remains to be seen, but Drupal 6 might be the last version that I actively develop with and support.

Alex (not verified):

It is a HUGE error that Drupal 7 has only "per-field-only storage approach". It must have in core "per-bundle storage model". But it is not :(( We created a lot of big social-oriented sites with Drupal 6. The sites have a lot of CCK-fields (100-200 fields), many content, many users. So in Drupal 7 the db queries would be huge: 20-30-40 joins??

It is very disapointed us. I think it is a architectural big error. So like @John H I think Drupal 6 is the last version that we develop with and support.

Dries:

It will be sufficient to drop-in a contributed module to optimize the storage. The storage model is pluggable.

Alex (not verified):

Dries thank you for the quick and encouraging answer! I will try to study the fields storage model. I hope it will allow us to have effective and smart field storage.

In any case it would be great to add an approach like "per-bundle storage model" (module pbs) to the Drupal7's core. It seems smarter and more effective for big projects (like social networks). And it would give developers more control on data storage scheme.

Max Antonov (not verified):

Data storage structure is the most important thing for each big site. But Drupal 7 core offers the worst field storage (per-field) approach. And there are no good solution to use another field storage model ("per-bundle storage model" module seems to be dead). It is very very disappointing. We have to change our preferred platform from Drupal to Yii framework.

PS: in any case Drupal is a great system. Dries, thank you for this.

Cliff (not verified):

Admittedly I have only been using the Fields API for a month or so, but I find it confusing.

As I see there are three phases in the acquisition and storage of form data:

  1. Create an HTML input field for the data, eg a text field or a checkbox.
  2. Process the form and hold data in variables so that it can be written to the database.
  3. Write the data to the database.

In the retrieval and display the phases are:

  1. Read the data from the database.
  2. Process the data to a form that can be displayed.
  3. Render the HTML.

Obviously I have skipped a lot of detail there.

We are told to create fields and fields instances and bundles and it will all happen automatically. I'm willing to believe but I'm struggling to relate the above to fields, fields instances and bundles.

Also, I find the suggestion that changes to fields should be done through the UI a bit odd. It's OK if you have one system, but if you have more than 2 or 3, then it's a chore and not easy to keep consistent across a number of systems. But the real disadvantage is that you can't quickly and reliably remove all vestiges of your changes and reapply a new version. I would have thought that the UI approach would fade slowly away. Online changes are fine for content but no good for structure.

Cheers,

Cliff