Dries Buytaert (creator of Drupal and CTO at Acquia) posted this tweet regarding the relationship between user experience and sentiment regarding Drupal:
I replied that unlike what the graphic indicated, we actually saw similar results in our research at Sitecore, and that much of this echos what Tony Byrne saw when he described a “Drupal-like “guild” culture around the Sitecore ecosystem”.
But it’s more than just disagreeing with the characterization of the comparison (because there is actually more similarities between Acquia/Drupal and Sitecore than there are differences, but that’s a whole other blog post) – there is actually a lot to unpack here, because usability in WCM is a pretty complicated issue which needs to be looked at through a few different angles:
What do we mean by user?
In WCM, regardless of your system, there are usually a number of personas involved in day-to-day usage; at a minimum you tend to have content creators and layout creators. In simple systems like SaaS site builders (Wix.com, Squarespace, etc.) these are often the same person. Similarly, for more complex systems, you can be into the dozens (content creators, information architects, editors, administrators, analytics viewers, SEO, optimization/testing, etc. etc.)
Similarly, even the task of “content authoring” there are a lot of different model and use cases. Some systems will be aligned around componentized content and place emphasis on page layout. Some are purely unstructured drag-and-drop. Some systems are completely structured (akin to form entry). Some do a “halfway” model (such as WordPress Gutenburg). Some assume authors don’t create in the CMS at all, and are really just copy/pasting from Word – so formatting is not important, aside from stripping out garbage HTML.
Also, in WCM, there is generally an inherent tension between content authors who want maximum flexibility and developers/information architects who want to lock things down (usually for good reason). Often, as a creator gets more experienced they start to understand why some limitations are put in place (“Ah, now I understand why <blink> is bad, or the fact that I couldn’t classify an MP3 as vegetarian“)
Lastly, in Enterprise CMS, there is actually what we would consider the “buyer persona” which means that usually the decision-maker is not necessarily the end user. Often there are a number of factors that go into this that are more related to how “usable” the business is, versus the actual software. In other words, was sales receptive? Did they help with a proof-of-concept? Did they clearly articulate a roadmap which matched the organizational directives? Did they provide good partner and client references?
All this to say, when we say user – we should actually be focusing on task. This level of detail gives us a clearer sense of what they are doing and trying to accomplish and therefore we can have a far clearer sense of success criteria.
What do we mean by “beginner/intermediate/expert”?
This is somewhat related to the roles above, and why we should be focusing on task. In many cases the skill level often reflects the needs of the tasks being accomplished and not nessesarily time. A very skilled content author (who is aware of content strategy, how to tag for SEO, when to deploy for maximum social effect, etc.) doesn’t need to know much more about the inner workings of the CMS. But something like interface UX speed and reduced clicks would be very important if they are entering mass amounts of content.
Interestingly enough, those UX factors don’t change much across the experience spectrum – so usually increased satisfaction (if user skill was the only factor) would simply indicate familiarity with tooling. This has product management implications unto itself – any redesign or refactoring will likely lead to a minor revolt of user base if the newer “better” iteration is not immediately intuitive.
So, again this comes down to tasks – what are the tasks expected by a “beginner” vs. an “expert” and how does the system enable those? Any usability research needs to be centered around that, otherwise you are chasing various forms of bias and assumptions about the “width” of a role vs. the “depth” – just because someone wants a simple interface, does not mean they are simple themselves or doing simple things.
Headless CMS vendors know this, as the model and interface for editing “core” content are simple, but often the content model is related to larger enterprise concepts (such as content strategy, content endpoints, etc.) which exist outside of that “simple” system, but are quite complex themselves. So “experience/time” is often too coarse a measure for categorizing usability (at least to the extent that it can actually influence product management).
What role does the software play versus the implementation?
One of the facts we discovered at Sitecore was that while there was some degree of correlation between user/developer skill and satisfaction with the software, there was a far higher degree of correlation between the partner implementation and the satisfaction with the software. As a result, resources were allocated towards partner enablement and training.
To some degree, this may also be subject to bias – many of the early partners were very experienced and technical, and if you move into newer markets and provide more interfaces to deal with those markets (Sitecore Experience Accelerator, Sitecore JavaScript Services, etc.) the partner skill level will obviously be lower since the technologies enabling those partners are so much newer. Not to mention the fact that the software may be pushing into markets and use cases less appropriate (as the “low hanging fruit” of ideal-fit starts to go away).
But there is a certain truth to the matter – the implementation matters substantially to overall usability, and this means having a partner who knows how to manage many different aspects; not just how to code against the system, but mapping functionality to the organizational needs and working with/around inherent strengths/weakness.
What does this mean for vendors
A distribution as outlined by Dries is actually not anything close to what you want in a growing market – it effectively means that most new users face significant hurdles and those experienced users who are fans are basically the result of survivorship bias. Any system which needs to target all experience levels also needs to have a high degree of usability at the beginning and continue to grow this – usually by a considered approach of tailoring or hiding interfaces and functionality as appropriate. As an example of this, one of the most highly rated implementors of Sitecore would go so far as to literally disable and remove large parts of the software to achieve this balance.
Similarly, you can over-rotate in that you have a system entirely focused around new users, but unable to address more complex requirements. This often limits a product to being a more SMB or departmental tool. This also have the effect of limiting the implementation partner community substantially (which is a great source of evangelism and code – both Drupal and Sitecore have very similar followings in this respect).
There is nothing wrong with that approach per se (Wix.com makes more revenue than any “Enterprise WCM” system by far) but it does mean that a vendor should know their strengths and not discard one revenue model and area of product focus for another.
In other words, the product strategy has to fit that of the business strategy. If the business model is aligned around a stable market, keeping the existing install base and community happy (i.e. nice big support margins, big moats making switching difficult) then that sort of curve of linear experience/happiness is acceptable. If the business model requires new growth above all, then rapid iteration towards more intuitive interfaces are the priority (and screw the existing users!). Apple among others is famous for this model – notice they have completely abandoned the much more intractable enterprise space and any partners that would along with that.
So, in conclusion, none of those vendor curves are “wrong”, but they should (hopefully) reflect the underlying business strategy for each.