Run the web as a business system.
A practical, vendor-neutral method to compare shortlisted CMS platforms through identical buyer-run publishing scenarios and failure paths.
WebChorus Editorial Team8 min read
Screen the CMS market with mandatory requirements, but decide among shortlisted platforms by asking representative users to run the same buyer-owned publishing scenarios. Fix the content, roles, starting state, expected result and failure variation beforehand. Then compare what actually happened, how much effort it took and which configuration, subscription tier, add-on, custom code, partner or external system made the result possible.
A polished demonstration may prove that a configured system can publish a page. It is less likely to reveal a stale translation, approval of the wrong revision, a validation failure at release time or a reusable item that needs one local exception. Comparable operational tests surface such dependencies before they become implementation surprises, manual controls or recurring costs.
The evaluation rules to carry into the shortlist
Requirements should first eliminate candidates that cannot meet non-negotiable architectural, security, accessibility, data, legal, commercial or support constraints. Feature lists, request-for-proposal responses and vendor demonstrations remain useful at this stage. They narrow the field and identify questions, but they do not show how the buyer's people, content and operating model behave when real work crosses roles, revisions, locales, channels and integrations.
Operational proof begins when every candidate receives the same versioned inputs, named actors, starting states, expected outcomes and failure variations. Government Digital Service guidance recommends testing assumptions about users, interfaces, data, compliance, security and technical constraints through prototypes before long-term commitment. The ten scenario families used here apply that principle as an adaptable editorial framework; they are not an official standard or a universal procurement prescription.
Every repeatable scenario must specify its purpose, buyer-owned sample, actors, exact starting state, normal task, meaningful variation and expected observable outcome before the session starts. The outcome should say both what must happen and what must not happen. Use realistic articles, assets, locale records, role rosters and integration payloads, with occasional users included wherever they perform the work in normal operations.
Define the evidence packet just as carefully: rendered pages, screens, audit entries, API responses, exports, timestamps, notifications and participant observations. Record elapsed time, steps, hand-offs, prompts, configuration, extensions, custom code, plan tier and external help separately from success. Mark the scenario failed or open when it misses a must-pass result, hides manual work, leaves state ambiguous, grants unsafe privilege, lacks required evidence or depends on unresolved work.
A feature claim says a CMS can do something; a representative scenario shows what your organisation must do to make it happen.
Authoring and review tests should prove that representative people can create accessible, structured content and release the intended revision without disturbing the live page. Ask a frequent author and an occasional author to build the same article with headings, links, an image, alternative text, metadata and a related-content reference. Include responsive previews, a keyboard-only critical path and one validation or accessibility mistake that each author must identify and correct.
W3C's ATAG guidance covers both accessibility of the authoring interface and support for producing accessible content, but this bounded exercise cannot establish ATAG or WCAG conformance. For review, keep the current version live while a working revision is returned, corrected and approved. Drupal documents this live-versus-working model. Create a newer parallel draft and verify precisely which revision was approved, what each role could change and what the history recorded.
Localisation and reuse tests should reveal which source, edition and shared item users are actually changing, previewing and delivering. Create a secondary-language or regional edition, review and publish it independently, and then change the source after translation starts. Drupal documents separately moderated translations that can begin from the published source instead of its latest working revision, illustrating why stale-state visibility and source identity must be observed rather than assumed.
Leave one localised field empty and inspect the API response and rendered result. Contentful documents requested locales, a default locale and configured fallback values, but those semantics are product- and configuration-specific. Next, reference one governed fact, profile or contact block from several destinations and update it once. References can propagate a published update, yet dependency previews, publication order, caches, rollback and one destination's explicit exception still require separate proof.
Permissions, scheduling and correction scenarios should prove effective control at the boundaries where publishing risk becomes visible. Assign least-privilege author, reviewer, translator, publisher and administrator roles, then attempt permitted and forbidden actions through visible controls, direct routes and relevant APIs. WordPress documentation distinguishes capabilities such as reading, editing one's own or others' content, publishing, importing, exporting and administration, showing why a reassuring role name is not sufficient evidence.
Schedule a coordinated publish and later unpublish action for related content and assets in a named time zone, such as Asia/Kolkata, then introduce a validation failure or last-minute change. Contentful documents dates, IANA time zones, permissions, notifications and validation failures for scheduled actions. Capture stored time zone, dependency scope, preflight result, execution timestamps, public state, notifications, partial-release behaviour and recovery steps rather than accepting a scheduled-status label.
For correction, publish a material fix through the smallest authorised path, verify every delivery channel and cache, and then restore the prior approved revision after making the correction fail. WordPress revision endpoints can expose earlier content, authors, timestamps and statuses. Those records are useful evidence, but their existence alone does not prove safe rollback, correct approval handling, a complete audit trail or convergence across cached delivery.
Archiving, integration and recovery should be tested as distinct operational outcomes, with every conclusion limited to what the exercise demonstrates. Define retirement first: retain the page with an explanation, unpublish and redirect it, restrict it, delete it or apply another explicit state. GOV.UK guidance, for example, distinguishes withdrawal that retains a URL from unpublishing that removes content and can redirect; these are examples, not universal business rules.
Reverse the retirement decision and inspect the URL response, links, search, feeds, APIs, attachments, history, permissions and downstream effects. For integration, create or update realistic content through the intended connector, then send invalid input, repeat a request and delay or fail the consumer. WordPress documents public API resources and authenticated management actions, while CMIS demonstrates that even a common repository model need not expose every underlying capability.
For recovery, export the agreed content, assets, models, relationships, identifiers, redirects and relevant operational state, then restore a representative set in isolation. Record completeness, gaps, elapsed effort, restored relationships and unresolved vendor dependencies. NIST describes contingency planning as coordinated plans, procedures and technical measures for recovering systems, operations and data. A CMS trial restore can inform that work, but it cannot certify production recovery or business continuity.
| Scenario family and buyer-run task | Expected observable outcome | Decisive evidence | Failure or exception variation |
|---|---|---|---|
| Authoring: create a structured article | Usable entry and accurate responsive preview | Structure, keyboard path, validation, time and assistance | Find and correct an accessibility or validation mistake |
| Review: return, revise, approve and publish | The intended revision goes live | Live and working states, comments, identity and history | Create a newer draft during approval |
| Localisation: publish a secondary edition | Locale state and delivery remain understandable | Source signal, fallback, metadata and API response | Change the source and omit one field |
| Reuse: update one shared item | Only intended destinations receive the change | Dependency view, previews, caches and rollback | One destination needs different context or timing |
| Permissions: attempt allowed and forbidden work | Least privilege holds at each boundary | Controls, direct routes, API denials and audit identity | Restrict a locale, field, type or transition |
| Scheduling: coordinate timed publication | Related items reach the intended public state | Time zone, preflight, timestamps and notifications | Trigger validation failure or change the time |
| Correction: fix and reverse a live error | Approved content converges across delivery channels | Revision comparison, timestamps, caches and audit record | Restore the prior approved revision |
| Archiving: retire and reinstate content | The defined URL and discovery outcome occurs | Response, redirect, search, API, assets and history | Reverse the retirement decision |
| Integration: exchange realistic content | Mapped content and status reconcile correctly | Requests, responses, events, logs and duplicates | Repeat invalid input or delay the consumer |
| Recovery: export and restore in isolation | The representative dataset is restored with known gaps | Export contents, relationships, validation and elapsed effort | Recover after deletion, corruption or unavailability |
Teams should decide with separate records for mandatory gates, demonstrated outcomes, operational effort, dependencies and unresolved risk. Do not let an aggregate score conceal a failed must-pass outcome. Attribute every success to native capability, configuration, plan tier, add-on, extension, custom code, partner service, external system or roadmap commitment. Set gates, weights and thresholds before testing because this method is an editorial procurement framework, not a universal scoring standard.
Convert unresolved migration, integration, training, configuration, testing and manual-control work into implementation scope, cost, contract terms, explicit risk or rejection. Retain the versioned scenario cards, buyer-owned samples, participant roles, observations, timestamps, screenshots, API records, exports, dependency assumptions, gate results and decision log. Seek qualified accessibility, security, privacy, legal, data, infrastructure and continuity judgement wherever conformance, compliance, threats, production resilience or recovery objectives must be established.
First, screen candidates against mandatory architecture, security, accessibility, data, legal, commercial and support constraints. Then ask representative users to run the same predefined, buyer-owned publishing scenarios in every shortlisted CMS. Compare outcomes, effort, dependencies and unresolved risk separately.
Include realistic content, representative users, an exact starting state, the normal task, a failure variation and an observable expected outcome. Specify the evidence to capture, including rendered pages, timestamps, audit entries, API responses and exports. Record effort and dependencies separately from success.
It should support the buyer's versioned scenario rather than replace it with a polished product tour. Let representative users attempt the default path before the vendor explains configuration or alternative approaches. Record any plan tier, extension, custom code, partner service or external system required.
A practical adaptable set covers authoring, review, localisation, reuse, permissions, scheduling, correction, archiving, integration and recovery. Each organisation should select variations that reflect its actual publishing risks. The set is an editorial framework, not a universal product standard.
Keep must-pass gates separate from observed outcomes, usability, effort, dependencies and open risks. A weighted total should not offset failure of a mandatory requirement. Define the organisation's own gates, weights and thresholds before testing, and retain the evidence behind every result.
This article was researched using the following sources:
Use a nine-field page-purpose brief to test a website content request, select the right decision and give writers a focused production contract.
Website Governance and Operations
Build an adaptable eight-domain website governance matrix that fixes decision owners, delegated limits, required input, escalation routes and records.
Trace representative user tasks across navigation, labels, links and search to diagnose findability issues before approving a redesign.
