Data-Driven Website Optimization Using MVT:
Watch the 5 minute project presentation and pitch.
Digital technologies and best practices evolve rapidly and modern web programs must have the ability to adapt quickly in this changing environment. However, new tools must be systematically and rigorously evaluated to ensure that they are adding value to the larger program. Multivariate testing (MVT) is an established protocol for sites that are designed to optimize sales or where conversations are clear and unambiguous. However the usefulness of MVT on sites that are largely informational in nature is still an open question. Moreover, MVT must be an efficient use of resources and compare favorably with other user experience approaches to justify its incorporation into the overall digital program.
Multivariate testing (MVT) is testing several elements within one webpage to determine which versions of those elements, put together, would best achieve the objective of the webpage. Adding it into our digital strategies integrates a lean and flexible approach to dealing with what could be only one flaw or error in design (what the website looks like), site architecture (how the webpages are organized), or content curation (how the content is presented).
This project is testing whether incorporating MVT to our digital process leads to efficiencies in how a website is optimized for communications. To calculate the impact of MVT, the team is tracking the following key performance indicators:
Total reach for each page version
- Change in traffic for final version (Has traffic increased or decreased since the winning version of testing been published?)
- Findability of page version (How is the page version likely to perform in search?)
- Total conversion rate for each page version (Is the user accomplishing the task?)
- Total effort hours spent on project by site improvement method
- Total cost of testing tools
Achaia Walton (Project Lead), Office of the Assistant Secretary for Public Affairs
Nicholas Garlow, Office of the Assistant Secretary for Public Affairs
Beth Liu, Office of the Assistant Secretary for Public Affairs
Karen Silver, Office of the Assistant Secretary for Public Affairs
Project Lead’s Approving Supervisor:
Andrew Wilson, Content Manager, Digital Strategy Group, Office of the Assistant Secretary for Public Affairs
HHS Ignite is the IDEA Lab’s incubator for Department staff with ideas on how to modernize government. Selected teams are introduced to startup methodologies for problem identification and project implementation. In the entrepreneurial spirit, Ignite projects are iterative, their impacts measurable, and their solutions scalable. This is one of 13 projects that participated in the beta year of Ignite which ran from June 2013 to February 2014.
Status update: 3 Months Later
As part of Ignite’s approach to evaluating it’s effectiveness, each Ignite team is asked to provide information on their project status three months following their pitch. The MVT team indicates that they are spending some time on the effort and that MVT is part of their larger website improvement toolkit, but it has not been incorporated into the workflow of the office. According to the project lead: “We need more clear use cases where this is the “right” solution for the problem facing us.”