We partner with the online university for working professionals in an ongoing basis to consistently improve the website through A/B Testing. Every three months, we build a roadmap of upcoming tests to increase their leads and improve their understanding of their users' journey through the site.
Initially partnered in 2018 for initial usability audit, and have been working on CRO Testing since then. Our partnership has evolved over the years, including a website redesign, research plan creation, and technology activation plan creation.
In this client project for R2i, I serve as team lead for this CRO testing retainer, managing our testing roadmap and test proposal strategies.
This conversion rate optimization project has been an ongoing initiative for the past 4 years.
Conversion Rate Optimization (CRO), A/B & Multi-variate Testing, Service Design, Internal Tools, Dashboard Design, Research Repository
Talk about the steps for how we got here. usability audit, Affinity diagram, whole site rebuild, etc.
As we entered each quarter, we collected our list of test ideas from the following sources:
To build our testing roadmaps, we typically seek input from various stakeholders to go along with our initial recommendations and create a more collaborative approach to testing that better takes into account business priorities and complements other Walden initiatives.
To collect ideas, we run a quarterly discovery session, inclusive of: Workshops, stakeholder interviews, comparative, usability review, analytics, personalization and audience segmentation
To prioritize ideas, we’ve tried many popular frameworks for these tests, (MSCW, PIE, Dot voting, Consensus-based decision making) and each one has shown their pros and cons.
Ultimately, we’ve settled on impact versus effort scoring model for this particular project, since we needed a more formula-based approach that could be reused in subsequent quarters and since we’ve already assessed priority during our discovery presentation, our roadmap is usually only comprised of high and mid priority items. Impact is determined using an Excel based calculator with estimated impact for different test types, and changed quarterly, based on latest Analytics traffic. Effort is determine by the effort needed to develop the test idea, and is typically roughly estimated in days by the development team and can remain fixed between quarters.
Once we’ve got an approved roadmap for the quarter, we can begin planning each individual test. For routine test planning, we use a testing brief using PowerPoint to share with stakeholders. We include background and rationale to explain why we're testing, a sketch of the proposal to guide the design updates, and analytics data to visualize how big the audience is.
preparing documentation for the entire team to collaborate on includes sketches, justification, inspiration, and analytics data (analytics is pulled with support of a team member)
work with the rest of the team to ensure they have everything they need to keep moving and ensure we launch tests on time. This requires working with project management to build an Agile timeline for all roles.
To support the rest of the team and ensure we're selecting the correct testing tool and timeline, we explore whether test ideas are feasible in Google Optimize before sending to a Sitecore dev to build, as Sitecore development takes longer and requires more resources. Additionally, as we're testing a new solution, we like to collect user data without fully building out the solution, so we don't sink the cost of building the full solution into the test without knowing it will win.
To create a more visible, accessible database of our testing results, a more easily navigable roadmap for stakeholder visibility, and a collaborative intake form for wider team input, we’ve begun work on an interactive roadmap and results repository.
For our proof of concept, we’ve built our app using the Django CMS framework for dynamic content population, the Notion API for database management, Google Developer scripts for the timeline functionality, and an embedded Google Form that automatically uploads submissions to the testing backlog.
Our proof of concept supports the following key user stories:
The current repository prototype is now LIVE.
We designed a new website to help the advocates for generic and biosimilar medicines and drive engagement with their users and bring their brand to life.
We designed and prototyped a Slackbot that streamlines project onboarding, delivering document summaries and task liststhat help our team find the right information at the right time. Our project tied for third in the 2025 Fearless Purple Hackathon despite an unfinished body of work.
We created customized views of coming of ager journeys to Medicare based on their scenarios. Our improvements resulted in improved access to Medicare applications, improved customer satisfaction, and more personalized guidance.
