Experimentation mindset for an experimentation culture
If you have a true optimization mindset, you (can) apply it to all areas in life.
One of those areas is within your organization to get to a culture of experimentation.
As in your A/B tests, there is no one-size-fits-all. You can test best practices on your website, but you will find that not all of them work.
The same holds for your organization. Occasionally you read “8 steps for cultural change” or “to involve the board, always talk money.” However, these tips might not work for your organization.
There are some things that will likely always work to create cultural change. For example, make experimentation accessible, educate people, make it easy, and give people sufficient autonomy. But for your colleagues to become truly involved and change their way of thinking and working, you need to experiment with your approach.
This article will give tips on incorporating this kind of experimentation in your weekly process, followed by three case studies for inspiration.
Plan your experiments and make them part of your stand-ups
As in CRO, to be successful with experimentation within your organization, you need a structured approach.
Ideation phase
Before you can run experiments, you need ideas. To get those ideas, you can consult different sources.
Get inspiration from others
Talk to other specialists in the market and exchange ideas. You can also read blog posts and, of course, visit events and learn from speakers on stage and people in the audience.
Brainstorm ideas
You will get many ideas by brainstorming with your team and experimentation enthusiasts. As a guideline, you can use my Culture Change Canvas, which I explain in this article: How to motivate your organization to adopt CRO.
Books and scientific articles
There are many books and scientific articles on change. My favorite books on this topic are:
- Switch — Chip & Dan Heath
- Break the Cycle! — Arend Ardon
- Experimentation Works — Stefan Thomke
- Nudge — Richard Thaler & Cass Sunstein
- Atomic habits — James Clear
- The science of organizational change — Paul Gibbons
- Drive — Daniel Pink
Sit with a board member and craft a plan
If you have a board member enthusiastic about experimentation, get him/her on board as soon as possible. Together, craft a plan to get experimentation higher on the company agenda.
Plan your experiments
Just like your online experiments, plan your organizational experiments in your documentation tool (like Airtable or Notion). Give it a start and end date, and make someone responsible for each experiment.
Incorporate organizational experiments in your weekly process
You probably have a stand-up (and perhaps stand-down) every week to discuss running and upcoming online experiments. Use the last 10 minutes to discuss the organizational experiments you planned in the previous step.
As with your online tests, see who is doing what, how it is going, and reflect on work not done. This will keep it top of mind and make people accountable for running and completing these tasks.
Reflect
After every experiment, reflect with the team. Was it successful? Should we do it more often? Should we improve the experiment or focus our attention elsewhere?
Again, just like you do for your online experiments.
Three case studies
Case 1: Creatively get the board involved
Being the first CRO specialist in one of my former jobs, I faced the challenge of getting colleagues and board members involved in this way of working.
I tried several things, like a which test won competition, update emails, presentations, and updates on the intranet. But the best solution was a strategically placed data dashboard.
You see a rough drawing of the office floor in the image below. The green arrow points to a central spot that everyone passes several times a day. There, I placed a big TV screen with a live Google Analytics dashboard displaying live data on page views, conversions, marketing channel performance, and overall data.
This had several advantages.
- First, everybody saw data several times a day. And as these were live numbers and flashes on updates, it got extra attention.
- Second, as it displayed the pageviews for the different product teams, an internal competition arose.
- The biggest advantage, however, was that visitors also saw the data and got the perception that the company has a data-driven way of working. They mentioned this at the start of almost every meeting.
As the board of directors was seeing visitors at the office daily, they received many compliments on the data-drivenness of their organization!
This got the board much more interested and involved.
Case 2: Get 50% of the organization to hand in test ideas
At Videoland (the second-largest Video On Demand platform in The Netherlands after Netflix), a goal of the CRO team was to receive more test ideas from colleagues outside the team.
I have been gladly involved at Videoland since early 2021. Last year, we tried several things to accomplish that goal.
We placed links to a form in update emails and mentioned it during sprint reviews, presentations, and meetings. But these all failed. We received one or two ideas.
What did work tremendously well was an idea competition.
With nice prices (such as team activities and workshops) at the individual and department level, we motivated colleagues to hand in their test ideas through an Airtable form.
We reminded our colleagues through different channels (email, Slack, and several team meetings) of the current leader board and the approaching deadline during the competition. All this, to create as much buzz and involvement as possible.
This resulted in a lot of internal discussion about various ideas and fierce competition between departments which put experimentation top of mind.
In the end, we got 133 ideas from almost 50% of the organization. But it did not stop there.
As we know who sent in which idea, we can keep people updated (through automated email triggers in Airtable). When an idea results in a winning experiment, we will bring a cake to that department to celebrate.
This keeps experimentation top of mind and people involved.
For Dutch people: you can read the complete case study here.
Case 3: Get people to share their successes and failures
The third excellent case comes from my colleague Desiree van der Horst.
At her former job, her goal was to create a culture of failure and experimentation.
She started with magnetic boards on the wall, where marketers could post their fails and successes. But it did not work. Possibly because walking over there, especially to the fail board, felt like a walk of shame.
To come up with a solution, she conducted user research. She asked all the marketers what was holding them back. The main reasons for not sharing fuck ups: fear and not wanting to stand out. And for successes: uncertainty if the accomplishment is interesting enough for others.
When she asked what a good way would be to share successes and failures, she repeatedly got: “It needs to be online, light and airy, and should not cost me any effort.”
Based on this research, Desiree set up a new Slack channel and named it “Bitching & Achieving” with some ground rules:
- The main goal of this channel is to learn from each other’s mistakes.
- This channel is to share successes and failures.
- Respect each other.
- The channel is for marketers only. So no management, sales, or other departments.
With the magnetic boards, nine fails were shared in three months. But using the Slack channel, 53 fails and 43 successes were shared!
You can read Desiree’s case here: https://www.linkedin.com/pulse/bitching-achieving-desiree-van-der-horst/
Use your mindset and knowledge to get to an experimentation culture
Within our work, our job is not just to run A/B tests. A large proportion of our work should be dedicated to change management.
We already have the mindset, knowledge, and skills to experiment and optimize. Now we just need to apply this to our organization.
Make experimentation a part of your weekly CRO process, get ideas, and start experimenting.
Do you experiment in your organization? I am very interested to hear about your successes and failures.
