Top Traits of Great A/B Testing Leaders
Having the right leader in place is the difference between an A/B testing program that thrives or fizzles. One of my passions is observing, understanding, and learning characteristics of effective and not so effective leaders.
Hiring, firing and promoting leaders is one of the most important decisions organizations must get right. Getting the right A/B testing leader can be the difference between a program that thrives or a program that fizzles. Throughout my career, I have witnessed both sides of the story.
>> If I were in an organization looking to hire or promote an A/B testing leader, I would look to these three traits in the following order: Influence, self confidence, and expertise.
Last year, I put a list of 100 testing leads I have encountered over the years from various companies and decided to do a little score sheet exercise. Note, these scorings are all subjective. I wanted to see based on four traits, which ones had the strongest correlation to the successful growth of a testing program within an organization. As shown in the table, Influence and Self Confidence tied for the highest correlation.
What do these traits mean? Consider these Naoshi definitions.
Likeability: how well people tend to get along with this person
Expertise: the person’s knowledge around ideation, processes, technology, stats
Influence: the ability for the person to influence others to take action
Self confidence: humbleness to admit failure and listen to others’ opinions, secure in themselves and speaks up when necessary, does not micromanage
Why these four? Over many years, these are the big buckets I look for in A/B testing leads.
A few example scenarios:
Scenario A the obvious (high likeability, medium expertise, low influence, low self confidence): Bob is part of a large financial organization. He is well versed in A/B testing. Has gone to many conferences, studied a lot around the subject, familiar with the testing tools, and have run some tests himself. He is very friendly and people love to chat with him. However, he has very little influence in getting different departments to work on testing. Also, he is very timid about job security and does not want to speak up and rock the boat. As we can all conclude, the A/B testing program will fizzle
Scenario B the less obvious (medium likeability, high expertise, medium influence, low self confidence): Gene comes into an organization with over four years of experience running tests, has a stats background, and on paper looks like a rock star. She comes into a large organization with a director level title, so has some built-in authority, and is tasked to grow the newly started experimentation program. At first, things are good. She seems very competent, sets up processes, and is able to work cross functionally with different departments to start to execute tests. However, over time, she starts to show signs of being rigid about her opinions on what to test and becoming very controlling over all aspects of the testing program and processes. People involved in testing start to resent working on the tests, culture suffers, and ultimately the testing program loses it’s influence within the organization.
Scenario C the less less obvious: (medium likeability, medium expertise, high influence, high self confidence): Conan joins a large organization with just a year’s worth of experience in A/B testing. He has a track record of moving up in different organizations before joining this one. Conan seeks to understand various parties’ goals and needs. As he builds out new processes and guidelines, he passionately researches and studies more on A/B testing, and proactively seeks tips and knowledge from different people in the organization that has any applicable knowledge. He creates buy-in from various departments because he clearly shows how his team’s efforts can further elevate their success.
Running A/B tests is often a highly cross functional activity. Yes, tactical expertise is important. That only goes so far. It is critical for A/B testing leaders to be able to influence action.
The pitfall many organizations fall into when seeking 'testing leads’ or ‘optimization leads’ is diving straight into competency and likeability of candidates. First, as we all know, it’s easy to fudge knowledge and accomplishments on a resume or LinkedIn profile and put on a friendly show during the interview process. Second, even if they are extremely knowledgeable in the space, it does not necessarily mean that person will get the desired result of an organization seeking to scale or grow their A/B testing program.
Success depends on the ability of the testing lead to influence action from others. This person needs to be self confident in his or her abilities, be humble, and empower others.
>> If I were in an organization looking to hire or promote an A/B testing leader, I would look to these three traits in the following order: Influence, self confidence, and expertise.