This morning at 8:45am, WhichTestWon published its 300th Case Study on A/B testing. As always, we included creative samples and real-life test results :-).
No, it wasn't easy. Contrary to what many people seem to think, companies like Avis and Yahoo! don't just hand over Case Studies tied up in a bow for us to post on our site. First our editors do a lot of research and networking to figure out who's actually conducting interesting tests. Then, they use all their powers of persuasion to beg, plead, and wheedle creative samples, results data, and most importantly, permission to write a story about the test.
It can be a tough sell. It's scary to hand over your data and 'secret sauce' to be revealed in public. Every single marketer, web analytics pro and usability expert who has allowed their data to be shared at WhichTestWon is a hero to the community. They took a professional risk for the greater good of all.
Lastly, actually writing Case Studies can be tough. At most, we have space for about 200 words -- compare that to MarketingSherpa where stories routinely ran 1,200+! Getting the technical details right is also a challenge, often involving hours of review, statistical analysis, emails and phone calls. Yes, we go through this process even on the rare occasions when a source hands us a pre-written story because we have to be sure every detail is clear and correct. We redo the math, re-examine the creative samples, query the testing strategy and process used, and rewrite the story to fit our criteria.
I've been incredibly lucky in having a strong editorial team to make our Case Study Library possible -- Natalie Myers Tomasso, Lisa Seaman, and Justin Rondeau all went above and beyond.
We've also received significant support from the testing services and testing community, including 42 testing tech companies and 115 conversion optimization agencies and consultancies. Everyone has recognized that a rising tide floats all our boats.
Thousands of readers have also contributed their insights to make each Case Study far more valuable -- we've published 5,634 reader notes and comments so far!
Back when we launched in mid-2009, fewer than 35% of companies marketing or publishing online had conducted a test or scientifically improved their conversion optimization in the prior 12 months. Our goal then, as it is today, was to double that number. We're on our way....
BTW: If you'd like to see all 300 Case Studies, here's a link to WhichTestWon's complete library.
No, it wasn't easy. Contrary to what many people seem to think, companies like Avis and Yahoo! don't just hand over Case Studies tied up in a bow for us to post on our site. First our editors do a lot of research and networking to figure out who's actually conducting interesting tests. Then, they use all their powers of persuasion to beg, plead, and wheedle creative samples, results data, and most importantly, permission to write a story about the test.
It can be a tough sell. It's scary to hand over your data and 'secret sauce' to be revealed in public. Every single marketer, web analytics pro and usability expert who has allowed their data to be shared at WhichTestWon is a hero to the community. They took a professional risk for the greater good of all.
Lastly, actually writing Case Studies can be tough. At most, we have space for about 200 words -- compare that to MarketingSherpa where stories routinely ran 1,200+! Getting the technical details right is also a challenge, often involving hours of review, statistical analysis, emails and phone calls. Yes, we go through this process even on the rare occasions when a source hands us a pre-written story because we have to be sure every detail is clear and correct. We redo the math, re-examine the creative samples, query the testing strategy and process used, and rewrite the story to fit our criteria.
I've been incredibly lucky in having a strong editorial team to make our Case Study Library possible -- Natalie Myers Tomasso, Lisa Seaman, and Justin Rondeau all went above and beyond.
We've also received significant support from the testing services and testing community, including 42 testing tech companies and 115 conversion optimization agencies and consultancies. Everyone has recognized that a rising tide floats all our boats.
Thousands of readers have also contributed their insights to make each Case Study far more valuable -- we've published 5,634 reader notes and comments so far!
Back when we launched in mid-2009, fewer than 35% of companies marketing or publishing online had conducted a test or scientifically improved their conversion optimization in the prior 12 months. Our goal then, as it is today, was to double that number. We're on our way....
BTW: If you'd like to see all 300 Case Studies, here's a link to WhichTestWon's complete library.
No comments:
Post a Comment