The A\/B Cemetery is where all those countless tests go to die, forgotten, undocumented, and often unreconciled with product strategy. It\u2019s what happens when teams run experiments but fail to learn from them, act on them, or even remember they happened.<\/p>\n\n\n\n
\nThe phrase plays on the idea of a graveyard for A\/B tests. In data-driven product cultures, it\u2019s easy to run lots of experiments, \u201clet\u2019s test everything!\u201d, but without discipline, results pile up, conflicting insights emerge, and nobody knows which changes were kept, reverted, or why decisions were made. The cemetery is full of old tests, dead ideas, and unlearned lessons.<\/p>\n<\/blockquote>\n\n\n\n
WHEN<\/h3>\n\n\n\n
You know you\u2019re working in an A\/B Cemetery when:<\/p>\n\n\n\n
\n
- Nobody can explain why the current version of a page looks the way it does.<\/li>\n\n\n\n
- Teams repeat experiments that were already run (and failed) in the past.<\/li>\n\n\n\n
- There\u2019s no central log of experiments or their outcomes.<\/li>\n\n\n\n
- Stakeholders cherry-pick winning variants without understanding tradeoffs.<\/li>\n\n\n\n
- Test results are ignored because they don\u2019t fit preconceived opinions.<\/li>\n<\/ul>\n\n\n\n
It\u2019s particularly common in high-velocity growth teams or organizations that celebrate \u201ctest velocity\u201d over insight quality.<\/p>\n\n\n\n
WHY<\/h3>\n\n\n\n
The A\/B Cemetery is filled up because:<\/p>\n\n\n\n
\n
- There\u2019s no process for documenting results and decisions.<\/li>\n\n\n\n
- Team turnover erases institutional memory.<\/li>\n\n\n\n
- Experimentation is treated as a numbers game rather than a learning opportunity.<\/li>\n\n\n\n
- Winning variants are deployed without analyzing side effects or long-term impact.<\/li>\n<\/ul>\n\n\n\n
Without a culture of learning, experiments just become busywork.<\/p>\n\n\n\n
HOW<\/h3>\n\n\n\n
Here\u2019s how to keep your experiments out of the cemetery:<\/p>\n\n\n\n
\n
- Log every test.<\/strong> Keep a centralized record of hypothesis, variants, metrics, results, and decisions.<\/li>\n\n\n\n
- Share learnings.<\/strong> Make sure findings are presented and discussed across the team.<\/li>\n\n\n\n
- Validate long-term.<\/strong> Revisit \u201cwinners\u201d to ensure they still perform over time.<\/li>\n\n\n\n
- Kill weak tests early.<\/strong> Not all tests deserve to run, be clear about hypotheses and success criteria.<\/li>\n\n\n\n
- Train teams.<\/strong> Help everyone understand how to design meaningful experiments.<\/li>\n<\/ul>\n\n\n\n
PRO TIP<\/h3>\n\n\n\n
\nName your tests descriptively (\u201cHomepage CTA placement test #2\u201d) so future teams know what was tested and why, instead of \u201cTest123\u201d buried in some analytics tool.<\/p>\n<\/blockquote>\n\n\n\n
EXAMPLES<\/h3>\n\n\n\n
\n
- A signup page with odd copy nobody can justify because it \u201cwon\u201d a test years ago.<\/li>\n\n\n\n
- A marketing team running yet another button color test that was already run twice.<\/li>\n\n\n\n
- A dashboard full of variants but no record of which metrics improved.<\/li>\n<\/ul>\n\n\n\n
CONCLUSION<\/h3>\n\n\n\n
The A\/B Cemetery reminds us that experiments are only as good as the knowledge they produce. Test less, learn more, and don\u2019t let your product become a patchwork of unexamined changes.<\/p>\n\n\n\n
Also known as: Test graveyard \u2022 Experiment entropy \u2022 Data without memory<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"
The A\/B Cemetery is where all those countless tests go to die, forgotten, undocumented, and often unreconciled with product strategy. It\u2019s what happens when teams run experiments but fail to learn from them, act on them, or even remember they happened.<\/p>\n","protected":false},"author":1,"featured_media":336,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[24],"tags":[64,19,21,20,62,39],"class_list":["post-253","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-the-testing-twilight-zone","tag-a-b-cemetery","tag-design","tag-funny","tag-idiom","tag-testing","tag-ux","entry","has-media","owp-thumbs-layout-horizontal","owp-btn-big","owp-tabs-layout-horizontal","has-no-thumbnails","has-product-nav"],"jetpack_featured_media_url":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-content\/uploads\/2025\/09\/20.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/posts\/253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/comments?post=253"}],"version-history":[{"count":1,"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/posts\/253\/revisions"}],"predecessor-version":[{"id":255,"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/posts\/253\/revisions\/255"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/media\/336"}],"wp:attachment":[{"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/media?parent=253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/categories?post=253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hitchhikersguidetodesign.com\/book\/wp-json\/wp\/v2\/tags?post=253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}