A/B testing & experiments skill

Design tests that can be trusted: hypothesis, sample size, duration, guardrails, and reading results correctly.

It's one of cmo.so's open Conversion skills: a SKILL.md file that tells an AI agent when to use it, what to ask you first, the steps to follow, the checks to pass and what to hand back, so the job is done the same careful way every time.

Reach for it when you'd otherwise say something like “Should I A/B test this”, “How long should this test run”, “Is this result significant”. Your agent reads the skill's description and picks it for requests like these, so you don't need to name it.

If a request is close but not quite right, the skill says so and points to the related skill that fits better, instead of forcing a poor answer.

It works best with the change being considered and why (data, research, hunch); baseline rate of the primary metric and daily traffic or sends; smallest improvement worth acting on; testing tool or platform (site tool, Google Ads experiments, Meta A/B test, email tool); and for a readout: visitors and conversions per variant, dates, any incidents.

You don't have to prepare everything up front. The skill asks for missing pieces one at a time and says plainly when it's working from an assumption instead of your data.

You get a test plan: hypothesis, metrics, sample size, duration, stopping rule; a readout with effect size, interval, guardrails and a plain decision; and a prioritised experiment backlog and a learnings log entry.

Every result comes from a written method with a quality bar, so numbers name their source, claims are checked and the output is ready to use or share rather than a rough draft.

Yes. The skill is open source under the MIT licence, so you can use it, change it and share it, including for client work.

Seeing, copying and downloading the whole file on cmo.so needs a free sign-in. Copying also upvotes and saves it, so you can find it again under Saved on the skills page.

Claude Code installs it from the cmo.so marketplace. Any agent that reads SKILL.md folders, such as Codex or Cursor, can use it once the folder is in its skills directory.

You can also paste the copied skill into ChatGPT, Claude or any assistant at the start of a conversation and ask it to follow the skill.

In Claude Code, run /plugin marketplace add AI-Marketing-OS-LTD/cmo-so-skills, then /plugin install cmo-so-cro@cmo-so. That installs this skill together with the rest of the Conversion group.

For other agents, sign in, press Copy or Download above and put the file in your agent's skills folder as SKILL.md inside a folder named after the skill.

No. The skill works on its own with whatever your agent can already see and do: your files, the pages you share and the data you paste in.

A free cmo.so account only matters if you also want Maggie to plan, write and publish your marketing every day.

It pairs well with Page conversion optimisation; and Ad creative. The agent can hand over from one to the next in the same conversation, for example research first, then the plan, then the writing.

Each of them has its own page with what it needs and what it returns, linked below.

Yes. It's a plain Markdown file, so you can edit the steps, add your own checks and examples, and keep your version in your own repo. The MIT licence allows it.

If your change would help others too, suggest it on the skills page or open a pull request on GitHub, and it may be folded into the library for everyone.

It's written by cmo.so and adapts open work by others, credited on this page: alirezarezvani/claude-skills (MIT); irinabuht12-oss/marketing-skills (none); and irinabuht12-oss/marketing-skills (none).

Only permissively licensed work is adapted; copyleft repos are used for ideas only and repos without a licence aren't used at all.

No. The skill only guides your own agent; your files, documents and data stay wherever your agent and its tools keep them.

cmo.so only sees what you connect to cmo.so. Your votes and saves on this page are stored against your cmo.so sign-in.