Set up A/B testing with tags

Use Umami's data-tag attribute to run simple A/B tests by tagging different page variants and comparing their metrics.

How it works

Tags let you label the tracker script with a variant name. All page views and events collected under that tag are grouped together, making it easy to compare performance between variants using Umami's filtering and breakdown tools.

Step 1: Create your variants

Serve different versions of your page and include the Umami script with a different data-tag for each variant:

Variant A:

<script
  defer
  src="https://your-umami.example.com/script.js"
  data-website-id="your-website-id"
  data-tag="homepage-v1"
></script>

Variant B:

<script
  defer
  src="https://your-umami.example.com/script.js"
  data-website-id="your-website-id"
  data-tag="homepage-v2"
></script>

Step 2: Assign users to variants

How you split traffic between variants is up to your application. Common approaches:

• Server-side routing: Use your backend or edge middleware to serve different HTML templates.
• Feature flags: Use a feature flag service to control which variant a user sees, and set the data-tag accordingly.
• JavaScript: Dynamically set the tag based on a cookie or random assignment.

Dynamic tag example

<script>
  var variant = document.cookie.includes('ab=v2') ? 'homepage-v2' : 'homepage-v1';
  var el = document.createElement('script');
  el.setAttribute('src', 'https://your-umami.example.com/script.js');
  el.setAttribute('data-website-id', 'your-website-id');
  el.setAttribute('data-tag', variant);
  document.head.appendChild(el);
</script>

Step 3: Compare results

Once both variants have collected data, use Umami's filtering to compare them.

Filter by tag

1. Open your website in Umami.
2. Click the Filter button.
3. Select Tag and enter the variant name (e.g., homepage-v1).
4. Note the key metrics: visitors, bounce rate, visit duration, and any conversion events.
5. Clear the filter and repeat for homepage-v2.

Use Breakdown

1. Open the Breakdown insight.
2. Select Tag as the field.
3. Run the insight to see visitors, views, bounce rate, and visit duration side by side for each variant.

Measure conversion differences

1. Create a Goal for your conversion event (e.g., signup).
2. Filter by each tag to compare the conversion rate between variants.

Tips

• Run the test long enough to get meaningful sample sizes. A few hundred visitors per variant is a reasonable minimum.
• Keep variant names descriptive: pricing-short-form vs. pricing-long-form is more useful than v1 vs. v2.
• Only test one change at a time to isolate what's driving the difference.