JavaScript AB Testing Script: 7 Steps to Build Your Own A/B Test

Ever shipped a new button color, eyeballed the numbers for a week, and just went with a gut feeling instead of real data? A proper javascript ab testing script fixes that, and everything below is real, working code with real captured output, not a theory.

At its core, an A/B test means splitting visitors into two groups, control sees the current version, variant sees the change, then comparing how each group behaves. A working javascript ab testing script needs exactly one thing to hold true for that comparison to mean anything: a visitor has to land in the same bucket every single time they show up.

If bucketing flips on a whim, a returning visitor might see control today and variant tomorrow. That contamination corrupts the result before a single conversion gets counted, and it is the single most common way a homemade test quietly produces garbage numbers.

A concrete example makes the stakes clearer. An online store testing a green checkout button against the existing blue one only learns something useful if every visitor who saw green keeps seeing green across their whole visit, and every visitor bucketed into blue never accidentally sees the other color partway through checkout. One flipped bucket in the middle of a session and that visitor's data belongs in neither group anymore.

The trick to consistent bucketing without a database or a server round trip is a deterministic hash. Feed it the same string twice and it returns the exact same number twice, every time, on every device.

function hashString(str) {
  let hash = 0x811c9dc5;
  for (let i = 0; i < str.length; i++) {
    hash ^= str.charCodeAt(i);
    hash = Math.imul(hash, 0x01000193); // FNV-1a prime
  }
  return hash >>> 0;
}

function assignVariant(userId, testName, weights = { control: 0.5, variant: 0.5 }) {
  const ratio = hashString(`${testName}:${userId}`) / 4294967295;
  let cumulative = 0;
  for (const [name, weight] of Object.entries(weights)) {
    cumulative += weight;
    if (ratio < cumulative) return name;
  }
}

That is the entire engine behind a javascript ab testing script. No server call, no database row, just a small well known hash called FNV-1a turning a string like homepage-cta-color:user_47291 into a number between zero and one, then checking which slice of that range it falls into.

Run that function against 100,000 simulated visitor IDs and the split holds up exactly where it should, close to even for a 50/50 test and close to the target ratio for a weighted one, with the exact same visitor landing in the exact same bucket no matter how many times the function gets called.

Actual output from the hash function above, run against 100,000 simulated visitor IDs.

A deterministic hash solves consistency in theory, but plenty of real scripts still get this wrong in practice by rolling a fresh Math.random() pick on every page load instead. Here is that mistake next to the fix, run through an actual browser five times each.

function assignNaive() {
  return Math.random() < 0.5 ? 'control' : 'variant';
}
// re-rolls a brand new variant on every single page load

const TEST_KEY = 'ab_test_homepage-cta-color';

function assignPersisted() {
  const stored = localStorage.getItem(TEST_KEY);
  if (stored) return stored;
  const fresh = Math.random() < 0.5 ? 'control' : 'variant';
  localStorage.setItem(TEST_KEY, fresh);
  return fresh;
}

Same visitor, five real page loads in Chromium via Playwright. The naive version flips, the persisted version never does.

Not every test should start at an even 50/50. A risky change to a checkout flow is safer ramped in slowly, 90/10 or 80/20, so a bug only touches a small slice of real traffic while the numbers are still being watched closely.

The assignVariant function from earlier already supports this. Passing { control: 0.8, variant: 0.2 } instead of the default weights shifts the cutoff point in the exact same cumulative check, no extra logic required, and the earlier screenshot already shows that split landing at 80.16% and 19.84% across 100,000 simulated visitors.

Open source feature flagging platforms like GrowthBook build entire product features around exactly this idea, gradually increasing a variant's weight over days instead of flipping a switch to 50/50 on day one. A homemade javascript ab testing script can borrow the same instinct even without adopting a whole platform.

Before trusting any result from either split, it is worth understanding statistical significance at a basic level. A weighted test with a small variant group needs proportionally more total traffic before the numbers stop being noise, since the variant bucket is smaller to begin with.

It also helps to periodically check that the actual observed split matches the configured weights. A large, persistent gap between the two, say a configured 50/50 test that somehow settles at 60/40 in practice, usually points to a bug in the bucketing code itself rather than ordinary random variation, and catching that early saves an entire test from being run on broken data.

Bucketing a visitor is only half the job. The other half is recording which bucket converted, and the simplest place to send that is wherever GA4 is already listening.

function trackConversion(testName, variant) {
  window.dataLayer = window.dataLayer || [];
  window.dataLayer.push({
    event: 'ab_test_conversion',
    test_name: testName,
    variant: variant
  });
}

// called once, right when the goal action actually happens
trackConversion('homepage-cta-color', window.currentVariant);

Pushing a plain object to dataLayer works with a GTM container already wired up per GA4 custom event tracking through Google Tag Manager, so the variant name rides along as an event parameter and shows up in GA4's own reporting without writing a single line of server code.

Google's own GA4 event reference covers the parameter naming rules worth following here, mainly that a custom parameter like variant needs to be registered as a custom dimension in GA4's admin panel before it shows up as a usable report column, not just in the raw event stream.

A homemade javascript ab testing script like the one above is genuinely enough for a single page, low stakes test. It is not a full replacement for a dedicated platform once the requirements grow past that.

One practical limitation worth planning around up front is what the page looks like for a split second before the script runs. If the control version renders first and the page then swaps in the variant a moment later, visitors briefly see a flash of the wrong version. Hiding the relevant section until the bucket decision is made, then revealing it, avoids that flash entirely at the cost of a tiny extra delay before that part of the page becomes visible.

Server rendered pages add a real wrinkle here too. If the variant decision has to be made before the page even reaches the browser, the same hash function needs to run again on the server using Node's built in crypto module or an equivalent, so the visitor gets the correct variant baked into the first response instead of an initial flash of the wrong one.

It is also worth knowing that Google's own free testing tool, Google Optimize, shut down in 2023, a change documented directly by Google. That gap is part of why a growing number of teams reach for either an open source option or a small script like this one for anything that does not need enterprise scale reporting.

Most of the ways a javascript ab testing script goes wrong trace back to one of these five habits.

• Rolling a fresh random pick on every page load: proven above to flip a visitor between buckets, which quietly poisons the entire result set.
• Swapping the variant's DOM in late, after the page has already painted: causes a visible flash, related to the layout shift concerns covered in web.dev's guide to Cumulative Layout Shift, since a late DOM swap can itself trigger a real layout shift penalty.
• Peeking at results too early and stopping the test the moment a number looks good: small sample sizes swing wildly and often revert once enough real traffic has flowed through.
• Testing on the same device used to build the script: the developer's own repeated visits get counted as real traffic unless that user is explicitly excluded from the sample.
• Forgetting to register the custom dimension in GA4: events still arrive in the raw stream, but the variant parameter never becomes a usable report column until that step happens.