Skip to main content

Run an Experiment

Enterprise beta

Experimentation is in beta on Enterprise plans. Get in touch to join.

Once an experiment is running, Flagsmith serves the variations automatically through the flag. Your application has two jobs:

  1. Record exposures: tell Flagsmith when an identity actually experienced a variation.
  2. Record conversion events: send the events your metrics aggregate.

Instrument your application​

Event collection is currently available in the JavaScript and Python SDKs, with more to follow. Enable it with the enableEvents / enable_events option.

Users must be identified. Exposures and conversion events are joined per identity, so use the same identifier for flags and events.

import flagsmith from '@flagsmith/flagsmith';

await flagsmith.init({
    environmentID: 'YOUR_CLIENT_SIDE_ENVIRONMENT_KEY',
    enableEvents: true,
});
await flagsmith.identify('user_42');

// Evaluate the flag and record an exposure in one call
const flag = flagsmith.getExperimentFlag('checkout_button');
// ...render based on flag.value

// Record a conversion event; the name must match your metric's event name
flagsmith.trackEvent('purchase', { value: 99.5 });

In React, the useExperiment hook resolves the flag and records the exposure:

const flag = useExperiment('checkout_button');

Exposures​

An exposure is the moment an identity is served a variation; it is what enrols them in the results. getExperimentFlag / get_experiment_flag evaluates the flag and records the exposure in one call.

Record the exposure where the user actually experiences the change; if you evaluate in one place and render in another, call trackExposureEvent / track_exposure_event at the point of display instead.

Exposures are deduplicated per identity: each identity is counted once, in the variation it saw first, so recording an exposure more than once is safe.

Conversion events​

trackEvent / track_event sends a named event, optionally with a numeric value, traits and metadata.

Any event whose name matches a metric's event name feeds that metric. Conversions are attributed post-exposure: an identity's events only count after its first exposure.

While it runs​

  • The flag is locked while the experiment is running.
  • Monitor enrolment on the experiment page's Exposures panel to confirm data is arriving.
  • You can edit the rollout from the experiment page, but changing it mid-experiment can affect the statistical validity of your results.