Prepare Your Environment for A/B Testing

There are a couple of steps you should take to prepare your A/B testing environment.

Set-up Click Analytics

Because Algolia uses click and conversion rates to do comparative A/B testing, you need to implement Click and Conversion Analytics first.

Add unique user identifiers to your searches

Make sure every search is associated with a unique user identifier. This serves two purposes:

• It ensures randomness. The engine creates two sets of users - one for variant A and another for variant B. Whenever a new user arrives, they’re randomly distributed into A or B, based on a hash of their unique user identifier.
• It maintains the same distribution. If a user was assigned to variant A once, they continue with A and never B. This “user-to-variant stickiness” is mandatory for the A/B test conclusion to be reliable.

Algolia uses unique user tokens to ensure that the same user is always directed to the same variant. By default, the user token is the user’s IP address. However, you should not rely on this behavior. We recommend that you explicitly create a unique user identifier that you can keep track of in your own application. The user identifier can be sent to Algolia with the userToken parameter, or with the X-Algolia-UserToken request header.

A/B Testing

Once these steps are completed, you can follow the dashboard’s process to start, stop, and view your tests.

If you want to AB test an index’s setting in the dashboard, select the same index for both variants. A prompt will appear asking you which settings you want to test.