What is GrowthBook?
GrowthBook is a modular platform. You can use it for either Feature Flags, Experiment Analysis, or both.
Feature Flags you create in the GrowthBook UI are published to our API as a JSON file. Pass this JSON into our SDKs and use the feature flags throughout your code. If a feature is running as part of an A/B test, we'll fire a tracking callback in the SDK so you can record that event in your data warehouse or analytics platform for later analysis.
Experiment Analysis queries your data warehouse for raw experiment data and runs it through our stats engine to produce a report. No raw user-level events or PII are ever sent to GrowthBook. We only get back aggregate info instead (sums, sums of squares, etc.).
Feature Flags
Feature flags are a very powerful developer tool. They give you deep control over how and when new functionality is released to your users.
GrowthBook supports 4 types of features:
- Boolean (on/off)
- Number
- String
- JSON
Using features in your code is easy. Here's an example from our Javascript SDK:
// For boolean features
if (growthbook.isOn("my-feature")) {
// ... Do something
}
// For number, string, and JSON features
const value = growthbook.getFeatureValue("other-feature", "fallback");
GrowthBook also supports multiple environments, so you can, for example, have a feature enabled in dev but not production.
You can also change the value of a feature with Override Rules. The following types of rules are supported:
- Forced Value - Choose a subset of users based on targeting attributes and assign them all the same value
- Percentage Rollout - Use random sampling to roll out a new feature value to a percent of users
- A/B Experiment - Run a controlled A/B test between 2 or more feature values
All features for an environment are packaged together into a single JSON file. All you need to do is pass this into our SDKs along with user targeting attributes.
Read more about features here.
Experiment Analysis
GrowthBook needs to connect to your Data Source in order to query experiment results. We support all of the popular SQL data warehouses in addition to Mixpanel and Google Analytics. GrowthBook is extremely flexible and can support almost any schema structure with a little bit of configuration.
Once connected to a data source, you need to build a re-usable library of Metrics. Metrics are what your experiments are trying to improve. Metrics are defined via SQL (if your data source supports it) or a simple query builder. The following types are supported:
- binomial - simple yes/no conversion metrics (e.g.
started trial
,bounce rate
,purchased
) - count - when the number or magnitude of conversions matters (e.g.
downloads per user
,points earned
) - revenue - the amount of revenue earned (e.g.
revenue per user
,average order value
) - duration - the time it takes to do something (e.g.
page load time
,time on site
)
Once these are metrics are set up, you can import experiments and start analyzing the results. GrowthBook uses a Bayesian statistics engine to determine the probability that a variation is better than the control, as well as how much better it is and how risky it is to stop the experiment now.
Your data team can drill down into results by custom dimensions, view the raw SQL that GrowthBook is running on your data warehouse, and export results to a Jupyter notebook for even deeper analysis.
Read more about experiments here.
Use Cases
There are typically three reasons that teams use GrowthBook.
1. Full Experimentation Platform
In this use case, companies use Feature Flags and our SDKs to run experiments in their applications. Then they use our Experiment Analysis to look at the results and decide on a winner.
This is best for companies that are either just starting with experimentation or want to completely switch away from their current way of doing things.
2. Feature Flags Only
In this use case, companies don't run experiments at all and just use GrowthBook feature flags within their engineering team.
This is best for companies that don't have enough traffic to run full experiments, but still want all of the benefits that feature flags provide. It's also good for companies that know they will want to run experiments in the future and want to start instrumenting their applications today to get ready.
3. Experiment Analysis Only
In this use case, companies are already running experiments and analyzing results usually with either a home-built reporting system or by manually creating Jupyter notebooks. They use GrowthBook to automate and improve the analysis process to save time and make better decisions.
This is best for companies that already have a robust process for running experiments and just need a little help analyzing results at scale.