Experimentation with Feature Flags
Experimentation and A/B testing cannot only be done using Feature Flags. You can also develop containers in different branches and use Kubernetes to run different versions in production; however, this will increase your complexity in Git and does not scale well. You don't have the context for the users either, so gathering the data to prove or diminish your hypothesis is much harder. Most of the solutions for Feature Flags have built-in support for experiments, so this is the fastest way to get started.
To experiment, you define a hypothesis, conduct an experiment, and then learn from the results. An experiment can be defined as follows (see Figure 10.4):
- Hypothesis: We believe {customer segment}, wants {product/feature} because {value prop}.
- Experiment: To prove or disprove the preceding, the team will conduct an experiment.
- Learning: The experiment will prove the hypothesis by impacting the following metrics.
...