The short answer
Record the page change, timing, measurement window and other events that might influence the outcome. An experiment log should make alternative explanations visible. This is especially important when content edits happen alongside product launches, advertising or a website migration.
A worked example
A fictional team rewrites a guide and starts a paid campaign in the same week. Later traffic rises. The log records both changes, so the report does not attribute the entire increase to the article's direct answer. The rewrite may still improve usability, but the available observations do not isolate its traffic effect.
A practical checklist
- Save the previous version and describe the specific edit. Avoid vague change labels such as 'full SEO optimization' that cannot be independently inspected.
- Record concurrent campaigns, tracking changes, outages and publication events. Note whether the comparison period contains a different seasonal or operational context.
- Review the result with those conditions in view. Distinguish a correlated change from a causal conclusion and mark inconclusive tests explicitly.
What to avoid
Avoid deleting negative periods or moving the reporting window after seeing the outcome. Do not use an uncontrolled before-and-after comparison as proof that one small copy change caused business growth. A trustworthy log includes observations that challenge the preferred explanation.
A useful follow-up
Should I keep an experiment that did not improve the metric?
Keep the record. Decide whether the change still serves readers, and report the measured result without reframing it as an unsupported win.