Have you worked on a data team? I've seen that bs a number of times, it's how I mentally grade different managers and PM/POs.
Re the unavailable data: Smart people ask before a big change, get told what devs are missing and need to instrument/record and then leverage those new metrics for the before/after comparison. Not-smart people yolo the changes, ask for the metrics after and go whoops it's too hard or impossible to check.
Re the unavailable data: Smart people ask before a big change, get told what devs are missing and need to instrument/record and then leverage those new metrics for the before/after comparison. Not-smart people yolo the changes, ask for the metrics after and go whoops it's too hard or impossible to check.