Add experiment results from lift tests to the model as priors for calibration

[yanhong-zhao-ef]

Hi @pabloduque0 thank you for the help so far and I have been looking for any documentation in the repository for this information but couldn't find any.

I have the conversion lift tests results from YouTube and I would like to input that into my model as a prior. How would you advise me to go about this?

[michevan]

This is still a topic of active research, so we don't have a definitive answer yet, but a reasonable starting point would be to keep the prior on the channel coefficient as a half-normal, but change the scale of that half-normal so that its mean is the same as the result of your experiment result.

To do this, you first have to convert your ROI for a given channel as measured from your experiment into a beta for that channel: ROI = beta x F(impressions) / spend, where F is your media transformation for that channel (you'll have to assume values here for the hill and adstock parameters). Once you have your point estimate for beta, you can solve for the corresponding scale parameter for the half-normal as: scale = beta x sqrt( pi / 2).

[queili]

Hello @michevan,
I've been looking for the same information, thanks a lot for the help so far.
Just to be sure, I pass those lift values to the media priors or do I have to input it as a custom prior?
I don't have test results for all channels, just for some of them, so for the rest should I pass the sum of costs as usual?

[michevan]

hi @kayleighdp6, thanks for the question! This is a complicated topic, and as I said above we're still researching it ourselves so our guidance here may change still.

But as a baseline approach, it should be fine to pass those lift values (converted into a scale parameter as described above) directly to the media prior argument, and they can be passed along with costs in the other channels where you don't have experiments. Make sure to re-scale the costs in advance though so that all the media priors are similar in magnitude!

The next level of complexity would be to use custom priors. There are a few ways to do this. You could use wider / less informative priors than the default for channels without experiments, and/or you could use either the half-normal default or the actual results of your experiment (including error bars) for the priors for channels with experiments.

The overall guidance is just to make sure your priors reflect a self-consistent (across all channels) picture of your estimates of the effectiveness of each channel, before you've run any data into your MMM. Our default priors in LMMM follow this approach but are arguably a bit conservative since the half-normals have zero as their likeliest value.