Policy learning with confidence

This paper introduces a framework for selecting policies that maximize expected welfare under estimation uncertainty. The proposed method explicitly balances the size of the estimated welfare against the uncertainty inherent in its estimation, ensuring that chosen policies meet a reporting guarantee, namely, that actual welfare is guaranteed not to fall below the reported estimate with a pre-specified confidence level. We produce the efficient decision frontier, describing policies that offer maximum estimated welfare for a given acceptable level of estimation risk. We apply this approach to a variety of settings, including the selection of policy rules that allocate individuals to treatments and the allocation of limited budgets among competing social programs.