The ability to estimate and control extreme tail risks, besides being an integral part of quantitative risk management, is central to running operations requiring high service levels and cyber-physical systems with high-reliability specifications. Despite this significance, scalable algorithmic approaches have remained elusive: This is due to the rarity with which relevant risky samples get observed, and the critical role experts play in devising variance reduction techniques based on instance-specific large deviations studies. Our goal in this talk is to examine if such tailored variance reduction benefits can be instead achieved by instance-agnostic algorithms capable of scaling well across multitude of tail estimation and optimisation tasks.

To this end, we identify an elementary transformation whose push-forward automatically induces efficient importance sampling distributions across a variety of models by replicating the concentration properties observed in less rare samples. This obviates the need to explicitly identify a good change of measure, thereby overcoming the primary bottleneck in the use of importance sampling beyond highly stylized models. Our novel approach is guided by developing a new large deviations based stochastic model which brings out the phenomenon of self-similarity of zero variance distributions. Being a nonparametric phenomenon, this self-similarity is manifest in a rich set of objectives modeled with tools such as linear programs, piecewise linear/quadratic objectives, feature maps specified in terms of neural networks, etc., together with a spectrum of light and heavy-tailed multivariate distributions.

Karthyek Murthy serves as an Assistant Professor in the Engineering Systems & Design pillar of Singapore University of Technology and Design. His research interests revolve around data-driven operations research, focusing on models and methods for translating data into large-scale planning and operational decisions that are robustly effective and reliable in the face of uncertainty. His research has been recognized with the biennial INFORMS Applied Probability Society best publication award (2023), Winter Simulation Conference best paper award (2019), and the INFORMS Junior Faculty JFIG paper competition Third Prize (2021). He earned his PhD in Systems Sciences from the Tata Institute of Fundamental Research in 2015 and worked as a postdoctoral researcher in Columbia University’s Industrial Engineering & Operations Research department from 2015 to 2017. He serves on the editorial boards of Operations Research, Stochastic Systems and Operations Research Letters.