Abstract

In this work, we develop a risk-averse Policy Gradient algorithm in a tail risk optimization problem. Our objective is to find the optimal policy that minimizes tail risk, given a risk measure such as Conditional Value at Risk (CVaR). We employed Extreme Value Theory, along with the automated threshold method to manage risks associated with extreme events. This paper is the first to integrate EVT within risk-averse policy gradient RL algorithms for sequential decision making. To evaluate our approach, we initially test it on simulated data generated from heavytailed distributions, including the Generalized Pareto distribution (GPD) and the Burr distribution. Subsequently, we applied our method to address a hedging problem, aiming to mitigate exceedingly high risks and finding optimal gamma hedging strategies within a highly volatile market where options are notably expensive. This involves identifying the optimal proportion of gamma to hedge, while minimizing costs and risk associated with gamma hedging errors. Also, we utilize the finite difference method to approximate the gradient of the estimated CVaR. The experimental results indicate convergence in the policy, CVaR estimation, and the gradient approximation of estimated CVaR. Moreover, integrating Extreme Value Theory into risk-averse policy gradient methods significantly improves performance, especially in markets characterized by an underlying asset following a Normal Inverse Gaussian distribution (NIG), known for its pure-jump semi-heavy tail distribution.