The main aim of this talk is to introduce and review some of the recent developments in the theoretical and mathematical aspects of the non-equilibrium statistical mechanics of geophysical flows. This domains, at the intersection between statistical mechanics, turbulence and climate applications is a wonderful new playground for theoretical and mathematical physicists. Path integrals, instanton theory, semiclassical approximations, large deviation theory, and diffusion Monte-Carlo algorithms are at the core of our approach. Many natural and experimental turbulent flows display a bistable behavior: rare and abrupt dynamical transitions between two very different subregions of the phase space. The most prominent natural examples are probably the Earth magnetic field reversals (over geological timescales), the Kuroshio current bistability. Recent results show that a similar bistability occurs also in the turbulent dynamics of atmosphere jets, for instance on Jupiter's troposphere. Those abrupt transitions are extremely rare events that change drastically the nature of the flow and are thus of paramount importance. We will discuss those phenomena from a theoretical physics point of view, mainly in relation with large deviation theory.