Mixed precision algorithms for high performance computing

04/01/2022
Intervenant(s) : Théo Mary
Today's floating-point arithmetic landscape is broader than ever. While scientific computing has traditionally used single precision and double precision floating-point arithmetics, half precision is increasingly available in hardware and quadruple precision is available in software. Lower precision arithmetic brings increased speed and reduced communication and energy costs, but it produces results of correspondingly low accuracy. Higher precisions are more expensive but can potentially provide great benefits, even if used sparingly. A variety of mixed precision algorithms have been developed that combine the superior performance of lower precisions with the better accuracy of higher precisions. Some of these algorithms aim to provide results of the same quality as algorithms running in a fixed precision but at a much lower cost; others use a little higher precision to improve the accuracy of an algorithm.
In this tutorial I will provide an introductory description of a broad range of mixed precision algorithms, primarily in linear algebra, aimed at anyone who wishes to develop or benefit from mixed precision numerical linear algebra algorithms.