Mathematical Physics is the study of what can be really proved about our theoretical models of physical systems. This differs from other kinds of theoretical physics because physicists often take the lack of experimental refutation of a mathematically invalid calculation as “proof” that the result is correct.
Statistical physics (aka Statistical Mechanics) is the study of physical systems having so many degrees of freedom that it is not feasible to measure a complete set of the individual observables (such as the positions and momenta of all the molecules in a volume of gas or the angular momenta of all the electrons in a crystal), but for which some observables (such as temperatures and pressures), defined as averages of those most naturally considered as forming a complete set, are expected to evolve in a way that does not depend on the specific values of all the variables needed for a complete description.
Many of the expected behaviours of these averages are assumed by physicists without any complete proof; and one important area of mathematical physics is the filling in of these missing proofs. A classic text of this sort is the book ‘Statistical Mechanics: Rigorous Results’ by David Ruelle.