Charles Hermite’s name has been attributed to several objects and results in mathematics, from Hermitian matrices to Hermite polynomials to Hermite’s identity or Hermite-Minkowski theorem. Despite his achievements and central role in the mathematical life of the 19th century, he often appears as an anti-hero, opposed to anything modern, be it ideals, non-Euclidean geometry or set theory. I will try to explain his point of view which is linked to a vision of mathematics as a natural, observational science, and show how this perspective shaped his mathematical work and his requirements on what good mathematics should be.