Approximation methods in physics: foundational, philosophical and historical questions

When attempting to describe the world using mathematical models, we almost inevitably encounter a problem of tractability: the equations that most faithfully describe the real-world system of interest are too complex to be solved exactly. There are different ways to respond to this situation. One approach is to move to a simpler set of equations; if all goes well, we may hit upon a more tractable representation of the target system which is nevertheless predictive, explanatory or otherwise useful. Another approach is to retain the more complex equations but broaden the range of methods used to draw inferences from them. Applied mathematicians have developed a dizzying array of approximation methods, which allow quantitative results to be obtained in the absence of exact solutions. In the context of quantum physics alone, the list of approximation methods is long and ever-expanding: perturbative expansions, WKB semi-classical expansions, Hartree-Fock variational approximations, Born-Oppenheimer approximations, numerical integration methods, Monte Carlo simulations, and the list goes on.