Large Language Models and the Rationalist-Empiricist Debate

ABSTRACT To many, Chomsky’s debates with Quine and Skinner are an updated version of the rationalist-empiricist debates in the 17th century, the consensus being that Chomsky’s rationalism was victorious. This dispute has re-emerged with the advent of Large Language Models, with some arguing that LLMs vindicate rationalism because of the necessity of building in innate biases to make them work. The necessity of building in innate biases is taken to prove that empiricism hasn’t got the conceptual resources to explain linguistic competence. Such claims depend on the nature of the empiricism one is endorsing. Externalized empiricism has no difficulties with innate apparatus once they are determined empirically (Quine 1969). Thus, externalized empiricism isn’t refuted because of the need to build in innate biases in LLMs. Furthermore, the relevance of LLMs for the rationalist-empiricist debate in relation to humans is dubious. For any claim about whether LLMs learn in an empiricist manner to be relevant to humans it needs to be shown that LLMs and humans learn in the same way. Two key features distinguish humans and LLMs. Humans learn despite a poverty of stimulus but LLMs learn because of an incredibly rich stimulus. Human’s linguistic outputs are grounded in sensory experience (via human specific cognitive capacities) but LLMs are not. These differences in how the two learn indicates that they both use different underlying competencies to produce their output. Therefore, any claims about whether LLMs learn in an empiricist manner are not relevant to whether humans learn in an empiricist manner.