J.W.Backus entitled his 1978 Turing Award lecture "Can Programming be Liberated from the von Neumann Style" (Communications of the ACM, August 1978, Volume 21, Number 8). John von Neumann was a brilliant pioneer in computing, as well as earning considerable renown for his work in game theory and its applications in economics. He never completed his 1945 paper, the "First Draft of a Report on EDVAC" which might have been the first description of the concept of a stored program computer as we currently understand the term.
As Backus stated:
When von Neumann and others conceived it [the von Neumann computer] over thirty years ago, it was an elegant, practical, and unifying idea that simplified a number of engineering and programming problems that existed then. Although the conditions that produced its architecture have changed radically, we nevertheless still identify the notion of "computer" with this thirty [(rpj:) now sixty!] year old concept.In its simplest form, a von Neumann computer has three parts: a central processing unit (or CPU), a store, and a connecting tube that can transmit a single word between the CPU and the store (and send an address to the store). I propose to call this tube the von Neumann bottleneck. The task of a program is to change the store in a major way; when one considers that this task must be accomplished entirely by pumping single words back and forth through the von Neumann bottleneck, the reason for its name becomes clear.
Backus goes on to discuss how the majority of programming languages today reflect this bottleneck architecture: They use variables to denote individual cells in the store and assignments to model the pushing of data and addresses through the bottleneck. Such languages are called von Neumann languages, or imperative languages.
Von Neumann languages have imperative statements -- requests to the CPU to make some change to the store, and expressions (conceptually simpler) to denote values that need to be calculated by the CPU. Making things worse, many languages allow the processing of expressions to include side-effects that change the contents of the store during expression evaluation. The main difficulty with imperative statements and side effects is that they do not commute: we cannot change their order without changing the meaning of our program.
Purely functional, also called "applicative", programming languages restore commutativity and preserve the rules of mathematics so that proofs concerning the correctness of functional programs are easier to establish.