Mingardi's book is the best short introduction to Spencer's social thought. It is part of a remarkable series of books on "Major Conservative and Libertarian Thinkers" edited by John Meadowcroft.
Mingardi points out the odd character of the history of Spencer's reputation. During his lifetime, Spencer was perhaps the most famous and respected philosopher of the last half of the 19th century. He was probably the first philosopher to sell over a million copies of his books before his death. And yet, by the 1930s and 1940s, hardly anyone was reading Spencer. Through the influence of people like Richard Hofstadter--through his book Social Darwinism in American Thought (1944)--Spencer's evolutionary classical liberalism was labeled as Social Darwinism, which suggested a morally repugnant claim that nothing should be done to help the weak and the poor who were unfit to live in a competitive society. Now, Barack Obama regularly pins this label of Social Darwinism on those who disagree with him about expanding the power of government for social reform.
As Mingardi indicates, even those leading the recent revival of evolutionary classical liberalism--those like Paul Seabright, Paul Rubin, Daniel Friedman, Matt Ridley, and me--rarely acknowledge that this is a revival of Spencer's thinking. Moreover, this recent evolutionary social thought draws from the influential ideas of Friedrich Hayek about how the spontaneous orders in complex modern societies are generated by adaptive cultural evolution; but while this corresponds exactly to what Spencer said about the spontaneous evolution of "industrial societies" as opposed to "militant societies," Hayek showed no knowledge of, or interest in, Spencer.
As I have indicated in some previous posts, I reject the unreasonable utopianism found occasionally in some of Spencer's writing--as, for example, when he writes in Social Statics: "so surely must the human faculties be moulded into complete fitness for the social state; so surely must the things we call evil and immorality disappear; so surely must man become perfect" (80). Mingardi never takes up this fantastic claim that human nature can be so changed as to be morally perfect.
Mingardi has convinced me, however, that, setting aside his occasional utopianism, Spencer defended an evolutionary classical liberalism that supports a rational optimism about historical progress towards increasing liberty and decreasing violence.
One good statement of this reasoning is at the end of Spencer's "Filiation of Ideas" (in David Duncan's Life and Letters of Herbert Spencer):
The assertion of the liberty of each limited only by the like liberties of all, was shown to imply the doctrine that each ought to receive the benefits and bear the evils entailed by his actions, carried on within those limits; and Biology had shown that this principle follows from the ultimate truth that each creature must thrive or dwindle, live or die, according as it fulfills well or ill the conditions of its existence--a principle which, in the case of social beings, implies that the activities of each must be kept within the bounds imposed by the like activities of others. So that, while among inferior creatures survival of the fittest is the outcome of aggressive competition, among men as socially combined it must be the outcome of non-aggressive competition: maintenance of the implied limits, and insurance of the benefits gained within the limits, being what we call justice. And thus, this ultimate principle of social conduct was affiliated upon the general process of organic evolution. (576)
Despite this optimism about evolutionary progress as favoring the expansion of liberty and social order based on voluntary cooperation, Spencer became deeply discouraged by the move--during the last few decades of his life--from the classical liberal principle of limited government protecting individual liberty to the new liberal principle of expanding government for promoting social reform. As this movement towards collectivist statism increased in the first half of the 20th century, some scholars claimed that Spencer had been overthrown by the very evolution that he had championed, because social evolution had turned away from his classical liberalism.
Actually, however, Spencer did not claim that evolutionary progress was an unswerving and inevitable line of ascent. Rather, he recognized the "rhythm of motion" in social evolution:
On recognizing the universality of rhythm, it becomes clear that it was absurd to suppose that the great relaxation of restraints--political, social, commercial--which culminated in free trade would continue. A re-imposition of restraints, if not of the same kind, then of other kinds, was inevitable; and it is now manifest that whereas during a long period there had been an advance from involuntary cooperation in social affairs to voluntary cooperation, there has commenced a reversal of the process. (Autobiography, II: 369)But if Spencer was right about societies with increasing liberty and decreasing violence being generally more functionally adaptive than other societies with less liberty and more violence, then we can expect that although there will be unpredictable swerves in history away from classical liberal principles, the general pattern of history over the long run must be towards classical liberalism.
Pinker offers a good visual model of this in his Better Angels of Our Nature. While surveying the evidence for a long historical evolutionary trend of declining violence and increasing liberty, which supports classical liberal culture, Pinker acknowledges that there can be variable deviations from this general trend. So he presents the trend of declining violence as a declining irregular sawtooth pattern, in which it's possible to have sudden peaks in violence like World War Two, caused by a few illiberal individual leaders like Hitler, Stalin, and Mao, but still the general trend downwards continues.
In 1971, Murray Rothbard made a similar point in his essay on "Social Darwinism Reconsidered". He observed that if one is persuaded by the evolutionary classical liberal argument of Spencer, then one can be reasonably optimistic that history is generally moving towards liberty, despite the many deviations and reversals in the process. So, for example, if one is persuaded by Ludwig von Mises' argument that socialism cannot calculate economic values, then socialism cannot work in a modern industrial society. Even Lenin recognized that when he saw the failure of his attempt to abolish money, in his attempt to put Marx's ideas into practice, and Lenin was forced to shift back to a limited free market economy (the "New Economic Policy"). Similarly, all of the attempts to establish pure socialism in an industrial society have eventually failed.
This vindicates the reasonable optimism of Social Darwinism as based on a scientific understanding of evolutionary natural law and of cause and effect, Rothbard concluded, because "over the long run, the dysfunctional must come to a bad end, must cleanse itself and wipe itself out, while the truly functional and proper can remain and prosper." "The eventual victory of liberty is inevitable," he declared, "because only liberty is functional for modern man."
If evolutionary classical liberalism is correct, then liberal societies must be evolutionarily more adaptive, more functional, or more productive--economically, morally, and intellectually--than illiberal societies. And, consequently, despite the occasional turns towards illiberal social orders, the arrow of history in the long run points to liberty.
Posts on some of these points can be found here, here, here, here, here, here, here, here, and here.