Software has changed, hardware has not
Computer hardware has undergone several evolutions. It began with the room size monsters such as ENIAC. Mainframes have been with us since the 1950s. Minicomputers arrived in the mid-1960s. Microcomputers came in the late 1970s and expanded greatly in the 1980s. Blade servers arrived in 2001.
As the hardware changed, the approach to programming changed with it. COBOL programming for mainframes separated procedure from data. Software developers embraced object oriented programming to handle the complexity of building user interfaces for desktop computers.
In the late 1990s, a subtle shift occurred; software started to drive changes in hardware rather than the other way around. Databases and client-server architecture demanded larger and larger servers. Web-based architectures pushed the demand even further.
Relational databases and other server applications are designed to run on a single server. As the size of data and applications have grown, the resources needed for these applications has grown and concomitantly the size of the servers has grown as well. It is now common to have servers with 16 processors or more. However, not all applications require so much hardware capacity. The servers are actually too big for most applications. As a result, it is now common to divide the large server into multiple virtual servers.
In this decade we are seeing a remarkable, dramatic shift in software development. Parallel processing was once the domain of university and defense research. Hadoop has made parallel processing common place. Even small and midsize businesses are looking at using the Hadoop zoo of technologies. With NodeJS and similar technologies, web applications are becoming microservices, smaller and even more distributed.
Meanwhile, where is the dramatic shift in hardware? Hardware vendors keep building bigger and bigger servers, rushing forward to meet a need of yesterday, not tomorrow. We spend so much developing software that can make big servers look and behave like lots of small servers.
Why not just create small servers?