I used to work at a company called Silicon Graphics (SGI) for a while in the year 2000, when it owned a company called Cray which had traditionally produced big iron super computers (often vector machines) for the government and specialized industries. SGI was mainly focused, at the time, on the computer graphics industry -- and their kit was sort of standard at the time. I worked mostly with the Origin product line which was a NUMA (non-uniform memory access)
At the time, the enemy at the gates for these types of machines were commodity linux "beowulf" clusters since they were very cheap to cobble together (in terms of parts), and could solve some HPC (high performance compute) jobs very easily. The niche left over for the NUMA and vector machines was made a bit thinner, but still existed. Some problems required the level of processor to processor communication and raw I/O they offered. The catch was this benefit had a huge cost: these machines were not mass produced, they were expensive and had to be engineered to be fault tolerant (at any given time some small percentage of processors would be dead/offline).
Fortunately clusters are still winning today, and even more so with the advent of high speed networking (e.g. Infiniband, 10Gbe, 40Gbe). Actually, they are pretty common place if you consider the data centers where most of the internet runs as just examples of clusters (hello google's map reduce). A decent node-to-node latency over these newer networking technologies is on the order of a few microseconds for a 1024 byte pay-load. The use of commodity (ok, mass produced) parts today make the construction of such useful machines pretty easy. Software has been written by communities of people to make them reliable, and faster.
And if its good enough to run the internet, maybe its good enough for your application?
No comments:
Post a Comment