Amazon AWS EC2 is a well-developed public cloud computing service in which most instances deliver the performance that would be expected from its "compute units" metric. However, in some cases, the measured performance of compute instances can have statistical aberrations when compared to theoretical benchmark performance. Our blog from a few weeks ago demonstrated that improper cloud performance modeling can result in erroneous assumptions. With Rackspace, some misestimates can arise as result of vendor design, but with Amazon EC2, a likely culprit for bottlenecks will be memory constraints, as is often the case in the adoption of modern compute architectures.
When a server is configured for a benchmark environment, it is designed so that that memory and other factors will not become a bottleneck for performance. In order to fill in the gaps in the sparse performance results of systems produced by benchmarks such as SPEC and others, it is possible to interpolate and extrapolate values to estimate performance of different systems (IDEAS RPE2 benchmark is an example of this methodology). It is also possible to apply that same procedure to public cloud services (as IDEAS CloudSizer has started to do in its free beta version). In the graph above, the triangles plot the theoretical assumption of expected benchmark results for a selection of Amazon AWS EC2 instances (the y-axis on the right side represents the range of RPE2 performance estimates for EC2 instances). The blue data points are empirical results, i.e. what actually happens when an end-user spins up a workload on Amazon EC2. Most of the points line up exactly, or are very close, but there are two points at which the theoretical expectation is very different from the empirical data. Both are in the High-CPU instances.
Amazon’s High-CPU instances are like a car that has a Mustang engine (performance), but no gas (memory) to power it. The lower X-axis and the bars in the graph represent the ratio of configured memory in EC2 instances to empirical performance ratio (scaled). Most of the instances have a consistent amount of memory to power their respective output. With the High-CPU instance, though, the memory configuration drops off a cliff, and may thus become a bottleneck for certain workloads. The clustered instance also has a lower ratio, but the total memory for that product, 23 GB, is not atypical for a server with two Intel quad-core Xeon X5570 (the hardware behind that particular instance). The two High-CPU instances, on the other hand, have 1.7 GB and 8 GB. Both values are much lower than typical benchmark enviroments but 1.7 GB is even less than the memory set aside for running an application in a 32-bit enviroment. For end-users interested in purchasing the High-CPU instance, it may be prudent for them to perform their own experiments first, in order to determine if there is any performance degradation. Additionally, it may be important to test out the other classes of instances, since it may actually be the case that there are much cheaper options benefitting from a more attractive $/performance ratio.
Comments