Pandering to the Masses: Does Engineering Still Matter?
By Joel Hruska
Date: September 27, 2001
Last month, the P4 hit 2 GHz. If you saw the ads (and you'd have to be hiding under a rock to miss them) Intel has been trumpeting this as a monumental achievement of engineering, asking such thought-provoking questions as, "Can time and space be bent?" and claiming "it took 29 years to reach 1 GHz -- and only 18 months to reach 2 GHz."
From a marketing standpoint, the P4 is a brilliant product. It adeptly exploits the ignorance of the masses, and allows Intel to trumpet from the heavens about the raw "MHz speed" of their new product. It also marks the first time since the introduction of the P4 that Intel moves ahead of AMD even slightly in both overall performance and MHz speed.
How much of an achievement is this, however? The Intel Pentium 4 at 2 GHz is 42% faster than the Athlon in terms of MHz! Furthermore, look at the FSB (Front-Side-Bus, or system interface) speeds. At 400 MHz the Pentium 4's FSB is clocked 50% higher than the 266 MHz Athlon bus, and the P4 itself possesses 35% more memory bandwidth. In short, the P4 requires both clockspeed and bandwidth advantages in excess of 30% in order to pull just slightly ahead of its main competitor. Furthermore, keep in mind, the Athlon core design is over two years old -- the Pentium 4 core isn't even one.
Given its speed and bandwidth advantages, it doesn't seem like much of an accomplishment for the 2 GHz Intel Pentium 4 to barely squeak past the Athlon 1.4 GHz in some benchmarks (to be honest, there are still almost as many tests that the much cheaper 1.4 GHz Athlon still wins). In fact, it seems almost sad -- especially coming from Intel. Let's take a look at the Pentium 4 processor minus the Intel marketing machine, without any references to Blue Men, and avoiding questions about the space-time continuum that belong in a bad sci-fi novel rather than in a CPU article.
===================================
Methodology, Benchmarks, and Results
"IPC" is an abbreviation for "Instructions Per Clock-cycle" and is a measure of the efficiency of a processor per clock tick. The primary issue we are going to examine in this article is the impact of the P4's IPC penalty when measured against the Pentium 3.
It's a known fact that the Intel Pentium 4 is less efficient per clock-cycle than the P3. The question is, how much of an impact does the P4's lack of efficiency have on its performance? Does its higher clock speed offset its IPC penalty?
In order to answer this question, we derived a simple test. We examined the P4's performance in several well known benchmarks who's results we gathered from well known sites [ed: and, with recent advertising decisions, perhaps Intel friendly] like Tom's Hardware and AnandTech. We also took a look at published SPECINT2000 scores for both processors. We then derived an "indicative IPC" number for comparison purposes by dividing the benchmark score by the clock-speed of the processor. This number will be abbreviated "iIPC" for the remainder of this article.
Rather than clutter the main article with the details on how the iIPC number was generated, we are providing all statistical information and methodology data here. Anyone curious as to how the iIPC numbers were generated can refer to this article for further explanation.
===================================
iIPC in Sandra 2001te Dhrystone ALU
Despite its much-hyped "double-pumped" ALU's, the Pentium4 takes a clear backseat to the Pentium 3 in terms of sheer efficiency in SiSoft Sandra. The Intel P4 comes in with an iIPC score only 69% as high as that of the P3. This means it takes an Intel Pentium 4 running at 1.7 to 1.8 GHz to equal the performance of the Pentium 3 Tualatin at 1200 MHz in SiSoft Sandra's Dhrystone.
===================================
iIPC in SiSoft Sandra 2001te Whetstone
In Sandra Whetstone, the difference is even more dramatic. Without its specialized FPU instructions, the Pentium 4's FPU is demolished by the Pentium 3's. The Intel Pentium 4 only performs at 39% of the P3's iIPC level when running pure x87 FPU code. Keep in mind that this is a synthetic benchmark, not a real-world performance test, but it indicates just how severe a performance hit the P4 can take under non-optimal conditions.
===================================
iIPC in Sysmark 2000
Unlike our first two tests, SysMark2000 is not a purely synthetic benchmark with no real world performance correlation. SysMark2000 is an application level benchmark that stresses certain tasks popular software titles. However, SysMark200 also has been shown by Van's Hardware to have a decided bias towards Pentium III and 4 processors with SSE support. Nevertheless, the same story is repeated here.
The Intel Pentium 4 comes in at about 60% of the performance level of Pentium 3. A clear pattern is starting to emerge -- and a clear reason for why Intel has ramped the P4 so aggressively. In all the cases we've looked at thus far, the Pentium 4 must command a 30-60% clock-speed advantage over the Pentium 3 in order to be competitive. Let's a look at our last benchmark, though, before we draw final judgment.
===================================
iIPC in SPECINT 2000
The P4 in this case comes in at 87% of the performance of the P3 in terms of iIPC executed. The P4's relatively better performance in SPEC is due to the benchmark's bandwidth sensitivity as well as SSE2 optimizations enabled by using Intel's latest compilers. With SPEC, an almost best-case circumstance for Intel's new flagship, the slower P4s creep ahead of the fastest Pentium IIIs, with 1.4 and 1.5 GHz P4s edging the Coppermine and Tualatin PIII cores.
But even here there is a dire caveat. One of its few true strengths, the P4 has phenomenal memory bandwidth -- it can transfer data faster than any other current processor. However, the Intel Pentium 4 needs high bandwidth to perform competitively. When saddled with SDRAM through Intel's new i845 chipset, suddenly a 2GHz P4 performs on the same level as a 1GHz Pentium III on many applications.
The average P4 iIPC penalty as derived from an average of all four of these benchmarks weighs in at just under 40%.
The Intel Pentium 4, in other words, performs at roughly 60% the iIPC rating of the Pentium 3.
===================================
Quite a Trendsetter: Pentium 4 "Firsts"
I want to make it clear that there's nothing inherently wrong about the design of the P4 from an engineering perspective. Lowering the IPC of a processor is a well-known and easy way to allow the processor to ramp to a higher speed, with the idea that eventually the higher speed grades available will more than make up for a decrease in IPC. This alone is not a problem -- but it's not the whole story either. The Pentium 4 is a whole series of 'firsts' for Intel. Let's look at some of them.
The Pentium 4
is the first mainstream, consumer-level processor from Intel to offer a lower
level of performance in current applications than its previous generation.
With the exception of the Pentium Pro (which was NOT a consumer-oriented chip)
Intel's next generation of processors have always and without exception
offered increased performance in nearly all circumstances. The Pentium 4
breaks this trend. Only recently (nearly a year after its release) have
Pentium 4 chips become available in speed grades high enough to begin showing
real-world performance benefits. The exceptions -- such as Quake 3 -- are few and
far between.
The Pentium 4
is the first processor who's performance Intel has made such excuses for.
When the Pentium 4 scores badly on modern-day software, the standard Intel
response has been: "The software isn't optimized to take advantage of the
Pentium 4." Well, that's all well and good -- interesting, even -- and notable only
because it sounds like something AMD would've said four years ago when the
Pentium II was cleaning the K6-2's clock. Since when does Intel processors
need optimizations to perform well? Pentium didn't. Pentium II didn't. Pentium
III didn't. I've seen Intel processors with good performance use enhanced
instruction sets to perform BETTER -- but the Pentium 4 is the first chip from
Intel that needs enhanced instructions to perform competitively at all.
The Pentium 4
is the first chip from Intel who's continued success in the marketplace relies
on the stupidity of customers.
Now, let me be clear on this one. I am not insinuating that buying Intel is
inherently stupid. The fact is, however, Intel has chosen to concentrate so
explicitly on MHz ratings for one reason -- marketing. Knowing that the average
customer rates a computer on MHz, Intel has chosen a strategy that allows them
to demonstrate MHz speed above all else, regardless of actual real-world
performance.
It's a dangerous move. When a customer moves from a computer running at 1 GHz to a computer running at 2 GHz, he/she expects to see a major leap in performance. While a 2 GHz P4 IS faster than a 1 GHz P3 [ed: if the P4 is not using Intel's SDRAM based i845.], its already-discussed IPC penalties will prevent it from demonstrating the major performance jump the customer may be looking for. By pushing marketing so aggressively, Intel risks alienating its own customer base when they discover they didn't get the performance they thought they paid for.
The i845 chipset may only exacerbate this problem. While it allows for cheaper P4's and a non-Rambus platform, it also clearly handicaps the P4's performance severely. It is possible that customers purchasing the P4+SDRAM combo may be so turned off by its performance/price ratio that they'd end up leaving the Intel camp altogether upon their next upgrade.
Now, I don't know about the rest of you, but I'd be a touch offended, realizing that Intel's marketing plan basically rested on assuming the stupidity of my fellow consumers and me. [Ed: Did someone mention "Celeron?"]
===================================
So why does all this
matter?
It's a valid question. I know there are reviewers and readers out there who will
ask: "What's it matter if the P4 is inefficient since it is ramping high enough
to begin overcoming those problems." Here's my response:
Even today, nine months after its release, the vast majority of software
available is not Pentium 4 optimized -- meaning the worst case IPC penalties discussed above
will become extremely important. Companies considering P4 upgrades must realize
that, unlike Intel processors in the past, they cannot count on a consistently
high level of performance from P4 -- as we and other reviewers have shown, the
chip's performance is all over the map, from extremely high to extremely low.
Furthermore, ordinary benchmarks cannot be relied upon. It's a known fact that
certain video card manufacturers now hyper-optimize their video card drivers for
Quake III in order to turn in exceptional scores in the benchmark. As our
readers know, VHJ recently did an expose on BAPCo, and the inclusion of SSE
instructions to benefit the Pentium 3 --while leaving the Athlon without similar
code. Intel has repeatedly demonstrated that they are not above tampering with
supposedly neutral benchmarks in order to obtain desired results.
===================================
Congratulations, Intel
I'd like to officially congratulate Intel on the release of the 2GHz Pentium 4. It's the first Intel P4 chip, which -- under heavily specialized conditions and wielding only a 30-40% advantage over its main competitor in terms of clock speed, memory bandwidth, and front-side bus speed -- has managed to finally compete with its older rival.
Now there's something to celebrate about.
===================================
Final Words
It is not the intent of this writer nor of Van's Hardware to imply that the Pentium 4 is an inherently bad product, nor that its IPC penalty is an insurmountable barrier when compared to its older and more efficient siblings. We feel a duty, however, to educate our readers beyond the questionable decisions made by Intel marketing, and feel that potential Intel customers have a right to know the performance level to expect from a Pentium 4 platform -- especially considering the price premium Intel has set for the P4.
We recommend that readers interested in purchasing any new computer or processor take the time to research the product, analyze its strengths and weaknesses, and take a careful look at what use the computer is intended for. In the case of the P4 it would also be a good idea to inquire about the level of SSE2 support in any software you intend to use, as this will make a major difference in the chip's performance.
===================================
Pssst! Our Shopping Page has been updated.
===================================