I received the information below from a source who wishes to remain anonymous. The claim it makes is pretty strong - that SIS have, with particular chipsets, been deliberately cheating in a popular benchmark by not rendering all the required frames. I should stress that I have NOT personally verified this information. The experiments below were conducted by my anonymous informant. I mulled over whether to share this information or not for a couple of days before deciding that I would make it public, and that people can make up their own minds - but again, I have no way to directly confirm these results myself. Coincidentally, there is apparently a motherboard that I think uses one of the affected chipsets on the way to me for review, so I will be able to perform the experiments described below myself and share the results. If SIS contact me with a response to the claims in this article, I will share that too. Again, I have not confirmed these results myself so take them with several large grains of salt.

This technique was successful for a while because it was impossible with the naked eye to see the discontinuity in frame numbers while running a benchmark program. However, once news of Frame Skipping spread within the industry, Ziff Davis put a stop to it by the simple expedient of including frame counters on all rendered frames starting with 3D Winbench ’99.

That should have been the end of the story. But when we started comparing the performance of the SIS 630 integrated chipset with that of the VIA Apollo PM133 chipset, we found a new twist to the tale. This is because, from the evidence we have gathered, it appears that SIS has developed new variation of this technique for inflating graphics benchmark performance, which (for want of a better name) we will call "Partial Frame Skipping".

With this technique, we believe that when the SIS 630 driver detects the execution of 3DWinmark it dynamically decides to render a partial frame or full frame depending on the rendering load of the graphics device. The driver ensures that the partial frame always has the frame counter meter drawn, in an attempt to avoid being detected by a video capture device that can see when consecutive frames do not have consecutive frame numbers. To illustrate this technique, we used a DirectX frame grabber utility and captured a sequence of frames in few 3DWinmark tests. The testing methodology and results are described below.

Testing Configuration & Methodology

To compare the performance of the SIS 630 with the PMI33, we ran benchmarks on two systems with the following almost identical configurations:

SIS 630 System

LeadTek Motherboard with SIS 630 chipset with a Pentium III 733MHz with 133MHz FSB, 128MB SDRAM at 133MHz using DirectX7 on Windows ME

PM133 System

PM133 Reference Motherboard with a Pentium III 733MHz with 133MHz FSB, 128MB SDRAM at 133MHz using DirectX7 on Windows ME

When we ran the ZD 3DWinmark test at the default resolution of 10x7x32 and obtained the following results, we were surprised to see that there was such a large gap in performance, with the SIS 630 achieving 15.0 points and the VIA PM133 achieving 10.3.

Our suspicions about the reliability of this result were strengthened when we ran SPEC ViewPerf benchmarks, in which the PM133 heavily outperformed the SIS 630 across the board by margins as high as over 400%.

To find out the reasons for this disparity, we conducted a closer inspection of the 3DWinbench test and found several inconsistencies in the rendering of images on the SIS 630 chipset. We believe that some of these inconsistencies result from efforts by SIS to deceive the benchmarks through Partial Frame Skipping.

Partial Frame Skipping Investigation

For our investigation, we captured the first 11 frames of the chamber scene in 3DWinmark test number 5 on the SIS 630 and VIA PM133 testing systems, as well as on a separate system configured with an NVidia Geforce 256 VGA adapter.

On the PM133 and NVidia Geforce systems, we saw a continuous camera movement from Frame 1 to Frame 11 with the background image changing from Frame 1 to Frame 11 along with the Frame Counter in the image. On the SIS 630 system, however, we noticed some very strange behavior: Although the frame counter changed from 1 to 11, the actual frames themselves only changed in the following sequence

The bitmap images captured on the three systems are available in Zip files that can be provided upon request. Any image playback utility (Adobe Photoshop) can be used to see the trick being performed by the SIS 630 drivers.

For the purpose of this report, we have analyzed the first four frames from the Chamber sequence taken from the SIS 630 and PM133 systems. As you can see in the highlighted area on Frame 2 of the PM133 system, the ringed planet has changed its position by moving away from the laser beam. In Frame 2 on the SIS 630 system, however, it has remained in exactly the same place as in Frame 1; in fact, the only thing that has changed in Frame 2 on the SIS 630 system is the frame counter!

This trend continues with Frame 3 and Frame 4. On Frame 3, the ringed planet is now in exactly the same position in both the SIS 630 and PM133 systems. But in Frame 4 on the PM133 system, the gap between the ringed planet and the laser beam has widened further, while on the SIS 630 system the gap has remained the same as on Frame 3. Again, the only change in Frame 4 on the SIS 630 system is the counter.

Conclusion

As a result of Partial Frame Skipping, the SIS 630 driver does not draw MORE THAN HALF the frames in this sequence. It just draws the frame counters, which are a minor percentage of the rendering required to draw the whole frame. The only reason to do this is to artificially inflate the 3DWinbench benchmark score.