Performance Numbers
As many developers know, providing performance numbers can be a bit a tricky. I have tried to pull together a bunch of information looking both at low-level primitives, which give a fairly good indication of compiler performance, and higher level applications which might give a better example of real-world speed improvements.The numbers below measure percentage improvements or decreases in performance between Release 3.2, Release 4 Metrowerks/PE and Release 4 EGCS/ELF. These will be denoted as 3.2, PE and ELF in the figures below.
All of these tests we performed on the same machine:
- Single Intel Pentium II 350mhz
- 440 BX motherboard (100 mhz bus)
- 64 mb PC-100 Ram
- 2 4.1 gig Ultra IDE drives
- Diamond Viper 330 Video Card (RIVA 128 chipset)
- NE2000 compatible PCI network card
I chose this hardware as it is a fairly standard, middle of the road new machine that costs roughly $1000. The performance numbers differ depending on exactly what hardware you have.
I do want to note that there are instances where the switch to ELF and gcc seems to marginally decrease performance. We think that this has to do with the slight increase in overhead when calling a function from a shared library. In those tests (like the Transaction Timings below) where much of the actual time is spent making these function calls, rather than doing lots of actual calculations, this speed difference will appear. In normal, non-benchmark applications this speed difference is negligible.
Coding Primitives:
|
Operation |
ELF Performance |
|
| vs. 3.2 | vs. PE | |
| memcpy() | + 141% | + 20% |
| notable improvement: memcpy 1024 bytes | + 547% | + 78% |
| sprintf() | + 50% | + 50% |
| notable improvement*: sprintf int64 | + 1425% | + 1509% |
| strcpy() | + 38% | + 37% |
| strlen() | + 106% | + 90% |
*Metrowerks had a particularly bad implementation
OS Primitives:
|
OS Primitive |
ELF Performance |
|
| vs. 3.2 | vs. PE | |
| Kernel Port Messaging | + 14% | + 3.5% |
| Graphic Primitives | + 40% | + 8% |
Transaction Timings
Transaction Timings (app_server communication)
Performance of resizing a window with 2000 views.
|
Release 3.2 APIs |
||
| Build | elapsed time (sec) | operations per second |
| 3.2 | 5.55 | .180 |
| PE | .75 | 1.333 |
| ELF | .79 | 1.266 |
ELF vs. 3.2: + 603%
ELF vs. PE : - 5%
|
New Release 4 APIs |
||
| Build | elapsed time (sec) | operations per second |
| 3.2 | 5.55 | .180 |
| PE | .136 | 7.735 |
| ELF | .139 | 7.194 |
ELF vs. 3.2: + 3997%
ELF vs. PE : - 2%
Semaphore Speeds
These improvements reflect overall kernel function calling improvements, the semaphore code remains the same.
ELF vs. 3.2: + 7%
ELF vs. PE : + 5%
Real Applications
These applications do real things in the BeOS, rather than testing the
same small function time and time again, like the various primitive
tests above. They should show the performance improvements of the
BeOS a little more realistically.
Fractal App
4 threads calculating a Mandlebrot set, passing information through
2.4 k BMessages to a BWindow displaying the information, while writing
to the disk.
| Build | elapsed time (sec) | operations per second |
| 3.2 | 12.6 | .079 |
| PE | 3.4 | .294 |
| ELF | 1.5 | .667 |
ELF vs. 3.2: + 774%
ELF vs. PE : + 127%
Query Test
Creates a new indexed attribute, creates 1000 files with that attribute,
performs a query to find those files, and writes their names into a file.
| Build | elapsed time (sec) | operations per second |
| 3.2 | 2.57 | .389 |
| PE | 3.03 | .330 |
| ELF | 2.14 | .467 |
ELF vs. 3.2: + 20%
ELF vs. PE : + 42%
Back to the Switch
