Difference between revisions of "Basic Benchmark"
(→Logic of the Benchmark) |
|||
| (8 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | == |
+ | == Benchmark Setup == |
| + | The benchmark is to identify a unit of scanning for a project. That unit is based on the following specifications and the benchmark program calls the Globalyzer API to execute the scan to measure the operations precisely: |
||
| − | |||
| − | == Logic of the Benchmark == |
||
| − | The benchmark is to identify a unit of scanning for a project. That unit is based on the following specifications: |
||
* 100 files |
* 100 files |
||
* 10,000 lines of code |
* 10,000 lines of code |
||
* The default Java rule set |
* The default Java rule set |
||
| + | * 10 scans are run in one batch. |
||
| − | |||
| + | ** The first time the scan is executed, the Java rule set is loaded |
||
| − | The benchmark is carried out using Lite. |
||
| + | ** the next 9 runs do not need to load the Java rule set and hence perform better. |
||
| + | * 3 batches of 10 scans each are run |
||
| + | ** On the first batch, the system must connect to Globalyzer, get the meta-data, etc., which lowers the performance |
||
| + | ** On the subsequent batches, the connection is established, and the first scan only reloads the rule sets as mentioned above. |
||
If you have, say, 100,000 lines of code, on the system specified, it would about take 10 times the duration of this test. |
If you have, say, 100,000 lines of code, on the system specified, it would about take 10 times the duration of this test. |
||
| Line 25: | Line 27: | ||
== Benchmark Results == |
== Benchmark Results == |
||
| + | |||
| + | <pre> |
||
| + | Execute Java Scan Benchmark (10K lines of code, 100 files): |
||
| + | Reloaded rule sets |
||
| + | Executing java scan(1): 11231 milliseconds |
||
| + | Executing java scan(2): 1140 milliseconds |
||
| + | Executing java scan(3): 800 milliseconds |
||
| + | Executing java scan(4): 736 milliseconds |
||
| + | Executing java scan(5): 768 milliseconds |
||
| + | Executing java scan(6): 656 milliseconds |
||
| + | Executing java scan(7): 624 milliseconds |
||
| + | Executing java scan(8): 587 milliseconds |
||
| + | Executing java scan(9): 576 milliseconds |
||
| + | Executing java scan(10): 580 milliseconds |
||
| + | |||
| + | Reloaded rule sets |
||
| + | Executing java scan(1): 1377 milliseconds |
||
| + | Executing java scan(2): 589 milliseconds |
||
| + | Executing java scan(3): 595 milliseconds |
||
| + | Executing java scan(4): 640 milliseconds |
||
| + | Executing java scan(5): 592 milliseconds |
||
| + | Executing java scan(6): 560 milliseconds |
||
| + | Executing java scan(7): 576 milliseconds |
||
| + | Executing java scan(8): 576 milliseconds |
||
| + | Executing java scan(9): 592 milliseconds |
||
| + | Executing java scan(10): 576 milliseconds |
||
| + | |||
| + | Reloaded rule sets |
||
| + | Executing java scan(1): 1361 milliseconds |
||
| + | Executing java scan(2): 576 milliseconds |
||
| + | Executing java scan(3): 592 milliseconds |
||
| + | Executing java scan(4): 544 milliseconds |
||
| + | Executing java scan(5): 576 milliseconds |
||
| + | Executing java scan(6): 574 milliseconds |
||
| + | Executing java scan(7): 577 milliseconds |
||
| + | Executing java scan(8): 577 milliseconds |
||
| + | Executing java scan(9): 592 milliseconds |
||
| + | Executing java scan(10): 576 milliseconds |
||
| + | </pre> |
||
Latest revision as of 18:43, 9 December 2016
Benchmark Setup
The benchmark is to identify a unit of scanning for a project. That unit is based on the following specifications and the benchmark program calls the Globalyzer API to execute the scan to measure the operations precisely:
- 100 files
- 10,000 lines of code
- The default Java rule set
- 10 scans are run in one batch.
- The first time the scan is executed, the Java rule set is loaded
- the next 9 runs do not need to load the Java rule set and hence perform better.
- 3 batches of 10 scans each are run
- On the first batch, the system must connect to Globalyzer, get the meta-data, etc., which lowers the performance
- On the subsequent batches, the connection is established, and the first scan only reloads the rule sets as mentioned above.
If you have, say, 100,000 lines of code, on the system specified, it would about take 10 times the duration of this test.
If you have, say, two rule sets, it would take about twice the duration of this test.
Benchmark System Characteristics
Operating System Type: amd64 Processor Identifier: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel Number of Processors: 4 Available processors (cores): 4 Free memory (bytes): 94958872 Maximum memory (bytes): 1407713280 Total memory (bytes): 96468992
Benchmark Results
Execute Java Scan Benchmark (10K lines of code, 100 files): Reloaded rule sets Executing java scan(1): 11231 milliseconds Executing java scan(2): 1140 milliseconds Executing java scan(3): 800 milliseconds Executing java scan(4): 736 milliseconds Executing java scan(5): 768 milliseconds Executing java scan(6): 656 milliseconds Executing java scan(7): 624 milliseconds Executing java scan(8): 587 milliseconds Executing java scan(9): 576 milliseconds Executing java scan(10): 580 milliseconds Reloaded rule sets Executing java scan(1): 1377 milliseconds Executing java scan(2): 589 milliseconds Executing java scan(3): 595 milliseconds Executing java scan(4): 640 milliseconds Executing java scan(5): 592 milliseconds Executing java scan(6): 560 milliseconds Executing java scan(7): 576 milliseconds Executing java scan(8): 576 milliseconds Executing java scan(9): 592 milliseconds Executing java scan(10): 576 milliseconds Reloaded rule sets Executing java scan(1): 1361 milliseconds Executing java scan(2): 576 milliseconds Executing java scan(3): 592 milliseconds Executing java scan(4): 544 milliseconds Executing java scan(5): 576 milliseconds Executing java scan(6): 574 milliseconds Executing java scan(7): 577 milliseconds Executing java scan(8): 577 milliseconds Executing java scan(9): 592 milliseconds Executing java scan(10): 576 milliseconds
