Basic Benchmark

From Lingoport Wiki
Jump to: navigation, search

Benchmark Setup

The benchmark is to identify a unit of scanning for a project. That unit is based on the following specifications and the benchmark program calls the Globalyzer API to execute the scan to measure the operations precisely:

  • 100 files
  • 10,000 lines of code
  • The default Java rule set
  • 10 scans are run in one batch.
    • The first time the scan is executed, the Java rule set is loaded
    • the next 9 runs do not need to load the Java rule set and hence perform better.
  • 3 batches of 10 scans each are run
    • On the first batch, the system must connect to Globalyzer, get the meta-data, etc., which lowers the performance
    • On the subsequent batches, the connection is established, and the first scan only reloads the rule sets as mentioned above.

If you have, say, 100,000 lines of code, on the system specified, it would about take 10 times the duration of this test.

If you have, say, two rule sets, it would take about twice the duration of this test.

Benchmark System Characteristics

   Operating System Type: amd64
   Processor Identifier: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
   Number of Processors: 4
   Available processors (cores): 4
   Free memory (bytes): 94958872
   Maximum memory (bytes): 1407713280
   Total memory (bytes): 96468992

Benchmark Results

Execute Java Scan Benchmark (10K lines of code, 100 files):
Reloaded rule sets
Executing java scan(1): 11231 milliseconds
Executing java scan(2): 1140 milliseconds
Executing java scan(3): 800 milliseconds
Executing java scan(4): 736 milliseconds
Executing java scan(5): 768 milliseconds
Executing java scan(6): 656 milliseconds
Executing java scan(7): 624 milliseconds
Executing java scan(8): 587 milliseconds
Executing java scan(9): 576 milliseconds
Executing java scan(10): 580 milliseconds

Reloaded rule sets
Executing java scan(1): 1377 milliseconds
Executing java scan(2): 589 milliseconds
Executing java scan(3): 595 milliseconds
Executing java scan(4): 640 milliseconds
Executing java scan(5): 592 milliseconds
Executing java scan(6): 560 milliseconds
Executing java scan(7): 576 milliseconds
Executing java scan(8): 576 milliseconds
Executing java scan(9): 592 milliseconds
Executing java scan(10): 576 milliseconds

Reloaded rule sets
Executing java scan(1): 1361 milliseconds
Executing java scan(2): 576 milliseconds
Executing java scan(3): 592 milliseconds
Executing java scan(4): 544 milliseconds
Executing java scan(5): 576 milliseconds
Executing java scan(6): 574 milliseconds
Executing java scan(7): 577 milliseconds
Executing java scan(8): 577 milliseconds
Executing java scan(9): 592 milliseconds
Executing java scan(10): 576 milliseconds