Difference between revisions of "Basic Benchmark"

From Lingoport Wiki
Jump to: navigation, search
(Benchmark Results)
(Benchmark Results)
Line 30: Line 30:
 
== Benchmark Results ==
 
== Benchmark Results ==
   
  +
<pre>
<code> Execute Java Scan Benchmark (10K lines of code, 100 files):
 
  +
Execute Java Scan Benchmark (10K lines of code, 100 files):
Reloaded rule sets
 
  +
Reloaded rule sets
Executing java scan(1): 11231 milliseconds
 
  +
Executing java scan(1): 11231 milliseconds
 
Executing java scan(2): 1140 milliseconds
 
Executing java scan(2): 1140 milliseconds
 
Executing java scan(3): 800 milliseconds
 
Executing java scan(3): 800 milliseconds
Line 42: Line 43:
 
Executing java scan(9): 576 milliseconds
 
Executing java scan(9): 576 milliseconds
 
Executing java scan(10): 580 milliseconds
 
Executing java scan(10): 580 milliseconds
  +
 
Reloaded rule sets
 
Reloaded rule sets
 
Executing java scan(1): 1377 milliseconds
 
Executing java scan(1): 1377 milliseconds
Line 53: Line 55:
 
Executing java scan(9): 592 milliseconds
 
Executing java scan(9): 592 milliseconds
 
Executing java scan(10): 576 milliseconds
 
Executing java scan(10): 576 milliseconds
  +
 
Reloaded rule sets
 
Reloaded rule sets
 
Executing java scan(1): 1361 milliseconds
 
Executing java scan(1): 1361 milliseconds
Line 64: Line 67:
 
Executing java scan(9): 592 milliseconds
 
Executing java scan(9): 592 milliseconds
 
Executing java scan(10): 576 milliseconds
 
Executing java scan(10): 576 milliseconds
<code>
+
</pre>

Revision as of 18:42, 9 December 2016

Rationale

Logic of the Benchmark

The benchmark is to identify a unit of scanning for a project. That unit is based on the following specifications and API is used to execute the scan to measure the operations precisely:

  • 100 files
  • 10,000 lines of code
  • The default Java rule set
  • 10 scans are run in one batch.
    • The first time the scan is executed, the Java rule set is loaded
    • the next 9 runs do not need to load the Java rule set and hence perform better.
  • 3 batches of 10 scans each are run
    • On the first batch, the system must connect to Globalyzer, get the meta-data, etc., which lowers the performance
    • On the subsequent batches, the connection is established, and the first scan only reloads the rule sets as mentioned above.

If you have, say, 100,000 lines of code, on the system specified, it would about take 10 times the duration of this test.

If you have, say, two rule sets, it would take about twice the duration of this test.

Benchmark System Characteristics

   Operating System Type: amd64
   Processor Identifier: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
   Number of Processors: 4
   Available processors (cores): 4
   Free memory (bytes): 94958872
   Maximum memory (bytes): 1407713280
   Total memory (bytes): 96468992

Benchmark Results

Execute Java Scan Benchmark (10K lines of code, 100 files):
Reloaded rule sets
Executing java scan(1): 11231 milliseconds
Executing java scan(2): 1140 milliseconds
Executing java scan(3): 800 milliseconds
Executing java scan(4): 736 milliseconds
Executing java scan(5): 768 milliseconds
Executing java scan(6): 656 milliseconds
Executing java scan(7): 624 milliseconds
Executing java scan(8): 587 milliseconds
Executing java scan(9): 576 milliseconds
Executing java scan(10): 580 milliseconds

Reloaded rule sets
Executing java scan(1): 1377 milliseconds
Executing java scan(2): 589 milliseconds
Executing java scan(3): 595 milliseconds
Executing java scan(4): 640 milliseconds
Executing java scan(5): 592 milliseconds
Executing java scan(6): 560 milliseconds
Executing java scan(7): 576 milliseconds
Executing java scan(8): 576 milliseconds
Executing java scan(9): 592 milliseconds
Executing java scan(10): 576 milliseconds

Reloaded rule sets
Executing java scan(1): 1361 milliseconds
Executing java scan(2): 576 milliseconds
Executing java scan(3): 592 milliseconds
Executing java scan(4): 544 milliseconds
Executing java scan(5): 576 milliseconds
Executing java scan(6): 574 milliseconds
Executing java scan(7): 577 milliseconds
Executing java scan(8): 577 milliseconds
Executing java scan(9): 592 milliseconds
Executing java scan(10): 576 milliseconds