第一時間打電話與BenchmarkDotNet

Question

我已經嘗試了BenchmarkDotNet用一個簡單的測試類

public class BenchTest 
{ 
    bool First = false; 

    [Benchmark] 
    public void FirstTimeInitEffect() 
    { 
     if (First == false) 
     { 
      Console.WriteLine("First called"); 
      First = true; 
      Thread.Sleep(1000); 
     } 
     else 
     { 
      Thread.Sleep(10); 
     } 

    } 
} 

讓他跟

static void Main(string[] args) 
    { 
     var cfg = ManualConfig.Create(DefaultConfig.Instance); 
     var job = new Job(); 
     job.Accuracy.RemoveOutliers = false; 
     cfg.Add(new Job[] { 
      job 
      } 
     ); 

     BenchmarkRunner.Run<BenchTest>(cfg); 
    } 

這將打印一個很好的總結，但我所來自的結果無法運行第一次調用FirstTimeInitEffect方法需要1秒。我希望從基準測試工具中可以看到第一個通話效果。我已經嘗試過一個自定義配置，以防止刪除異常值，但這並沒有把戲。我是否使用了錯誤的工具，或者這是否超出了工具的範圍？

// * Detailed results * 
BenchTest.FirstTimeInitEffect: Job-LQPFTL(RemoveOutliers=False) 
Runtime = Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0; GC = Concurrent Workstation 
Mean = 10.8548 ms, StdErr = 0.0169 ms (0.16%); N = 15, StdDev = 0.0654 ms 
Min = 10.7158 ms, Q1 = 10.8058 ms, Median = 10.8963 ms, Q3 = 10.9011 ms, Max = 10.9029 ms 
IQR = 0.0953 ms, LowerFence = 10.6628 ms, UpperFence = 11.0440 ms 
ConfidenceInterval = [10.8217 ms; 10.8879 ms] (CI 95%) 
Skewness = -1.01049139924314, Kurtosis = 2.40561202562778 


Total time: 00:00:21 (21.92 sec) 

// * Summary * 

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.2.9200.0 
Processor=Intel(R) Core(TM) i7-4770K CPU 3.50GHz, ProcessorCount=8 
Frequency=3417979 Hz, Resolution=292.5706 ns, Timer=TSC 
    [Host]  : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 
    Job-LQPFTL : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 

RemoveOutliers=False Allocated=0 B 

       Method |  Mean | StdDev | 
-------------------- |----------- |---------- | 
FirstTimeInitEffect | 10.8548 ms | 0.0654 ms | 

Answer 1

BenchmarkDotNet做了很多熱身迭代;它可以在我們開始收集實際測量數據之前達到穩定狀態。此外，它還會執行一些其他的東西，這些東西也會調用您的方法（jitting，pilot迭代等等）。所以，第一個電話被省略了，沒有被包含在摘要中。

如果您想要根據所有測量（無需接觸，試驗和熱身迭代）獲得統計數據，則應使用RunStrategy.ColdStart（而不是RunStrategy.Throughput這是默認設置）。自BenchmarkDotNet v0.10.2以來它工作正常。

源代碼：

[SimpleJob(RunStrategy.ColdStart, targetCount: 5)] 
[MinColumn, MaxColumn, MeanColumn, MedianColumn] 
public class BenchTest 
{ 
    private bool firstCall; 

    [Benchmark] 
    public void FirstTimeInitEffect() 
    { 
     if (firstCall == false) 
     { 
      firstCall = true; 
      Console.WriteLine("// First call"); 
      Thread.Sleep(1000); 
     } 
     else 
      Thread.Sleep(10); 
    } 
} 

測量：

Result 1: 1 op, 1000582715.59 ns, 1.0006 s/op 
Result 2: 1 op, 10190609.23 ns, 10.1906 ms/op 
Result 3: 1 op, 10164930.24 ns, 10.1649 ms/op 
Result 4: 1 op, 10604238.53 ns, 10.6042 ms/op 
Result 5: 1 op, 10420930.04 ns, 10.4209 ms/op 

詳細結果：

Mean = 208.4175 ms, StdErr = 198.1331 ms (95.07%); N = 5, StdDev = 443.0390 ms 
Min = 10.1163 ms, Q1 = 10.1203 ms, Median = 10.4233 ms, Q3 = 505.7118 ms, Max = 1,000.9497 ms 
IQR = 495.5915 ms, LowerFence = -733.2670 ms, UpperFence = 1,249.0991 ms 
ConfidenceInterval = [-179.9233 ms; 596.7583 ms] (CI 95%) 
Skewness = 1.07, Kurtosis = 2.08 

總結：

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.2.9200.0 
Processor=Intel(R) Core(TM) i7-6700HQ CPU 2.60GHz, ProcessorCount=8 
Frequency=2531252 Hz, Resolution=395.0614 ns, Timer=TSC 
    [Host]  : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 
    Job-IQRRGS : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1586.0 

RunStrategy=ColdStart TargetCount=5 Allocated=1.64 kB 

       Method |  Mean |  StdErr |  StdDev |  Min |   Max |  Median | 
-------------------- |------------ |------------ |------------ |----------- |-------------- |----------- | 
FirstTimeInitEffect | 208.4175 ms | 198.1331 ms | 443.0390 ms | 10.1163 ms | 1,000.9497 ms | 10.4233 ms |