Compare commits

..
Author SHA1 Message Date
David Peter fa89267547 Describe the default benchmark duration as an estimate 2026-10-02 11:29:15 +02:00
2 changed files with 7 additions and 9 deletions

No files matched your search

+3 -2
View File
@@ -33,8 +33,9 @@ hyperfine 'sleep 0.3'
```
Hyperfine will automatically determine the number of runs to perform for each command. By default,
it will perform *at least* 10 benchmarking runs and measure for at least 3 seconds. To change this,
you can use the `-r`/`--runs` option:
it will perform *at least* 10 benchmarking runs and estimate a run count targeting roughly
3 seconds, including shell overhead and preparation and conclusion commands. To set an exact
number of runs, you can use the `-r`/`--runs` option:
```sh
hyperfine --runs 5 'sleep 0.3'
```
+4 -7
View File
@@ -7,7 +7,8 @@
# ///
"""This script performs Welch's t-test on a JSON export file with two
benchmark results to test for a difference in their population mean runtimes."""
benchmark results to test whether or not the two distributions are
the same."""
import argparse
import json
@@ -39,10 +40,6 @@ print(f"t = {t:.3}, p = {p:.3}")
print()
if dispose:
print(
f"A statistically significant difference in mean runtime was detected (p < {th})."
)
print(f"There is a difference between the two benchmarks (p < {th}).")
else:
print(
f"No statistically significant difference in mean runtime was detected (p >= {th})."
)
print(f"The two benchmarks are almost the same (p >= {th}).")