Compare commits

..
2 changed files with 26 additions and 10 deletions

No files matched your search

+22 -3
View File
@@ -3,7 +3,8 @@
hyperfine \- command\-line benchmarking tool
.SH SYNOPSIS
.B hyperfine
.RB [ \-ihVN ]
.RB [ \-hVN ]
.RB [ \-\-ignore\-failure[=MODE] ]
.RB [ \-\-warmup
.IR NUM ]
.RB [ \-\-min\-runs
@@ -14,6 +15,10 @@ hyperfine \- command\-line benchmarking tool
.IR NUM ]
.RB [ \-\-setup
.IR CMD ]
.RB [ \-\-reference
.IR CMD ]
.RB [ \-\-reference\-name
.IR NAME ]
.RB [ \-\-prepare
.IR CMD ]
.RB [ \-\-conclude
@@ -109,6 +114,16 @@ for compiling your software or with the provided parameters, or to do any
other work that should happen once before a series of benchmark runs,
not every time as would happen with the \fB\-\-prepare\fR option.
.HP
\fB\-\-reference\fR \fICMD\fP
.IP
Benchmark \fICMD\fP as the reference for the relative comparison of results.
If this option is not specified, the fastest command is used as the reference.
.HP
\fB\-\-reference\-name\fR \fINAME\fP
.IP
Give a meaningful \fINAME\fP to the reference command.
Requires \fB\-\-reference\fR.
.HP
\fB\-p\fR, \fB\-\-prepare\fR \fICMD...\fP
.IP
Execute \fICMD\fP before each timing run. This is useful for clearing disk caches,
@@ -207,9 +222,13 @@ things like "sleep 0.1; sleep 0.2" are not possible without a shell.
.IP
An alias for '\-\-shell=none'.
.HP
\fB\-i\fR, \fB\-\-ignore\-failure\fR
\fB\-i\fR, \fB\-\-ignore\-failure\fR[=\fIMODE\fP]
.IP
Ignore non\-zero exit codes of the benchmarked programs.
Ignore failures of the benchmarked programs. Without a value or with
\fBall\-non\-zero\fR, all non\-zero exit codes are ignored.
Alternatively, provide a comma\-separated list of exit codes to ignore,
for example \fB\-\-ignore\-failure=1,2\fR.
An explicit value must be attached to the option with an equals sign.
.HP
\fB\-\-style\fR \fITYPE\fP
.IP
+4 -7
View File
@@ -7,7 +7,8 @@
# ///
"""This script performs Welch's t-test on a JSON export file with two
benchmark results to test for a difference in their population mean runtimes."""
benchmark results to test whether or not the two distributions are
the same."""
import argparse
import json
@@ -39,10 +40,6 @@ print(f"t = {t:.3}, p = {p:.3}")
print()
if dispose:
print(
f"A statistically significant difference in mean runtime was detected (p < {th})."
)
print(f"There is a difference between the two benchmarks (p < {th}).")
else:
print(
f"No statistically significant difference in mean runtime was detected (p >= {th})."
)
print(f"The two benchmarks are almost the same (p >= {th}).")