Compare commits

..
Author SHA1 Message Date
David Peter d3d9be3f9f Fix documentation typos and man-page formatting 2026-10-02 11:29:13 +02:00
Marshall Porter 1f5a2ff70c ci: add Linux ARM64 musl release target (#924) 2026-10-02 11:26:01 +02:00
c0b3834277 Handle broken pipe errors on stdout gracefully (#932)
Replace `println!` calls on the results paths with `writeln!` on
`io::stdout()` and propagate the `io::Error` to `main`, where
`ErrorKind::BrokenPipe` is turned into a quiet `exit(0)`. This matches
the conventional behavior of command-line tools when their output pipe
is closed early (e.g. `hyperfine ... | head -n 0`), and fixes panics in
all run-output paths: the per-benchmark header and results, the speed
comparison summary, and `-`-style exports to stdout. The warnings and
"Note" outputs to stderr are converted the same way.


---------

Co-authored-by: Mathjk <Mathjk@users.noreply.github.com>
Co-authored-by: David Peter <mail@david-peter.de>
2026-10-02 11:18:56 +02:00
David Peter 4d073e08ae Dependency and MSRV update (#934) 2026-10-02 11:05:02 +02:00
12 changed files with 132 additions and 44 deletions

No files matched your search

+1
View File
@@ -79,6 +79,7 @@ jobs:
matrix:
job:
- { target: aarch64-unknown-linux-gnu , os: ubuntu-24.04, use-cross: true }
- { target: aarch64-unknown-linux-musl , os: ubuntu-24.04, use-cross: true }
- { target: arm-unknown-linux-gnueabihf , os: ubuntu-24.04, use-cross: true }
- { target: arm-unknown-linux-musleabihf, os: ubuntu-24.04, use-cross: true }
- { target: i686-pc-windows-msvc , os: windows-2022 }
+4 -4
View File
@@ -5,7 +5,7 @@
- Add `--reference-name` option to give a meaningful name to the reference command, see #808 (@niklasdewally)
- The `--ignore-failure` option now supports a comma-separated list of exit codes to ignore (e.g., `--ignore-failure=1,2`), see #836 (@sharkdp)
- Python scripts: Add `--time-unit` option to `advanced_statistics.py` (@sharkdp)
- Python scripts: Add new `plot_benchmarks.py` script for plotting collections of benchmarks, see #806 (@marxin)
- Python scripts: Add new `plot_benchmark_comparison.py` script for plotting collections of benchmarks, see #806 (@marxin)
## Bugfixes
@@ -99,7 +99,7 @@
## Other
- Thanks to @berombau for working on dependency upgrades, see #584
- Fixed installationm on Windows, see #595 and #596 (@AntoniosBarotsis)
- Fixed installation on Windows, see #595 and #596 (@AntoniosBarotsis)
# v1.15.0
@@ -244,7 +244,7 @@ Package authors: note that Hyperfine now comes with a set of shell completion fi
- The new `--parameter-list <VAR> <VALUES>` option can be used to run
a parametrized benchmark on a user-specified list of values.
This is similar to `--parameter-scan <VAR> <MIN> <MAX>`, but doesn't
necessarily required numeric arguments.
necessarily require numeric arguments.
``` bash
hyperfine --parameter-list compiler "gcc,clang" \
@@ -415,7 +415,7 @@ for reviewing pull requests and for giving very valuable feedback.
## Other
- [Arch Linux package](https://aur.archlinux.org/packages/hyperfine) for Hyperfine (@jD91mZM2).
- Ubuntu/Debian packages are now are available.
- Ubuntu/Debian packages are now available.
# v0.2.0
+1 -1
View File
@@ -334,7 +334,7 @@ The name *hyperfine* was chosen in reference to the hyperfine levels of caesium
## Citing hyperfine
Thank you for considering to cite hyperfine in your research work. Please see the information
Thank you for considering citing hyperfine in your research work. Please see the information
in the sidebar on how to properly cite hyperfine.
## License
+6 -7
View File
@@ -105,7 +105,7 @@ Perform exactly \fINUM\fP runs for each command. If this option is not specified
\fB\-s\fR, \fB\-\-setup\fR \fICMD...\fP
.IP
Execute \fICMD\fP once before each set of timing runs. This is useful
for compiling your software or with the provided parameters, or to do any
for compiling your software with the provided parameters, or to do any
other work that should happen once before a series of benchmark runs,
not every time as would happen with the \fB\-\-prepare\fR option.
.HP
@@ -129,8 +129,8 @@ run after the corresponding benchmark command.
.IP
Execute \fICMD\fP after the completion of all benchmarking runs for each individual
command to be benchmarked. This is useful if the commands to be benchmarked
produce artifacts that need to be cleaned up. It only runs once a series of
benchmark runs, as opposed to \fB\-\-conclude\fR option which runs after
produce artifacts that need to be cleaned up. It only runs once per series of
benchmark runs, as opposed to the \fB\-\-conclude\fR option which runs after
every run.
.HP
\fB\-P\fR, \fB\-\-parameter\-scan\fR \fIVAR\fP \fIMIN\fP \fIMAX\fP
@@ -148,7 +148,7 @@ Example:
This performs benchmarks for 'make \-j 1', 'make \-j 2', ..., 'make \-j 8'.
.IP
To have the value increase following different patterns, use shell
arithmetics.
arithmetic.
.IP
.RS
Example:
@@ -309,9 +309,9 @@ Read from \fI\,/dev/null\/\fP (the default).
Read the input from the given file.
.RE
.HP
\fB\-n\fR, \fB\-\-command\-name\fR \fiNAME\fP
\fB\-n\fR, \fB\-\-command\-name\fR \fINAME\fP
.IP
Give a meaningful \fiNAME\fP to a command. This can be specified multiple times
Give a meaningful \fINAME\fP to a command. This can be specified multiple times
if several commands are benchmarked.
.HP
\fB\-h\fR, \fB\-\-help\fR
@@ -363,7 +363,6 @@ Demonstrate when each of \fB\-\-setup\fR, \fB\-\-prepare\fR, \fB\-\-conclude\fR,
'echo command n={n}'
.fi
.RE
.RE
.SH AUTHOR
.LP
David Peter <mail@david-peter.de>
+2 -2
View File
@@ -1,4 +1,4 @@
This folder contains scripts that can be used in combination with hyperfines `--export-json` option.
This folder contains scripts that can be used in combination with hyperfine's `--export-json` option.
### Example:
@@ -7,7 +7,7 @@ hyperfine 'sleep 0.020' 'sleep 0.021' 'sleep 0.022' --export-json sleep.json
./plot_whisker.py sleep.json
```
### Pre-requisites
### Prerequisites
To make these scripts work, you will need `numpy`, `matplotlib` and `scipy`.
+19 -11
View File
@@ -5,6 +5,7 @@ pub mod scheduler;
pub mod timing_result;
use std::cmp;
use std::io::{self, Write};
use crate::benchmark::executor::BenchmarkIteration;
use crate::command::Command;
@@ -12,6 +13,7 @@ use crate::options::{
CmdFailureAction, CommandOutputPolicy, ExecutorKind, Options, OutputStyleOption,
};
use crate::outlier_detection::{modified_zscores, OUTLIER_THRESHOLD};
use crate::output::console_writeln;
use crate::output::format::{format_duration, format_duration_unit};
use crate::output::progress_bar::get_progress_bar;
use crate::output::warnings::{OutlierWarningOptions, Warnings};
@@ -140,12 +142,13 @@ impl<'a> Benchmark<'a> {
/// Run the benchmark for a single command
pub fn run(&self) -> Result<BenchmarkResult> {
if self.options.output_style != OutputStyleOption::Disabled {
println!(
console_writeln!(
io::stdout(),
"{}{}: {}",
"Benchmark ".bold(),
(self.number + 1).to_string().bold(),
self.command.get_name_with_unused_parameters(),
);
)?;
}
let mut times_real: Vec<Second> = vec![];
@@ -358,19 +361,22 @@ impl<'a> Benchmark<'a> {
let system_str = format_duration(system_mean, Some(time_unit));
if self.options.output_style != OutputStyleOption::Disabled {
let mut stdout = io::stdout().lock();
if times_real.len() == 1 {
println!(
console_writeln!(
stdout,
" Time ({} ≡): {:>8} {:>8} [User: {}, System: {}]",
"abs".green().bold(),
mean_str.green().bold(),
" ", // alignment
user_str.blue(),
system_str.blue()
);
)?;
} else {
let stddev_str = format_duration(t_stddev.unwrap(), Some(time_unit));
println!(
console_writeln!(
stdout,
" Time ({} ± {}): {:>8} ± {:>8} [User: {}, System: {}]",
"mean".green().bold(),
"σ".green(),
@@ -378,16 +384,17 @@ impl<'a> Benchmark<'a> {
stddev_str.green(),
user_str.blue(),
system_str.blue()
);
)?;
println!(
console_writeln!(
stdout,
" Range ({} … {}): {:>8} … {:>8} {}",
"min".cyan(),
"max".purple(),
min_str.cyan(),
max_str.purple(),
num_str.dimmed()
);
)?;
}
}
@@ -430,15 +437,16 @@ impl<'a> Benchmark<'a> {
}
if !warnings.is_empty() {
eprintln!(" ");
let mut stderr = io::stderr().lock();
console_writeln!(stderr, " ")?;
for warning in &warnings {
eprintln!(" {}: {}", "Warning".yellow(), warning);
console_writeln!(stderr, " {}: {}", "Warning".yellow(), warning)?;
}
}
if self.options.output_style != OutputStyleOption::Disabled {
println!(" ");
console_writeln!(io::stdout(), " ")?;
}
self.run_cleanup_command(self.command.get_parameters().iter().cloned(), output_policy)?;
+22 -13
View File
@@ -3,10 +3,12 @@ use super::executor::{Executor, MockExecutor, RawExecutor, ShellExecutor};
use super::{relative_speed, Benchmark};
use colored::*;
use std::cmp::Ordering;
use std::io::{self, Write};
use crate::command::{Command, Commands};
use crate::export::ExportManager;
use crate::options::{ExecutorKind, Options, OutputStyleOption, SortOrder};
use crate::output::console_writeln;
use anyhow::Result;
@@ -58,13 +60,13 @@ impl<'a> Scheduler<'a> {
Ok(())
}
pub fn print_relative_speed_comparison(&self) {
pub fn print_relative_speed_comparison(&self) -> Result<()> {
if self.options.output_style == OutputStyleOption::Disabled {
return;
return Ok(());
}
if self.results.len() < 2 {
return;
return Ok(());
}
let reference = self
@@ -79,17 +81,19 @@ impl<'a> Scheduler<'a> {
reference,
self.options.sort_order_speed_comparison,
) {
let mut stdout = io::stdout().lock();
match self.options.sort_order_speed_comparison {
SortOrder::MeanTime => {
println!("{}", "Summary".bold());
console_writeln!(stdout, "{}", "Summary".bold())?;
let reference = annotated_results.iter().find(|r| r.is_reference).unwrap();
let others = annotated_results.iter().filter(|r| !r.is_reference);
println!(
console_writeln!(
stdout,
" {} ran",
reference.result.command_with_unused_parameters.cyan()
);
)?;
for item in others {
let stddev = if let Some(stddev) = item.relative_speed_stddev {
@@ -114,18 +118,20 @@ impl<'a> Scheduler<'a> {
stddev
),
};
println!(
console_writeln!(
stdout,
"{} {}",
comparator,
item.result.command_with_unused_parameters.magenta()
);
)?;
}
}
SortOrder::Command => {
println!("{}", "Relative speed comparison".bold());
console_writeln!(stdout, "{}", "Relative speed comparison".bold())?;
for item in annotated_results {
println!(
console_writeln!(
stdout,
" {}{} {}",
format!("{:10.2}", item.relative_speed).bold().green(),
if item.is_reference {
@@ -136,12 +142,13 @@ impl<'a> Scheduler<'a> {
" ".into()
},
item.result.command_with_unused_parameters,
);
)?;
}
}
}
} else {
eprintln!(
console_writeln!(
io::stderr(),
"{}: The benchmark comparison could not be computed as some benchmark times are zero. \
This could be caused by background interference during the initial calibration phase \
of hyperfine, in combination with very fast commands (faster than a few milliseconds). \
@@ -149,8 +156,10 @@ impl<'a> Scheduler<'a> {
--shell=none/-N option. If it does not help either, you command is most likely too fast \
to be accurately benchmarked by hyperfine.",
"Note".bold().red()
);
)?;
}
Ok(())
}
pub fn final_export(&self) -> Result<()> {
+4
View File
@@ -3,6 +3,10 @@ use std::num::{self, ParseFloatError, ParseIntError};
use rust_decimal::Error as DecimalError;
use thiserror::Error;
#[derive(Debug, Error)]
#[error(transparent)]
pub struct ConsoleOutputError(pub std::io::Error);
#[derive(Debug, Error)]
pub enum ParameterScanError {
#[error("Error while parsing parameter scan arguments ({0})")]
+5 -3
View File
@@ -1,5 +1,5 @@
use std::fs::{File, OpenOptions};
use std::io::Write;
use std::io::{self, Write};
mod asciidoc;
mod csv;
@@ -18,6 +18,7 @@ use self::orgmode::OrgmodeExporter;
use crate::benchmark::benchmark_result::BenchmarkResult;
use crate::options::SortOrder;
use crate::output::console_writeln;
use crate::util::units::Unit;
use anyhow::{Context, Result};
@@ -144,8 +145,9 @@ impl ExportManager {
}
ExportTarget::Stdout => {
if !intermediate {
println!();
println!("{}", String::from_utf8(content()?).unwrap());
let mut stdout = io::stdout().lock();
console_writeln!(stdout)?;
console_writeln!(stdout, "{}", String::from_utf8(content()?).unwrap())?;
}
}
}
+16 -2
View File
@@ -4,10 +4,12 @@
)]
use std::env;
use std::io::{self, Write};
use benchmark::scheduler::Scheduler;
use cli::get_cli_arguments;
use command::Commands;
use error::ConsoleOutputError;
use export::ExportManager;
use options::Options;
@@ -44,17 +46,29 @@ fn run() -> Result<()> {
let mut scheduler = Scheduler::new(&commands, &options, &export_manager);
scheduler.run_benchmarks()?;
scheduler.print_relative_speed_comparison();
scheduler.print_relative_speed_comparison()?;
scheduler.final_export()?;
Ok(())
}
fn caused_by_broken_console_pipe(error: &anyhow::Error) -> bool {
error
.downcast_ref::<ConsoleOutputError>()
.is_some_and(|e| e.0.kind() == io::ErrorKind::BrokenPipe)
}
fn main() {
match run() {
Ok(_) => {}
Err(e) => {
eprintln!("{} {:#}", "Error:".red(), e);
// Exit quietly when a reader stops consuming console output early.
if caused_by_broken_console_pipe(&e) {
std::process::exit(0);
}
// The write to stderr can itself fail if stderr is closed; the
// error message cannot be shown in that case anyway.
let _ = writeln!(io::stderr(), "{} {:#}", "Error:".red(), e);
std::process::exit(1);
}
}
+10
View File
@@ -1,3 +1,13 @@
pub mod format;
pub mod progress_bar;
pub mod warnings;
/// Mark console write errors so broken pipes can be handled separately from file errors.
macro_rules! console_writeln {
($writer:expr $(, $($args:tt)*)?) => {
writeln!($writer $(, $($args)*)?)
.map_err($crate::error::ConsoleOutputError)
};
}
pub(crate) use console_writeln;
+42 -1
View File
@@ -1,5 +1,5 @@
mod common;
use common::hyperfine;
use common::{hyperfine, hyperfine_raw_command};
use predicates::prelude::*;
@@ -33,6 +33,47 @@ fn one_run_is_supported() {
.success();
}
/// Regression test: hyperfine must not panic when writing to a closed
/// stdout pipe (e.g. `hyperfine ... | head -n 1` or a downstream process
/// that terminates early). A `BrokenPipe` error should result in a quiet
/// exit with code 0 instead of a `println!` panic.
#[test]
fn exits_quietly_when_stdout_is_closed() {
use std::process::Stdio;
for args in [
vec!["--runs=1", "echo dummy benchmark"],
vec!["--runs=1", "--export-json=-", "echo dummy benchmark"],
vec!["--runs=1", "echo dummy benchmark", "echo second benchmark"],
] {
// A pipe whose read end has been closed before hyperfine is even
// started: every write to it will fail with `BrokenPipe`.
let (reader, writer) = std::io::pipe().expect("failed to create pipe");
drop(reader);
let output = hyperfine_raw_command()
.args(&args)
.stdout(writer)
.stderr(Stdio::piped())
.spawn()
.expect("failed to spawn hyperfine")
.wait_with_output()
.expect("failed to wait for hyperfine");
let stderr = String::from_utf8_lossy(&output.stderr);
assert!(
!stderr.contains("panicked"),
"hyperfine panicked while writing to a closed pipe:\n{}",
stderr
);
assert!(
output.status.success(),
"hyperfine did not exit cleanly; stderr:\n{}",
stderr
);
}
}
#[test]
fn can_run_commands_without_a_shell() {
hyperfine()