Compare commits

..
Author SHA1 Message Date
David Peter 7cf08c30dc Export selected metrics side by side in CSV 2026-10-07 21:17:25 +02:00
18 changed files with 44 additions and 356 deletions

No files matched your search

+22 -23
View File
@@ -1,49 +1,48 @@
# v2.0.0
# v2.0.0-alpha.2
See the breaking changes below when migrating from hyperfine 1.x.
## Changes
- Always run `--setup`, `--prepare`, `--conclude`, and `--cleanup` commands in a shell, even when benchmarked commands run directly. Improve error messages for these commands, see #990 (@sharkdp).
- Require an explicit shell choice for commands containing unquoted, standalone shell operators such as `&&` or `>`: use `-S` to interpret shell syntax or `-N` to pass operators as literal arguments (@sharkdp).
- Use a compact output layout on narrow terminals (@sharkdp).
- Make plotting and analysis scripts runnable directly from GitHub with `uvx --from 'git+https://github.com/sharkdp/hyperfine' <script>` (@sharkdp).
# v2.0.0-alpha.1
This is the first alpha release of hyperfine 2.0. Command-line options and export formats may change before the stable release. See the breaking changes below when migrating from hyperfine 1.x.
## Features
- Add support for alternative performance metrics (peak memory usage, instructions, CPU cycles, cache misses, branch misses, ...). By default, hyperfine will now display wall-clock time and peak memory usage, but users can use the [new `--metrics` option](README.md#choosing-metrics-and-units) to select different performance metrics. The terminal output now shows an overview of all selected performance metrics, including relative changes.
- Add `--env` to set environment variables even without an intermediate shell (which is the new default behavior, see below). For example:
```sh
# Parameterized benchmark, comparing different thread counts:
hyperfine -P threads 1 8 --env 'OMP_NUM_THREADS={threads}' 'my_command'
```
- The plotting and analysis scripts now support different metrics as well, see #981 and #984 (@sharkdp).
- Add support for alternative performance metrics (peak memory usage, CPU cycles, instructions, cache misses, branch misses, ...). By default, hyperfine will now display wall-clock time and peak memory usage (wall-clock time only on Windows),
but users can use the new `--metrics` option to select different performance metrics.
- The terminal output now shows an overview of all selected performance metrics, including relative changes.
- Python plotting and analysis scripts support the new JSON format and can select performance metrics with `--metric`, see #981 and #984 (@sharkdp).
## Breaking changes
- Benchmarked commands are now executed directly by default, without an intermediate shell (`--shell=none`). Use `-S` (an alias for `--shell=default`) to restore the previous behavior (`sh` on Unix, `cmd.exe` on Windows), or select a shell with `--shell <SHELL>`. See [the rationale for this change](https://github.com/sharkdp/hyperfine/issues/787#issuecomment-6038898252).
- Always run `--setup`, `--prepare`, `--conclude`, and `--cleanup` commands in a shell, even when using `--shell=none` (the new default behavior), see #990 (@sharkdp).
- The JSON format for `--export-json` has changed (schema version 2). The [new format](README.md#json) now includes metadata, per-run measurements, and statistical summaries for all measured quantities, see #790 (@sharkdp). Code that reads those exported JSON files must be updated. For example, `results[i].mean` is now `results[i].summary.time_wall_clock.mean`, and `results[i].times` is replaced by `results[i].measurements[j].time_wall_clock.value`. The new structure looks like this:
- Benchmarked commands are now executed directly by default, without an intermediate shell (`--shell=none`). Use `-S` (an alias for `--shell=default`) to restore the previous behavior (`sh` on Unix, `cmd.exe` on Windows), or select a shell with `--shell <SHELL>`.
- The JSON format for `--export-json` has changed (schema version 2). The new format now includes metadata, per-run measurements, and statistical summaries for all measured quantities, see #790 (@sharkdp). Code that reads those exported JSON files must be updated. For example, `results[i].mean` is now `results[i].summary.time_wall_clock.mean`, and `results[i].times` is replaced by `results[i].measurements[j].time_wall_clock.value`. The new structure looks like this:
```python
{
"schema_version": 2,
"primary_metric": "time_wall_clock",
"metadata": {
"hyperfine_version": "2.0.0",
"hyperfine_version": "2.0.0-alpha.1",
"start_time": "2026-10-06T12:00:00Z",
"platform": {"os": "Linux", "architecture": "x86_64"}
},
"results": [
{
"command": "sleep 1", # Actual command after parameter substitution
"name": "wait 1s", # Optional custom or automatically generated name
"name": "wait 1s", # Optional custom name, separate from the command
"parameters": {"duration": {"value": "1"}}, # Values are now objects
"environment": {"OMP_NUM_THREADS": "8"}, # Only present with --env overrides
"measurements": [ # One entry per run, excluding warmups
{
"time_wall_clock": {"value": 1.0, "unit": "second"},
"time_cpu": {"value": 0.0, "unit": "second"},
"time_user": {"value": 0.0, "unit": "second"},
"time_system": {"value": 0.0, "unit": "second"},
"memory_peak_resident": {"value": 1048576.0, "unit": "byte"},
"cpu_cycles": {"value": 5000000}, # Hardware counters, when available
"instructions": {"value": 10000000},
"exit_code": 0
},
... # Further runs omitted
@@ -65,11 +64,11 @@ See the breaking changes below when migrating from hyperfine 1.x.
}
```
Times are always exported in seconds and memory in bytes. Hardware counters and their summaries have no `unit` field. Unavailable memory measurements and hardware counters are omitted; other available counters may also be included. `stddev` is `null` for a single run.
Times are always exported in seconds and memory in bytes.
- `--time-unit`/`-u` has been removed. Units can now be selected using the `--metrics METRIC[:UNIT],…` option, e.g. `--metrics time_wall_clock:ms,memory_peak_resident:MiB`.
- `--sort` has been removed. It complicated hyperfine significantly and doesn't make too much sense with the new output format.
- `--reference` and `--reference-name` have been removed (for now). The first command is always considered as the reference command. Invocations using `--reference` now show migration guidance to put the reference command first.
- The format of [Markdown](README.md#markdown), AsciiDoc and org-mode, and [CSV](README.md#csv) exports has also been changed. Markdown, AsciiDoc, and org-mode contain only the primary metric (the first metric selected by `--metrics`). CSV exports all selected metrics side by side.
- `--reference` and `--reference-name` have been removed (for now). The first command is always considered as the reference command.
- The format of Markdown, AsciiDoc, org-mode and CSV exports has also been changed. These formats contain only the primary metric (the first metric selected by `--metrics`). CSV now includes columns with the name and the unit of the primary metric.
# v1.21.0
+1 -1
View File
@@ -13,5 +13,5 @@ authors:
repository-code: 'https://github.com/sharkdp/hyperfine'
abstract: A command-line benchmarking tool.
license: MIT
version: 2.0.0
version: 2.0.0-alpha.2
date-released: '2026-10-07'
Generated
+1 -1
View File
@@ -377,7 +377,7 @@ checksum = "15cdd26707701c53297e2fa6afb323d55fbc1d0810c3aec078ae3ef0424c3c15"
[[package]]
name = "hyperfine"
version = "2.0.0"
version = "2.0.0-alpha.2"
dependencies = [
"anyhow",
"approx",
+1 -1
View File
@@ -7,7 +7,7 @@ license = "MIT OR Apache-2.0"
name = "hyperfine"
readme = "README.md"
repository = "https://github.com/sharkdp/hyperfine"
version = "2.0.0"
version = "2.0.0-alpha.2"
edition = "2018"
build = "build.rs"
rust-version = "1.97.0"
+3 -29
View File
@@ -2,8 +2,7 @@
A command-line benchmarking tool.
<img width="800" height="195" alt="hyperfine-demo-800" src="https://github.com/user-attachments/assets/9a5a45d3-df3f-4e0e-8cc9-5d8153f39e48" />
<img width="839" height="205" alt="hyperfine demo" src="https://github.com/user-attachments/assets/13c20b1d-39c1-4da1-8fcb-0c775dd032f5" />
*(hyperfine in action, benchmarking [`mypy`](https://mypy-lang.org/) and [`ty`](https://ty.dev/))*
@@ -155,31 +154,6 @@ Otherwise, inline the function into the benchmarked command:
hyperfine -S 'my_function() { sleep 1; }; my_function'
```
### Environment variables
Use `--env` to set environment variables for benchmark commands and their respective setup, prepare, conclude,
and cleanup commands. Assignments are space-separated and support shell-style quoting,
but no shell expansion:
```sh
hyperfine --env 'OMP_NUM_THREADS=8 CFLAGS="-O3 -march=native"' 'my_command'
```
Specify once for all commands or once per benchmark command:
```sh
hyperfine \
--env 'OMP_NUM_THREADS=8' './benchmark-cpp' \
--env 'RAYON_NUM_THREADS=8' './benchmark-rust'
```
Use an empty `--env=''` for no overrides. Values can contain parameter placeholders.
For example, the following benchmarks `my_command` with `OMP_NUM_THREADS` ranging from 1 to 8:
```sh
hyperfine -P threads 1 8 --env 'OMP_NUM_THREADS={threads}' 'my_command'
```
### Choosing metrics and units
By default, hyperfine displays wall-clock time and memory usage
@@ -331,8 +305,8 @@ apt install hyperfine
Alternatively, for the latest version, you can download the appropriate `.deb` package from the [Release page](https://github.com/sharkdp/hyperfine/releases) and install it via `dpkg`:
```
wget https://github.com/sharkdp/hyperfine/releases/download/v2.0.0/hyperfine_2.0.0_amd64.deb
sudo dpkg -i hyperfine_2.0.0_amd64.deb
wget https://github.com/sharkdp/hyperfine/releases/download/v1.21.0/hyperfine_1.21.0_amd64.deb
sudo dpkg -i hyperfine_1.21.0_amd64.deb
```
### On Fedora
-11
View File
@@ -192,17 +192,6 @@ This performs benchmarks for 'g++ \-O2 main.cpp' and 'clang++ \-O2 main.cpp'.
The option can be specified multiple times to run benchmarks for all
possible parameter combinations.
.HP
\fB\-\-env\fR \fIASSIGNMENTS\fP
.IP
Set environment variables for benchmark commands and their respective setup, prepare, conclude,
and cleanup commands. Assignments are space-separated and support shell-style quoting,
but no shell expansion, e.g. \fB\-\-env\fR 'OMP_NUM_THREADS=8 CFLAGS="\-O3 \-march=native"'.
Specify once for all commands or once per benchmark command, like \fB\-\-prepare\fR.
Use an empty string for no overrides. Values can contain parameter placeholders.
.IP
For example, \fBhyperfine \-P threads 1 8 \-\-env\fR 'OMP_NUM_THREADS={threads}' 'my_command'
benchmarks 'my_command' with OMP_NUM_THREADS ranging from 1 to 8.
.HP
\fB\-\-shell\fR \fISHELL\fP
.IP
Set the shell to use for executing benchmarked commands, including warmup runs
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "hyperfine-scripts"
version = "2.0.0"
version = "0.1.0"
description = "Plot and analyze hyperfine JSON benchmark results"
requires-python = ">=3.10"
license-files = ["LICENSE-APACHE", "LICENSE-MIT"]
+6 -8
View File
@@ -18,13 +18,15 @@ pub struct BenchmarkResult {
/// regardless of any custom name.
pub command: String,
/// The custom or automatically generated name after parameter substitution,
/// if it differs from `command`. Automatic names include environment overrides.
/// The custom name after parameter substitution, if it differs from `command`.
/// For example, `--command-name="wait {duration}s"` with `duration=1`
/// produces `Some("wait 1s")`. Without a custom name, or when the expanded
/// name equals `command`, this is `None`.
#[serde(skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
/// The name, followed by parameters not used in the command or environment
/// templates, in command-line order. For example, `sleep {duration}`
/// The custom name (or `command`), followed by parameters not used in the
/// command template, in command-line order. For example, `sleep {duration}`
/// with `duration=1`, `branch=main`, and the custom name `wait {duration}s`
/// produces `wait 1s (branch = main)`. Without the custom name, this is
/// `sleep 1 (branch = main)`. Used in summaries and table exports.
@@ -38,10 +40,6 @@ pub struct BenchmarkResult {
/// Parameter values for this benchmark
#[serde(skip_serializing_if = "BTreeMap::is_empty")]
pub parameters: BTreeMap<String, Parameter>,
/// Explicit environment overrides after parameter substitution.
#[serde(skip_serializing_if = "BTreeMap::is_empty")]
pub environment: BTreeMap<String, String>,
}
impl BenchmarkResult {
-1
View File
@@ -215,7 +215,6 @@ impl Executor for ShellExecutor<'_> {
) -> Result<Measurement> {
let on_windows_cmd = cfg!(windows) && *self.shell == Shell::Default("cmd.exe");
let mut command_builder = self.shell.command();
command_builder.envs(command.get_environment());
command_builder.arg(if on_windows_cmd { "/C" } else { "-c" });
// Windows needs special treatment for its behavior on parsing cmd arguments
+1 -7
View File
@@ -61,9 +61,8 @@ impl<'a> Benchmark<'a> {
output_policy: &CommandOutputPolicy,
iteration: executor::BenchmarkIteration,
) -> Result<Measurement> {
let command = command.clone().with_environment_from(self.command);
self.executor
.run_intermediate_command_and_measure(&command, iteration, output_policy)
.run_intermediate_command_and_measure(command, iteration, output_policy)
.map_err(|error| {
let hint = if cfg!(windows) {
"Append ' || exit /b 0' to the command if this failure can be ignored."
@@ -444,11 +443,6 @@ impl<'a> Benchmark<'a> {
Ok(BenchmarkResult {
command,
name,
environment: self
.command
.get_environment()
.map(|(name, value)| (name.to_owned(), value))
.collect(),
display_name: self.command.get_name_with_unused_parameters(),
measurements,
parameters: self
-35
View File
@@ -50,19 +50,6 @@ fn build_command() -> Command {
.value_hint(ValueHint::CommandString)
.value_parser(NonEmptyStringValueParser::new()),
)
// Retain the removed option to provide migration guidance for old invocations.
.arg(
Arg::new("reference")
.long("reference")
.hide(true)
.value_name("CMD")
.num_args(0..=1)
.default_missing_value("")
.value_parser(|_: &str| -> Result<String, String> {
Err("The --reference option has been removed. Put your reference command first instead.".into())
})
.help("Removed: put your reference command first instead."),
)
.arg(
Arg::new("metrics")
.long("metrics")
@@ -132,28 +119,6 @@ fn build_command() -> Command {
.help("Perform exactly NUM runs for each command. If this option is not specified, \
hyperfine automatically determines the number of runs."),
)
.arg(
Arg::new("env")
.long("env")
.action(ArgAction::Append)
.num_args(1)
.value_name("ASSIGNMENTS")
.help("Set environment variables for benchmark commands and their respective setup, \
prepare, conclude, and cleanup commands. Assignments are space-separated and \
support shell-style quoting, but no shell expansion:\n\n \
hyperfine \\\n \
--env 'OMP_NUM_THREADS=8 CFLAGS=\"-O3 -march=native\"' \\\n \
'my_command'\n\n\
Specify once for all commands or once per benchmark command:\n\n \
hyperfine \\\n \
--env 'OMP_NUM_THREADS=8' './benchmark-cpp' \\\n \
--env 'RAYON_NUM_THREADS=8' './benchmark-rust'\n\n\
Use an empty --env='' for no overrides. Values can contain parameter placeholders. \
For example, the following benchmarks 'my_command' with OMP_NUM_THREADS ranging \
from 1 to 8:\n\n \
hyperfine -P threads 1 8 \\\n \
--env 'OMP_NUM_THREADS={threads}' 'my_command'"),
)
.arg(
Arg::new("setup")
.long("setup")
+6 -91
View File
@@ -14,11 +14,9 @@ use crate::{
use clap::{parser::ValuesRef, ArgMatches};
use anyhow::{bail, ensure, Context, Result};
use anyhow::{bail, Context, Result};
use rust_decimal::Decimal;
type Environment = Vec<(String, String)>;
/// A command that should be benchmarked.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Command<'a> {
@@ -30,9 +28,6 @@ pub struct Command<'a> {
/// Zero or more parameter values.
parameters: Vec<ParameterNameAndValue<'a>>,
/// Environment overrides, before parameter substitution.
environment: Environment,
}
impl<'a> Command<'a> {
@@ -41,7 +36,6 @@ impl<'a> Command<'a> {
name,
expression,
parameters: Vec::new(),
environment: Vec::new(),
}
}
@@ -54,26 +48,14 @@ impl<'a> Command<'a> {
name,
expression,
parameters: parameters.into_iter().collect(),
environment: Vec::new(),
}
}
pub fn get_name(&self) -> String {
match self.name {
Some(name) => self.replace_parameters_in(name),
None => {
let environment = self
.get_environment()
.map(|(name, value)| format!("{name}={}", shell_words::quote(&value)))
.collect::<Vec<_>>()
.join(" ");
if environment.is_empty() {
self.get_command_line()
} else {
format!("{environment} {}", self.get_command_line())
}
}
}
self.name.map_or_else(
|| self.get_command_line(),
|name| self.replace_parameters_in(name),
)
}
pub fn get_name_with_unused_parameters(&self) -> String {
@@ -105,7 +87,6 @@ impl<'a> Command<'a> {
if let Some(program_name) = tokens.next() {
let mut command_builder = std::process::Command::new(program_name);
command_builder.args(tokens);
command_builder.envs(self.get_environment());
Ok(command_builder)
} else {
bail!("Can not execute empty command")
@@ -116,24 +97,8 @@ impl<'a> Command<'a> {
&self.parameters
}
pub fn get_environment(&self) -> impl Iterator<Item = (&str, String)> {
self.environment
.iter()
.map(move |(name, value)| (name.as_str(), self.replace_parameters_in(value)))
}
pub fn with_environment_from(mut self, command: &Command<'_>) -> Self {
self.environment = command.environment.clone();
self
}
pub fn is_parameter_unused(&self, parameter: &str) -> bool {
let placeholder = format!("{{{parameter}}}");
!self.expression.contains(&placeholder)
&& !self
.environment
.iter()
.any(|(_, value)| value.contains(&placeholder))
!self.expression.contains(&format!("{{{parameter}}}"))
}
pub fn get_unused_parameters(&self) -> impl Iterator<Item = &(&'a str, ParameterValue)> {
@@ -172,58 +137,8 @@ impl<'a> Command<'a> {
/// A collection of commands that should be benchmarked
pub struct Commands<'a>(Vec<Command<'a>>);
fn parse_environment(value: &str) -> Result<Environment> {
let mut environment = Environment::new();
for assignment in shell_words::split(value).context("Failed to parse '--env' assignments")? {
let (name, value) = assignment.split_once('=').with_context(|| {
format!(
"Invalid value '{assignment}' for '--env': expected an assignment like NAME=value or NAME=\"quoted value\"."
)
})?;
ensure!(
!name.is_empty()
&& name.bytes().enumerate().all(|(index, byte)| {
byte == b'_'
|| byte.is_ascii_alphabetic()
|| (index > 0 && byte.is_ascii_digit())
}),
"Invalid environment variable name '{name}' in '--env'"
);
// Keep only the last assignment, including case variants on Windows.
environment.retain(|(key, _)| {
if cfg!(windows) {
!key.eq_ignore_ascii_case(name)
} else {
key != name
}
});
environment.push((name.to_owned(), value.to_owned()));
}
Ok(environment)
}
impl<'a> Commands<'a> {
pub fn from_cli_arguments(matches: &'a ArgMatches) -> Result<Commands<'a>> {
let mut commands = Self::build_from_cli_arguments(matches)?;
if let Some(values) = matches.get_many::<String>("env") {
let environments = values
.map(|value| parse_environment(value))
.collect::<Result<Vec<_>>>()?;
let num_commands = commands.num_commands();
ensure!(
environments.len() == 1 || environments.len() == num_commands,
"The '--env' option has to be provided just once or N times, where N={num_commands} is the \
number of benchmark commands."
);
for (index, command) in commands.0.iter_mut().enumerate() {
command.environment =
environments[if environments.len() == 1 { 0 } else { index }].clone();
}
}
Ok(commands)
}
fn build_from_cli_arguments(matches: &'a ArgMatches) -> Result<Commands<'a>> {
let command_names = matches.get_many::<String>("command-name");
let command_strings = matches
.get_many::<String>("command")
-2
View File
@@ -91,7 +91,6 @@ fn test_csv() {
let results = vec![
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("echo command_a"),
name: Some(String::from("command_a")),
display_name: String::from("command_a"),
@@ -142,7 +141,6 @@ fn test_csv() {
},
},
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("command_b"),
name: None,
display_name: String::from("command_b"),
-8
View File
@@ -35,7 +35,6 @@ fn get_output<E: Exporter + Default>(results: &[BenchmarkResult], unit: Option<U
fn test_markup_export_auto_ms() {
let results = [
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 0.1"),
name: None,
display_name: String::from("sleep 0.1"),
@@ -71,7 +70,6 @@ fn test_markup_export_auto_ms() {
parameters: BTreeMap::new(),
},
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 2"),
name: None,
display_name: String::from("sleep 2"),
@@ -149,7 +147,6 @@ fn test_markup_export_auto_ms() {
fn test_markup_export_auto_s() {
let results = [
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 2"),
name: None,
display_name: String::from("sleep 2"),
@@ -185,7 +182,6 @@ fn test_markup_export_auto_s() {
parameters: BTreeMap::new(),
},
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 0.1"),
name: None,
display_name: String::from("sleep 0.1"),
@@ -263,7 +259,6 @@ fn test_markup_export_auto_s() {
fn test_markup_export_manual_ms() {
let timing_results = [
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 2"),
name: None,
display_name: String::from("sleep 2"),
@@ -299,7 +294,6 @@ fn test_markup_export_manual_ms() {
parameters: BTreeMap::new(),
},
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 0.1"),
name: None,
display_name: String::from("sleep 0.1"),
@@ -376,7 +370,6 @@ fn test_markup_export_manual_ms() {
fn test_markup_export_manual_s() {
let results = [
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 2"),
name: None,
display_name: String::from("sleep 2"),
@@ -412,7 +405,6 @@ fn test_markup_export_manual_s() {
parameters: BTreeMap::new(),
},
BenchmarkResult {
environment: BTreeMap::new(),
command: String::from("sleep 0.1"),
name: None,
display_name: String::from("sleep 0.1"),
-8
View File
@@ -47,14 +47,6 @@ fn run() -> Result<()> {
{
for command in commands.iter() {
let command_line = command.get_command_line();
if shell_syntax::starts_with_assignment(&command_line) {
bail!(
"Cannot run '{command_line}' directly because it starts with a variable assignment.\n\
Use the --env option to set environment variables, for example:\n\n \
hyperfine --env 'OMP_NUM_THREADS=8 CFLAGS=\"-O3 -march=native\"' 'my_command'\n\n\
Alternatively, use -S / --shell=default to run the command in a shell."
);
}
if let Some(operator) = shell_syntax::first_unquoted_operator(&command_line) {
bail!(
concat!(
+2 -49
View File
@@ -1,32 +1,8 @@
//! Conservative detection of shell syntax in commands run without a shell.
//! Conservative detection of shell operators in commands run without a shell.
use std::iter::Peekable;
use std::str::CharIndices;
/// Detects the variable assignment at the start of a command like
/// `OMP_NUM_THREADS=8 my_command`.
pub fn starts_with_assignment(command: &str) -> bool {
let Some(token) = tokenize(command).next() else {
return false;
};
let mut bytes = token.bytes();
if !bytes
.next()
.is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
{
return false;
}
for byte in bytes {
if byte == b'=' {
return true;
}
if !byte.is_ascii_alphanumeric() && byte != b'_' {
return false;
}
}
false
}
/// Find the first standalone operator that is neither quoted nor escaped.
///
/// Operators attached to another word (such as `hello>out`), expansions, and
@@ -114,30 +90,7 @@ impl<'a> Iterator for Tokens<'a> {
#[cfg(test)]
mod tests {
use super::{first_unquoted_operator, starts_with_assignment, tokenize};
#[test]
fn detects_leading_assignments() {
assert!(starts_with_assignment("A=1 command"));
assert!(starts_with_assignment("A=$HOME command"));
assert!(starts_with_assignment("A= command"));
assert!(starts_with_assignment("_A2='hello world' B=2 command"));
assert!(starts_with_assignment("A=1"));
assert!(starts_with_assignment(" # comment\n A=1 command"));
assert!(!starts_with_assignment(""));
assert!(!starts_with_assignment("# A=1 command"));
assert!(!starts_with_assignment("command A=1"));
assert!(!starts_with_assignment("env A=1 command"));
assert!(!starts_with_assignment("'A=1' command"));
assert!(!starts_with_assignment("\"A\"=1 command"));
assert!(!starts_with_assignment(r"A\=1 command"));
assert!(!starts_with_assignment("./A=1 command"));
assert!(!starts_with_assignment("1A=1 command"));
assert!(!starts_with_assignment("A-B=1 command"));
assert!(!starts_with_assignment("=1 command"));
assert!(!starts_with_assignment("A"));
}
use super::{first_unquoted_operator, tokenize};
#[test]
fn tokenizer_preserves_words_quotes_and_escapes() {
-1
View File
@@ -3,7 +3,6 @@ use std::process::Command;
pub fn hyperfine_raw_command() -> Command {
let mut cmd = Command::new(assert_cmd::cargo::cargo_bin!("hyperfine"));
cmd.current_dir("tests/");
cmd.env_remove("CLICOLOR_FORCE");
cmd
}
-79
View File
@@ -44,25 +44,6 @@ fn runs_successfully() {
.success();
}
#[test]
fn rejects_removed_reference_option_with_migration_guidance() {
for args in [
vec!["--reference", "echo reference", "echo comparison"],
vec!["echo comparison", "--reference=echo reference"],
vec!["--reference", "echo reference"],
vec!["--reference"],
] {
hyperfine()
.args(args)
.assert()
.code(2)
.stdout("")
.stderr(predicate::str::contains(
"The --reference option has been removed. Put your reference command first instead.",
));
}
}
#[test]
fn one_run_is_supported() {
hyperfine()
@@ -72,66 +53,6 @@ fn one_run_is_supported() {
.success();
}
#[test]
fn environment_basic_usage() {
let command = if cfg!(windows) {
"echo %HYPERFINE_TEST%"
} else {
"echo \"$HYPERFINE_TEST\""
};
hyperfine()
.args(["--shell=default", "--runs=1", "--show-output"])
.arg("--command-name=env-test")
.args(["--env", r#"HYPERFINE_TEST="hello, world""#])
.arg(command)
.assert()
.success()
.stdout(predicate::str::contains("hello, world"));
}
#[test]
fn environment_parameter_expansion() {
let command = if cfg!(windows) {
"echo %HYPERFINE_TEST%"
} else {
"echo \"$HYPERFINE_TEST\""
};
hyperfine()
.args(["--shell=default", "--runs=1", "--show-output"])
.arg("--command-name=env-test")
.args(["-P", "value", "1", "2"])
.args(["--env", "HYPERFINE_TEST=hello-{value}"])
.arg(command)
.assert()
.success()
.stdout(predicate::str::contains("hello-1"))
.stdout(predicate::str::contains("hello-2"));
}
#[test]
fn rejects_invalid_environment() {
hyperfine()
.args(["--env", "OMP_NUM_THREADS"])
.arg("my_command")
.assert()
.failure()
.stderr(predicate::str::contains(
"Invalid value 'OMP_NUM_THREADS' for '--env': expected an assignment like NAME=value or NAME=\"quoted value\".",
));
}
#[test]
fn requires_explicit_shell_for_environment_assignments() {
// Without an explicit shell choice, report the leading variable assignment.
hyperfine()
.args(["--runs=1", "SOME_ENV_VAR=$HOME my_command"])
.assert()
.failure()
.stderr(predicate::str::contains(
"starts with a variable assignment",
));
}
/// Regression test: hyperfine must not panic when writing to a closed
/// stdout pipe (e.g. `hyperfine ... | head -n 1` or a downstream process
/// that terminates early). A `BrokenPipe` error should result in a quiet