Benchmarks

Benchmarks for the AD-safe machinery, reading the analytic gamma-CDF hooks against the bare Distributions.jl functions and timing the per-backend gradients of each hook.

Quick start

Install the benchpkg CLI once:

bash

task benchmark-install
# Or: julia -e 'using Pkg; Pkg.add("AirspeedVelocity")'

then run the suite:

bash

# Benchmark current state
task benchmark

# Compare main branch vs current state
task benchmark-compare

# Filter to specific benchmarks
task benchmark -- --filter=Hooks
task benchmark-compare -- --filter="AD gradients"

Benchmark structure

Baseline/
  Gamma/               (cdf, logcdf, ccdf, logccdf, pdf)

Hooks/
  Gamma/               (cdf_ad_safe, logcdf_ad_safe, ccdf_ad_safe,
                        logccdf_ad_safe, pdf_ad_safe — analytic path)
  Normal (fall-through)/ (cdf_ad_safe, logcdf_ad_safe, pdf_ad_safe —
                        generic method, collapses to Distributions)
  _gamma_cdf/          (scalar, broadcast)

AD gradients/
  <every test/ADFixtures scenario>/
    ForwardDiff, ReverseDiff (tape), Enzyme forward, Enzyme reverse,
    Mooncake forward, Mooncake reverse

Primal cost and gradient cost

The Baseline group times the stock Distributions.jl functions each hook wraps. On a Gamma the hooks route through the analytic gamma-CDF path (_gamma_cdf) rather than SpecialFunctions.gamma_inc, so the gap between Hooks/Gamma and Baseline/Gamma is the cost of the AD-safe detour on a plain (non-AD) call. On a Normal the generic method falls straight through to Distributions, so Hooks/Normal (fall-through) should sit on top of the baseline with no measurable overhead.

The AD gradients group is the point of the package: it times DifferentiationInterface.gradient for every scenario in test/ADFixtures across the ForwardDiff, ReverseDiff, Enzyme, and Mooncake backends, which is where the analytic derivative rules earn their keep.

CI integration

Pull requests benchmark head and base in separate jobs via the benchmark workflow and post a single comparison comment. Pushes to main and tagged releases append to the performance history below.

Performance history

The summary tracks each benchmark suite's headline timing across recent revisions.

Benchmark summary (overall)

Suite	Median ratio	Trend	Status
AD gradients	0.16	↘	ok
Baseline	0.24	↘	ok
Hooks	0.28	↘	ok
time_to_load	1.99	↗	⚠ reg

Ratio: latest vs oldest shown revision (1.00 = no change, higher = slower/larger). ⚠ reg = at/above the regression threshold.

<details> <summary>Per-suite detail</summary>

Ratio summary

Most recent 3 revisions, columns labelled by commit date.

AD gradients

Time

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
_gamma_cdf direct/Enzyme forward	7.57 ± 0.078 μs	7.52 ± 0.065 μs	1.01 ± 0.014
_gamma_cdf direct/Enzyme reverse	0.308 ± 0.018 μs	0.303 ± 0.018 μs	1.01 ± 0.084
_gamma_cdf direct/ForwardDiff	0.749 ± 0.097 μs	0.744 ± 0.097 μs	1.01 ± 0.19
_gamma_cdf direct/Mooncake forward	6.12 ± 0.79 μs	6.03 ± 0.75 μs	1.01 ± 0.18
_gamma_cdf direct/Mooncake reverse	5.05 ± 0.37 μs	5.02 ± 0.38 μs	1.01 ± 0.11
_gamma_cdf direct/ReverseDiff (tape)	0.862 ± 0.19 μs	0.868 ± 0.19 μs	0.993 ± 0.31
ccdf_ad_safe Gamma/Enzyme forward	9.53 ± 0.09 μs	9.48 ± 0.089 μs	1.01 ± 0.013
ccdf_ad_safe Gamma/Enzyme reverse	4.41 ± 0.063 μs	4.4 ± 0.06 μs	1 ± 0.02
ccdf_ad_safe Gamma/ForwardDiff	1.51 ± 0.013 μs	1.52 ± 0.014 μs	0.995 ± 0.013
ccdf_ad_safe Gamma/Mooncake forward	7.67 ± 0.14 μs	7.67 ± 0.14 μs	1 ± 0.026
ccdf_ad_safe Gamma/Mooncake reverse	21.7 ± 8.6 μs	21.7 ± 9 μs	0.997 ± 0.57
ccdf_ad_safe Gamma/ReverseDiff (tape)	6.51 ± 0.19 μs	6.53 ± 0.19 μs	0.996 ± 0.042
cdf_ad_safe Gamma/Enzyme forward	9.53 ± 0.09 μs	9.52 ± 0.1 μs	1 ± 0.014
cdf_ad_safe Gamma/Enzyme reverse	4.38 ± 0.067 μs	4.4 ± 0.067 μs	0.995 ± 0.022
cdf_ad_safe Gamma/ForwardDiff	1.52 ± 0.013 μs	1.51 ± 0.012 μs	1.01 ± 0.012
cdf_ad_safe Gamma/Mooncake forward	7.77 ± 0.16 μs	7.74 ± 0.15 μs	1 ± 0.028
cdf_ad_safe Gamma/Mooncake reverse	22.1 ± 9 μs	22 ± 8.9 μs	1.01 ± 0.58
cdf_ad_safe Gamma/ReverseDiff (tape)	4.7 ± 0.089 μs	4.63 ± 0.086 μs	1.01 ± 0.027
logccdf_ad_safe Gamma/Enzyme forward	9.59 ± 0.08 μs	9.55 ± 0.09 μs	1 ± 0.013
logccdf_ad_safe Gamma/Enzyme reverse	4.55 ± 0.066 μs	4.54 ± 0.076 μs	1 ± 0.022
logccdf_ad_safe Gamma/ForwardDiff	1.53 ± 0.014 μs	1.54 ± 0.012 μs	0.994 ± 0.012
logccdf_ad_safe Gamma/Mooncake forward	8.04 ± 0.17 μs	8.06 ± 0.17 μs	0.998 ± 0.029
logccdf_ad_safe Gamma/Mooncake reverse	22.3 ± 9 μs	22.2 ± 9.1 μs	1 ± 0.58
logccdf_ad_safe Gamma/ReverseDiff (tape)	5.39 ± 0.86 μs	5.35 ± 0.89 μs	1.01 ± 0.23
logcdf_ad_safe Gamma/Enzyme forward	9.54 ± 0.09 μs	9.57 ± 0.091 μs	0.997 ± 0.013
logcdf_ad_safe Gamma/Enzyme reverse	4.52 ± 0.064 μs	4.57 ± 0.057 μs	0.989 ± 0.019
logcdf_ad_safe Gamma/ForwardDiff	1.53 ± 0.013 μs	1.56 ± 0.011 μs	0.978 ± 0.011
logcdf_ad_safe Gamma/Mooncake forward	7.94 ± 0.15 μs	7.92 ± 0.14 μs	1 ± 0.026
logcdf_ad_safe Gamma/Mooncake reverse	22 ± 9 μs	22.2 ± 9.2 μs	0.991 ± 0.58
logcdf_ad_safe Gamma/ReverseDiff (tape)	5.11 ± 0.36 μs	5.07 ± 0.82 μs	1.01 ± 0.18
pdf_ad_safe Gamma/Enzyme forward	8.37 ± 0.073 μs	8.46 ± 0.074 μs	0.989 ± 0.012
pdf_ad_safe Gamma/Enzyme reverse	3.41 ± 0.055 μs	3.42 ± 0.055 μs	0.997 ± 0.023
pdf_ad_safe Gamma/ForwardDiff	0.837 ± 0.09 μs	0.849 ± 0.09 μs	0.986 ± 0.15
pdf_ad_safe Gamma/Mooncake forward	6.19 ± 0.38 μs	6.22 ± 0.31 μs	0.994 ± 0.079
pdf_ad_safe Gamma/Mooncake reverse	27.9 ± 11 μs	27.9 ± 11 μs	1 ± 0.57
pdf_ad_safe Gamma/ReverseDiff (tape)	15.4 ± 0.42 μs	15.2 ± 0.38 μs	1.01 ± 0.038

Memory

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
_gamma_cdf direct/Enzyme forward	30 allocs: 1.06 kB	30 allocs: 1.06 kB	1
_gamma_cdf direct/Enzyme reverse	2 allocs: 0.0781 kB	2 allocs: 0.0781 kB	1
_gamma_cdf direct/ForwardDiff	7 allocs: 0.359 kB	7 allocs: 0.359 kB	1
_gamma_cdf direct/Mooncake forward	0.064 k allocs: 3.06 kB	0.064 k allocs: 3.06 kB	1
_gamma_cdf direct/Mooncake reverse	0.064 k allocs: 2.9 kB	0.064 k allocs: 2.9 kB	1
_gamma_cdf direct/ReverseDiff (tape)	16 allocs: 0.656 kB	16 allocs: 0.656 kB	1
ccdf_ad_safe Gamma/Enzyme forward	0.036 k allocs: 1.11 kB	0.036 k allocs: 1.11 kB	1
ccdf_ad_safe Gamma/Enzyme reverse	24 allocs: 1 kB	24 allocs: 1 kB	1
ccdf_ad_safe Gamma/ForwardDiff	7 allocs: 0.266 kB	7 allocs: 0.266 kB	1
ccdf_ad_safe Gamma/Mooncake forward	0.058 k allocs: 2.91 kB	0.058 k allocs: 2.91 kB	1
ccdf_ad_safe Gamma/Mooncake reverse	0.217 k allocs: 26 kB	0.217 k allocs: 26 kB	1
ccdf_ad_safe Gamma/ReverseDiff (tape)	0.082 k allocs: 3.55 kB	0.082 k allocs: 3.55 kB	1
cdf_ad_safe Gamma/Enzyme forward	0.036 k allocs: 1.11 kB	0.036 k allocs: 1.11 kB	1
cdf_ad_safe Gamma/Enzyme reverse	24 allocs: 1 kB	24 allocs: 1 kB	1
cdf_ad_safe Gamma/ForwardDiff	7 allocs: 0.266 kB	7 allocs: 0.266 kB	1
cdf_ad_safe Gamma/Mooncake forward	0.058 k allocs: 2.91 kB	0.058 k allocs: 2.91 kB	1
cdf_ad_safe Gamma/Mooncake reverse	0.217 k allocs: 26 kB	0.217 k allocs: 26 kB	1
cdf_ad_safe Gamma/ReverseDiff (tape)	0.061 k allocs: 2.47 kB	0.061 k allocs: 2.47 kB	1
logccdf_ad_safe Gamma/Enzyme forward	0.036 k allocs: 1.11 kB	0.036 k allocs: 1.11 kB	1
logccdf_ad_safe Gamma/Enzyme reverse	24 allocs: 1.03 kB	24 allocs: 1.03 kB	1
logccdf_ad_safe Gamma/ForwardDiff	7 allocs: 0.266 kB	7 allocs: 0.266 kB	1
logccdf_ad_safe Gamma/Mooncake forward	0.058 k allocs: 2.91 kB	0.058 k allocs: 2.91 kB	1
logccdf_ad_safe Gamma/Mooncake reverse	0.218 k allocs: 26.2 kB	0.218 k allocs: 26.2 kB	1
logccdf_ad_safe Gamma/ReverseDiff (tape)	0.093 k allocs: 3.75 kB	0.093 k allocs: 3.75 kB	1
logcdf_ad_safe Gamma/Enzyme forward	0.036 k allocs: 1.11 kB	0.036 k allocs: 1.11 kB	1
logcdf_ad_safe Gamma/Enzyme reverse	24 allocs: 1.03 kB	24 allocs: 1.03 kB	1
logcdf_ad_safe Gamma/ForwardDiff	7 allocs: 0.266 kB	7 allocs: 0.266 kB	1
logcdf_ad_safe Gamma/Mooncake forward	0.058 k allocs: 2.91 kB	0.058 k allocs: 2.91 kB	1
logcdf_ad_safe Gamma/Mooncake reverse	0.218 k allocs: 26.2 kB	0.218 k allocs: 26.2 kB	1
logcdf_ad_safe Gamma/ReverseDiff (tape)	0.078 k allocs: 3.28 kB	0.078 k allocs: 3.28 kB	1
pdf_ad_safe Gamma/Enzyme forward	0.036 k allocs: 1.11 kB	0.036 k allocs: 1.11 kB	1
pdf_ad_safe Gamma/Enzyme reverse	24 allocs: 1.03 kB	24 allocs: 1.03 kB	1
pdf_ad_safe Gamma/ForwardDiff	7 allocs: 0.266 kB	7 allocs: 0.266 kB	1
pdf_ad_safe Gamma/Mooncake forward	0.058 k allocs: 2.91 kB	0.058 k allocs: 2.91 kB	1
pdf_ad_safe Gamma/Mooncake reverse	0.283 k allocs: 0.0323 MB	0.283 k allocs: 0.0323 MB	1
pdf_ad_safe Gamma/ReverseDiff (tape)	0.243 k allocs: 10.4 kB	0.243 k allocs: 10.4 kB	1

Baseline

Time

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
Gamma/ccdf	3.59 ± 0.37 μs	3.57 ± 0.38 μs	1 ± 0.15
Gamma/cdf	3.6 ± 0.37 μs	3.57 ± 0.38 μs	1.01 ± 0.15
Gamma/logccdf	5.68 ± 0.065 μs	5.66 ± 0.035 μs	1 ± 0.013
Gamma/logcdf	5.52 ± 0.065 μs	5.56 ± 0.044 μs	0.993 ± 0.014
Gamma/pdf	4.15 ± 0.21 μs	4.18 ± 0.2 μs	0.991 ± 0.068

Memory

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
Gamma/ccdf	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/cdf	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/logccdf	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/logcdf	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/pdf	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1

Hooks

Time

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
Gamma/ccdf_ad_safe	3.65 ± 0.36 μs	3.63 ± 0.36 μs	1.01 ± 0.14
Gamma/cdf_ad_safe	3.6 ± 0.38 μs	3.57 ± 0.37 μs	1.01 ± 0.15
Gamma/logccdf_ad_safe	5.34 ± 0.067 μs	5.33 ± 0.067 μs	1 ± 0.018
Gamma/logcdf_ad_safe	4.95 ± 0.21 μs	4.94 ± 0.21 μs	1 ± 0.06
Gamma/pdf_ad_safe	4.15 ± 0.21 μs	4.17 ± 0.21 μs	0.993 ± 0.07
Normal (fall-through)/cdf_ad_safe	1.7 ± 0.32 μs	1.5 ± 0.31 μs	1.14 ± 0.32
Normal (fall-through)/logcdf_ad_safe	3.98 ± 0.35 μs	3.94 ± 0.35 μs	1.01 ± 0.13
Normal (fall-through)/pdf_ad_safe	1.08 ± 0.33 μs	1.09 ± 0.33 μs	0.995 ± 0.43
_gamma_cdf/broadcast	3.59 ± 0.36 μs	3.57 ± 0.36 μs	1 ± 0.14
_gamma_cdf/scalar	1.55 ± 0.01 ns	1.55 ± 0.009 ns	1 ± 0.0087

Memory

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
Gamma/ccdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/cdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/logccdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/logcdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Gamma/pdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Normal (fall-through)/cdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Normal (fall-through)/logcdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
Normal (fall-through)/pdf_ad_safe	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
_gamma_cdf/broadcast	2 allocs: 0.906 kB	2 allocs: 0.906 kB	1
_gamma_cdf/scalar	0 allocs: 0 B	0 allocs: 0 B

time_to_load

Time

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
time_to_load	0.522 ± 0.0037 s	0.501 ± 0.007 s	1.04 ± 0.016

Memory

Benchmark	v0.1.0	8400c41028f79b...	v0.1.0 / 8400c41028f79b...
time_to_load	0.15 k allocs: 11.7 kB	0.15 k allocs: 11.7 kB	1

Per-benchmark timelines

<details> <summary>Show 3 plots</summary>

</details>

Benchmarks ​

Quick start ​

Benchmark structure ​

Primal cost and gradient cost ​

CI integration ​

Performance history ​

Benchmark summary (overall) ​

Ratio summary ​

AD gradients ​

Time ​

Memory ​

Baseline ​

Time ​

Memory ​

Hooks ​

Time ​

Memory ​

time_to_load ​

Time ​

Memory ​

Per-benchmark timelines ​

Benchmarks

Quick start

Benchmark structure

Primal cost and gradient cost

CI integration

Performance history

Benchmark summary (overall)

Ratio summary

AD gradients

Time

Memory

Baseline

Time

Memory

Hooks

Time

Memory

time_to_load

Time

Memory

Per-benchmark timelines