Skip to content

Commit

Permalink
Add Janet and Rust Script, add a bar-chart, add more related lists
Browse files Browse the repository at this point in the history
  • Loading branch information
ad-si committed Aug 25, 2024
1 parent 294e581 commit ce33434
Show file tree
Hide file tree
Showing 9 changed files with 253 additions and 94 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ jobs:
- name: Set up Guile
run: brew install guile

- name: Set up Janet
run: brew install janet

- name: Set up Julia
run: brew install julia

Expand Down
33 changes: 6 additions & 27 deletions makefile
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ print-versions:
@printf '"racket": "%s",\n' "$$(racket --version)"
@printf '"roc": "%s",\n' "$$(roc --version | head -n1)"
@printf '"ruby": "%s",\n' "$$(ruby --version)"
@printf '"rust": "%s",\n' "$$(rustc --version), $$(rust-script --version)"
@printf '"scala": "%s",\n' "$$(scala-cli --version | tr '\n' ',')"
@printf '"stack": "%s",\n' "$$(stack --version)"
@printf '"swift": "%s",\n' \
Expand All @@ -77,31 +78,9 @@ run-shebangs:
--shell none \
--warmup 10 \
--export-json result.json \
'./bash' \
'./bun' \
'./dart' \
'./dash' \
'./elixir' \
'./elvish' \
'./fish' \
'./fsharp.fsx' \
'./guile' \
'./haskell' \
'./julia' \
'./ksh' \
'./lua' \
'./luajit' \
'./nushell' \
'./ocaml' \
'./osh' \
'./perl' \
'./php' \
'./python' \
'./racket' \
'./roc.roc' \
'./ruby' \
'./scala' \
'./swift' \
'./v.vsh'
$$(cat _all_.txt)


shebang-scripts/today/chart.svg:
bun run ./shebang-scripts/generate-chart.ts

# TODO: Save result.json
168 changes: 101 additions & 67 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,83 @@

Benchmark for interpreted languages.

![Barchart for relative execution speed](shebang-scripts/today/chart.svg)

The values for Rust, V, and Haskell are not really representative
as they are compiled on the first run
and Haskell unfortunately does not even cache the compiled binary.


## Languages

- [Bash]
- [Bun]
- [Dart]
- [Dash]
- [Elixir]
- [Elvish]
- [F#]
- [Fish]
- [Guile]
- [Haskell]
- JavaScript via
- [Node.js]
- [Deno]
- [Bun]
- [Julia]
- [Ksh]
- [Lua]
- [Lua]
- [Nickel]
- [Nix Language]
- [Nushell]
- [OCaml]
- [Osh]
- [Perl]
- [PHP]
- TypeScript via
- [Deno]
- [Bun]
- [Python]
- [Lua]
- [Nickel]
- [Nix Language]
- [Python]
- [Racket]
- [Roc.roc]
- [Ruby]
- [Scala]
- [Swift]
- [Typst]
- [V]

[Bash]: https://www.gnu.org/software/bash/
[Bun]: https://bun.sh/
[Dart]: https://dart.dev/
[Dash]: https://wiki.archlinux.org/title/Dash
[Deno]: https://deno.com/
[Elixir]: https://elixir-lang.org/
[Elvish]: https://elv.sh/
[F#]: https://fsharp.org/
[Fish]: https://fishshell.com/
[Guile]: https://www.gnu.org/software/guile/
[Haskell]: https://www.haskell.org/
[Julia]: https://julialang.org/
[Ksh]: https://www.kornshell.com/
[Lua]: https://www.lua.org/
[Nickel]: https://nickel-lang.org/
[Nix Language]: https://nixos.org/manual/nix/stable/language/
[Node.js]: https://nodejs.org/
[Nushell]: https://www.nushell.sh/
[OCaml]: https://ocaml.org/
[Osh]: https://www.oilshell.org/
[Perl]: https://www.perl.org/
[PHP]: https://www.php.net/
[Python]: https://www.python.org/
[Racket]: https://racket-lang.org/
[Roc.roc]: https://roc-lang.org/
[Ruby]: https://www.ruby-lang.org/
[Scala]: https://www.scala-lang.org/
[Swift]: https://swift.org/
[Typst]: https://typst.app/docs/
[V]: https://vlang.io/


### Workarounds
Expand All @@ -37,86 +90,67 @@ Benchmark for interpreted languages.

## Result

### MacBook Pro 14"
Check out the workflow runs for the latest benchmark results.

```yaml
Year: 2021
Chip: Apple M1 Pro
Memory: 16 GB
macOS: 13.6.2
```

On 2024-01-29:
### GitHub's MacOS 14 Runner

#### Bucket Calc

```txt
nickel-lang-cli nickel 1.3.0 (rev Homebre)
Python 3.11.6
Lua 5.4.6 Copyright (C) 1994-2023 Lua.org, PUC-Rio
deno 1.39.1 (release, aarch64-apple-darwin)
v8 12.0.267.8
typescript 5.3.3
bun --version
1.0.20
node --version
v20.7.0
hyperfine 1.18.0

===== Running Benchmark =====

Summary
lua bin-calculation.lua ran
4.71 ± 0.39 times faster than nickel export bin-calculation.ncl
5.89 ± 0.50 times faster than bun run bin-calculation.js
6.00 ± 0.51 times faster than bun run bin-calculation.ts
11.09 ± 1.11 times faster than python3 bin-calculation.py
11.82 ± 0.92 times faster than deno run bin-calculation.ts
11.86 ± 1.07 times faster than deno run bin-calculation.js
17.90 ± 1.33 times faster than node bin-calculation.js
lua bucket-calc/main.lua ran
2.96 ± 3.97 times faster than nickel export bucket-calc/main.ncl
3.05 ± 3.98 times faster than bun run bucket-calc/main.js
3.09 ± 4.07 times faster than bun run bucket-calc/main.ts
4.71 ± 5.77 times faster than python3 bucket-calc/main.py
4.76 ± 5.83 times faster than deno run bucket-calc/main.ts
4.80 ± 5.91 times faster than deno run bucket-calc/main.js
8.56 ± 10.56 times faster than node bucket-calc/main.js
24.21 ± 29.52 times faster than
typst query --field=text --one bucket-calc/main.typ
```


### Mac mini

```yaml
Year: 2018
Processor: 3.2 GHz 6-Core Intel Core i7
Memory: 32 GB 2667 MHz DDR4
macOS: 13.6.3
```
On 2024-01-29:
#### Shebang Scripts

```txt
nickel-lang-cli nickel 1.4.0 (rev Homebre)
Python 3.11.7
Lua 5.4.6 Copyright (C) 1994-2023 Lua.org, PUC-Rio
deno 1.39.4 (release, x86_64-apple-darwin)
v8 12.0.267.8
typescript 5.3.3
bun --version
1.0.23
node --version
v20.10.0
hyperfine 1.18.0

===== Running Benchmark =====

Summary
lua bin-calculation.lua ran
4.09 ± 0.52 times faster than nickel export bin-calculation.ncl
4.70 ± 0.54 times faster than bun run bin-calculation.ts
4.93 ± 0.75 times faster than bun run bin-calculation.js
8.36 ± 1.00 times faster than python3 bin-calculation.py
8.83 ± 1.21 times faster than deno run bin-calculation.js
8.85 ± 1.05 times faster than deno run bin-calculation.ts
11.39 ± 1.33 times faster than node bin-calculation.js
./luajit ran
1.06 ± 0.22 times faster than ./lua
1.61 ± 0.27 times faster than ./bash
1.62 ± 0.29 times faster than ./dash
2.12 ± 0.49 times faster than ./ksh
2.66 ± 0.55 times faster than ./osh
3.77 ± 0.63 times faster than ./elvish
3.91 ± 0.67 times faster than ./v.vsh
4.92 ± 0.82 times faster than ./fish
6.16 ± 1.02 times faster than ./bun
9.65 ± 1.72 times faster than ./guile
13.94 ± 2.29 times faster than ./python
14.20 ± 2.39 times faster than ./nushell
17.43 ± 2.86 times faster than ./php
23.01 ± 3.94 times faster than ./ocaml
35.43 ± 5.99 times faster than ./ruby
52.94 ± 9.08 times faster than ./racket
56.77 ± 9.59 times faster than ./perl
89.13 ± 14.62 times faster than ./dart
106.23 ± 18.77 times faster than ./swift
143.96 ± 24.03 times faster than ./julia
199.84 ± 33.40 times faster than ./roc.roc
224.83 ± 47.56 times faster than ./elixir
483.43 ± 87.95 times faster than ./haskell
507.16 ± 114.14 times faster than ./scala
528.19 ± 94.47 times faster than ./fsharp.fsx
```


## Related

- [jinyus/related_post_gen] - Data Processing benchmark
- [plb2] - A programming language benchmark
- [Programming-Language-Benchmarks][PLB]
- [script-bench-rs] - Rust embedded scripting languages benchmark

[jinyus/related_post_gen]: https://github.com/jinyus/related_post_gen
[PLB]: https://github.com/hanabi1224/Programming-Language-Benchmarks
[plb2]: https://github.com/attractivechaos/plb2
[script-bench-rs]: https://github.com/khvzak/script-bench-rs
86 changes: 86 additions & 0 deletions shebang-scripts/generate-chart.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
#! /usr/bin/env bun

import fs from "fs"
import path from "path"
import vega from "vega"
import vl from "vega-lite"

// Load data from the JSON file
const data = JSON.parse(
fs.readFileSync(path.join(__dirname, "/today/result.json"), "utf8"),
)

// Extract data from the JSON
const results: { command: string; mean_time: number }[] = data.results.map(
(result) => {
return {
command: result.command.replace("./", ""),
mean_time: result.mean,
}
},
)

console.log({ results })

// Convert results to a DataFrame-like structure
const df = results.map((result, index) => ({
...result,
relative_speed: 0, // Placeholder for now
}))

// Find the fastest entry (smallest mean_time)
const fastestEntry = df.reduce((min, entry) => {
return entry.mean_time < min.mean_time ? entry : min
}, df[0])

const fastestCommand = fastestEntry.command
const fastestTime: number = fastestEntry.mean_time

// Calculate times faster relative to the fastest entry
df.forEach((entry) => {
entry.relative_speed = fastestTime / entry.mean_time
})

// Sort the DataFrame by relative_speed
df.sort((a, b) => b.relative_speed - a.relative_speed)

console.log(df)

// Define the Vega-Lite chart specification
const spec = {
$schema: "https://vega.github.io/schema/vega-lite/v5.json",
title: `Relative Execution Speed (Higher is Better)`,
mark: "bar",
width: 800,
height: 600,
encoding: {
x: {
field: "command",
type: "nominal",
sort: "-y",
title: "Command",
axis: {
labelAngle: -45,
}
},
y: {
field: "relative_speed",
type: "quantitative",
title: `Relative Speed in Comparison to ${fastestCommand}`,
},
tooltip: [{ field: "command" }, { field: "relative_speed" }],
},
data: {
values: df,
},
}

// Create a Vega view for rendering
const runtime = vl.compile(spec).spec
const view = new vega.View(vega.parse(runtime), { renderer: "none" })

// Export to SVG
view.toSVG().then((svg) => {
fs.writeFileSync(path.join(__dirname, "today/chart.svg"), svg)
console.log("Chart generated successfully!")
})
1 change: 1 addition & 0 deletions shebang-scripts/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{ "dependencies": { "vega": "^5.30.0", "vega-canvas": "^1.2.7", "vega-lite": "^5.20.1" } }
28 changes: 28 additions & 0 deletions shebang-scripts/today/_all_.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
./bash
./bun
./dart
./dash
./elixir
./elvish
./fish
./fsharp.fsx
./guile
./haskell
./janet
./julia
./ksh
./lua
./luajit
./nushell
./ocaml
./osh
./perl
./php
./python
./racket
./rust-script
./roc.roc
./ruby
./scala
./swift
./v.vsh
1 change: 1 addition & 0 deletions shebang-scripts/today/chart.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit ce33434

Please sign in to comment.