Tracking the frame reference

framecpp is a reimplementation of FRaME, so its worth rests entirely on reproducing the original model. To keep that honest, the package ships a corpus of golden references — real frame model runs — and checks itself against them. This vignette documents that corpus and re-runs the check live.

library(framecpp)
library(dplyr)
library(tidyr)

Provenance: everything traces to original FRaME

The references live in inst/extdata/ref_*.rds. Each was produced by frame v0.5.3 itself — the parameter table was fed to frame’s JVM runner (ffm.runner.CSVRunner) and the resulting model output saved. The capture scripts are in data-raw/. So every golden output is genuine FRaME; framecpp must match it bit-for-bit.

corpus <- tribble(
  ~tag,      ~community,
  "mild",    "FRaME's shipped example community (4 strata, flat leaves) — control weather",
  "extreme", "FRaME's shipped example community — severe weather",
  "alt",     "Hand-authored round/flat-leaf community (tuart_traits species)",
  "tuart",   "Survey-derived, non-layered community (frame's k-means strata from tuart_survey)"
)
corpus
#> # A tibble: 4 × 2
#>   tag     community                                                             
#>   <chr>   <chr>                                                                 
#> 1 mild    FRaME's shipped example community (4 strata, flat leaves) — control w…
#> 2 extreme FRaME's shipped example community — severe weather                    
#> 3 alt     Hand-authored round/flat-leaf community (tuart_traits species)        
#> 4 tuart   Survey-derived, non-layered community (frame's k-means strata from tu…

How each maps to FRaME:

mild / extreme — the community FRaME distributes as its own worked example, run under two weather regimes. The most direct “does framecpp equal frame?” test.
alt — a structurally different community (round-leaf species, non-degenerate crowns) authored in FRaME’s paramBuilder input format; its golden output is a real frame run, so it exercises the round/flat-leaf paths against FRaME.
tuart — strata derived by frame’s own k-means stratifier from the tuart_survey transect data (data-raw/tuartref_reference.R), then run through frame. A genuinely non-layered community, again validated against FRaME.

Each ref_*.rds holds the frame parameter table (params) and the full model output tables (db), plus the captured run metadata.

ref <- readRDS(system.file("extdata", "ref_extreme.rds", package = "framecpp"))
names(ref)
#> [1] "params" "db"     "run"
head(ref$db$ROS, 4)          # frame's own rate-of-spread output
#>   repId runIndex       level      ros
#> 1     1        1     Surface 0.097200
#> 2     1        1 NearSurface 0.197637
#> 3     1        1    Elevated 0.133488
#> 4     1        1   MidStorey 0.000000

The check

For each reference we run framecpp on the same parameter table frame consumed (run_fire_model() chains scenario_from_params() → the compiled model → parse_fire_result()), then compare every per-stratum rate of spread and flame summary against frame’s recorded output (runIndex == 1).

relerr <- function(a, b) abs(a - b) / pmax(abs(b), 1e-9)

check_tag <- function(tag) {
  ref <- readRDS(system.file("extdata", paste0("ref_", tag, ".rds"), package = "framecpp"))
  res <- run_fire_model(ref$params)

  # rate of spread (frame reports levels that spread; framecpp keyed by name)
  ros_ref <- ref$db$ROS |> filter(runIndex == 1)
  ros_err <- relerr(res$run1$ros[as.character(ros_ref$level)], ros_ref$ros)

  # flame summaries: length / angle / height per stratum
  fl_ref <- ref$db$FlameSummaries |> filter(runIndex == 1)
  fs <- res$run1$flameSummaries
  fl_err <- unlist(lapply(seq_len(nrow(fl_ref)), function(i) {
    s <- fs[[as.character(fl_ref$level[i])]]
    relerr(c(s$flameLength, s$flameAngle, s$flameHeight),
           c(fl_ref$flameLength[i], fl_ref$flameAngle[i], fl_ref$flameHeight[i]))
  }))

  tibble(tag = tag,
         comparisons = length(ros_err) + length(fl_err),
         worst_rel_err = max(ros_err, fl_err, na.rm = TRUE))
}

results <- bind_rows(lapply(corpus$tag, check_tag))
results
#> # A tibble: 4 × 3
#>   tag     comparisons worst_rel_err
#>   <chr>         <int>         <dbl>
#> 1 mild             16      3.81e-16
#> 2 extreme          16      2.00e-16
#> 3 alt              16      2.73e-15
#> 4 tuart            16      2.69e-16

stopifnot(all(results$worst_rel_err < 1e-9))
sprintf("framecpp reproduces frame across the corpus — worst relative error %.1e",
        max(results$worst_rel_err))
#> [1] "framecpp reproduces frame across the corpus — worst relative error 2.7e-15"

The residuals are at the level of floating-point rounding (~1e-15), not model disagreement: framecpp is classic FRaME, recomputed in C++.

A concrete comparison

The tuart community crowns hard. Side by side, framecpp against frame’s own flame-height output:

ref <- readRDS(system.file("extdata", "ref_tuart.rds", package = "framecpp"))
res <- run_fire_model(ref$params)

fl_ref <- ref$db$FlameSummaries |> filter(runIndex == 1) |> select(level, frame = flameHeight)
fs <- res$run1$flameSummaries
fl_am <- tibble(level = names(fs),
                framecpp = vapply(fs, `[[`, numeric(1), "flameHeight"))

inner_join(fl_ref, fl_am, by = "level") |>
  mutate(rel_err = relerr(framecpp, frame))
#>         level     frame  framecpp     rel_err
#> 1 NearSurface  0.365881  0.365881 1.51719e-16
#> 2    Elevated  2.036800  2.036800 0.00000e+00
#> 3   MidStorey  2.163215  2.163215 0.00000e+00
#> 4      Canopy 19.769518 19.769518 0.00000e+00

Beyond the golden references

The golden corpus checks framecpp against complete frame model runs. A separate differential harness (tools/parity/, needs a JDK + frame) drives the real frame_scala classes stage by stage over thousands of random inputs, and the testthat suite mirrors FRaME’s own ScalaTest specs. Run the golden check any time with make parity.