Package 'toolero'

Description

Takes a syntactic tree and renders it using Quarto's Typst engine, exporting the result as a PNG image. Supports two rendering backends controlled by tree_notation:

Usage

arborize(
  tree,
  output = "syntactic-tree.png",
  dpi = 300,
  tree_notation = c("simple", "structured"),
  papersize = "a5",
  margin = "0.5cm",
  provenance = TRUE,
  overwrite = FALSE
)

Arguments

Details

• "simple" uses ⁠@preview/syntree⁠ and accepts a bracket notation string, e.g. "[S [NP [Det the] [N cat]] [VP [V sat]]]". This is the most compact input format and suits basic linguistic trees.

• "structured" uses ⁠@preview/lingotree⁠ and accepts a nested tree() call string. This backend supports per-node styling, movement arrows, and multi-dominant trees.

The function is useful for producing standalone tree figures that can be embedded in any document format – LaTeX, Word, HTML, or presentations – without requiring a full LaTeX installation.

arborize() performs the following steps:

1. Validates inputs and resolves the Typst package from tree_notation.

2. Builds a minimal .qmd document via .build_arborize_qmd().

3. Writes the document and renders it inside a self-cleaning temporary directory managed by withr::with_tempdir().

4. Calls quarto::quarto_render() to produce an intermediate PDF via Typst.

5. Converts the PDF to PNG using pdftools::pdf_convert().

6. Reads the PNG bytes into memory before the temporary directory is deleted, then writes them to the specified output path.

7. If provenance = TRUE, writes a companion .yml file recording the tree string and all rendering arguments.

On first use, Typst will download the required package from the Typst package registry. This requires an internet connection. Subsequent renders use the locally cached package.

Requires Quarto 1.4 or later with Typst support, and the pdftools package for PDF-to-PNG conversion. Install pdftools with install.packages("pdftools").

Value

Invisibly returns the path to the output PNG file.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

References

syntree Typst package (v0.2.1): https://typst.app/universe/package/syntree

lingotree Typst package (v1.0.0): https://typst.app/universe/package/lingotree

Examples

## Not run: 
# Simple bracket notation (default) -- also writes tree-1.yml
arborize("[NP [Det the] [N cat]]", output = "my-trees/tree-1.png")

# Suppress provenance file
arborize("[NP [Det the] [N cat]]", provenance = FALSE)

# Wider tree with print-quality output
arborize(
  paste0(
    "[Aspectual Classes ",
    "[Statives [States]] ",
    "[Dynamic ",
    "[Atelic [Activities]] ",
    "[Telic ",
    "[Instantaneous [Achievements]] ",
    "[Durative [Accomplishments]]]]]"
  ),
  output    = "aspectual-classes.png",
  dpi       = 600,
  papersize = "a4"
)

# Structured notation using lingotree
arborize(
  "tree(
    tag: [VP],
    tree(
      tag: [DP],
      [every],
      [farmer]
    ),
    [smiled]
  )",
  tree_notation = "structured",
  output        = "vp-tree.png"
)

## End(Not run)

Description

check_project() audits a project directory and reports whether it follows the structure and conventions that init_project() creates. It is useful both for projects initialized with init_project() and for existing projects that were created independently.

Usage

check_project(path = ".", config = NULL, error = TRUE)

Arguments

Details

Each check records one of four status values: "pass" (the expected artifact was found), "fail" (a required artifact is missing), "warn" (a recommended artifact is missing or a problematic file was found), or "info" (a file was found that warrants attention but is not necessarily a problem).

README detection is case-insensitive and extension-agnostic: any file whose stem matches readme (in any capitalization) counts, regardless of extension or the absence of one. init_project() uses the same detection when deciding whether it would overwrite an existing README.

Value

A tibble with columns check, status, and message, returned invisibly. Assign the result to use it programmatically.

Where the folder set comes from

Three sources, in order of precedence.

An explicit config argument wins. Folders it declares and the project lacks are reported as "fail": the caller named a file and that file states what the project should look like.

Failing that, a ⁠_toolero.yml⁠ at the project root is used. init_project() writes one recording the structure it actually created, so a folder listed there and missing from disk means something removed it. That is also a "fail".

Failing both, the built-in standard set is used – ⁠data-raw/⁠, ⁠data/⁠, ⁠R/⁠, ⁠scripts/⁠, ⁠output/figures/⁠, ⁠output/tables/⁠, and ⁠reports/⁠. Missing folders here are "warn", not "fail": nobody declared anything, so the standard set is a suggestion rather than a contract.

A ⁠_toolero.yml⁠ that exists but cannot be parsed is reported as a failing check and the audit continues against the built-in set. A config that cannot be parsed is an error, since the caller asked for that file specifically.

The renv checks

Beyond the presence of renv.lock, two checks guard the failure mode that costs the most to discover late: a lockfile that does not describe the analysis, which produces a container image that builds cleanly and then cannot run.

A .renvignore excluding .qmd files is reported, and the advice is to remove the entry. It stops renv from seeing the library() calls in a project whose Quarto document is the source of truth. That the document will eventually be purled to a .R file does not make up for it: the snapshot you containerize from may be taken before the purl, and the .qmd is the file being maintained either way. Versions of init_project() before v0.5.0 wrote one; projects created by those versions still carry it.

A renv.lock recording no packages is reported only when the project also has .R or .qmd source files. A newly scaffolded project legitimately has an empty lockfile – renv::scaffold() does no dependency discovery, because there is nothing yet to discover – so the pairing is what makes the observation worth printing.

Stale purled scripts

Every .qmd under path whose header declares purl: true (see create_qmd()'s use_purl argument) gets its own row, comparing it against the .R script R/purl.R is expected to have derived from it. In an ordinary project that script is flat, reports/analysis.qmd to R/analysis.R. In a Quarto website or book (⁠project: type:⁠ in ⁠_quarto.yml⁠) it mirrors the document's path, posts/one/index.qmd to R/posts/one/index.R. A script found at the other layout's path, which happens when R/purl.R is an older copy, is reported with the path it was found at and how to refresh the hook. Missing entirely, or older than the .qmd it was purled from, is reported as "warn": the .qmd is the source of truth, so an ⁠R/⁠ script older than the document it came from means an edit was made and not yet re-rendered, and a container or cluster job that bakes in the .R file would run the old analysis without any error to say so. Documents never opted into purl produce no row – nothing to check.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

init_project(), generate_project_config()

Examples

# Audit the current working directory

check_project()


# Audit a specific project directory

project_dir <- withr::local_tempdir()
check_project(path = project_dir)


# Audit against a custom folder structure

project_dir <- withr::local_tempdir()
config_path <- file.path(tempdir(), "my-config.yml")
generate_project_config("my-config.yml", path = tempdir())
check_project(path = project_dir, config = config_path)


# Access results programmatically

project_dir <- withr::local_tempdir()
out <- check_project(path = project_dir)

Description

Creates a new Quarto document in the specified directory. Optionally copies a sample dataset and a worked analysis example, wires up custom branding assets from a directory of standardized files, and scaffolds a post-render purl hook for extracting R code.

Usage

create_qmd(
  filename = NULL,
  path = ".",
  header_defaults = NULL,
  overwrite = FALSE,
  use_purl = FALSE,
  include_examples = TRUE,
  use_style = FALSE,
  yaml_data = lifecycle::deprecated()
)

Arguments

Details

create_qmd() performs the following steps:

1. Validates that filename is supplied and path exists.

2. If include_examples = TRUE: creates ⁠data-raw/⁠ under path and copies sample.csv there. Creates ⁠assets/⁠ if needed and copies the generic placeholder logo as logo.png, unless a logo already exists there. Uses the example template for the .qmd.

3. If include_examples = FALSE: uses the skeleton template for the .qmd. No sample data or logo is copied.

4. If use_style is TRUE or a directory path: looks for styles.css, header.html, and footer.html by name and injects whichever are present into the YAML header.

5. Stamps purl: true or purl: false into the document's own YAML header, reflecting use_purl.

6. If header_defaults (or the deprecated yaml_data) is provided, reads the YAML file and substitutes its values into the document header, descending into nested mappings so a sibling key it doesn't mention survives. This runs after style injection and the purl stamp, so header_defaults can override any auto-generated YAML key, including purl itself.

7. If use_purl = TRUE, ensures R/purl.R exists. Then, unless ⁠_quarto.yml⁠ already exists and declares ⁠project: type:⁠ as website, book, or manuscript (in which case wiring is skipped with a warning explaining why), ensures ⁠_quarto.yml⁠ has the post-render hook – creating ⁠_quarto.yml⁠ from the package template if absent, or merging the hook into the existing file's ⁠project:⁠ block if present.

8. The sample dataset bundled with the template, data-raw/sample.csv, is a subset of the Palmer Archipelago penguin data, taken from an earlier version of the palmerpenguins package than the one on CRAN today. Treat it as teaching material rather than as a citable copy of the data.

   Since R 4.5.0 the same data ships with base R, so ?datasets::penguins is the most convenient reference, with datasets::penguins_raw carrying the uncleaned form. One difference matters when reading the two side by side: base R shortened four of the column names, so bill_length_mm, bill_depth_mm, flipper_length_mm and body_mass_g here are bill_len, bill_dep, flipper_len and body_mass there. species, island, sex and year are spelled the same in both. This template keeps the longer names, which carry their units.

   Original data: Gorman KB, Williams TD, Fraser WR (2014). Ecological sexual dimorphism and environmental variability within a community of Antarctic penguins (genus Pygoscelis). PLoS ONE 9(3): e90081. doi:10.1371/journal.pone.0090081. R package: Horst AM, Hill AP, Gorman KB (2020). palmerpenguins: Palmer Archipelago (Antarctica) Penguin Data. doi:10.5281/zenodo.3960218. Collected by Palmer Station Antarctica LTER, a member of the Long Term Ecological Research Network.

Value

Invisibly returns path.

Documents in subfolders

path is the project root; filename may place the document below it, as in create_qmd("reports/analysis.qmd"). The sample data, the placeholder logo, R/purl.R, and ⁠_quarto.yml⁠ still go to the project root. Paths in the document follow two rules:

• Code paths start at the project root. The example template writes its results to "output/..." through toolero functions, and its params: input_file: "data-raw/sample.csv" is read from the root by resolve_input_path(), so the same code works at the root, in ⁠reports/⁠, and on an HTCondor execute node.

• Header paths are relative to the document, because that is how Quarto resolves them. For a document in ⁠reports/⁠, use_style writes css: ../assets/styles.css, and likewise for the two ⁠include-*⁠ entries. When a header include is wired in, the document's resource-path also lists the project root, so the logo header.html refers to as assets/logo.png is found.

Every edit to the document's YAML header is made line by line rather than by parsing the header and writing it back out. Keys the edit does not touch keep the template's own quoting, indentation, comments, and ordering, so the document a reader opens is the template we shipped plus the keys they asked for.

Note: filename has no default value and must always be supplied explicitly. Use tempdir() for temporary output during testing or exploration.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

Examples

# Minimal blank document -- no examples, no styling, no purl
create_qmd(path = tempdir(), filename = "analysis.qmd",
           include_examples = FALSE)

# Full worked example with sample data and placeholder logo
create_qmd(path = tempdir(), filename = "analysis.qmd",
           overwrite = TRUE)

# Opt this document into purl: stamps purl: true and wires up
# R/purl.R + the _quarto.yml post-render hook (merged if the file
# already exists, e.g. inside a larger Quarto website project)
create_qmd(path = tempdir(), filename = "analysis.qmd",
           overwrite = TRUE, use_purl = TRUE)

# Blank document wired to branding assets (assumes assets/ exists,
# e.g. from init_project(branding = "uw-madison"))
create_qmd(path = tempdir(), filename = "report.qmd",
           include_examples = FALSE, use_style = TRUE,
           overwrite = TRUE)

# Blank document with custom branding from a different directory
create_qmd(path = tempdir(), filename = "report.qmd",
           include_examples = FALSE, use_style = "my-branding/",
           overwrite = TRUE)

# Pre-populated YAML header, typically from generate_profile()
profile_file <- tempfile(fileext = ".yml")
writeLines("author:\n  - name: 'Your Name'", profile_file)
create_qmd(path = tempdir(), filename = "analysis.qmd",
           header_defaults = profile_file, overwrite = TRUE)

Description

Identifies which of three execution environments the code is currently running in: an interactive R session, a ⁠quarto render⁠ call, or a plain Rscript invocation. This is useful for writing code that behaves correctly across all three contexts, such as choosing figure dimensions or deciding whether to report progress.

Usage

detect_execution_context(interactive_fn = interactive)

Arguments

Details

To resolve an input data path across the same three contexts, use resolve_input_path(), which calls this function and then validates what the chosen branch produced.

Detection follows a priority order:

1. If interactive() is TRUE, returns "interactive".

2. If the environment variable QUARTO_DOCUMENT_PATH is set and non-empty, returns "quarto".

3. Otherwise, returns "rscript".

The order is unobservable in practice, and that is worth recording so nobody has to re-derive it. QUARTO_DOCUMENT_PATH is set only by Quarto rendering a document, and every path that renders one – ⁠quarto render⁠, ⁠quarto preview⁠, quarto::quarto_render(), the RStudio Render button – runs the R code in a spawned process that is not interactive. Running chunks inline in RStudio is the reverse case: the session is interactive and the variable is unset (checked in RStudio, September 2026). The two tests therefore never fire together, so there is no case in which the priority arbitrates anything.

Value

A character string, one of "interactive", "quarto", or "rscript".

See Also

resolve_input_path() for the input-path case specifically.

Examples

# Behave differently depending on where the code is running. Progress
# output is useful at a console and only clutters a cluster job log.
context <- detect_execution_context()

if (context == "rscript") {
  options(cli.progress_show_after = Inf)
}

Description

Writes a Citation File Format (CITATION.cff) skeleton, optionally pre-filled with author information from a profile written by generate_profile(), so a project's citation metadata doesn't mean retyping the same name, affiliation, and ORCID a third time.

Usage

generate_citation(
  filename = "CITATION.cff",
  path = ".",
  profile = NULL,
  overwrite = FALSE
)

Arguments

Details

title, version, repository-code, url, and license are project facts a personal profile has no way to know, so they are left as placeholders (some commented out) regardless of profile. date-released is filled in with today's date, since that much is always knowable at generation time; edit it later if the actual release date differs.

The given-names/family-names split needed by the Citation File Format is done by splitting a profile's single name field on its last space, which is right for the ordinary case and wrong for some real names. Review the generated file's given-names/family-names fields before relying on them, especially for a multi-word family name.

Value

The path to the written file, invisibly.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

generate_profile(), the function that writes the file this one can read from.

Examples

generate_citation(path = tempdir())

profile_file <- tempfile(fileext = ".yml")
writeLines(
  paste(
    'author:',
    '  - name: "Erwin Lares"',
    '    affiliation: "RCI, UW-Madison"',
    '    orcid: "0000-0002-3284-828X"',
    sep = "\n"
  ),
  profile_file
)
generate_citation(path = tempdir(), profile = profile_file, overwrite = TRUE)

Description

Writes a Markdown file documenting a single dataset, with the standard sections a data-management plan asks for – source, date obtained, license and usage terms, collection method, variables, and known issues – pre-filled where they can be, and left as placeholders where they can't.

Usage

generate_data_doc(dataset, path = "data", overwrite = FALSE)

Arguments

Details

Recall that a research compendium's ⁠data/⁠ and ⁠data-raw/⁠ folders hold the data itself, but not where it came from or what its columns mean. generate_data_doc() fills that gap the same way generate_citation() fills in for a missing CITATION.cff: a skeleton with a few fields already filled in, ready for you to complete by hand.

Value

Invisibly returns the full path to the written file.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

init_project(), generate_license(), generate_citation()

Examples

## Not run: 
generate_data_doc("survey_responses.csv")

generate_data_doc("weather_2024.csv", path = "data-raw")

## End(Not run)

Description

Takes a Quarto document and produces an XML file that is directly importable into a UW-Madison Knowledge Base (KB) article. The function re-renders the .qmd with embed-resources: true so all visual assets are self-contained, extracts the HTML body, and wraps it in the KB XML structure along with metadata drawn from the document's YAML header.

Usage

generate_kb_xml(html_path, qmd_path = NULL, output_dir = NULL)

Arguments

Details

generate_kb_xml() performs the following steps:

1. Validates that html_path exists.

2. Infers qmd_path from html_path if not supplied, then validates it.

3. Extracts title, description, and categories from the .qmd YAML header and maps them to kb_title, kb_summary, and kb_keywords.

4. Re-renders the .qmd in an isolated temporary directory with embed-resources: true so all CSS, images, and JS are self-contained. The ⁠data/⁠ and ⁠assets/⁠ folders are copied alongside the .qmd to ensure the render succeeds.

5. Extracts the ⁠<body>⁠ from the embedded HTML.

6. Escapes HTML entities in the body for XML compatibility, as required by the UW-Madison KB import format.

7. Builds the XML structure with kb_title, kb_keywords, kb_summary, and kb_body nodes.

8. Writes the .xml file to output_dir.

Temporary files are managed via withr::local_tempdir() and are automatically cleaned up when the function exits, even on error.

When importing the resulting XML into the KB, check the Decode HTML entity in body content option.

Value

Invisibly returns the path to the written .xml file.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

Examples

# Infer qmd_path automatically, write XML alongside the HTML
# generate_kb_xml(html_path = "docs/analysis.html")

# Supply qmd_path explicitly and write to a specific output directory
# generate_kb_xml(
#   html_path  = "docs/analysis.html",
#   qmd_path   = "analysis.qmd",
#   output_dir = "exports"
# )

Description

Writes a plain-text LICENSE file at the project root, using one of a small set of common license templates with the copyright holder and year filled in.

Usage

generate_license(
  license = "MIT",
  holder = NULL,
  year = format(Sys.Date(), "%Y"),
  path = ".",
  overwrite = FALSE
)

Arguments

Details

Note that this is different from how an R package normally carries a license. A package's DESCRIPTION declares ⁠License: MIT + file LICENSE⁠ and pairs a two-line CRAN stub (LICENSE) with the full text in a separate LICENSE.md. The projects init_project() scaffolds are research compendia, not packages – there is no DESCRIPTION to reference a license from – so generate_license() instead writes one self-contained file carrying the license text directly, the way a plain GitHub repository does.

Value

Invisibly returns the full path to the written file.

GPL-3

The GNU General Public License's full text runs to several hundred lines. Rather than risk an inaccurate transcription of it, license = "GPL-3" writes the license notice the Free Software Foundation itself recommends attaching to a program, together with a link to the canonical full text at https://www.gnu.org/licenses/gpl-3.0.txt. "MIT" and "CC0" are both short enough to reproduce here in full, and are written that way.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

init_project(), generate_data_doc()

Examples

## Not run: 
generate_license(license = "MIT", holder = "Jane Researcher")

generate_license(license = "CC0", holder = "Example Lab",
                  path = "~/Documents/my-project")

## End(Not run)

Description

generate_manifest() reads the accumulator written by save_output() over the course of an analysis, collapses it to one row per output file, and writes the output record, project-manifest.json, describing every artifact the project produced.

Usage

generate_manifest(
  output_dir = NULL,
  filename = "project-manifest.json",
  overwrite = FALSE,
  config = NULL,
  git_root = "."
)

Arguments

Details

The output record opens with schema_version, then holds execution_context, generated_at, and commit once at the top level, followed by an artifacts array with one entry per output file, ordered chronologically. Context, generation time, and commit are facts about the run as a whole rather than about any individual artifact, so they are not repeated per entry. Package and R versions are deliberately absent: that is renv's job, and duplicating it here would create a second record to keep in sync.

Field names are toolero's own rather than RO-Crate vocabulary. The translation to ⁠@id⁠, dateCreated, and the rest belongs in encapsulr::describe() as a thin mapping layer, so that toolero's public interface does not inherit a downstream package's data model.

Checksums are likewise excluded. RO-Crate defers fixity to BagIt and OCFL, and rocrateR::bag_rocrate() computes manifest-sha512.txt automatically at bagging time.

A missing accumulator is an error: no save was ever recorded, and an empty output record would present that as a finished result. An accumulator holding no rows is different – the file exists, so the machinery was wired up – and produces an empty output record with a warning.

Value

The path to the output record, invisibly.

Provenance

The output record also holds commit: the git commit checked out in git_root at the moment the record was written, or null when the project is not a git repository, has no commits yet, or git is not installed. This is deliberately the one piece of "which version of the code produced this" that package versions cannot supply – renv.lock already answers which package versions were in play, but nothing else records which revision of the analysis script itself ran. Like execution_context and generated_at, it describes the run as a whole and is not repeated per artifact.

config is entirely opt-in and affects output_dir only, not commit. Without it, output_dir defaults to ⁠output/⁠ under the project root. When config is supplied but cannot be read, this aborts with the same message init_project() gives for a bad config, rather than silently falling back to "output".

Format

The output record is a single JSON object with these keys, in this order:

• schema_version – integer, currently 1.

• execution_context – "interactive", "quarto", or "rscript", as returned by detect_execution_context().

• generated_at – when the record was written, in UTC with millisecond precision ("2026-09-29T18:04:12.345Z").

• commit – a 40-character git commit SHA, or null.

• artifacts – an array, empty rather than absent when nothing was saved. Each entry carries the seven accumulator fields: file_path (relative to the project root when the file is inside the project, as given to save_output() otherwise), r_class (the object's classes joined with "|"), timestamp (same format as generated_at), function_used, status ("success" or "failure"), error_message, and note. A field with no value is written as null, never as an empty string.

schema_version increments only when an existing key is removed, renamed, or changes meaning or type. A record with no schema_version was written by toolero 0.5.x and has the version 1 shape without the key. The full specification, including the rules for readers, is in the family's CONVENTIONS.md.

The output record is toolero's own format. Read it with read_output_records(); other packages should treat the file as opaque rather than parse it.

The output record and the job manifest

This is the output record: a record of outputs from a computation that has already happened. It is distinct from the job manifest produced by write_by_group() and consumed by submitr::htc_gen_submit(), which lists inputs to a computation about to happen. The file name, project-manifest.json, and this function's name predate the family's vocabulary and are kept for compatibility; the file defaults to project-manifest.json rather than manifest.json so the two documents cannot be confused on disk.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

save_output(), which feeds the accumulator, and read_output_records(), which reads the output record back.

Examples

output_dir <- withr::local_tempdir()

save_output(
  object = mtcars,
  file_path = fs::path(output_dir, "mtcars.rds"),
  .f = saveRDS,
  output_dir = output_dir
)

generate_manifest(output_dir = output_dir)

Description

Writes a YAML skeleton, pre-filled with placeholders and explanatory comments, covering the author information and formatting preferences that tend to be identical across every document you create with create_qmd(). Edit the result with your own values, then pass its path to create_qmd(header_defaults = ) instead of retyping the same author block and format options on every call.

Usage

generate_profile(filename, path = fs::path_home(), overwrite = FALSE)

Arguments

Details

The written file has two sections. Personal information becomes the document's ⁠author:⁠ block: name, affiliation, ORCID, email, and a website URL, the fields Quarto's default HTML title block already knows how to render. Document settings covers the recurring, non-personal choices create_qmd() doesn't otherwise remember for you: date/ date-modified, categories, lang, execute options for quiet and reproducible rendering, and a ⁠format: html:⁠ block of layout preferences. Neither section touches css, include-before-body, or include-after-body – those are create_qmd()'s use_style argument's job, reading from a project's own ⁠assets/⁠ folder, and a profile that also tried to set them would collide with a specific project's branding rather than complementing it.

The file is a plain copy of the packaged template, not a form filled in programmatically – this function does not prompt for values. Open the written file, replace the placeholder values with your own, and delete any key you don't want pre-filled; create_qmd() leaves a deleted key's template placeholder untouched.

A phone number and mailing address are deliberately not among the placeholders. Documents built from this file are the kind that tend to get rendered to HTML and published, and neither belongs in something public.

Value

The path to the written file, invisibly.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

create_qmd(), whose header_defaults argument reads the file this writes; generate_citation(), which can also draw on it.

Examples

generate_profile("my-profile.yml", path = tempdir())

Description

Writes a YAML configuration file pre-filled with the standard toolero folder structure and naming conventions. Edit the file to define a custom project layout, then pass its path to init_project() via the config argument.

Usage

generate_project_config(filename, path = ".", overwrite = FALSE)

Arguments

Details

The file uses the same schema as the ⁠_toolero.yml⁠ that init_project() writes into a project, so a config you author by hand and the project config a project carries are the same kind of document. The only difference is who wrote it.

Value

Invisibly returns the full path to the written file.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

init_project(), check_project()

Examples

## Not run: 
# Write to the current working directory
generate_project_config("my-project.yml")

# Write to home directory for easy reuse across projects
generate_project_config("linguistics-project.yml", path = "~")

# Overwrite an existing config
generate_project_config("my-project.yml", overwrite = TRUE)

## End(Not run)

Description

init_project() creates a new R project at the given path with an opinionated folder structure suited for research workflows. It optionally initializes renv for package management and git for version control, and records the structure it resolved in a project config, ⁠_toolero.yml⁠, at the project root.

Usage

init_project(
  path,
  use_renv = TRUE,
  use_git = TRUE,
  custom_folders = NULL,
  config = NULL,
  open = FALSE,
  branding = "none",
  uw_branding = deprecated(),
  use_readme = TRUE,
  use_rprofile = FALSE,
  scaffold_fn = renv::scaffold
)

Arguments

Value

Called for its side effects. Invisibly returns path.

The project config

init_project() writes ⁠_toolero.yml⁠ to the project root, recording the folder set it resolved and the naming conventions in force. The file records the resolved structure, never the inputs that produced it, so a project built from a config, one built with custom_folders, and one built from the defaults all produce the same shape of file and a reader never has to replay anything to learn what the project looks like.

It exists because the structure is configurable. check_project() can audit a customized project without being handed the same config again, and containr and submitr can resolve where code, data, and outputs live rather than assuming. Commit the file: it describes the project, not the machine it was created on.

The format is experimental and may gain keys before it settles. The schema_version field exists so that a reader can tell whether it understands what it is holding.

The .here marker

init_project() also writes an empty .here file at the project root. It marks the project root for toolero's own functions, which read a relative path from there (see the Paths section), and for here::here() in any code of yours that still calls it. An .Rproj file or a .git folder would serve too, but a project created outside RStudio and without git has neither, so .here guarantees a marker. Commit it. It is not needed on an execute node, where toolero takes the job's scratch directory as the root. An existing .here is left as it is.

Empty folders and git

Each folder init_project() creates that is still empty when the call finishes receives a zero-byte .gitkeep.

git tracks files rather than directories, so without this a scaffolded structure survives nothing: the opening commit contains the files at the project root and none of the layout, and a collaborator cloning the repository gets a project with no folders in it. The placeholders are written whether or not use_git = TRUE, since a project can be git-initialized at any point afterwards.

Folders that already have content are left alone – ⁠assets/⁠ holds branding files by then and is tracked on the strength of those.

The active project

init_project() makes the new project the active usethis project for the duration of the call, and restores whichever project was active before when it returns. Nothing is left pointing somewhere the caller did not ask for.

This matters more than it sounds. usethis::create_project() sets the active project only for its own duration – it uses usethis::local_project() internally and restores the caller's project on exit when open = FALSE. Any step that resolves paths through the active project therefore has to set it again explicitly. In v0.5.0 and earlier init_project() did not, so with use_git = TRUE the git initialization and its opening commit ran against whatever project happened to be active in the calling session rather than the project just created.

Dependency discovery and renv

When use_renv = TRUE, init_project() calls renv::scaffold(), which creates renv/library, renv/activate.R, renv/.gitignore, an .Rprofile that activates the project in future sessions, and an initial renv.lock.

The call is made through the injectable scaffold_fn argument rather than by name, purely so tests can substitute a stand-in and never load renv's namespace into the test process. Loading it there is what broke covr::package_coverage(): renv's load hook takes over .libPaths(), and every package a test reaches with :: afterwards (rather than one already loaded) becomes unresolvable. Two environment variables papered over the symptom before this argument existed; see covr-renv-incident.md for the incident this closes out.

Three things changed here in v0.5.0, and they are worth understanding together.

renv::scaffold() replaces renv::init(). init() loads the new project into the calling session, repointing .libPaths() at a library that is empty apart from renv itself – so every package the caller had available vanishes until they restart R. Its restart argument suppresses the restart, not the activation. That is reasonable behavior for someone adopting renv in the project they are sitting in, and the wrong behavior for a function whose job is to scaffold a project somewhere else. scaffold() builds the same infrastructure and leaves the caller's session untouched.

The .renvignore containing ⁠*.qmd⁠ is gone. It excluded Quarto documents from renv's dependency discovery, which meant that a project whose library() calls live in its .qmd source – the arrangement this package recommends – could snapshot a lockfile with none of the analysis packages in it, and containr::generate_dockerfile() would then build an image that could not run the analysis. At the point the file was written the project contained no .qmd files at all, so it never affected the snapshot taken at creation time; its only effect was on every snapshot the user took afterwards.

The snapshot at creation time is gone too, for the same underlying reason the .renvignore was pointless there: a project that has just been created has no code in it, so there is nothing to discover and nothing worth recording. Take a snapshot yourself once the project has code, and before containerizing:

renv::snapshot()

Personal .Rprofile and renv

R reads exactly one .Rprofile per session: the project's own if the working directory has one, a personal one only if it does not. When use_renv = TRUE, renv::scaffold() writes a project .Rprofile containing source("renv/activate.R"), and from that point on a user's own startup customizations – aliases, options, personal helper functions – are shadowed for every session opened in this project. Nothing warns about this; it simply stops loading.

use_rprofile = TRUE appends a guarded block to the project's .Rprofile, after renv's own activation line, that sources ⁠~/.Rprofile⁠ if it exists. Pass a character string instead of TRUE to source a different file – for instance a profile tracked in a dotfiles repository rather than kept at the platform default location. Either way, the existence check happens inside the written block, at every session start, rather than being baked in once, so a file written or edited after the project is created is still picked up. If the named file does not exist yet when init_project() itself runs, a warning says so – most often a sign of a typo – but the guard is written regardless, since the file may simply not exist yet.

R has no mechanism for merging profiles, so the two files stack rather than combine: the personal file runs in the same session, top to bottom, at the point where the block appears. Settings that do not overlap coexist without trouble. If both files set the same option (repos, for example), the later line wins, and because the block is appended at the end of the project .Rprofile, the personal value overrides anything set above it, including values renv established during activation. A personal profile that must not touch such options inside a project can guard them, for instance with ⁠if (!file.exists("renv.lock"))⁠.

The personal file runs under renv's restricted library paths. Packages it attaches or relies on – devtools, or a clipboard helper that uses clipr – are found only if they are installed in the project library, so a startup line such as require(devtools) may warn, and a helper that depends on an optional package should fall back gracefully when that package is absent.

Defaults to FALSE because it cuts against renv's own isolation goal: a project that automatically re-sources a personal environment is no longer fully isolated from it. The TRUE mode is also written generically – the block sources whichever file sits at ⁠~/.Rprofile⁠ for whoever opens the project, not a specific person's file – so a collaborator who clones the project gets the same behavior the project's original author chose, rather than one tied to a specific person's home directory. A character path is written into the guard exactly as given, so the same portability holds as long as the path itself is written portably (~-relative, say, rather than an absolute path tied to one machine).

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

check_project(), generate_project_config()

Examples

## Not run: 
init_project(path = file.path(tempdir(), "project1"),
             use_renv = FALSE, use_git = FALSE)

# Generic placeholder branding
init_project(path = file.path(tempdir(), "project2"),
             branding = TRUE, use_renv = FALSE, use_git = FALSE)

# UW-Madison RCI branding
init_project(path = file.path(tempdir(), "project2b"),
             branding = "uw-madison", use_renv = FALSE, use_git = FALSE)

# Add a folder and suppress one from the standard set
init_project(path = file.path(tempdir(), "project3"),
             custom_folders = c("models", "-output/figures"),
             use_renv = FALSE, use_git = FALSE)

# Drive structure entirely from a config file
init_project(path = file.path(tempdir(), "project4"),
             config = "~/linguistics-project.yml",
             use_renv = FALSE, use_git = FALSE)

# Plain-text README instead of Markdown (same content, README.txt)
init_project(path = file.path(tempdir(), "project5"),
             use_readme = "plain", use_renv = FALSE, use_git = FALSE)

# Skip the README entirely
init_project(path = file.path(tempdir(), "project6"),
             use_readme = FALSE, use_renv = FALSE, use_git = FALSE)

# Keep loading your own ~/.Rprofile customizations under renv (the
# scenario this argument exists for -- renv's own generated .Rprofile
# would otherwise shadow ~/.Rprofile entirely)
init_project(path = file.path(tempdir(), "project7"),
             use_rprofile = TRUE, use_git = FALSE)

# Source a personal profile kept somewhere other than ~/.Rprofile
init_project(path = file.path(tempdir(), "project8"),
             use_rprofile = "~/dotfiles/rprofile", use_git = FALSE)

## End(Not run)

Description

qmd_to_r() extracts R code chunks from a .qmd file and writes them to a standalone .R script using knitr::purl(). It works on any .qmd file regardless of whether it was created with create_qmd().

Usage

qmd_to_r(input, output = NULL, documentation = 1L, quiet = TRUE)

Arguments

Details

The parent directory of output is created if it does not already exist, and the creation is reported. knitr::purl() does not do this itself, so writing a derived script into ⁠R/⁠ from a project that was not created by init_project() would otherwise fail.

Value

Invisibly returns the path to the output .R file.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

Examples

# Extract R code from a qmd file
qmd <- tempfile(fileext = ".qmd")
writeLines(c(
  "---",
  "title: Analysis",
  "---",
  "",
  "```{r}",
  "x <- 1 + 1",
  "```"
), qmd)

# Default output path: same directory, .R extension
qmd_to_r(input = qmd)

# Explicit output path. R/ is where toolero expects derived scripts;
# the directory is created if it does not exist yet.
out <- tempfile(fileext = ".R")
qmd_to_r(input = qmd, output = out)

# Strip all documentation
qmd_to_r(input = qmd, output = out, documentation = 0L)

Description

read_clean_csv() reads a CSV file, standardizes column names, optionally handles missing values, and optionally prints an ingest summary. It combines readr::read_csv(), janitor::clean_names(), and tidyr::drop_na() into a single, reproducibility-friendly step.

Usage

read_clean_csv(
  path,
  na = c("", "NA"),
  drop_na = FALSE,
  summary = FALSE,
  verbose = FALSE,
  ...
)

Arguments

Value

A tibble with cleaned column names.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

Examples

sample_path <- system.file("templates", "sample.csv", package = "toolero")

# Basic usage
data <- read_clean_csv(sample_path)

# Explicit missing-value codes
data <- read_clean_csv(sample_path, na = c("", "NA", "N/A", ".", "-999"))

# Drop rows missing in any column
data <- read_clean_csv(sample_path, drop_na = TRUE)

# Drop rows missing in specific columns
data <- read_clean_csv(sample_path, drop_na = c("bill_length_mm", "sex"))

# Print ingest summary
data <- read_clean_csv(sample_path, summary = TRUE)

# Combine arguments
data <- read_clean_csv(
  sample_path,
  na      = c("", "NA", "N/A", "."),
  drop_na = TRUE,
  summary = TRUE
)

Description

Reads the output record, project-manifest.json, that generate_manifest() writes at the end of an analysis, and returns what it describes as a table with one row per artifact. Given several output folders, it reads each and stacks the results, so that the outputs of a multi-job run (one folder per job, as submitr::htc_collect() leaves them) can be inspected as one table.

read_output_records() is the reader that goes with generate_manifest(). The output record is toolero's own format, and this is the one place that interprets it, so other code (including other packages) can read the table this returns rather than parse the file.

Usage

read_output_records(
  path = NULL,
  filename = "project-manifest.json",
  config = NULL
)

Arguments

Value

A tibble with one row per artifact and these columns, in order:

• source – the name given to the folder in path, or the path itself.

• file_path, r_class, timestamp, function_used, status, error_message, note – the seven artifact fields, exactly as recorded (see generate_manifest()). All character; timestamps are kept as the strings written, and a null in the record becomes NA.

• read_from – "output record", or "accumulator" when the rows came from the fallback described below.

• schema_version – integer; NA for rows read from an accumulator.

• execution_context, generated_at, commit – facts about the run as a whole, repeated on each of its rows; NA for rows read from an accumulator, which does not record them.

An output record with no artifacts contributes no rows. When nothing at all can be read, the result is a tibble with these columns and no rows.

Versions

A record written by toolero 0.5.x carries no schema_version and is read as version 1, which is the shape it has. A record with a version this toolero does not know is read as far as possible, with a warning: the fields listed above are taken where they exist and have a single value, and anything else is ignored. The rules are the family's, set out in its CONVENTIONS.md.

When there is no usable output record

An analysis that stopped before generate_manifest() ran, or whose record cannot be parsed, may still have left its accumulator behind. For each folder, read_output_records() reads the output record when it can; otherwise it reads accumulator.csv from the same folder, keeping the latest row per file_path exactly as generate_manifest() would, and warns that it did so. A folder with neither contributes no rows, also with a warning. One incomplete folder therefore never stops the others from being read. A path that does not exist at all is an error, since that is a mistake in the call rather than a fact about a run.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

generate_manifest(), which writes the output record, and save_output(), which feeds it.

Examples

output_dir <- withr::local_tempdir()

save_output(
  object = mtcars,
  file_path = fs::path(output_dir, "mtcars.rds"),
  .f = saveRDS,
  output_dir = output_dir
)
generate_manifest(output_dir = output_dir, git_root = output_dir)

read_output_records(output_dir)

## Not run: 
# Several jobs brought back by submitr: label each job's rows by group.
jobs <- submitr::htc_collect()
jobs <- jobs[!is.na(jobs$output_dir), ]
records <- read_output_records(stats::setNames(jobs$output_dir, jobs$group_id))

## End(Not run)

Description

Picks an input file path according to where the code is running, then checks that the result is usable and explains what went wrong when it is not. It is the pattern this family of packages recommends everywhere, packaged so that it lives in one place rather than being retyped into every document.

Usage

resolve_input_path(
  interactive = NULL,
  quarto = NULL,
  rscript = NULL,
  must_exist = TRUE,
  context = detect_execution_context()
)

Arguments

Details

Each of the three arguments is an ordinary R argument and therefore a promise, so only the branch matching context is ever evaluated. Under Rscript, where params does not exist, passing quarto = params$input_file is safe because that expression is never forced.

The reason this is a function rather than a documented switch() follows from the same mechanism. When a branch is selected and its expression fails, the failure happens inside this function, where it can be caught and explained. A document that declares no ⁠params:⁠ block raises ⁠object 'params' not found⁠, which says nothing about YAML headers; a hand-written switch() in the document evaluates that expression in the document's own frame, where nothing is in a position to intercept it.

The three ways a branch produces something unusable, and what this function says about each:

• Under Rscript with no argument passed, commandArgs(trailingOnly = TRUE)[1] is NA_character_. Reading that produces an error about NA rather than about a missing argument.

• Under Quarto, params exists only if the YAML header declares it, and params$input_file is NULL if the block exists without that key.

• In any context, the resolved path may simply not be there, which is most often a working directory that is not what the author assumed.

Value

A single character string: the resolved path. In the interactive and quarto contexts, a relative path is returned joined to the project root (see the section below).

Relative paths start at the project root

In the interactive and quarto contexts, a relative path such as "data-raw/sample.csv" is read from the project root, not from the working directory. So a document under ⁠reports/⁠ can declare input_file: data-raw/sample.csv in its header and find the file whether it is rendered (Quarto runs the code in ⁠reports/⁠) or run chunk by chunk in RStudio. How the root is found, and the fallback for documents written before this rule, are described in the Paths section below.

The rscript context is left alone. On an HTCondor execute node the path arrives as a command line argument that already points at the file: an absolute path to data baked into the image, or a bare subset file name in the job's scratch directory.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

detect_execution_context(), which decides the branch.

Examples

# Resolving for a named context. must_exist = FALSE because there is no
# such file here; in a real document you want the default.
resolve_input_path(
  rscript    = "data-raw/sample.csv",
  context    = "rscript",
  must_exist = FALSE
)

## Not run: 
# A path per context, which is what a scaffolded document shows. Only
# the branch matching the current context is evaluated, so the params
# reference is safe under Rscript, where params does not exist.
input_file <- resolve_input_path(
  interactive = "data-raw/sample.csv",
  quarto      = params$input_file,
  rscript     = commandArgs(trailingOnly = TRUE)[1]
)

# The defaults cover the mechanical branch and fall back to the
# document's own params for the other two, so a document whose YAML
# header declares input_file needs no arguments at all.
input_file <- resolve_input_path()

## End(Not run)

Description

run_by_group() applies a function to each subset of a dataset and collects the results. Subsets can be supplied in two ways: as files listed in a job manifest produced by write_by_group(), or as a named list of data frames already in memory. When the function returns tabular output (a data frame or tibble), the results are automatically unnested into a flat tibble with a group-id column. When the function returns non-tabular output (a model, a plot, a file path), the results are returned as a nested tibble with a group-id column and a results list-column.

Usage

run_by_group(
  manifest = NULL,
  .f,
  ...,
  groups = NULL,
  .id = "group_id",
  .read_fn = read_clean_csv,
  workers = 1L,
  seed = NULL,
  verbose = FALSE
)

Arguments

Value

A tibble. If .f returns tabular output, the tibble is flat with a .id column prepended. If .f returns non-tabular output, the tibble has two columns: .id and results (a list-column).

The split-apply pattern

run_by_group() is the apply half of the split-apply workflow in toolero. The split half is write_by_group(), which partitions a data frame by a grouping column and writes one file per group along with a job manifest.

# Split to disk
write_by_group(penguins, group_col = "species",
               output_dir = "data/jobs", manifest = TRUE)

# Apply from disk via the job manifest
results <- run_by_group(
  manifest = "data/jobs/manifest.csv",
  .f       = my_analysis
)

# Apply from memory via named list
subsets <- penguins |>
  dplyr::group_split(species) |>
  setNames(c("Adelie", "Chinstrap", "Gentoo"))

results <- run_by_group(
  groups = subsets,
  .f     = my_analysis
)

The split is done once. The apply step can be run many times as you iterate on the analysis function.

What .f receives and returns

.f receives a single data frame as its first argument. It can return anything, but the return type must be consistent across all groups. Consistency is evaluated by bucket: either all groups return a data frame (tabular) or none do (non-tabular). Mixed returns cause an error identifying which groups returned unexpected types.

Common return types and their output shape:

• A one-row tibble of summary statistics – unnested into a flat table

• A multi-row tibble (e.g. model coefficients) – unnested with the group ID repeated per row

• A model object – returned as a list-column

• A ggplot object – returned as a list-column

• A file path – returned as a list-column

Passing arguments through to .f

Most arguments need nothing special. A number, a string, a logical, a file path, a function: these are ordinary values, and they reach .f in both modes exactly as you would expect. run_by_group() imposes no requirement on how .f is written.

scale_group <- function(data, multiplier) {
  dplyr::summarise(data, total = sum(x) * multiplier)
}

run_by_group(groups = subsets, .f = scale_group, multiplier = 10)

One case is different, and only one: an argument that is a bare column name. A symbol like flipper_length_mm has no meaning until it meets the data, so a function that accepts one has to capture it rather than evaluate it, which is what {{ }} does. That is a property of how .f is written rather than anything run_by_group() asks for – calling such a function directly has the same requirement.

plot_group <- function(data, x, y) {
  ggplot2::ggplot(data, ggplot2::aes(x = {{ x }}, y = {{ y }})) +
    ggplot2::geom_point()
}

run_by_group(
  groups = subsets,
  .f     = plot_group,
  x      = flipper_length_mm,
  y      = body_mass_g
)

Run sequentially, which is the default, ... is forwarded unevaluated, so flipper_length_mm stays a symbol until {{ }} captures it inside plot_group() and evaluates it against the group's data. Versions before 0.5.0 evaluated ... in run_by_group()'s own frame, where that symbol means nothing, so the call above failed with ⁠object 'flipper_length_mm' not found⁠ before plot_group() was reached.

If you would rather not write .f that way, pass the column name as a string and index with it. A string is an ordinary value, so it needs no tidy evaluation and works in both modes:

plot_group <- function(data, x, y) {
  ggplot2::ggplot(data, ggplot2::aes(x = .data[[x]], y = .data[[y]])) +
    ggplot2::geom_point()
}

run_by_group(
  groups = subsets,
  .f     = plot_group,
  x      = "flipper_length_mm",
  y      = "body_mass_g"
)

Bare column names do not survive workers > 1. Parallel execution sends the work to separate R sessions, which means every argument has to be materialized and serialized first. An argument whose value exists only inside the data mask .f builds has nothing to serialize, so it cannot make the trip. run_by_group() reports this directly rather than letting it surface from inside future's globals inspection. Ordinary values, strings included, are unaffected.

The portable form for a bare column name moves it inside .f, which works in both modes:

run_by_group(
  groups  = subsets,
  .f      = \(d) plot_group(d, x = flipper_length_mm, y = body_mass_g),
  workers = 4
)

Two smaller consequences of forwarding rather than forcing, both matching what a direct call to .f does, and both sequential-only: an argument with a side effect is evaluated at most once for the whole call rather than once per group, and an argument .f never touches is never evaluated at all.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

Examples

sample_path <- system.file("templates", "sample.csv", package = "toolero")
penguins <- read_clean_csv(sample_path)

# Split the data to disk
tmp <- tempdir()
write_by_group(penguins, group_col = "species",
               output_dir = tmp, manifest = TRUE)

# Define an analysis function
summarise_species <- function(data) {
  dplyr::summarise(data,
    n            = dplyr::n(),
    mean_mass    = mean(body_mass_g, na.rm = TRUE),
    mean_flipper = mean(flipper_length_mm, na.rm = TRUE)
  )
}

# Apply via the job manifest -- returns a flat tibble
results <- run_by_group(
  manifest = file.path(tmp, "manifest.csv"),
  .f       = summarise_species
)

# Apply via named list in memory
subsets <- penguins |>
  dplyr::group_split(species) |>
  setNames(c("Adelie", "Chinstrap", "Gentoo"))

results <- run_by_group(
  groups = subsets,
  .f     = summarise_species
)

# Apply a function that returns a model -- returns a nested tibble
fit_model <- function(data) {
  lm(body_mass_g ~ flipper_length_mm, data = data)
}

models <- run_by_group(
  manifest = file.path(tmp, "manifest.csv"),
  .f       = fit_model
)

# Parallel execution using available cores
workers <- max(1L, parallelly::availableCores() - 1L)

results <- run_by_group(
  manifest = file.path(tmp, "manifest.csv"),
  .f       = summarise_species,
  workers  = workers
)

# Reproducible parallel execution with a fixed seed
random_summary <- function(data) {
  tibble::tibble(val = sample(seq_len(nrow(data)), 1))
}

results <- run_by_group(
  manifest = file.path(tmp, "manifest.csv"),
  .f       = random_summary,
  workers  = workers,
  seed     = 1234
)

Description

save_output() writes object to file_path via .f(object, file_path, ...), then – when manifest = TRUE (the default) – appends a row to the project-level accumulator at output_dir/accumulator.csv recording what was saved, how, and whether the write succeeded. The accumulator is the working file later consumed by generate_manifest(), which deduplicates it and reshapes it into the output record, project-manifest.json.

Usage

save_output(
  object,
  file_path,
  .f,
  ...,
  manifest = TRUE,
  note = NULL,
  output_dir = NULL,
  config = NULL
)

Arguments

Details

The call to .f is wrapped in a narrowly-scoped tryCatch() – only the .f(object, file_path, ...) call itself, not the rest of save_output()'s body. On failure, a row is still appended recording status = "failure" and the caught message, after which the original condition is rethrown unmodified. Its class, message, and call are preserved as caught, so downstream handlers behave as though .f() had been called directly. This is the one place in the package where the cli convention is deliberately not followed: cli::cli_abort() would construct a new condition and discard the original class.

r_class records class(object) as a single pipe-separated field, captured before the write, since class cannot be reliably recovered from the file afterward. Timestamps are recorded in UTC with millisecond precision so that they sort lexicographically – generate_manifest() relies on this when keeping the latest row per file_path.

Value

object, invisibly. Called for its side effects.

Project conventions

The accumulator for a document under ⁠reports/⁠ lands in the project's own ⁠output/⁠ rather than in ⁠reports/output/⁠, because paths start at the project root. config is opt-in: without it, output_dir defaults to ⁠output/⁠ under the project root. When config is supplied but cannot be read, this aborts with the same message init_project() gives for a bad config, rather than silently falling back to "output".

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

generate_manifest()

Examples

output_dir <- withr::local_tempdir()

save_output(
  object = mtcars,
  file_path = fs::path(output_dir, "mtcars.rds"),
  .f = saveRDS,
  note = "Unmodified example data.",
  output_dir = output_dir
)

Description

Splits a data frame by one or more grouping columns and writes each group to a separate CSV file. Optionally writes a job manifest listing the output files, their group values, and row counts.

Usage

write_by_group(
  data,
  group_col,
  output_dir = NULL,
  manifest = FALSE,
  drop_na = TRUE,
  prefix = NULL,
  config = NULL
)

Arguments

Details

Output filenames are derived from the group values of group_col. Each value is sanitized independently: converted to lowercase, runs of non-alphanumeric characters replaced with a single -, and leading/trailing dashes stripped. When group_col has more than one element, the sanitized values are joined with ⁠--⁠ in the order supplied, so group_col = c("species", "sex") on an Adelie male produces adelie--male.csv. A prefix, if supplied, is sanitized the same way and joined to the front with a single -.

Value

Invisibly returns output_dir.

Why the separators differ

Because a run of non-alphanumeric characters collapses to exactly one dash, a sanitized value can contain a single - but never two in a row. That is what makes ⁠--⁠ safe between columns: it can only ever appear where this function put it.

The alternative would lose data rather than merely look untidy. Joined with a single dash, the groups ⁠("a-b", "c")⁠ and ⁠("a", "b-c")⁠ both produce the key a-b-c, and since the split is performed on that key the two groups would be merged into one file and reported as one manifest row. Joined with ⁠--⁠ they are a-b--c and a--b-c, and stay distinct.

prefix is joined with a single - instead, because it is constant across every file in a call and so cannot create a collision: prepending the same string to two keys leaves them exactly as distinct as they were. It is a namespace for the whole split rather than another field of the group, and reads better as one.

The job manifest

If manifest = TRUE, a manifest.csv is written to output_dir. This is the job manifest: a list of inputs to a computation that has not happened yet, and the file consumed by run_by_group() and by submitr::htc_gen_submit() in multiple-job mode. It is a different document from the output record that generate_manifest() writes, which records outputs from a computation that already has.

The schema is one column per grouping variable, holding the raw unsanitized value, followed by group_value, n_rows, and file_path. file_path is relative to the project root when the file is inside the project ("data/jobs/adelie.csv"), and the absolute path otherwise, so the manifest means the same thing wherever it is read from. Grouping on one column therefore produces a job manifest whose first column repeats group_value exactly. That redundancy is deliberate: one schema with a varying column count is easier to read, validate and rely on than two schemas selected by how many columns you happened to group on.

group_value is a human-readable composite of the raw values joined by " | ", so "Adelie | male" for two columns and simply "Adelie" for one.

Row order

Groups are written, and job manifest rows recorded, in order of first appearance in data. This matters downstream: submitr writes its subdatasets.csv in job manifest order, HTCondor assigns ProcId in that order, and log filenames are reconstructed from position, so job manifest row order is the mapping from a job number back to a group.

Missing values

With drop_na = TRUE (the default), rows with a missing value in any grouping column are removed before splitting and a message reports how many.

With drop_na = FALSE, missing values are coerced to the string "NA" so that they form their own group rather than being dropped silently by split(). A column that also contains a literal "NA" value – North America, Not Applicable, a country code – would then have two semantically different groups collapse into one file. Rather than merge them, write_by_group() aborts and names the column.

Note: output_dir has no default value of its own. Supply it directly, or supply config and let it resolve from the project's split_dir convention. Use tempdir() for temporary output during testing or exploration.

Project conventions

config is entirely opt-in. A project never scaffolded by init_project() behaves exactly as before: pass output_dir and nothing about config changes. When config is supplied but the file cannot be read, this aborts with the same message init_project() gives for a bad config, rather than silently falling back to a built-in default – a config you asked for and didn't get should never look identical to not asking.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

See Also

run_by_group(), the apply half of this pair.

Examples

# Split a small data frame by group and write to a temp directory
data <- data.frame(
  species = c("Adelie", "Adelie", "Gentoo"),
  mass    = c(3750, 3800, 5000)
)
write_by_group(data, group_col = "species", output_dir = tempdir())

# Same but also write a job manifest
write_by_group(data, group_col = "species",
               output_dir = tempdir(), manifest = TRUE)

# Namespace the filenames -- adelie.csv becomes penguins-adelie.csv
write_by_group(data, group_col = "species", prefix = "penguins",
               output_dir = tempdir())

# Group by more than one column
data2 <- data.frame(
  species = c("Adelie", "Adelie", "Gentoo"),
  sex     = c("male", "female", "male"),
  mass    = c(3750, 3550, 5000)
)
write_by_group(data2, group_col = c("species", "sex"),
               output_dir = tempdir(), manifest = TRUE)

# Let a project's own _toolero.yml supply output_dir via split_dir.
# generate_project_config() writes the default conventions, including
# split_dir = "data/jobs" -- a relative path meant to be resolved against
# a project's own root, so this example runs from config_dir via
# withr::with_dir() rather than writing into the working directory.
config_dir <- tempfile()
dir.create(config_dir)
generate_project_config("_toolero.yml", path = config_dir)
withr::with_dir(config_dir, {
  write_by_group(
    data,
    group_col = "species",
    config    = file.path(config_dir, "_toolero.yml")
  )
})

Description

write_clean_csv() writes a data frame to a CSV file using readr::write_csv() and emits a cli confirmation message reporting the number of rows and columns written. It is the natural counterpart to read_clean_csv(), reinforcing the convention that ⁠data-raw/⁠ holds original inputs and ⁠data/⁠ holds cleaned, analysis-ready outputs.

Usage

write_clean_csv(data, path, overwrite = FALSE, ...)

Arguments

Details

If column names are not already clean, write_clean_csv() applies janitor::clean_names() before writing and emits a warning listing the affected columns.

Value

Invisibly returns path, resolved from the project root when it was relative.

Paths

A relative path is read from the project root, not from the working directory. So save_output(fit, "output/fit.rds", saveRDS) writes into the project's own ⁠output/⁠ folder whether you call it from the project root, from a document under ⁠reports/⁠, or from a job on an HTCondor execute node. You never need to wrap a path in here::here(). Doing so still works, because here::here() returns an absolute path and absolute paths are used as given.

The project root is found afresh on every call, in this order:

1. The job's scratch directory, when ⁠_CONDOR_SCRATCH_DIR⁠ names one. HTCondor sets that variable itself, inside every job.

2. The nearest folder above the working directory that carries a project marker: the .here file init_project() writes, an .Rproj file, a package DESCRIPTION, or a version control folder such as .git.

3. The working directory, when neither of the above applies.

Absolute paths (including those starting with ~, or a drive letter on Windows) and URLs such as ⁠https://⁠ or ⁠s3://⁠ are used as given. .. and ⁠./⁠ are not interpreted, so "../data/x.csv" means the folder above the root; write an absolute path to leave the project. A bare "." means the root itself.

Functions that read are forgiving about older code. A relative path that does not exist at the project root but does exist relative to the working directory is read from there, and a message says so. Functions that write never guess: a relative path is always written under the root.

A few arguments are deliberately left alone:

• init_project() reads path, config and use_rprofile from the working directory, because the project does not exist yet and so has no root.

• Under Rscript, resolve_input_path() reads the command line argument as given. Run Rscript from the project root or pass an absolute path.

• Arguments passed on through ... to another function reach it untouched.

• Where a default is "beside the input" (qmd_to_r() and generate_kb_xml()), the output follows wherever the input was read from.

Two limits are worth knowing. A project nested inside another project is not supported: the nearest marker wins, which can surprise you when you work from the outer one. And on an execute node ~ and ⁠$HOME⁠ cannot be trusted (⁠$HOME⁠ can be the submit node's path), so prefer relative paths in job scripts.

Examples

sample_path <- system.file("templates", "sample.csv", package = "toolero")
data <- read_clean_csv(sample_path)

# Write to a temp file
out <- tempfile(fileext = ".csv")
write_clean_csv(data, out)

# Overwrite an existing file
write_clean_csv(data, out, overwrite = TRUE)

# Dirty names are cleaned automatically with a warning
dirty <- data.frame("First Name" = "Jane", "Last Name" = "Doe",
                    check.names = FALSE)
write_clean_csv(dirty, tempfile(fileext = ".csv"))