Skip to contents

Create Rush Plan

Source: R/rush_plan.R
rush_plan.Rd

Stores the number of workers and Redis configuration options (redux::redis_config) for Rush. Instead of a single number of workers, the workers can be distributed over the compute profiles of mirai with the profiles argument. The function tests the connection to Redis and throws an error if the connection fails. This function is usually used in third-party packages to setup how workers are started.

Usage

rush_plan(
  n_workers = NULL,
  config = NULL,
  lgr_thresholds = NULL,
  lgr_buffer_size = NULL,
  large_objects_path = NULL,
  worker_type = "mirai",
  start_worker_timeout = NULL,
  profiles = NULL
)

Arguments

n_workers

(integer(1))
Number of workers to be started.

config

(redux::redis_config)
Configuration options used to connect to Redis. If NULL, the REDIS_URL environment variable is parsed. If REDIS_URL is not set, a default configuration is used. See redux::redis_config for details.

lgr_thresholds

(named character() | named numeric())
Logger threshold on the workers e.g. c("mlr3/rush" = "debug").

lgr_buffer_size

(integer(1))
By default (lgr_buffer_size = 0), the log messages are directly saved in the Redis data store. If lgr_buffer_size > 0, the log messages are buffered and saved in the Redis data store when the buffer is full. This improves the performance of the logging.

large_objects_path

(character(1))
The path to the directory where large objects are stored. These files are not removed automatically and the caller is responsible for cleaning them up, e.g. by pointing this at a subdirectory of tempdir().

worker_type

(character(1))
The type of worker to use. Options are "mirai" to start with mirai, "processx" to use processx or "script" to get a script to run.

start_worker_timeout

(numeric(1))
Default timeout in seconds used by $wait_for_workers() of Rush when no timeout is passed. If NULL, $wait_for_workers() waits indefinitely by default. A timeout of 0 checks once and errors immediately if the workers are not yet registered.

profiles

(named integer())
Number of workers to be started on each mirai compute profile, e.g. c(cpu = 2, gpu = 2). The names are the compute profiles created with mirai::daemons() and the values are the number of workers started on the daemons of the respective profile. Cannot be combined with n_workers.

Value

list() with the stored configuration.

Examples

if (redux::redis_available()) {
  config_local = redux::redis_config()
  rush_plan(config = config_local, n_workers = 2)

  rush = rsh(network_id = "test_network")
  rush
}
#> 
#> ── <Rush> ──────────────────────────────────────────────────────────────────────
#> • Running Workers: 0
#> • Queued Tasks: 0
#> • Running Tasks: 0
#> • Finished Tasks: 0
#> • Failed Tasks: 0