Skip to contents

rush

Package website: release | dev

rush is an R package for asynchronous and decentralized optimization. It uses a database-centric architecture in which workers communicate through a shared Redis database. To support high-throughput workloads, rush combines sub-millisecond per-task overhead with caching strategies that reduce database operations. The package integrates with the mlr3 ecosystem and serves as the backend for asynchronous optimization algorithms in bbotk and mlr3tuning.

Features

  • Database-centric architecture for decentralized optimization without a central controller.
  • Asynchronous communication between workers using Redis.
  • Sub-millisecond per-task overhead for high-throughput workloads.
  • Efficient caching mechanism that minimizes database read/write operations.
  • Task queue support for centralized tasks distribution when needed.
  • Fast data transformation from Redis to data.table.
  • Scales to large remote worker networks via the mirai package.
  • Worker logging directly into the Redis database using lgr.
  • Minimal dependencies for lightweight integration.

Install

Install the latest release from CRAN.

Install the development version from GitHub.

pak::pak("mlr-org/rush")

And install Redis.

Test with Redis

To test the package, set the RUSH_TEST_USE_REDIS environment variable to true. The test suite deletes the Redis database before execution, so never run it against a production server.

Sys.setenv(RUSH_TEST_USE_REDIS = "true")

Related Work