Skip to main content

​
Overview

The Malbox scheduler is a multi-tier asynchronous scheduling system that coordinates task queuing, worker pool allocation, resource provisioning, and plugin execution lifecycle. Think of the scheduler as the central coordinator that orchestrates the entire task management system.

​
Architecture

The scheduler implements an event-driven architecture built on Tokio’s asynchronous runtime. Rather than a single event loop, it uses a spawn-based fan-out pattern with three concurrent tasks:
  • Task ingestion - a spawned task that receives new submissions from the HTTP layer via an mpsc channel, caches them in the task store, and enqueues them for processing
  • Worker event listener - a spawned task that receives completion and error events from workers via an mpsc channel
  • Shutdown coordinator - the main task awaits a CancellationToken, then triggers graceful shutdown of the worker pool and aborts the ingestion/event tasks
On startup, the scheduler also recovers orphaned tasks from the database — tasks left in transient states from a prior daemon crash. It then loads any pending tasks back into the queue. The scheduler maintains references to three subsystems: the task store, task queue, and worker pool.

​
Task store

The task store provides persistent database storage combined with an in-memory cache for efficient retrieval of tasks loaded during the daemon’s lifetime.

​
Task queue

The task queue implements priority-based ordering using a binary heap. Task priority determines processing order when resources become available.
The task queue stores only task IDs. Actual task contents are retrieved from the task store.

​
Worker pool

The worker pool manages the lifecycle of worker instances. It maintains a baseline of min_workers that never idle out, and spawns additional workers (up to max_workers) when all active workers are busy. Additional workers have an idle timeout and exit automatically when work dries up. See Workers for more details.

​
Communication

Components communicate through typed channels and coordination primitives:
  • Task submissions - external systems submit tasks via an mpsc channel
  • Worker events - workers report completion and errors through a dedicated mpsc channel
  • Task availability - workers are notified of new tasks via a shared Notify
  • Shutdown coordination - a CancellationToken propagates shutdown signals from the scheduler to all workers via child tokens