Crate scx_rustsched_loader 

Loading and running a Rust BPF sched_ext scheduler: the object, the struct_ops map the kernel can take, the configuration table’s kernel entries resolved from BTF, the map sizes set before the load, the attach, the exit record, the arena statistics, the per-callback profiler, the sampler of the per-CID queues, and the CPU capacity and packing tiers the runtime ranks CIDs by. Nothing here is particular to a scheduling policy.

Re-exports

pub use tiers::Tiers;

pub(crate) use collections::BTreeMap; 🔒

pub(crate) use CString; 🔒

pub(crate) use OsStr; 🔒

pub(crate) use AsFd; 🔒

pub(crate) use AsRawFd; 🔒

pub(crate) use FromRawFd; 🔒

pub(crate) use OwnedFd; 🔒

pub(crate) use ptr::NonNull; 🔒

pub(crate) use Arc; 🔒

pub(crate) use AtomicBool; 🔒

pub(crate) use Ordering; 🔒

pub(crate) use Duration; 🔒

pub(crate) use Instant; 🔒

pub(crate) use Context; 🔒

pub(crate) use bail; 🔒

pub(crate) use ensure; 🔒

pub(crate) use Btf; 🔒

pub(crate) use Link; 🔒

pub(crate) use MapCore; 🔒

pub(crate) use MapFlags; 🔒

pub(crate) use ObjectBuilder; 🔒

pub(crate) use OpenObject; 🔒

pub(crate) use ProgramType; 🔒

pub(crate) use BtfType; 🔒

pub(crate) use HasSize; 🔒

pub(crate) use ReferencesType; 🔒

pub(crate) use Composite; 🔒

pub(crate) use Enum; 🔒

pub(crate) use Enum64; 🔒

pub(crate) use Func; 🔒

pub(crate) use FuncProto; 🔒

pub(crate) use Int; 🔒

pub(crate) use IntEncoding; 🔒

pub(crate) use MemberAttr; 🔒

pub(crate) use Ptr; 🔒

pub(crate) use Struct; 🔒

pub(crate) use info; 🔒

pub(crate) use warn; 🔒

pub use scx_rustsched_abi as config;

pub(crate) use UserExitInfo; 🔒

pub(crate) use libbpf_clap_opts::LibbpfOpts; 🔒

Modules

compat

The kernel’s sched_ext flags, 0 where it lacks one, for a scheduler’s user space to choose the ops flags it loads with, see Params::add_flags().

edq

The eligible deadline queue, from the source the BPF program is built from: the loader reads the queues in the arena through the same code. Eligible Deadline Queue: an intrusive red-black tree, a port of scx_eevdf’s lib/edq.bpf.c.

kfunc_protos 🔒

Match the return types of the object’s kfunc prototypes to the running kernel’s.

queues 🔒

The per-CID queues, observed from the loader. The arena the BPF program keeps them in is mapped into this process too, at the addresses the program itself uses, so the loader reads a queue through the same edq code the program runs, over a memory that can only load.

tiers

CPU capacities and asymmetric packing priorities, the loader’s half of scx_rustsched’s tiers.rs and topology.rs. The capacities and the scheduling-domain spans come from scx_utils’ Topology. The ITMT priority and SD_ASYM_PACKING come from the kernel through the rust_cpu_priority() syscall program, since neither has a user-space ABI. The result goes to the BPF program per CPU, which translates it to CIDs at init.

Macros

main

The main() of a scheduler: the objects its build script left in OUT_DIR, loaded and run by cli(). The scheduler’s options are a clap::Args structure, each field named as the tunable it sets, a time in microseconds as the tunable in nanoseconds, and what else the scheduler adds to the loader is its Scheduler implementation:

param_values 🔒

Structs

ArenaView

The arena, read through the mapping libbpf made when it created the map. The addresses the BPF program stores are user addresses in that mapping, so they are read as they are. An address outside of it reads as 0. The view keeps the object, and with it the mapping, alive.

Attached

The run of an attached scheduler, see Scheduler::attached(): one method for each thing the loader knows about it.

Capacity

How the CPUs’ capacities and packing ranks are read for the scheduler, see tiers.rs.

CidTableReader

A per-CID table of the scheduler’s, read from the loader: the one whose header is in the .data section section, see scx_rustsched’s cid_slot!, with words 64-bit words per CID. For counters each CPU keeps for itself, which cost the scheduling paths no shared write.

Cli 🔒

The options every scheduler takes, beside its own.

Loaded

The attached scheduler’s object, shared. What reads the arena keeps a clone, and with it the arena’s mapping, which goes with the object.

NoOpts

The options of a scheduler with no tunables.

Object

Represents a loaded BPF object file.

OpStats

One struct_ops program’s run count, run time and recursion misses, as the kernel accounts them while the profiler is enabled.

OpsProfiler

Each callback is its own BPF program and the kernel keeps a run count and a run time per program once the accounting is switched on. That costs two clock reads around every callback, so it is on only while a stats client is polling and goes away by itself once nobody has asked for a while.

Params

The tunables of an object that is open and not loaded yet, the ones it declares with scx_rustsched::params!, to set before the load, and the ops flags it is loaded with. It records which tunables were set: run() refuses to start a scheduler that has one nobody set.

QueueSampler

Samples the depth of every CID’s queue while a stats client is polling, and parks once nobody has asked for a while, as the callback profiler drops its accounting.

QueueStats

What the sampler saw of one CID’s queue, cumulative.

RunOptions

How run() loads and runs a scheduler.

Stats

A scheduler’s statistics: what it serves to scx_stats clients, and how --stats and --monitor print it. A scheduler has none unless its Scheduler::stats returns this, built from its two callbacks.

Topology 🔒

What Attached reads off scx_utils’ Topology, once asked.

Constants

ARENA_PAGES 🔒

The arena’s size in pages, as scheds/include/lib/arena_map.h declares it: address space only, backed by the pages actually allocated. An arena spans at most 4 GiB, which 2^16 pages keep to with 64 KiB pages on arm64.

EXIT_DUMP_DEFAULT

SCX_EXIT_DUMP_DFL_LEN, the dump length the kernel uses for 0.

EXIT_UNREG_KERN 🔒

SCX_EXIT_UNREG_KERN, the last of the graceful exit kinds.

PARAM_PREFIX 🔒

The section of a tunable: scx_rustsched::params! puts each in a read-only section of its own, named PREFIX NAME “.” TYPE.

POOL_PAGES

Pages the pool of task slots starts with. It grows as tasks need more.

UEI_BYTES 🔒

Size of struct user_exit_info: kind, exit_cpu, exit_code, a 128-byte reason and a 1024-byte message.

WATCHDOG_MAX_TIMEOUT_MS

The longest watchdog timeout the kernel takes, SCX_WATCHDOG_MAX_TIMEOUT, which is also what it applies when the timeout is 0.

Traits

ParamValue

A value a tunable can hold: an integer or a bool, alone or as the element of an array.

Scheduler

What a scheduler adds to the loader, on its options. Every method has a default, so impl Scheduler for Opts {} is a scheduler with nothing but the tunables its options name.

Functions

arena_stats

The arena statistics the BPF side keeps, see pool.rs: slots taken and released, validation errors, failed and successful page allocations, and idle-mask updates that ran out of retries.

arena_usage

What the arena holds, from the map’s memlock accounting, which kernels from 7.3 keep for the allocated pages, and the size of the map.

attach

Attach the scheduler through its struct_ops map ops.

cgroup_ops

The struct_ops map of the BPF object whose cgroup callbacks have the names the running kernel gives them: cpuctl_, or cgroup_ on a kernel from before the cid form renamed them. None for a kernel built without group scheduling, which has neither.

check_arena_stats

What the statistics must say after a clean run.

cli

The whole loader of a scheduler: parse the command line, then run() until Ctrl-C. main! calls this with the scheduler’s name, version and objects, and with its options as S.

configuration

The configuration table’s kernel entries: the offsets and enumerators resolved from the running kernel’s BTF, the pool size, the dump length, the tick period and the runtime’s options, a mask of config::runtime’s bits.

declared_dump_len

The length of the debug dump the scheduler’s struct_ops map asks for, dump_len of #[scheduler].

enumerator

The value of the enumerator name in kernel BTF, if the kernel has it.

exit_report

The exit record ops.exit() keeps, reported as scx_utils reports it: the debug dump and the exit reason, as an error for an error exit.

exit_summary 🔒

A graceful unregister as one line, “reason” or “reason (msg)”, read off the exit record. None when the scheduler has not exited, or was kicked out by an error, which keeps the standard report and its debug dump.

init_logger 🔒

Log at info level to stderr, each line with the local time and its level, as the C-form schedulers do.

member

missing_callbacks

The byte ranges, in the BPF object’s struct_ops type ops, of the cgroup callbacks the running kernel’s sched_ext_ops_cid lacks, read off both BTFs: all of them on a kernel built without group scheduling, the newer ones on an older kernel. libbpf skips a member the kernel lacks only when it is zero.

named 🔒

The options of A with their values as given or defaulted, by name.

open

Open the object object as name, with libbpf’s open options that ObjectBuilder does not take, such as a BPF token path, as the C schedulers pass them.

ops_flags

The ops flags the struct_ops map ops asks the kernel for, as #[scheduler] declares them.

ops_member

The byte range of the member name of the BPF object’s struct_ops type ops, read off its BTF.

ops_type 🔒

The BPF object’s struct_ops type of the map ops.

param_element 🔒

The element type of an array tunable, whose type is its element’s in brackets.

param_name 🔒

A tunable’s name as the command line takes it.

param_of 🔒

The name, in lower case, and the type of the tunable the map section holds, if it holds one.

param_parse 🔒

The value text gives a tunable of type ty and size bytes: true or false for a bool, and otherwise an integer, decimal or 0x hexadecimal, in the range of the type. None if it is not one.

publish_tiers 🔒

Read the CPUs’ capacities and packing priorities and hand them to the loaded object, see tiers.rs.

run

Load, attach and run the scheduler name until shutdown is set, the duration is over or the kernel ejects it. legacy and sized are the two objects scx_rustsched_build leaves in OUT_DIR, of which the running kernel takes one. matches is the command line the options scheduler were parsed from: its options set the tunables they name, and the scheduler’s own settings the rest.

select_ops

Pick the struct_ops map of the object name the kernel can take: the one whose cgroup callbacks it knows, cgroup_ops, or rust_ops without the callbacks it lacks, which are zeroed in the map’s initial value. The exit dump buffer is sized to dump_len and the map told so. Returns the map’s name.

set_config

Fill the frozen configuration table with config.

set_ops_flags

Load the struct_ops map ops with the ops flags.

set_ops_timeout

Load the struct_ops map ops with the watchdog timeout timeout_ms, in place of timeout_ms of #[scheduler].

size_maps

Size the arena and, when the object has them, the hrtick timers, one per CID. CIDs are dense and never exceed nr_cpu_ids.

sized_topology

Does the running kernel’s scx_bpf_cid_topo() take the structure’s size? Such a kernel copies only the fields it has and leaves the rest at -1, so its structure may still be the legacy one, without the cluster fields.

spawn_monitor 🔒

The stats client, in its own thread: beside the scheduler with --stats, started once its server is up.

tick_ns 🔒

TICK_NSEC. The coarse clocks advance only from the timer interrupt, so their resolution is the tick period.

time_unit 🔒

The tunable in nanoseconds that an option in microseconds or milliseconds sets, and the factor: slice_us sets slice_ns, 1000.

Type Aliases

Idle 🔒

What a scheduler does while no stats client asks, see Stats::idle.

Result

Result<T, Error>

Derive Macros

Opts

The derive that lets a field of the loader’s options carry #[opts(no_bpf)]: an option that only the loader uses, for which opts!() declares no tunable on the BPF side.