Module tokio_io 

Async drivers for tokio (and any executor): decode with no thread blocked on I/O and no async in codec code.

decode_async keeps the decoder, the sink and the input window in the async task. It awaits the AsyncSource for bytes, then hands one decode batch (steps until the decoder needs input or finishes) to a Compute pool, and awaits that. While the client uploads, nothing occupies a thread; CPU work runs on the pool in batches as large as the input on hand. A source returns whatever bytes have arrived, so rows come out while the upload is still running.

transcode_async adds an encoder: rows go through an EncoderSink and encoded bytes are written to an AsyncWrite after every batch. The decoder and encoder stay alive while a write waits; their raw input and intermediate storage are not bounded by the encoded-output queue. Choose this coupling for configurations whose retained state and upstream production are bounded.

pull_decode_on_thread runs a pull-native decoder (one written against DecodeJob::push_decoder_from_source) on one pool thread, with a DynReadAt whose reads block on the same AsyncSource. Such a codec blocks on reads by design, so it holds a thread for the whole decode.

Cancellation

Two mechanisms, for the two places a decode can be waiting:

• Inside a compute batch, the [SharedPulse] is checked at every step, so a long batch stops within one step of the pulse tripping. The pulse is also checked before each batch starts.
• While awaiting the source, cancel the way tokio code always does: drop the future, or race it in tokio::select! against your cancellation token or a timeout. Nothing runs between batches, so the decoder and sink are dropped at once. A batch already on the pool finishes it (bounded by the input on hand) unless the pulse also trips.

pull_decode_on_thread differs: the decode thread re-checks the pulse while it waits for a read, returns AsyncDriveError::Cancelled, and the async side stops waiting on the source when the thread finishes.

Structs

AsyncCursor

Drives any ByteStepper from an AsyncSource: the async twin of ReadAtCursor.

AsyncReadSource

A sequential AsyncSource over any AsyncRead (an HTTP body, a socket). Reads ahead of the stream position skip forward; reads behind it fail with SourceError::BeyondWindow, and the length is known only once the stream has ended.

BridgeError

An AsyncDriveError carried through a pull-native codec’s error.

BytesSource

An in-memory AsyncSource (tests, already-buffered bodies).

ChannelRowSink

A sink that streams finished strips to the async side over a bounded channel. Each strip is an owned [PixelBuffer]; provide_* hands the decoder a fresh strip and ships the previous one, blocking the decode thread while the channel is full, so a slow consumer throttles the decoder instead of the decoder filling memory.

Inline

Runs each batch on the calling task. For single-threaded hosts (WASM) and small images, where moving work to another thread costs more than it saves; it blocks the executor for one batch.

Enums

AsyncDriveError

Failure of an async drive.

StripEvent

What a ChannelRowSink sends to the async side.

Traits

AsyncSource

Where the async drivers get bytes: a socket, an HTTP body, an object store with range requests.

Compute

Where decode batches run. Implemented for any Fn(Job): |job| rayon::spawn(job), |job| { tokio::task::spawn_blocking(job); }, or Inline to run on the calling task (single-threaded hosts, tiny images).

Functions

decode_async

Decode from an async source with no thread blocked on I/O. See the module documentation.

pull_decode_on_thread

Run a pull-native decode on one compute thread. run receives a DynReadAt whose reads block on source, so a codec written against push_decoder_from_source streams from an async transport, with read_many_at becoming one concurrent AsyncSource::read_ranges.

transcode_async

Decode from an async source straight into a streaming encoder, writing encoded bytes to out after every batch. This does not establish an end-to-end memory bound: codec-owned raw/intermediate state and caller-owned input can remain live while the output is stalled. The current driver also drains pending output before finalizing the encoder. Select an operation strategy based on the actual configuration’s retention and release behavior.

write_pending_async

Write every byte enc has pending to w, then consume it: the async twin of write_pending. Returns how many bytes were written.

Type Aliases

Job

Work handed to a Compute.