Fast Encoder Core (quality 0 and quality 1)

Scope: src/compressor/core/fast/, and the parts of src/shared/ it uses. This is the implementation of the two fast Brotli qualities: a one-pass encoder (quality 0) and a two-pass encoder (quality 1), both ported from Google Brotli v1.2.0, commit 028fb5a, and both byte-identical to it.

Everything here is private. No type, error or SIMD detail from this tree appears in the public API; compressor.md describes the surface that wraps it.

1. Module map

graph TD
    modmod["fast/mod.rs<br/>FastEncoder, dispatch"]
    q0["fast/q0.rs<br/>one-pass scan and meta-blocks"]
    q1["fast/q1.rs<br/>two-pass scan, replay, TwoPassState"]
    cmd["fast/commands.rs<br/>insert/copy/distance mapping"]
    hist["fast/histogram.rs<br/>chunked byte counting"]
    tabs["fast/tables.rs<br/>quality 0 and 1 command tables"]
    consts["fast/constants.rs<br/>quality 0 and 1 constants"]
    ws["fast/workspace.rs<br/>OnePassArena, TwoPassArena"]

    subgraph shared["shared"]
        huff["huffman.rs<br/>tree build, canonical codes, serialisation"]
        ml["match_len.rs<br/>hybrid scalar/SIMD match length"]
        bits["bits.rs<br/>LSB-first bit writer"]
        log["fast_log.rs<br/>table + native or libm logarithms"]
        stabs["tables.rs<br/>entropy-coding tables"]
        sconsts["constants.rs<br/>format constants"]
    end

    modmod --> q0
    modmod --> q1
    modmod --> ws
    q0 --> cmd
    q0 --> huff
    q0 --> hist
    q0 --> ml
    q0 --> bits
    q1 --> cmd
    q1 --> huff
    q1 --> hist
    q1 --> ml
    q1 --> bits
    cmd --> bits
    cmd --> tabs
    cmd --> log
    huff --> bits
    huff --> stabs
    q0 --> consts
    q1 --> consts
    consts --> sconsts
    log --> stabs
    stabs --> sconsts

The bit writer, the Huffman builders, the match-length scan, the reference logarithms and the entropy-coding tables live in shared rather than in this tree, because the greedy encoder needs exactly the same implementations; see greedy-encoder.md.

2. Ownership and reuse

FastEncoder retains its hash table, entropy workspace and scratch across fragments. One-shot append output uses the caller’s reserved vector through a statically specialized bit writer. Large initialized slices use the fixed writer directly. Sessions with smaller or uninitialized destinations encode non-flush blocks into the compressor’s initialized pending backing through that same fixed writer; a separate completed-end cursor bounds delivery. Drains and session resets retain its initialized length, so later fragments neither copy through encoder scratch nor reinitialize the whole buffer. Flushes still use the encoder’s scratch. Bit output specifies initialization, reservation, overflow and partial-byte handling.

flowchart LR
    Session[Fast non-flush session block] --> Backing[Retained initialized pending backing]
    Backing --> Fixed[encode_block_into with selected backend]
    Fixed --> Completed[Publish completed byte count]
    Completed --> Caller[Copy only completed bytes to caller]
    Caller --> Retain[Reset delivery cursors; retain initialized backing]
    Retain --> Backing

The quality 0 arena is not built by the constructor. FastCore::OnePass holds an Option<Box<OnePassArena>> that the first fragment needing a match scan fills in; a stream whose only fragment is stored verbatim (§5) never allocates it, never clears a hash table, and never touches the Huffman node pool. Whether the encoder serves an independent worker is its const parameter INDEPENDENT, because that decides whether the verbatim shortcut applies; a serial build never instantiates the fragment-only kernels.

classDiagram
    class FastEncoder {
        -Backend backend
        -FastCore core
        -usize block_size_limit
        -u16 last_bytes
        -u32 last_bytes_bits
        -Vec~i32~ table
        -Vec~u8~ storage
        -bool finished
        +encode_block(input, is_last) BrotliResult~&[u8]~
    }
    class FastCore {
        <<enum>>
        OnePass: Option~Box~OnePassArena~~
        TwoPass: Box~TwoPassState~
    }
    class OnePassArena {
        +lit_depth [u8;256]
        +lit_bits [u16;256]
        +cmd_depth [u8;128]
        +cmd_bits [u16;128]
        +cmd_histo [u32;128]
        +cmd_code [u8;512]
        +cmd_code_numbits usize
        +tree Vec~HuffmanNode~
        +histogram [u32;256]
        +tmp_depth [u8;704]
        +tmp_bits [u16;64]
    }
    class TwoPassState {
        +arena Box~TwoPassArena~
        +commands Vec~u32~
        +literals Vec~u8~
    }
    class TwoPassArena {
        +lit_histo [u32;256]
        +lit_depth [u8;256]
        +lit_bits [u16;256]
        +cmd_histo [u32;128]
        +cmd_depth [u8;128]
        +cmd_bits [u16;128]
        +tmp_tree Vec~HuffmanNode~
        +tmp_depth [u8;704]
        +tmp_bits [u16;64]
    }

    FastEncoder *-- FastCore
    FastCore o-- OnePassArena : Q0, built on first scan
    FastCore *-- TwoPassState : Q1
    TwoPassState *-- TwoPassArena

The hash table and the scratch output buffer grow but never shrink. Only the active table range is cleared between fragments; unused capacity is left untouched. Both grow through shared::fixed::grow_zeroed: the first allocation comes from the allocator’s zeroing path, and a later, larger fragment extends the same vector with resize, zero-filling only the new tail. The Huffman node pools (tree, tmp_tree) start empty and are sized by each build to the symbols it actually uses, see encoder-workspace.md.

3. Fragment lifecycle

Every call to encode_block handles one fragment of at most 1 << lgwin bytes, mirroring BrotliEncoderCompressStreamFast.

stateDiagram-v2
    [*] --> Sized: reserve 2 * len + 503 + 8 bytes
    Sized --> Seeded: storage[0..2] = last_bytes
    Seeded --> Verbatim: FastCore::stores_verbatim (final q0 fragment within the bound)
    Verbatim --> Encoded: raw meta-block and closing bits, no table, no arena
    Seeded --> Tabled: clear the active hash table range
    Tabled --> Dispatched: retained kernel, #[simd] encode_fragment
    Dispatched --> Encoded: q0 or q1 writes meta-blocks
    Encoded --> Checked: writer overflow?
    Checked --> Failed: yes
    Checked --> Carried: no
    Carried --> [*]: emit position >> 3 bytes,\ncarry the partial byte
    Failed --> [*]: BufferOverflow

The trailing partial byte is never emitted: it is kept in last_bytes and re-seeded into the next fragment’s scratch or append destination, exactly as the reference carries s->last_bytes_.

4. Quality 0: one pass

stateDiagram-v2
    [*] --> OpenBlock
    OpenBlock --> Scan: header, 13 zero bits,\nliteral code, command code
    state Scan {
        [*] --> Probe
        Probe --> Probe: no match, skip += 1 >> 5
        Probe --> Emit: five byte match
        Emit --> Chain: literals, distance, copy
        Chain --> Chain: immediate match, no literals
        Chain --> Probe: no immediate match
        Emit --> Uncompressed: insert >= 6210 and\nShouldUseUncompressedMode
    }
    Scan --> Remainder: ip reaches ip_limit
    Remainder --> Merge: ShouldMergeBlock and\ntotal + next <= 1 MiB
    Merge --> Scan: patch MLEN, keep the meta-block open
    Remainder --> OpenBlock: input left, new meta-block
    Remainder --> NextCode: input exhausted
    Uncompressed --> OpenBlock
    NextCode --> [*]: build the next fragment's command code\nwhen is_last is false

The scan follows this order:

• the repeat candidate (ip - last_distance) is tested before the hash table candidate; the reference subtracts without a sign test and rejects a candidate at or past ip after comparing, and as both tests are pure the port tests repeated < ip first, which reads nothing for the initial distance of minus one;
• the word at the next position is loaded once: it is hashed for the table lookup and kept as the word the candidates are compared against when the scan reaches that position (words_match on two loaded words), so the table store between the two candidate tests never forces a reload;
• the hash table is written at the same points and in the same order;
• post-copy hash updates cover ip - 3, ip - 2, ip - 1 and ip, taken from a single word load;
• ShouldMergeBlock uses stride 43 and the current literal depths;
• the final size guard rewrites the whole fragment verbatim when the compressed form exceeds 31 + 8 * len bits.

For a serial final fragment, q0::stores_verbatim checks len <= cmd_code_numbits / 8 (an empty final fragment always qualifies: it is just the closing block). The compressed form needs at least 19 header bits, 13 block/context bits, and the retained command code, already exceeding the rewrite threshold at that length. q0::store_final_verbatim therefore emits the same uncompressed block and the closing bits immediately, without building literal codes or scanning matches. The initial command code has 448 bits, giving a 56-byte bound; a trained command code supplies its own bound. Non-final fragments still train the successor’s code, and independent parallel fragments retain their separate command tree.

The decision is taken twice, from the same predicate: FastCore::stores_verbatim answers it before prepare_table, so no hash table is cleared, and again in encode_fragment before the arena is created, so a fresh encoder whose only fragment qualifies allocates nothing beyond the output. Without an arena the predicate uses the initial 448-bit bound, which is exactly what a fresh arena would carry. compress_fragment itself still applies the predicate, so the per-fragment kernels behave identically when they are called directly.

flowchart TD
    Input[Quality 0 fragment] --> Empty{Empty?}
    Empty -->|yes| Final[Write final empty marker and align]
    Empty -->|no| Bound{Serial, final, and length within command-code bound?}
    Bound -->|yes| Raw[Emit uncompressed block]
    Bound -->|no| Encode[Build codes and scan matches]
    Encode --> Guard{Compressed bits exceed size threshold?}
    Guard -->|yes| Raw
    Guard -->|no| Finish[Finalize if last]
    Raw --> Finish

5. Quality 1: two passes

sequenceDiagram
    participant Blk as block loop
    participant P1 as CreateCommands
    participant SC as ShouldCompress
    participant P2 as StoreCommands
    participant W as BitWriter

    loop each 128 KiB block
        Blk->>P1: scan, fill command and literal buffers
        P1-->>Blk: num_commands, num_literals
        Blk->>SC: literals < 98% of block, else sampled entropy
        alt compressible
            Blk->>W: meta-block header, 13 zero bits
            Blk->>P2: exact histograms, prefix codes, replay
            P2->>W: command bits, extra bits, literals
        else incompressible
            Blk->>W: uncompressed meta-block
        end
    end

The first pass stores each command as a packed u32: the low eight bits are the fast command code (0..128) and the high bits carry the extra value. The second pass builds exact literal and command histograms from those buffers, seeds command codes 1, 2, 64 and 84, and replays the buffer.

5.1. Post-copy hash updates

The pinned reference contains an asymmetry that changes the command stream: with min_match == 4, the first post-match update path hashes offsets 0, 1, 0 where the chained path hashes 0, 1, 2.

flowchart TD
    A["copy emitted"] --> B{min_match}
    B -->|4| C{first update after literals?}
    C -->|yes| D["offsets 0, 1, 0 from the word at ip - 3"]
    C -->|no| E["offsets 0, 1, 2 from the word at ip - 3"]
    B -->|6| F["offsets 0, 1, 2 from ip - 5,<br/>then 0, 1 from ip - 2"]
    D --> G["candidate = table[hash at offset 3]"]
    E --> G
    F --> H["candidate = table[hash at offset 2 of ip - 2]"]

update_hashes_after_copy takes a FIRST_UPDATE const parameter that reproduces it. A targeted unit test pins the difference so the quirk cannot be “fixed” by accident.

6. Bitstream layer

graph LR
    pos["position: bit index"] --> w["BitWriter"]
    w -->|write| store["storage[pos >> 3 .. +8] as one word"]
    w -->|update| patch["patch an already emitted MLEN field"]
    w -->|rewind| back["drop everything after a saved position"]
    w -->|align| byte["advance to the next byte boundary"]
    w -->|write_bytes| raw["copy an uncompressed meta-block verbatim"]

write materialises a whole 64-bit word, which is what clears the bits above the new position — the same trick the reference uses — so the buffer keeps eight bytes of headroom past the largest position the encoder reaches. Writes past the end of the buffer set an overflow flag instead of panicking, and the encoder turns that into BufferOverflow.

Meta-block layout for a compressed fast-path block:

6.1. Prefix-code construction

shared::huffman builds the prefix codes of every encoder, not only these two qualities. Both builders list the used symbols by descending index, order them least popular first, merge them (merge_nodes) and walk the tree for depths (set_depth); a code deeper than its limit raises the minimum count (count_limit, doubled per retry) and starts again. They differ in how equal counts are ordered, and so in how they sort:

flowchart TD
    H["histogram, count_limit"] --> W{"builder"}
    W -->|"create_huffman_tree (q0/q1 commands, q2-q11, code-length codes)"| C["count used symbols per bucket: exact below 64, one per power of two above"]
    C --> S{"fewer than 13 leaves?"}
    S -->|yes| I["list by descending symbol, insertion sort"]
    S -->|no| P["bucket starts from the counts; place leaves by descending symbol"]
    P --> B["sort_items::<true> inside each power-of-two bucket with 2+ leaves"]
    W -->|"build_and_store_huffman_tree_fast (q0/q1 literals, q2 fast codes)"| F["list by descending symbol, sort_items::<false>"]
    I --> M["merge_nodes, set_depth"]
    B --> M
    F --> M
    M -->|"too deep: count_limit *= 2"| H

• create_huffman_tree breaks ties by descending symbol, so its order is total and any sort that reaches it gives the reference’s bytes. sort_leaves reaches it with a bucket sort: one pass counts the leaves of each bucket, prefix sums turn the counts into starts, and a second pass places leaves by descending symbol, which leaves a single-count bucket in order. Only the power-of-two buckets, from count 64 up, are sorted afterwards, by the reference’s shell sort. One array holds both start and cursor: after placement, bucket b spans rank[b - 1]..rank[b]. Fewer than 13 leaves go straight to the reference’s insertion sort, where the bucket pass would cost more than it saves.
• build_and_store_huffman_tree_fast compares counts only, as the reference’s literal builder does. Shell sort is not stable, so its order among equal counts is whatever SortHuffmanTreeItems leaves, and that order decides which of two equal symbols gets the shorter code. That path keeps the reference’s shell sort unchanged.

The scans around the tree go through shared::scan: they compare sixteen entries at once at the target’s baseline SIMD level (shared::baseline: SSE2, NEON or scalar lanes, chosen at compile time, since this code runs outside the encoders’ dispatched entries) and walk the bit mask. The bucket sort’s two passes visit only used symbols (for_each_nonzero_u32, ascending and descending); convert_bit_depths_to_symbols collects the used symbols once and assigns codes from that list; write_huffman_tree finds run ends and the trailing zeros with run_end_u8 and trim_zeros_u8; optimize_huffman_counts_for_rle takes its used-symbol count, last used symbol and smallest count from one walk and its run lengths from run_end_u32. Each reports exactly what the plain loop would, in the same order, so the code lengths and bytes do not change; the tests run every scan on the scalar, baseline and detected levels against the plain loop.

flowchart LR
    T["u32 counts / u8 depths"] --> C["16 lanes: simd_eq(0 or value)"]
    C --> M["to_bitmask"]
    M -->|"all lanes set (dense)"| D["straight loop over the chunk"]
    M -->|sparse| W["tzcnt / lzcnt walk over set bits"]
    M -->|"run end"| E["first differing lane"]

6.2. Two-queue Huffman merge

shared::huffman::merge_nodes uses the caller’s node pool for both sorted leaves and newly built parents. The leaf queue starts at index 0, ends at n, and has a sentinel at tree[n]. The parent queue starts at n + 1 with another sentinel. Each iteration selects two children, writes their parent to the next parent slot, then writes the following sentinel. It returns root index 2 * n - 1; the pool needs 2 * n + 1 entries. The builders handle a single used symbol before calling this merge.

For each child selection, the merge loads the two queue-front counts and compares them with <=: equal counts select the leaf queue. It carries the selected index and its already loaded count together, then advances that queue. The second child is selected from the updated fronts. Writing the parent uses those carried counts, without loading the selected children from the pool again. The weight is left_count.wrapping_add(right_count), matching the reference’s unsigned 32-bit addition even when a histogram overflows. Child indices, sentinel positions, tie order and all later depth/code choices remain unchanged.

flowchart TD
    Start["Sorted leaves + leaf/parent sentinels"] --> Left["Load both front counts; select left index + count"]
    Left --> TieL{"leaf count <= parent count?"}
    TieL -->|yes| LeafL["Advance leaf queue"]
    TieL -->|no| ParentL["Advance parent queue"]
    LeafL --> Right["Load updated fronts; select right index + count"]
    ParentL --> Right
    Right --> TieR{"leaf count <= parent count?"}
    TieR -->|yes| LeafR["Advance leaf queue"]
    TieR -->|no| ParentR["Advance parent queue"]
    LeafR --> Store["Store parent: wrapping sum + carried child indices"]
    ParentR --> Store
    Store --> Sentinel["Write next parent sentinel"]
    Sentinel --> More{"More parents needed?"}
    More -->|yes| Left
    More -->|no| Root["Return root index 2*n - 1"]

The helper stays private, scalar and allocation-free; it adds no API, error or SIMD dispatch boundary. Its callers retain and grow the node workspace, and their depth-limit retry loop is unchanged. Differential tests compare the complete node pool with the original merge for tied, mixed and overflowing counts, including an unused suffix that must remain untouched.

7. SIMD dispatch points

graph TD
    select["Backend detected once, retained on construction"] --> A["FastEncoder<INDEPENDENT>::encode_block / encode_block_append"]
    A --> kernel["dispatch!(backend) once per block"]
    kernel -->|"#[simd] entry, no re-selection"| B["encode_fragment<S: Simd, INDEPENDENT>"]
    B --> C["q0::compress_fragment<S>"]
    B --> D["q1::compress_fragment<S>"]
    C --> E0["match_len_at: first word settled scalar"]
    D --> E0
    E0 -->|"first word equal"| E["match_len_windows<S> over the rest"]
    E --> F{"S::u8s::LEN"}
    F -->|16| G["u8x16 loop, stride 16"]
    F -->|32| H["u8x32 loop, stride 32"]
    F -->|64| I["u8x64 loop, stride 64"]
    F -->|other| J["byte scan, stride 1"]

    classDef scalar fill:#fce5cd,stroke:#b45f06;
    class C,D,J scalar;

encode_fragment, the specialized q0 compress_fragment_impl and the q1 create_commands are #[simd] functions: each body becomes an always-inlined closure called through a feature-enabled entry, with the arguments passed separately. This keeps the native comparison loop inside a feature-enabled function even when LLVM keeps the large scan separate from the outer dispatcher. It introduces no feature detection inside the scan and retains independent table-width specializations.

Only the exact match-length scan is vectorised. Hash lookups, candidate selection, skip logic, command encoding, the bit writer and the Huffman builder all stay scalar, in reference order. The lane count is resolved into a const parameter at monomorphisation time so the vector loop can split its windows with as_chunks and load them with load_array_ref, which leaves neither a bounds check nor a length assertion in the loop.

find_match_length is a four-stage pipeline: a 16-byte scalar word prefix, then native vectors, then whole words, then single bytes. The staging only changes how a length is discovered, never which length it is, so every backend emits identical bytes.

match_len_at, the entry point the q0, q1 and greedy scans call, settles the first word before any window is cut: it bounds the candidate window once, loads eight bytes from each side, and answers a difference inside them with a trailing-zero count — the reference’s whole scan for the short matches that dominate. Only a match running through the whole word pays for the slicing of the staged pipeline, which then continues from byte eight. q0 calls match_len_at_outlined, which keeps that first-word compare inline and calls the vector tail out of line, so q0’s fragment loop does not carry the long-scan code and its registers; q1 and the greedy scans keep the fully inlined form.

7.1. The stride invariant

Each stage decides “did every step match?” by comparing what the scan reported against the window rounded down to a whole number of steps, and returns early when the scan stopped short. That test is only exact when the size it rounds by is the step the scan really takes, so native_vector_stride reports the stride rather than the lane count: every width without a vector loop degrades to the byte scan, whose stride is one. Rounding by a lane count the scan does not use would round the window up past what the scan can report and truncate a match that ran to the limit. No backend fearless_simd ships reaches that arm — NEON, SSE2 and the fallback all have u8s::LEN == 16 — so it is an invariant kept honest rather than a live path.

7.2. Vector entry

The scalar prefix checks the first 16 bytes. Longer matching runs reach the vector stage, followed by a scalar tail at the first mismatch or input limit.

8. Table-bit specialisation

flowchart LR
    A["fragment length"] --> B["HashTableSize:<br/>256, doubling"]
    B --> C{quality}
    C -->|0| D["force an odd shift:<br/>9, 11, 13, 15"]
    C -->|1| E["8 .. 17"]
    D --> F["q0::TableBits"]
    E --> G["q1::TableBits"]
    F --> H["compress_fragment_impl&lt;S, TABLE_BITS, ENTRIES&gt;(table: &amp;mut [i32; ENTRIES])"]
    G --> I["compress_fragment_impl&lt;S, TABLE_BITS, ENTRIES, MIN_MATCH&gt;(table: &amp;mut [i32; ENTRIES])"]

The table width and, for quality 1, the minimum match length are const parameters, so the shift and the match predicate are compile-time constants inside the hot loop. The table itself is handed to the specialised body as an array reference &mut [i32; 1 << TABLE_BITS] (first_chunk_mut on the prepared slice), so every hash index — a 64 - TABLE_BITS shift — is provably inside it and no lookup carries a bounds check. The array type preserves this bound inside the backend-specific entry #[simd] generates.

Inside q1’s create_commands the pass-one command and literal vectors are moved into locals for the duration of the scan and moved back at its end: behind the caller’s references their headers live in memory the compiler cannot tell apart from the table, so every push would reload the length and capacity after each table store.

9. Data-processing loops

Bulk loops are written as chunked iterators rather than indexed loops, which removes their bounds checks without any unsafe code:

9.1. Literal packing width

The tree builder caps a literal depth at fourteen bits, so four codes are at most 56 bits and always fit one BitWriter::write; two fit with room to spare. Quality 0 emits literals between matches. Runs of at least eight literals use quadruples, followed by pairs and singles; shorter runs start with pairs. Quality 1 replays a meta-block’s literals in quadruples too (pack_literals), then writes the up to three left over in one more call. It used to accumulate codes until the next would overflow 56 bits; that per-literal test cost more than the calls it saved, about 10% of quality 1 on binary input, where most bytes are literals.

flowchart LR
    R["literal run"] --> Q{"4 or more left?"}
    Q -->|yes| F["prefix-sum 4 widths, OR 4 codes, one write (<= 56 bits)"]
    F --> Q
    Q -->|no| T["OR the 0-3 left, one write (<= 42 bits)"]

Flushing

FastEncoder::flush_block is the fast path’s BROTLI_OPERATION_FLUSH. These qualities already close a meta-block on every call, so the flush is only the empty metadata block that pushes the stream back onto a byte boundary — and, when there is buffered input, a short non-final fragment ahead of it. An empty input skips the fragment entirely, exactly as the reference does when a flush arrives with nothing buffered, and nothing at all is emitted when the stream was already aligned. See compressor flushing.

Quality 1 emits a code description per meta-block. Frequent flushes create smaller blocks and can substantially increase compressed size.

Known gaps

• No attached prefix. These qualities carry no compound-dictionary search in the reference either, so a PreparedDictionary is refused rather than ignored.
• No static dictionary matching, no block splitting and no context modelling: out of scope for both fast qualities, as in the reference.
• Every word load in the scan is bounds-checked. The reference reads eight bytes at the hash position, the repeat candidate and the table candidate unchecked; the port proves each load with a compare, and the compiler cannot fold them against the loop’s ip_limit, which holds ip + 16 inside the input by construction. Carrying the loaded word between iterations removes one of the four; the scan on binary input still executes about a third more instructions per position than the C build and measures 85% of it, against 96–99% on incompressible input.
• ShouldMergeBlock and ShouldCompress use floating point. They reproduce the reference’s double arithmetic, including its single-precision log2 table, because the decisions are observable in the output.
• The command prefix code carries stale depths for symbols with a zero count. create_huffman_tree writes depths only for symbols that appear, exactly as the reference does; the seed histogram guarantees every symbol that can be emitted has a non-zero count.

Independent parallel fragments

begin_fragment removes the standalone header and fragment_aligned checks the flush boundary. The encoder’s const parameter INDEPENDENT (FastEncoder<true>) specializes q0/q1 command generation: independent fragments emit full distances, including the remainder after an insert command’s two-byte copy. q0 seeds the otherwise unused short-copy symbols; q1 preserves the explicit copy-two symbol during Huffman alphabet expansion. Serial specializations retain their original trees and cache coding. See parallel compression for prefix ownership, aligned assembly, and the minimized regressions.