Cranelift Pulley ISA assertion failure: exception_payload_regs ignores the calling convention

This crash is triggered by a mismatch between the calling convention used by a function signature and the register set returned by Pulley’s exception_payload_regs. When a try_call-style path expects exception payload values in convention-specific registers but the ISA helper returns a hard-coded or convention-agnostic register list, Cranelift eventually hits an internal assertion failure during verification or lowering.

Understanding the Root Cause

The issue appears when the Pulley ISA provides exception payload registers without consulting the active CallConv. In Cranelift, register assignment is not universally identical across conventions. A function declared with winch, system_v, or another convention may require different ABI behavior for arguments, returns, and exceptional control flow payloads.

In the failing case, the test uses a signature like this:

test compile
target pulley64

function %trigger_bug(i32) -> i32 {
    sig0 = (i32) -> i32 winch
    fn0 = %callee(i32) -> i32 winch

    block0(v0: i32):
        ; try_call ...

The critical detail is winch. If Pulley’s exception register lookup assumes the default ABI instead of the signature’s ABI, then the registers assigned to the exception path no longer match the rest of the call-lowering logic. That inconsistency can surface as:

• an assertion that expected registers and actual registers differ,
• incorrect value mapping for exception payloads, or
• downstream lowering failures because the machine environment becomes internally inconsistent.

Technically, this happens because exception payload handling is part of the ABI contract. If exception_payload_regs ignores the calling convention, Cranelift may lower the normal call with one convention and the exception continuation with another implicit convention. The result is a broken invariant: all register decisions for a call site must be derived from the same ABI rules.

In short, the root cause is not the .clif syntax itself. The real bug is that the Pulley backend returns exception payload registers in a way that is not parameterized by the call convention, even though the call site clearly is.

Step-by-Step Solution

The fix is to make Pulley’s exception payload register selection call-convention aware and ensure every caller passes the active convention through the relevant ABI path.

1. Find the Pulley ABI helper that returns exception payload registers

Look for code in the Pulley ISA backend or ABI layer with a shape similar to one of these patterns:

fn exception_payload_regs(...) -> ...

fn get_exception_payload_registers(...) -> ...

impl TargetIsa for Pulley { ... }

If the function currently returns a fixed register list without reading a CallConv or ABI-specific signature object, that is the bug.

2. Change the function signature to accept the active calling convention

If the current implementation looks conceptually like this:

fn exception_payload_regs() -> [Reg; 2] {
    [X0, X1]
}

Refactor it so the convention is explicit:

fn exception_payload_regs(call_conv: CallConv) -> [Reg; 2] {
    match call_conv {
        CallConv::Winch => {
            // return the registers defined for Winch exception payloads
            [REG_A, REG_B]
        }
        CallConv::SystemV | CallConv::Fast | CallConv::Cold | CallConv::Tail => {
            // return the registers used by Pulley for these conventions
            [REG_X, REG_Y]
        }
        _ => {
            // use the correct fallback for Pulley, not a generic hard-coded default
            [REG_X, REG_Y]
        }
    }
}

The exact register names depend on Pulley’s register file and existing ABI definitions. The important part is that the helper must derive its result from the same convention used when lowering the call.

3. Thread the calling convention through all call sites

Any code that handles exceptions, try_call, unwind edges, or payload extraction must pass the active signature convention into the helper. For example:

let call_conv = sig.call_conv;
let payload_regs = exception_payload_regs(call_conv);

If the code previously did something like this, update it:

let payload_regs = exception_payload_regs();

to this:

let payload_regs = exception_payload_regs(sig.call_conv);

That ensures normal returns and exceptional returns follow the same ABI contract.

4. Reuse existing ABI classification logic if available

If Pulley already has central ABI logic for argument and return registers, do not duplicate convention-specific behavior in a second place. Instead, route exception payload register selection through the same ABI decision layer:

fn exception_payload_regs(call_conv: CallConv) -> [Reg; 2] {
    pulley_abi::exception_payload_for(call_conv)
}

This reduces drift and prevents future mismatches where one convention gets updated for normal calls but not for exception paths.

5. Add or update regression tests

Create a regression test that specifically uses a non-default calling convention such as winch together with a throwing or exceptional call path.

test compile
target pulley64

function %trigger_bug(i32) -> i32 {
    sig0 = (i32) -> i32 winch
    fn0 = %callee(i32) -> i32 winch

    block0(v0: i32):
        ; invoke or try_call path that exercises exception payload handling
        ; ensure lowering completes without assertion failure
}

Also add a control test for the default convention so you verify both:

• the original ABI path still works, and
• the Winch ABI now uses the correct exception payload registers.

6. Validate with a full backend test run

After patching the helper and its callers, run the relevant test suites for Cranelift and Pulley. Typical validation should include:

cargo test -p cranelift-codegen

cargo test -p cranelift-filetests

If you have a focused filetest command in your workflow, run the specific regression as well. The success criteria are:

• no assertion failure during compile,
• identical or expected register allocation for non-exception paths, and
• correct lowering for exception payload values under winch.

7. Keep the ABI invariant explicit in comments

This bug is easy to reintroduce later. Add a short code comment near the helper explaining that exception payload registers are ABI-dependent:

// Exception payload registers must be selected from the active calling
// convention. Do not return a convention-agnostic register set here.

That small note can prevent future refactors from accidentally restoring the broken behavior.

Common Edge Cases

• Default convention appears to work: This bug can hide for a long time if most tests use the default calling convention. The failure only becomes visible when a different ABI like winch is used.
• Normal calls succeed, exception paths fail: Because the mismatch is localized to exceptional control flow, standard call/return tests may all pass while try_call or unwind tests crash.
• Multiple backends share assumptions: If Pulley copies logic from another ISA but does not adapt convention handling, the register mapping may be accidentally correct for one ABI and wrong for another.
• Verifier failures instead of assertions: Depending on where the mismatch surfaces, you may get verifier complaints about value locations rather than a direct backend assertion.
• Signature-level versus function-level convention: Make sure the code reads the convention from the correct source. Exception payloads should follow the convention of the actual call signature being lowered, not a global backend default.
• Future ABI extensions: If new calling conventions are added later, a wildcard fallback may silently choose the wrong payload registers again. Prefer exhaustive handling where practical.

FAQ

Why does this bug only show up with winch in the test case?

Because winch exercises a calling convention path that differs from the backend’s implicit default. If exception_payload_regs returns registers as though the default ABI were in use, the mismatch becomes visible as soon as lowering reaches the exception path.

Is the problem in the .clif file or in the Pulley backend?

The .clif file is only exposing the bug. The underlying defect is in the Pulley backend ABI logic, specifically the fact that exception payload registers are selected without respecting the active calling convention.

What is the safest long-term fix?

The safest fix is to centralize ABI decisions so arguments, returns, and exception payloads all derive from the same call-convention-aware layer. That prevents drift and makes new calling conventions much easier to support correctly.

Once Pulley’s exception_payload_regs is driven by the active CallConv, the assertion failure disappears because exceptional control flow once again uses the same ABI model as the call it belongs to.