Cranelift riscv64 Compile Failure with sext/uext on Floating-Point Arguments: Root Cause and Fix

This failure happens because the compiler pipeline is trying to apply integer argument extension semantics to a floating-point value, which is invalid for the affected backends. On riscv64 and similarly on s390x, a call site that marks a floating-point argument with sign extension (sext) or zero extension (uext) can survive too far into lowering, where backend code expects extensions to apply only to integer-like types. The result is a backend compile failure instead of a clean validation error or normalized IR.

Understanding the Root Cause

In Cranelift, argument extension flags such as sext and uext describe how a value narrower than the machine calling convention width should be passed. That makes sense for integer arguments like i8, i16, or i32 when the ABI requires widening. It does not make sense for f32 or f64, because floating-point values are not widened through integer sign or zero extension rules.

The bug appears when one of these invalid extension markers is attached to a floating-point argument in the IR or imported signature metadata. Instead of rejecting or stripping the invalid flag early, the compiler allows it to reach the backend lowering phase. Backends like riscv64 and s390x contain logic that assumes extension flags only appear on integer-compatible values. Once that assumption is broken, instruction selection or ABI argument assignment can panic, assert, or fail compilation.

Technically, the failure is usually rooted in one of these layers:

• IR construction: a signature parameter is created with ArgumentExtension::Sext or ArgumentExtension::Uext even though its type is floating-point.
• Verifier gap: the verifier does not reject the invalid combination early enough.
• ABI legalization gap: signature normalization does not clear or error on impossible extension flags for float parameters.
• Backend assumption: target-specific lowering code trusts the signature metadata and crashes when it encounters an illegal state.

The correct behavior is one of two things: either reject the invalid IR during verification, or sanitize the signature before backend lowering so that floating-point arguments never carry integer extension attributes.

Step-by-Step Solution

The most reliable fix is to enforce a rule in verification and ABI preparation: only integer scalar parameters may use sext/uext. Here is a practical implementation strategy for Cranelift maintainers or contributors debugging the issue.

1. Reproduce the failure with a minimal test

Create a regression test that passes a floating-point argument with an extension attribute. The exact syntax depends on the test harness, but the idea is to encode an invalid signature and ensure the compiler does not crash.

; Pseudocode / illustrative CLIF-style example only conceptually showing the bug shape
sig0 = (f32 uext) -> i32
fn %caller() -> i32 {
  v0 = f32const 1.0
  v1 = call %callee(v0)
  return v1
}

If the parser does not allow this exact syntax, construct the signature programmatically in a unit test instead.

2. Guard signature creation or verification

Add validation where function signatures or call signatures are verified. The rule should reject extension flags on anything other than integer scalar arguments.

fn verify_param_extension(param: &AbiParam) -> Result<(), VerifierError> {
    match param.extension {
        ArgumentExtension::None => Ok(()),
        ArgumentExtension::Uext | ArgumentExtension::Sext => {
            if param.value_type.is_int() {
                Ok(())
            } else {
                Err(VerifierError::InvalidArgumentExtension {
                    ty: param.value_type,
                    extension: param.extension,
                })
            }
        }
    }
}

If Cranelift already has a verifier pass for ABI parameters, place the check there so the invalid state is blocked early and consistently across all architectures.

3. Add a defensive normalization layer in ABI legalization

Even with verifier coverage, defensive backend-independent normalization helps avoid crashes when malformed IR enters through tests, fuzzing, or external embedding layers.

fn normalize_abi_param(param: &mut AbiParam) -> Result<(), CodegenError> {
    if !param.value_type.is_int() {
        match param.extension {
            ArgumentExtension::Sext | ArgumentExtension::Uext => {
                return Err(CodegenError::UnsupportedSignature(
                    "integer extension on floating-point argument"
                ));
            }
            ArgumentExtension::None => {}
        }
    }
    Ok(())
}

This ensures that even if invalid metadata escapes verification, the compiler fails gracefully with a clear error message instead of backend-specific breakage.

4. Audit backend lowering assumptions

Search for places in the riscv64 and s390x backend implementations that match on parameter extension flags without checking the value type first. These call paths often live near ABI argument assignment, call lowering, or machine signature translation.

match (param.value_type, param.extension) {
    (ty, ArgumentExtension::None) => lower_normally(ty),
    (ty, ArgumentExtension::Sext) if ty.is_int() => lower_sext(ty),
    (ty, ArgumentExtension::Uext) if ty.is_int() => lower_uext(ty),
    (_, ArgumentExtension::Sext | ArgumentExtension::Uext) => {
        return Err(CodegenError::UnsupportedSignature(
            "non-integer argument extension encountered during lowering"
        ));
    }
}

This step is important because backend assertions should not be the first line of defense.

5. Add architecture regression tests

Once the fix is in place, add tests that prove the issue is handled for both target backends named in the report.

#[test]
fn reject_float_arg_with_sext_on_riscv64() {
    let isa = isa_for("riscv64");
    let sig = make_sig_with_float_param_and_sext();
    let err = compile_with_sig(isa, sig).unwrap_err();
    assert!(err.to_string().contains("argument extension"));
}

#[test]
fn reject_float_arg_with_uext_on_s390x() {
    let isa = isa_for("s390x");
    let sig = make_sig_with_float_param_and_uext();
    let err = compile_with_sig(isa, sig).unwrap_err();
    assert!(err.to_string().contains("argument extension"));
}

6. If you are consuming Cranelift, fix the IR producer too

If your project generates Cranelift IR or imports signatures from another compiler layer, make sure you never annotate f32 or f64 parameters with integer extension flags. A safe mapping rule is:

• Integer narrow arguments: may use sext or uext when required by ABI or frontend semantics.
• Floating-point arguments: always use no extension.
• Vector arguments: do not reuse integer extension metadata unless explicitly supported by the ABI model.

fn map_frontend_param_to_abi(ty: Type, ext: FrontendExt) -> AbiParam {
    let mut param = AbiParam::new(ty);
    param.extension = if ty.is_int() {
        match ext {
            FrontendExt::SignExtend => ArgumentExtension::Sext,
            FrontendExt::ZeroExtend => ArgumentExtension::Uext,
            FrontendExt::None => ArgumentExtension::None,
        }
    } else {
        ArgumentExtension::None
    };
    param
}

7. Commit message guidance

A good patch summary for this issue would clearly state the invariant being restored:

cranelift: reject sext/uext on floating-point ABI params

Argument extension flags are only valid for integer scalar arguments.
Reject invalid float param extensions during verification and add
backend-independent guards to avoid riscv64/s390x lowering failures.

Common Edge Cases

• Imported LLVM-style signatures: Some tooling layers preserve calling convention attributes mechanically. If those attributes are copied onto float arguments, Cranelift must reject them before lowering.
• Bitcasts hiding intent: If a frontend bitcasts a float to an integer and then passes it with extension, that is a different case. The type at the ABI boundary matters. A true integer argument can be extended; a float cannot.
• Variadic or platform-specific ABI paths: Some backends have separate code for variadic calls or special calling conventions. Apply the same validation there.
• Return values: If similar extension metadata exists for returns, validate those too. The same type rule should hold unless the IR explicitly defines otherwise.
• Vectors and aggregates: Do not assume the fix is only about floats. Any non-integer scalar type carrying integer extension metadata should be reviewed.
• Fuzz-generated malformed IR: Fuzzers often produce impossible combinations. Verification errors are better than backend crashes and make fuzzing more productive.

FAQ

Can sext or uext ever be valid for floating-point arguments?

No. These are integer extension operations used to widen smaller integer values according to ABI or frontend requirements. Floating-point arguments follow different representation and calling convention rules.

Why does this fail on riscv64 and s390x specifically?

Those backends expose the bug because their lowering paths assume the signature metadata is already valid. Other architectures may reject it earlier, ignore it accidentally, or simply not hit the same assertion path. The real issue is target-independent: invalid extension metadata should never survive to backend lowering.

Should the fix be in the verifier, the frontend, or the backend?

Ideally in all three layers defensively, but the most important fix is in the verifier or ABI normalization layer. The frontend should avoid generating invalid signatures, and the backend should still fail gracefully if malformed IR slips through.

Once this invariant is enforced, Cranelift should stop crashing on this class of signatures and instead provide a predictable, diagnosable error path for invalid ABI metadata on Cranelift/Wasmtime-related codegen flows.