Chapter 4. Version 3.5.0-ea.1 release notes

AMD ROCm deployments on MI300X encounter memory access faults during CUDA graph capture.

When running models such as meta-llama/Llama-3.2-1B-Instruct or ibm-granite/granite-3.2-2b-instruct on AMD MI300X accelerators, GPU memory access faults occur with the error "Memory access fault by GPU node-2". This issue is caused by an incompatibility between vLLM CUDA Graph FULL mode and MI300X VF memory behavior under ROCm.

To work around this issue, set cudagraph_mode=PIECEWISE in your vLLM configuration:

--compilation-config '{"cudagraph_mode":"PIECEWISE"}'

Phi-4 Vision model fails to load with SigLIP2 compatibility error.

The RedHatAI/Phi-4-reasoning-vision-15B model requires a filter_out_non_signature_kwargs() function that is missing from the transformers library bundled with vLLM. The vLLM API server crashes at startup and cannot serve the model.

Sarvam-105B model fails to load with RoPE validation error.

The RedHatAI/sarvam-105b-FP8-Dynamic model configuration calls validate_rope(ignore_keys=…​), but the ignore_keys parameter was removed in Transformers v5. The vLLM API server crashes during model initialization.

AMD Zen CPU deployments crash due to zentorch plugin incompatibility.

The zentorch vLLM plugin supports vLLM versions 0.15.0 through 0.18.0, but Red Hat AI Inference 3.5.0-ea.1 ships vLLM 0.19.1. The engine crashes during model loading with a segfault.

To work around this issue, unset the VLLM_PLUGINS environment variable:

unset VLLM_PLUGINS

Structured output generation fails on IBM Z when using the llguidance backend.

When using the llguidance structured decoding backend on IBM Z (s390x), JSON Schema-constrained generation may produce invalid output or become stuck generating whitespace indefinitely.

Gemma 4 12B model deployment might fail due to unsupported gemma4_unified architecture in the bundled Transformers version.

Gemma 4 12B models use the gemma4_unified architecture, which is not supported by the version of Transformers included in the Red Hat AI Inference container image. Deployment of the google/gemma-4-12B-it model fails during vLLM initialization with the error "The checkpoint you are trying to load has model type gemma4_unified but Transformers does not recognize this architecture." Other Gemma 4 variants deploy successfully because they use different model architectures that are already supported.