Chapter 2. New features and enhancements

A new migration guide, Assess and plan for migration from Red Hat OpenShift AI 2.25.9 and later releases in the 2.25 series to 3.5, is now available. This guide walks administrators through the pre-upgrade migration assessment process by using the rhai-cli command-line tool, including guidance for side-by-side and in-place migration approaches, Kueue management-state prerequisites, and component-specific rhai-cli migrate actions for Kueue/RHBOK, AI Pipelines, Model Serving, Workbenches, TrustyAI, Training, Llama Stack/OGX, and Ray.

The existing 3.3 migration guide remains published for customers migrating to 3.3. This new guide is a parallel edition for customers migrating to OpenShift AI 3.5.

After upgrading to 3.5, move away from the support-required-upgrade-3.5 update channel that was used to trigger the migration. Select an appropriate supported update channel for your environment, such as stable-3.5, stable-3.x, or eus-3.5, according to your update and lifecycle requirements.

For information about available update channels and their lifecycle, see Red Hat OpenShift AI Self-Managed Life Cycle.

You can configure Models-as-a-Service to authenticate users with an external OpenID Connect (OIDC) identity provider. This feature enables enterprise-wide access to large language models without requiring OpenShift accounts for every user.

Key capabilities include:

OpenShift AI includes the EvalHub client SDK and command-line interface (CLI). The EvalHub SDK provides a Python client library and CLI for integrating evaluation workflows into development environments, automation pipelines, and notebook-based experimentation.

This introduces the following capabilities:

New models added to the Red Hat AI validated models catalog now undergo automated adversarial vulnerability scanning as part of the validation process. This extends the existing validation pipeline with a behavioral security assessment of each model.

The scans use garak, an open-source LLM vulnerability scanner, to probe each model’s responses against a broad range of adversarial attack patterns.

Scans are run by the Red Hat model validation team by using Red Hat EvalHub. EvalHub provides a consistent, repeatable validation methodology across newly validated models. The resulting quantitative vulnerability scores are published with each model entry in the Hugging Face catalog, alongside existing accuracy and performance data.

Vulnerability scores provide quantitative insight into a model’s security posture, supporting model selection and providing input to internal model risk management and AI security review processes.

Feast Feature Store supports a SparkApplication batch engine. You can run feature transformations on your own Apache Spark clusters managed by the Kubeflow Spark Operator. You can configure batch_engine.type: spark_application in your featurestore.yaml to submit materialization jobs as SparkApplication custom resources for distributed batch materialization and Spark user-defined function (UDF) execution in feature views. This capability supports enterprise retrieval-augmented generation (RAG) pipelines by providing batch document embedding at scale through Spark, with materialization to both offline and online feature stores including vector-enabled stores such as Milvus.

For more information, see SparkApplication batch engine in Feature Store.

When Kueue manages workload scheduling in a project, the workbenches overview page displays Kueue-derived scheduling states for each workbench. You can view states such as Queued, Starting, Preempted, Evicted, and Requeued, with human-readable messages that explain the current scheduling status.

If you have RBAC access to the Kueue Visibility API, you can see your queue position for pending workloads. The startup progress modal uses a tree-based view that shows per-container startup steps and a Kueue admission sub-step.

This release includes the following capabilities:

Distributed Inference with llm-d is generally available on Azure Kubernetes Service (AKS), CoreWeave Kubernetes Service (CKS), and OpenShift. This release supports multi-model serving, intelligent inference scheduling, and disaggregated serving to improve GPU utilization for generative AI models.

Additionally, Distributed Inference with llm-d includes Istio as the supported gateway implementation and features the Gateway API Inference Extension (GAIE). Both of these components are fully supported as part of the solution.

Flow control for Distributed Inference with llm-d is generally available. Platform Operators can define priority tiers using InferenceObjective resources, configure saturation detection, and apply per-band queuing policies to ensure that latency-sensitive interactive requests are served ahead of throughput-oriented batch traffic. Multiple workload profiles can share a single Distributed Inference with llm-d deployment without requiring dedicated GPU pools per priority level. Additionally, flow control features starvation protection through holdback policies, per-band capacity limits, and sheddable request eviction.

If you configured flow control during the Technology Preview in Red Hat OpenShift AI 3.4, note the following breaking changes:

With this update, you can use controlled deployment with Distributed Inference with llm-d to validate engine upgrades, model version rotations, and configuration changes on a fraction of production traffic before a full promotion. You can deploy two or more versions of an inference workload side-by-side on the same model endpoint, control the traffic distribution using declarative, weight-based traffic splitting, and compare per-version Prometheus metrics in real time. Active requests complete without disruption during promotions and rollbacks.

For more information, see Validate inference workload changes with controlled deployment.

You can automate the optimization of prompts for your agentic systems by using Training Hub. By providing a seed prompt and an evaluation dataset, the automated workflow systematically evolves and optimizes prompts using evolutionary search and LLM-driven reflection. This capability replaces manual, trial-and-error prompt engineering, allowing you to achieve production-quality agent performance without modifying model weights. The optimization process provides visibility into how specific prompt elements affect performance across iterations, and it is compatible with both hosted and self-hosted models.

For more information about Training Hub, see Train the model by using your prepared data.

Namespace administrators can deploy a dedicated EvalHub instance in their own namespace without cluster administrator or OpenShift AI administrator involvement. Set spec.tenancy: single in the EvalHub custom resource to deploy a per-tenant instance. The TrustyAI Operator automatically provisions all required RBAC resources, including Roles, ServiceAccounts, and RoleBindings.

Namespace administrators can also define tenant-scoped evaluation providers and collections by creating labeled ConfigMap resources in the instance namespace. Tenant-scoped configurations appear alongside system configurations and do not affect other tenants or the shared system configuration.

Administrators can configure OAuth proxy sidecar resource requests and limits directly in the DataScienceCluster CR under spec.components.kserve.oauthProxy.resources, without changing any component state from Managed to Unmanaged. You can explicitly set the fields to override the defaults. The default value persists when you omit the field.

Before this update, adjusting OAuth proxy sidecar CPU and memory for KServe InferenceServices required manually editing the inferenceservice-config ConfigMap and setting opendatahub.io/managed: "false" to prevent the Operator from overwriting the changes. This unmanaged state meant configuration mistakes were not corrected by reconciliation, risking production stability, and required manual intervention during upgrades.

The Red Hat AI Model Catalog includes a new Safety and Security Insights tab that displays AI security evaluation results for each model. The tab shows security scan results organized by category, including safety testing and security testing. These results cover risk vectors such as prompt injection, jailbreak resistance, and harmful content generation. As a result, you can evaluate a model’s security posture directly from the catalog without conducting independent security evaluations.

To access the Safety and Security Insights tab, open the Red Hat AI Model Catalog, select a model, and click the Safety and security insights tab on the model detail page.

The model catalog displays operational metrics for validated models, including cold-start load time, minimum vRAM requirements, and the runtime command used for benchmarking. Cold-start load time and runtime commands are shown per GPU configuration in the Performance Insights tab, while minimum vRAM and container size are displayed in the Model Details section.

You can filter and sort models by cold-start load time to identify models that meet specific startup latency requirements, and filter by minimum vRAM and container size. The cold-start load time metric measures only the vLLM model load phase after weights are downloaded and copied into the container, not the full end-to-end startup time.

With this update, you can view your Models-as-a-Service subscription assignments, browse associated models, and check token rate limits from the Subscriptions tab on the API keys page in Gen AI studio.

You can toggle between a subscription-grouped view and a model-grouped view, expand rows to see details, search by name, and sort alphabetically. You can also click an individual subscription to view its details, including the models available, their token rate limits, and any API keys you have for that subscription.

For more information, see View your Models-as-a-Service subscriptions.

Red Hat OpenShift AI 3.5 retains the 2025.2 workbench and pipeline runtime images alongside the new images that default to the Red Hat Python index. In the dashboard workbench creation flow, 2025.2 workbench images remain selectable but are marked as outdated and not recommended. In JupyterLab, the corresponding 2025.2 pipeline runtime images for Data Science, Minimal, PyTorch, PyTorch LLM Compressor, TensorFlow, and ROCm variants are still available for pipeline nodes.

The 2025.2 images are supported on both fresh 3.5 installations and upgrades from 3.4, including disconnected environments. This transition period allows you to validate your existing workflows against the new images before you migrate.

You can use a PersistentVolumeClaim (PVC) as a storage source for custom test data in EvalHub evaluation jobs. When you specify a PVC in the test_data_ref field of a benchmark configuration, EvalHub mounts the PVC read-only at /test_data inside the evaluation job pod. The adapter reads the data directly from that path without an init container or S3 credentials.

You can specify a sub_path to mount a specific subdirectory within the PVC, for example, when a single PVC contains data for multiple benchmarks.

For more information, see Provide evaluation test data from a PVC.

The universal workbench image includes all dependencies required for Reinforcement Learning from Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO) training workflows. These dependencies, provided by Training Hub, ensure out-of-the-box compatibility on CUDA backends, with ROCm and CPU image variants also available. With this update, you can immediately begin developing and experimenting with single-GPU GRPO (using Kubeflow TrainJob) without manual dependency installation or environment setup.

Multi-GPU distributed GRPO (using KubeRay RayJob) can also be achieved with the new Training Hub Ray runtime image, allowing you to immediately begin developing without manual dependency installation or environment setup.

For more information, see the guided examples for GRPO with Trainer and GRPO with Ray.

You can send inference requests to the standard OpenAI /v1/chat/completions endpoint with the model name in the request body, and MaaS applies subscription, rate-limiting, and authorization policies automatically. This enables drop-in compatibility with OpenAI-compatible SDKs and clients such as the Python openai library, LangChain, LlamaIndex, and OpenWebUI without requiring custom URL paths or non-standard headers. Legacy path-based routing remains supported for backward compatibility.

For more information, see Models-as-a-Service API overview.

This enhancement combines the Subscriptions and Authorization policies pages in Settings into a single MaaS governance page, accessible from Settings MaaS governance. The unified page uses a tabbed layout with Subscriptions and Authorization policies tabs, allowing administrators to manage both resources from one location. The subscription and authorization policy tables include expandable rows for the Groups and Models columns. You can click these columns to view group names, model names, and token limits inline.

For more information, see View subscriptions and View authorization policies.