{"id":973,"date":"2026-05-25T06:51:05","date_gmt":"2026-05-24T22:51:05","guid":{"rendered":"https:\/\/connectword.dpdns.org\/?p=973"},"modified":"2026-05-25T06:51:05","modified_gmt":"2026-05-24T22:51:05","slug":"stepfun-releases-stepaudio-2-5-realtime-an-end-to-end-voice-model-with-roleplay-specific-rlhf-and-paralinguistic-comprehension","status":"publish","type":"post","link":"https:\/\/connectword.dpdns.org\/?p=973","title":{"rendered":"StepFun Releases StepAudio 2.5 Realtime: An End-to-End Voice Model with Roleplay-Specific RLHF and Paralinguistic Comprehension"},"content":{"rendered":"

StepFun, the Shanghai-based AI lab, released StepAudio 2.5 Realtime. It is an end-to-end real-time speech large language model with fully customizable persona capabilities. <\/p>\n

StepAudio 2.5 Realtime is a voice model that operates in real time. Unlike pipeline-based systems that separate speech recognition, reasoning, and synthesis into sequential steps, this is an end-to-end model. Audio goes in and audio comes out through a single unified system. The model supports Chinese and English.<\/p>\n

It connects via a WebSocket API. The endpoint is wss:\/\/api.stepfun.com\/v1\/realtime<\/code> using the model string step-2.5-realtime<\/code>. <\/p>\n

The Three Technical Pillars<\/strong><\/h2>\n

StepFun research team describes three core architectural innovations behind the model:<\/strong><\/p>\n

1. Million-Scale Persona Data Augmentation<\/strong><\/h4>\n

Starting from 10,000+ high-quality natively authored personas, StepFun applied algorithmic augmentation to build a million-scale persona feature matrix. This was combined with millions of real-world conversational samples for training. The intent is generalization \u2014 specifically, stable performance on difficult, long-tail conversational topics. <\/p>\n

Instead of manually labeling millions of persona samples, StepFun team used algorithmic expansion from a curated seed set.<\/p>\n

2. Roleplay-Specific RLHF Alignment<\/strong><\/h4>\n

A known failure mode in conversational AI is \u201cout-of-character\u201d (OOC) behavior \u2014 when a model drifts away from its defined persona mid-conversation. StepFun team conducted dedicated RLHF (Reinforcement Learning from Human Feedback) optimization specifically for persona consistency in roleplay scenarios. RLHF is a training technique where human preference signals are used to train a reward model, which then guides language model behavior. Applying it specifically to roleplay stability is a targeted design choice.<\/p>\n

3. Unified Speech Understanding and Generation<\/strong><\/h4>\n

StepAudio 2.5 Realtime inherits the StepAudio 2.5 TTS capabilities and deeply fuses speech understanding and generation through reinforcement learning. This enables what StepFun calls \u201cglobal scene-level tonal setting\u201d and \u201cintra-sentence detail sculpting.\u201d The model can set an overall emotional register for a response while adjusting finer acoustic details within individual sentences. <\/p>\n

Paralinguistic Understanding<\/strong><\/h2>\n

A technically distinct area of this model is paralinguistic perception. Paralinguistics refers to non-verbal acoustic information in speech \u2014 things like tone, speaking rate, pauses, sighs, and laughter. By analyzing these elements, the model can perceive the user\u2019s mood and underlying intentions. For example, it can identify fatigue from a low tone or frustration from a rapid speech rate. Capturing these signals requires the model to operate on audio features rather than transcribed text alone.<\/p>\n

StepAudio 2.5 Realtime scored 82.18 on the paralinguistic comprehension benchmark, demonstrating perception of vocal speed, emotion, age, and other acoustic features.<\/p>\n

\n
\"\"
https:\/\/stepaudiollm.github.io\/step-audio-2.5-realtime\/<\/figcaption><\/figure>\n<\/div>\n

Benchmark Results<\/strong><\/h2>\n

StepFun research team conducted a comprehensive suite of subjective and objective evaluations, benchmarking StepAudio 2.5 Realtime against leading real-time voice models across five dimensions. <\/p>\n

Human evaluation is conducted through real mobile app conversations scored by human raters. The scores:<\/strong><\/p>\n