What MOSS-Audio Actually Does<\/strong><\/h3>\nMOSS-Audio supports speech understanding, environmental sound understanding, music understanding, audio captioning, time-aware QA, and complex reasoning<\/strong> over real-world audio. Its capability set breaks down into several distinct areas. Speech & Content Understanding<\/strong> accurately recognizes and transcribes spoken content, supporting both word-level and sentence-level timestamp alignment. Speaker, Emotion & Event Analysis<\/strong> identifies speaker characteristics, analyzes emotional states based on tone, timbre, and context, and detects key acoustic events within the audio. Scene & Sound Cue Extraction<\/strong> pulls meaningful signals from background sounds, environmental noise, and non-speech signals to infer scene context and atmosphere. Music Understanding<\/strong> analyzes musical style, emotional progression, and instrumentation. Audio Question Answering & Summarization<\/strong> handles questions and summaries across speech, podcasts, meetings, and interviews. Finally, Complex Reasoning<\/strong> performs multi-hop reasoning over audio content, powered by both chain-of-thought training and reinforcement learning.<\/p>\nIn practical terms, a single MOSS-Audio model can do all of the above without switching between different specialized systems.<\/p>\n
Four Model Variants<\/strong><\/h3>\nThe team released four variants at launch: MOSS-Audio-4B-Instruct<\/strong>, MOSS-Audio-4B-Thinking<\/strong>, MOSS-Audio-8B-Instruct<\/strong>, and MOSS-Audio-8B-Thinking<\/strong>. The naming convention is worth understanding if you\u2019re deciding which to use. The Instruct<\/strong> variants are optimized for direct instruction following, making them well-suited for production pipelines where you want predictable, structured outputs. The Thinking<\/strong> variants provide stronger chain-of-thought reasoning capabilities, better suited for tasks requiring multi-hop inference. The 4B models use Qwen3-4B<\/strong> as the LLM backbone, and the 8B models use Qwen3-8B<\/strong>, resulting in total model sizes of approximately 4.6B and 8.6B parameters respectively.<\/p>\n\n![\"\"](\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2026\/04\/Screenshot-2026-04-27-at-11.34.12-AM.png\")
https:\/\/github.com\/OpenMOSS\/MOSS-Audio<\/figcaption><\/figure>\n<\/div>\nThe Architecture: Three Components Working Together<\/strong><\/h3>\nMOSS-Audio follows a modular design comprising three components: an audio encoder, a modality adapter, and a large language model<\/strong>. Raw audio is first encoded by the MOSS-Audio-Encoder<\/strong> into continuous temporal representations at 12.5 Hz<\/strong>. Those representations are then projected into the language model\u2019s embedding space through the adapter, and finally consumed by the LLM for auto-regressive text generation.<\/p>\nThe research team trained the encoder from scratch rather than relying on off-the-shelf audio frontends. Their reasoning: a dedicated encoder delivers more robust speech representations, tighter temporal alignment, and better extensibility across acoustic domains.<\/p>\n
Two architectural innovations inside MOSS-Audio are worth understanding in detail.<\/p>\n
DeepStack Cross-Layer Feature Injection<\/strong>: A common weakness in audio models is that relying only on the encoder\u2019s top-layer features tends to lose low-level acoustic information, things like prosody, transient events, and local time-frequency structure. MOSS-Audio addresses this with a DeepStack<\/strong>-inspired cross-layer injection module between the encoder and the language model: in addition to the encoder\u2019s final-layer output, features from earlier and intermediate layers are selected, independently projected, and injected into the language model\u2019s early layers. This preserves multi-granularity information ranging from low-level acoustic details to high-level semantic abstractions, helping the model retain rhythm, timbre, transients, and background structure that a single high-level representation cannot fully capture.<\/p>\nTime-Aware Representation<\/strong>: Time is a critical dimension in audio that text models aren\u2019t naturally equipped to handle. MOSS-Audio addresses this through a time-marker insertion<\/strong> strategy during pretraining: explicit time tokens are inserted between audio frame representations at fixed time intervals to indicate temporal positions. This lets the model learn \u2018what happened when\u2019 within a unified text generation framework, naturally supporting timestamp ASR, event localization, time-based QA, and long-audio retrospection \u2014 without requiring a separate localization head or post-processing pipeline.<\/p>\nBenchmark Performance<\/strong><\/h3>\nThe numbers are strong. On general audio understanding, MOSS-Audio-8B-Thinking achieves an average accuracy of 71.08<\/strong> across four benchmarks \u2014 77.33 on MMAU<\/strong>, 64.92 on MMAU-Pro<\/strong>, 66.53 on MMAR<\/strong>, and 75.52 on MMSU<\/strong>, outperforming majority of open-source models. That includes larger models: Step-Audio-R1 at 33B scores 70.67, and Qwen3-Omni-30B-A3B-Instruct at 30B scores 67.91. For further context, Kimi-Audio (7B) scores 61.14 and MiMo-Audio-7B scores 62.97 on the same average. The 4B Thinking variant scores 68.37, meaning the smaller model with chain-of-thought training beats all larger open-source instruct-only competitors.<\/p>\nOn speech captioning<\/strong>, evaluated with an LLM-as-a-Judge methodology across 13 fine-grained dimensions including gender, age, accent, pitch, volume, speed, texture, clarity, fluency, emotion, tone, personality, and summary, MOSS-Audio-Instruct variants lead across 11 out of 13 dimensions, with MOSS-Audio-8B-Instruct achieving the best overall average score of 3.7252<\/strong>.<\/p>\nOn automatic speech recognition (ASR)<\/strong> spanning 12 evaluation dimensions \u2014 including health condition, code-switching, dialect, singing, and non-speech scenarios \u2014 MOSS-Audio-8B-Instruct achieves the lowest overall CER (Character Error Rate) of 11.30<\/strong> across all tested models.<\/p>\n\n![\"\"](\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2026\/04\/Screenshot-2026-04-27-at-11.34.43-AM-1.png\")
https:\/\/github.com\/OpenMOSS\/MOSS-Audio<\/figcaption><\/figure>\n<\/div>\nKey Takeaways<\/strong><\/h3>\n\n- Single Model, Full Audio Stack<\/strong>: MOSS-Audio unifies speech transcription, speaker and emotion analysis, environmental sound understanding, music analysis, audio captioning, time-aware QA, and complex reasoning into one open-source model, eliminating the need to chain multiple specialized systems together.<\/li>\n
- Two Architectural Innovations Drive Performance<\/strong>: DeepStack Cross-Layer Feature Injection preserves multi-granularity acoustic information by injecting features from intermediate encoder layers directly into the LLM\u2019s early layers, while time-marker insertion during pretraining gives the model explicit temporal awareness for timestamp-grounded tasks.<\/li>\n
- Best-in-Class Benchmark Results at Efficient Scale<\/strong>: MOSS-Audio-8B-Thinking achieves an average accuracy of 71.08 on general audio understanding benchmarks, outperforming all open-source models including 30B+ systems, while the 4B Thinking variant alone beats every larger open-source instruct-only competitor.<\/li>\n
- Dominant Timestamp ASR Accuracy<\/strong>: MOSS-Audio-8B-Instruct scores 35.77 AAS on AISHELL-1 and 131.61 AAS on LibriSpeech, dramatically outperforming both Qwen3-Omni-30B-A3B-Instruct (833.66) and the closed-source Gemini-3.1-Pro (708.24) on the same benchmark.<\/li>\n<\/ul>\n
\nCheck out\u00a0the\u00a0Model Weights<\/a><\/strong>\u00a0and\u00a0Repo<\/a><\/strong>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0130k+ ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0now you can join us on telegram as well.<\/a><\/strong><\/p>\nNeed to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0Connect with us<\/mark><\/a><\/strong><\/p>\nThe post OpenMOSS Releases MOSS-Audio: An Open-Source Foundation Model for Speech, Sound, Music, and Time-Aware Audio Reasoning<\/a> appeared first on MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"Understanding what\u2019s happening…<\/p>\n","protected":false},"author":1,"featured_media":808,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-807","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=807"}],"version-history":[{"count":0,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/media\/808"}],"wp:attachment":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=807"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=807"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=807"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
In practical terms, a single MOSS-Audio model can do all of the above without switching between different specialized systems.<\/p>\n
Four Model Variants<\/strong><\/h3>\nThe team released four variants at launch: MOSS-Audio-4B-Instruct<\/strong>, MOSS-Audio-4B-Thinking<\/strong>, MOSS-Audio-8B-Instruct<\/strong>, and MOSS-Audio-8B-Thinking<\/strong>. The naming convention is worth understanding if you\u2019re deciding which to use. The Instruct<\/strong> variants are optimized for direct instruction following, making them well-suited for production pipelines where you want predictable, structured outputs. The Thinking<\/strong> variants provide stronger chain-of-thought reasoning capabilities, better suited for tasks requiring multi-hop inference. The 4B models use Qwen3-4B<\/strong> as the LLM backbone, and the 8B models use Qwen3-8B<\/strong>, resulting in total model sizes of approximately 4.6B and 8.6B parameters respectively.<\/p>\n\nhttps:\/\/github.com\/OpenMOSS\/MOSS-Audio<\/figcaption><\/figure>\n<\/div>\nThe Architecture: Three Components Working Together<\/strong><\/h3>\nMOSS-Audio follows a modular design comprising three components: an audio encoder, a modality adapter, and a large language model<\/strong>. Raw audio is first encoded by the MOSS-Audio-Encoder<\/strong> into continuous temporal representations at 12.5 Hz<\/strong>. Those representations are then projected into the language model\u2019s embedding space through the adapter, and finally consumed by the LLM for auto-regressive text generation.<\/p>\nThe research team trained the encoder from scratch rather than relying on off-the-shelf audio frontends. Their reasoning: a dedicated encoder delivers more robust speech representations, tighter temporal alignment, and better extensibility across acoustic domains.<\/p>\n
Two architectural innovations inside MOSS-Audio are worth understanding in detail.<\/p>\n
DeepStack Cross-Layer Feature Injection<\/strong>: A common weakness in audio models is that relying only on the encoder\u2019s top-layer features tends to lose low-level acoustic information, things like prosody, transient events, and local time-frequency structure. MOSS-Audio addresses this with a DeepStack<\/strong>-inspired cross-layer injection module between the encoder and the language model: in addition to the encoder\u2019s final-layer output, features from earlier and intermediate layers are selected, independently projected, and injected into the language model\u2019s early layers. This preserves multi-granularity information ranging from low-level acoustic details to high-level semantic abstractions, helping the model retain rhythm, timbre, transients, and background structure that a single high-level representation cannot fully capture.<\/p>\nTime-Aware Representation<\/strong>: Time is a critical dimension in audio that text models aren\u2019t naturally equipped to handle. MOSS-Audio addresses this through a time-marker insertion<\/strong> strategy during pretraining: explicit time tokens are inserted between audio frame representations at fixed time intervals to indicate temporal positions. This lets the model learn \u2018what happened when\u2019 within a unified text generation framework, naturally supporting timestamp ASR, event localization, time-based QA, and long-audio retrospection \u2014 without requiring a separate localization head or post-processing pipeline.<\/p>\nBenchmark Performance<\/strong><\/h3>\nThe numbers are strong. On general audio understanding, MOSS-Audio-8B-Thinking achieves an average accuracy of 71.08<\/strong> across four benchmarks \u2014 77.33 on MMAU<\/strong>, 64.92 on MMAU-Pro<\/strong>, 66.53 on MMAR<\/strong>, and 75.52 on MMSU<\/strong>, outperforming majority of open-source models. That includes larger models: Step-Audio-R1 at 33B scores 70.67, and Qwen3-Omni-30B-A3B-Instruct at 30B scores 67.91. For further context, Kimi-Audio (7B) scores 61.14 and MiMo-Audio-7B scores 62.97 on the same average. The 4B Thinking variant scores 68.37, meaning the smaller model with chain-of-thought training beats all larger open-source instruct-only competitors.<\/p>\nOn speech captioning<\/strong>, evaluated with an LLM-as-a-Judge methodology across 13 fine-grained dimensions including gender, age, accent, pitch, volume, speed, texture, clarity, fluency, emotion, tone, personality, and summary, MOSS-Audio-Instruct variants lead across 11 out of 13 dimensions, with MOSS-Audio-8B-Instruct achieving the best overall average score of 3.7252<\/strong>.<\/p>\nOn automatic speech recognition (ASR)<\/strong> spanning 12 evaluation dimensions \u2014 including health condition, code-switching, dialect, singing, and non-speech scenarios \u2014 MOSS-Audio-8B-Instruct achieves the lowest overall CER (Character Error Rate) of 11.30<\/strong> across all tested models.<\/p>\n\nhttps:\/\/github.com\/OpenMOSS\/MOSS-Audio<\/figcaption><\/figure>\n<\/div>\nKey Takeaways<\/strong><\/h3>\n\n- Single Model, Full Audio Stack<\/strong>: MOSS-Audio unifies speech transcription, speaker and emotion analysis, environmental sound understanding, music analysis, audio captioning, time-aware QA, and complex reasoning into one open-source model, eliminating the need to chain multiple specialized systems together.<\/li>\n
- Two Architectural Innovations Drive Performance<\/strong>: DeepStack Cross-Layer Feature Injection preserves multi-granularity acoustic information by injecting features from intermediate encoder layers directly into the LLM\u2019s early layers, while time-marker insertion during pretraining gives the model explicit temporal awareness for timestamp-grounded tasks.<\/li>\n
- Best-in-Class Benchmark Results at Efficient Scale<\/strong>: MOSS-Audio-8B-Thinking achieves an average accuracy of 71.08 on general audio understanding benchmarks, outperforming all open-source models including 30B+ systems, while the 4B Thinking variant alone beats every larger open-source instruct-only competitor.<\/li>\n
- Dominant Timestamp ASR Accuracy<\/strong>: MOSS-Audio-8B-Instruct scores 35.77 AAS on AISHELL-1 and 131.61 AAS on LibriSpeech, dramatically outperforming both Qwen3-Omni-30B-A3B-Instruct (833.66) and the closed-source Gemini-3.1-Pro (708.24) on the same benchmark.<\/li>\n<\/ul>\n
\nCheck out\u00a0the\u00a0Model Weights<\/a><\/strong>\u00a0and\u00a0Repo<\/a><\/strong>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0130k+ ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0now you can join us on telegram as well.<\/a><\/strong><\/p>\nNeed to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0Connect with us<\/mark><\/a><\/strong><\/p>\nThe post OpenMOSS Releases MOSS-Audio: An Open-Source Foundation Model for Speech, Sound, Music, and Time-Aware Audio Reasoning<\/a> appeared first on MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"Understanding what\u2019s happening…<\/p>\n","protected":false},"author":1,"featured_media":808,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-807","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=807"}],"version-history":[{"count":0,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/media\/808"}],"wp:attachment":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=807"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=807"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=807"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The Architecture: Three Components Working Together<\/strong><\/h3>\nMOSS-Audio follows a modular design comprising three components: an audio encoder, a modality adapter, and a large language model<\/strong>. Raw audio is first encoded by the MOSS-Audio-Encoder<\/strong> into continuous temporal representations at 12.5 Hz<\/strong>. Those representations are then projected into the language model\u2019s embedding space through the adapter, and finally consumed by the LLM for auto-regressive text generation.<\/p>\nThe research team trained the encoder from scratch rather than relying on off-the-shelf audio frontends. Their reasoning: a dedicated encoder delivers more robust speech representations, tighter temporal alignment, and better extensibility across acoustic domains.<\/p>\n
Two architectural innovations inside MOSS-Audio are worth understanding in detail.<\/p>\n
DeepStack Cross-Layer Feature Injection<\/strong>: A common weakness in audio models is that relying only on the encoder\u2019s top-layer features tends to lose low-level acoustic information, things like prosody, transient events, and local time-frequency structure. MOSS-Audio addresses this with a DeepStack<\/strong>-inspired cross-layer injection module between the encoder and the language model: in addition to the encoder\u2019s final-layer output, features from earlier and intermediate layers are selected, independently projected, and injected into the language model\u2019s early layers. This preserves multi-granularity information ranging from low-level acoustic details to high-level semantic abstractions, helping the model retain rhythm, timbre, transients, and background structure that a single high-level representation cannot fully capture.<\/p>\nTime-Aware Representation<\/strong>: Time is a critical dimension in audio that text models aren\u2019t naturally equipped to handle. MOSS-Audio addresses this through a time-marker insertion<\/strong> strategy during pretraining: explicit time tokens are inserted between audio frame representations at fixed time intervals to indicate temporal positions. This lets the model learn \u2018what happened when\u2019 within a unified text generation framework, naturally supporting timestamp ASR, event localization, time-based QA, and long-audio retrospection \u2014 without requiring a separate localization head or post-processing pipeline.<\/p>\nBenchmark Performance<\/strong><\/h3>\nThe numbers are strong. On general audio understanding, MOSS-Audio-8B-Thinking achieves an average accuracy of 71.08<\/strong> across four benchmarks \u2014 77.33 on MMAU<\/strong>, 64.92 on MMAU-Pro<\/strong>, 66.53 on MMAR<\/strong>, and 75.52 on MMSU<\/strong>, outperforming majority of open-source models. That includes larger models: Step-Audio-R1 at 33B scores 70.67, and Qwen3-Omni-30B-A3B-Instruct at 30B scores 67.91. For further context, Kimi-Audio (7B) scores 61.14 and MiMo-Audio-7B scores 62.97 on the same average. The 4B Thinking variant scores 68.37, meaning the smaller model with chain-of-thought training beats all larger open-source instruct-only competitors.<\/p>\nOn speech captioning<\/strong>, evaluated with an LLM-as-a-Judge methodology across 13 fine-grained dimensions including gender, age, accent, pitch, volume, speed, texture, clarity, fluency, emotion, tone, personality, and summary, MOSS-Audio-Instruct variants lead across 11 out of 13 dimensions, with MOSS-Audio-8B-Instruct achieving the best overall average score of 3.7252<\/strong>.<\/p>\nOn automatic speech recognition (ASR)<\/strong> spanning 12 evaluation dimensions \u2014 including health condition, code-switching, dialect, singing, and non-speech scenarios \u2014 MOSS-Audio-8B-Instruct achieves the lowest overall CER (Character Error Rate) of 11.30<\/strong> across all tested models.<\/p>\n\nhttps:\/\/github.com\/OpenMOSS\/MOSS-Audio<\/figcaption><\/figure>\n<\/div>\nKey Takeaways<\/strong><\/h3>\n\n- Single Model, Full Audio Stack<\/strong>: MOSS-Audio unifies speech transcription, speaker and emotion analysis, environmental sound understanding, music analysis, audio captioning, time-aware QA, and complex reasoning into one open-source model, eliminating the need to chain multiple specialized systems together.<\/li>\n
- Two Architectural Innovations Drive Performance<\/strong>: DeepStack Cross-Layer Feature Injection preserves multi-granularity acoustic information by injecting features from intermediate encoder layers directly into the LLM\u2019s early layers, while time-marker insertion during pretraining gives the model explicit temporal awareness for timestamp-grounded tasks.<\/li>\n
- Best-in-Class Benchmark Results at Efficient Scale<\/strong>: MOSS-Audio-8B-Thinking achieves an average accuracy of 71.08 on general audio understanding benchmarks, outperforming all open-source models including 30B+ systems, while the 4B Thinking variant alone beats every larger open-source instruct-only competitor.<\/li>\n
- Dominant Timestamp ASR Accuracy<\/strong>: MOSS-Audio-8B-Instruct scores 35.77 AAS on AISHELL-1 and 131.61 AAS on LibriSpeech, dramatically outperforming both Qwen3-Omni-30B-A3B-Instruct (833.66) and the closed-source Gemini-3.1-Pro (708.24) on the same benchmark.<\/li>\n<\/ul>\n
\nCheck out\u00a0the\u00a0Model Weights<\/a><\/strong>\u00a0and\u00a0Repo<\/a><\/strong>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0130k+ ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0now you can join us on telegram as well.<\/a><\/strong><\/p>\nNeed to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0Connect with us<\/mark><\/a><\/strong><\/p>\nThe post OpenMOSS Releases MOSS-Audio: An Open-Source Foundation Model for Speech, Sound, Music, and Time-Aware Audio Reasoning<\/a> appeared first on MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"Understanding what\u2019s happening…<\/p>\n","protected":false},"author":1,"featured_media":808,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-807","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=807"}],"version-history":[{"count":0,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/media\/808"}],"wp:attachment":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=807"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=807"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=807"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The research team trained the encoder from scratch rather than relying on off-the-shelf audio frontends. Their reasoning: a dedicated encoder delivers more robust speech representations, tighter temporal alignment, and better extensibility across acoustic domains.<\/p>\n
Two architectural innovations inside MOSS-Audio are worth understanding in detail.<\/p>\n
DeepStack Cross-Layer Feature Injection<\/strong>: A common weakness in audio models is that relying only on the encoder\u2019s top-layer features tends to lose low-level acoustic information, things like prosody, transient events, and local time-frequency structure. MOSS-Audio addresses this with a DeepStack<\/strong>-inspired cross-layer injection module between the encoder and the language model: in addition to the encoder\u2019s final-layer output, features from earlier and intermediate layers are selected, independently projected, and injected into the language model\u2019s early layers. This preserves multi-granularity information ranging from low-level acoustic details to high-level semantic abstractions, helping the model retain rhythm, timbre, transients, and background structure that a single high-level representation cannot fully capture.<\/p>\n Time-Aware Representation<\/strong>: Time is a critical dimension in audio that text models aren\u2019t naturally equipped to handle. MOSS-Audio addresses this through a time-marker insertion<\/strong> strategy during pretraining: explicit time tokens are inserted between audio frame representations at fixed time intervals to indicate temporal positions. This lets the model learn \u2018what happened when\u2019 within a unified text generation framework, naturally supporting timestamp ASR, event localization, time-based QA, and long-audio retrospection \u2014 without requiring a separate localization head or post-processing pipeline.<\/p>\n The numbers are strong. On general audio understanding, MOSS-Audio-8B-Thinking achieves an average accuracy of 71.08<\/strong> across four benchmarks \u2014 77.33 on MMAU<\/strong>, 64.92 on MMAU-Pro<\/strong>, 66.53 on MMAR<\/strong>, and 75.52 on MMSU<\/strong>, outperforming majority of open-source models. That includes larger models: Step-Audio-R1 at 33B scores 70.67, and Qwen3-Omni-30B-A3B-Instruct at 30B scores 67.91. For further context, Kimi-Audio (7B) scores 61.14 and MiMo-Audio-7B scores 62.97 on the same average. The 4B Thinking variant scores 68.37, meaning the smaller model with chain-of-thought training beats all larger open-source instruct-only competitors.<\/p>\n On speech captioning<\/strong>, evaluated with an LLM-as-a-Judge methodology across 13 fine-grained dimensions including gender, age, accent, pitch, volume, speed, texture, clarity, fluency, emotion, tone, personality, and summary, MOSS-Audio-Instruct variants lead across 11 out of 13 dimensions, with MOSS-Audio-8B-Instruct achieving the best overall average score of 3.7252<\/strong>.<\/p>\n On automatic speech recognition (ASR)<\/strong> spanning 12 evaluation dimensions \u2014 including health condition, code-switching, dialect, singing, and non-speech scenarios \u2014 MOSS-Audio-8B-Instruct achieves the lowest overall CER (Character Error Rate) of 11.30<\/strong> across all tested models.<\/p>\n Check out\u00a0the\u00a0Model Weights<\/a><\/strong>\u00a0and\u00a0Repo<\/a><\/strong>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0130k+ ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0now you can join us on telegram as well.<\/a><\/strong><\/p>\n Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0Connect with us<\/mark><\/a><\/strong><\/p>\n The post OpenMOSS Releases MOSS-Audio: An Open-Source Foundation Model for Speech, Sound, Music, and Time-Aware Audio Reasoning<\/a> appeared first on MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":" Understanding what\u2019s happening…<\/p>\n","protected":false},"author":1,"featured_media":808,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-807","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=807"}],"version-history":[{"count":0,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/posts\/807\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=\/wp\/v2\/media\/808"}],"wp:attachment":[{"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=807"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=807"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connectword.dpdns.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=807"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Benchmark Performance<\/strong><\/h3>\n
Key Takeaways<\/strong><\/h3>\n
\n
\n