Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

Abstract

We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemotron 3 Super is the first model in the Nemotron 3 family to 1) be pre-trained in NVFP4, 2) leverage LatentMoE, a new Mixture-of-Experts architecture that optimizes for both accuracy per FLOP and accuracy per parameter, and 3) include MTP layers for inference acceleration through native speculative decoding. We pre-trained Nemotron 3 Super on 25 trillion tokens followed by post-training using supervised fine tuning (SFT) and reinforcement learning (RL). The final model supports up to 1M context length and achieves comparable accuracy on common benchmarks, while also achieving up to 2.2x and 7.5x higher inference throughput compared to GPT-OSS-120B and Qwen3.5-122B, respectively. Nemotron 3 Super datasets, along with the base, post-trained, and quantized checkpoints, are open-sourced on HuggingFace.

Workflow Status

Review status

pending

Role

unreviewed

Read priority

now

Vote

Not set.

Saved

no

Collections

Not filed yet.

Next action

Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

Similar Papers

BibTeX

@misc{nvidia2026nemotron,
  title = {Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning},
  author = {NVIDIA and Aakshita Chandiramani and Aaron Blakeman and Abdullahi Olaoye and Abhibha Gupta and Abhilash Somasamudramath and Abhinav Khattar and Adeola Adesoba and Adi Renduchintala and Adil Asif and Aditya Agrawal and Aditya Vavre and Ahmad Kiswani and Aishwarya Padmakumar and Ajay Hotchandani and Akanksha Shukla and Akhiad Bercovich and Aleksander Ficek and Aleksandr Shaposhnikov and Alex Gronskiy and Alex Kondratenko and Alex Neefus and Alex Steiner and Alex Yang and Alexander Bukharin and Alexander Young and Ali Hatamizadeh and Ali Taghibakhshi and Alina Galiautdinova and Alisa Liu and Alok Kumar and Ameya Sunil Mahabaleshwarkar and Amir Klein and Amit Zuker and Amnon Geifman and Anahita Bhiwandiwalla and Ananth Subramaniam and Andrew Tao and Anjaney Shrivastava and Anjulie Agrusa and Ankur Srivastava and Ankur Verma and Ann Guan and Anna Shors and Annamalai Chockalingam and Anubhav Mandarwal and Aparnaa Ramani and Arham Mehta and Arti Jain and Arun Venkatesan and Asha Anoosheh and Ashwath Aithal and Ashwin Poojary and Asif Ahamed and Asit Mishra and Asli Sabanci Demiroz and Asma Kuriparambil Thekkumpate and Atefeh Sohrabizadeh and Avinash Kaur and Ayush Dattagupta and Barath Subramaniam Anandan and Bardiya Sadeghi and Barnaby Simkin and Ben Lanir and Benedikt Schifferer and Benjamin Chislett and Besmira Nushi and Bilal Kartal and Bill Thiede and Bita Darvish Rouhani and Bobby Chen and Boris Ginsburg and Brandon Norick and Branislav Kisacanin and Brian Yu and Bryan Catanzaro and Buvaneswari Mani and Carlo del Mundo and Chankyu Lee and Chanran Kim and Chantal Hwang and Chao Ni and Charles Wang and Charlie Truong and Cheng-Ping Hsieh and Chenhan Yu and Chenjie Luo and Cherie Wang and Chetan Mungekar and Chintan Patel and Chris Alexiuk and Chris Holguin and Chris Wing and Christian Munley and Christopher Parisien and Chuck Desai and Chunyang Sheng and Collin Neale and Cyril Meurillon and Dakshi Kumar and Dan Gil and Dan Su and Dane Corneil and Daniel Afrimi and Daniel Burkhardt Eliuth Triana and Daniel Egert and Daniel Fatade and Daniel Lo and Daniel Rohrer and Daniel Serebrenik and Daniil Sorokin and Daria Gitman and Daria Levy and Darko Stosic and David Edelsohn and David Messina and David Mosallanezhad and David Tamok and Deena Donia and Deepak Narayanan and Devin O'Kelly and Dheeraj Peri and Dhruv Nathawani and Di Wu and Dima Rekesh and Dina Yared and Divyanshu Kakwani and Dmitry Konyagin Brandon Tuttle and Dong Ahn and Dongfu Jiang and Dorrin Poorkay and Douglas O'Flaherty and Duncan Riach and Dusan Stosic and Dustin Van Stee and Edgar Minasyan and Edward Lin and Eileen Peters Long and Elad Segal and Elena Lantz and Elena Lewis and Ellie Evans and Elliott Ning and Eric Chung and Eric Harper and Eric Pham-Hung and Eric W. Tramel and Erick Galinkin and Erik Pounds and Esti Etrog and Evan Briones and Evan Wu and Evelina Bakhturina and Evgeny Tsykunov and Ewa Dobrowolska and Farshad Saberi Movahed and Farzan Memarian and Fay Wang and Fei Jia and Felipe Soares and Felipe Vieira Frujeri and Feng Chen and Fengguang Lin and Ferenc Galko and Fortuna Zhang and Frankie Siino and Frida Hou and Gantavya Bhatt and Gargi Prasad and Geethapriya Venkataramani and Geetika Gupta and George Armstrong and Gerald Shen and Giulio Borghesi and Gordana Neskovic and Gorkem Batmaz and Grace Lam and Grace Wu and Greg Pauloski and Greyson Davis and Grigor Nalbandyan and Guoming Zhang and Guy Farber and Guyue Huang and Haifeng Qian and Haran Kumar Shiv Kumar and Harry Kim and Harsh Sharma and Hayate Iso and Hayley Ross and Herbert Hum and Herman Sahota and Hexin Wang and Himanshu Soni and Hiren Upadhyay and Huy Nguyen and Iain Cunningham and Ido Galil and Ido Shahaf and Igino Padovani and Igor Gitman and Igor Shovkun and Ikroop Dhillon and Ilya Loshchilov and Ingrid Kelly and Itamar Schen and Itay Levy and Ivan Moshkov and Izik Golan and Izzy Putterman and Jain Tu and Jan Baczek and Jan Kautz and Jane Polak Scowcroft and Janica Rosenberg and Jared Casper and Jarrod Pflum and Jason Grant and Jason Sewall and Jatin Mitra and Jeffrey Glick and Jenny Chen and Jesse Oliver and Jiacheng Xu and Jiafan Zhu and Jialin Song and Jian Zhang and Jiaqi Zeng and Jie Lou and Jill Milton and Jim Chow and Jimmy Zhang and Jinhang Choi and Jining Huang and Jocelyn Huang and Joel Caruso and Joey Conway and Joey Guman and Johan Jatko and John Kamalu and Johnny Greco and Jonathan Cohen and Jonathan Raiman and Joseph Jennings and Joyjit Daw and Juan Yu and Julio Tapia and Junkeun Yi and Jupinder Parmar and Jyothi Achar and Kari Briski and Kartik Mattoo and Katherine Cheung and Katherine Luna and Keith Wyss and Kevin Shih and Kezhi Kong and Khanh Nguyen and Khushi Bhardwaj and Kirill Buryak and Kirthi Shankar Sivamani and Konstantinos Krommydas and Kris Murphy and Krishna C. Puvvada and Krzysztof Pawelec and Kumar Anik and Laikh Tewari and Laya Sleiman and Leo Du and Leon Derczynski and Li Ding and Lilach Ilan and Lingjie Wu and Lizzie Wei and Luis Vega and Lun Su and Maarten Van Segbroeck and Maer Rodrigues de Melo and Magaret Zhang and Mahan Fathi and Makesh Narsimhan Sreedhar and Makesh Sreedhar and Makesh Tarun Chandran and Manuel Reyes Gomez and Maor Ashkenazi and Marc Cuevas and Marc Romeijn and Margaret Zhang and Mark Cai and Mark Gabel and Markus Kliegl and Martyna Patelka and Maryam Moosaei and Matthew Varacalli and Matvei Novikov and Mauricio Ferrato and Mehrzad Samadi and Melissa Corpuz and Meng Xin and Mengdi Wang and Mengru Wang and Meredith Price and Micah Schaffer and Michael Andersch and Michael Boone and Michael Evans and Michael Z Wang and Miguel Martinez and Mikail Khona and Mike Chrzanowski and Mike Hollinger and Mingyuan Ma and Minseok Lee and Mohammad Dabbah and Mohammad Shoeybi and Mostofa Patwary and Nabin Mulepati and Nader Khalil and Najeeb Nabwani and Nancy Agarwal and Nanthini Balasubramaniam and Narimane Hennouni and Narsi Kodukula and Natalie Hereth and Nathaniel Pinckney and Nave Assaf and Negar Habibi and Nestor Qin and Neta Zmora and Netanel Haber and Nick Reamaroon and Nickson Quak and Nidhi Bhatia and Nikhil Jukar and Nikki Pope and Nikolai Ludwig and Nima Tajbakhsh and Nir Ailon and Nirmal Juluru and Nirmalya De and Nowel Pitt and Oleg Rybakov and Oleksii Hrinchuk and Oleksii Kuchaiev and Olivier Delalleau and Oluwatobi Olabiyi and Omer Ullman Argov and Omri Almog and Omri Puny and Oren Tropp and Otavio Padovani and Ouye Xie and Parth Chadha and Pasha Shamis and Paul Gibbons and Pavlo Molchanov and Peter Belcak and Peter Jin and Pinky Xu and Piotr Januszewski and Pooya Jannaty and Prachi Shevate and Pradeep Thalasta and Pranav Prashant Thombre and Prasoon Varshney and Prerana Gambhir and Pritam Gundecha and Przemek Tredak and Qing Miao and Qiyu Wan and Quan Tran Minh and Rabeeh Karimi Mahabadi and Rachel Oberman and Rachit Garg and Rahul Kandu and Raina Zhong and Ran El-Yaniv and Ran Zilberstein and Rasoul Shafipour and Renee Yao and Renjie Pi and Richard Mazzarese and Richard Wang and Rick Izzo and Ridhima Singla and Rima Shahbazyan and Rishabh Garg and Ritika Borkar and Ritu Gala and Riyad Islam and Robert Clark and Robert Hesse and Roger Waleffe and Rohit Varma Kalidindi and Rohit Watve and Roi Koren and Ron Fan and Ruchika Kharwar and Ruisi Cai and Ruoxi Zhang and Russell J. Hewett and Ryan Prenger and Ryan Timbrook and Ryota Egashira and Sadegh Mahdavi and Sagar Singh Ashutosh Joshi and Sahil Modi and Samuel Kriman and Sandeep Pombra and Sanjay Kariyappa and Sanjeev Satheesh and Santiago Pombo and Saori Kaji and Satish Pasumarthi and Saurav Mishra and Saurav Muralidharan and Scott Hara and Sean Narenthiran and Sebastian Rogawski and Seonjin Na and Seonmyeong Bak and Sepehr Sameni and Seth Poulos and Shahar Mor and Shantanu Acharya and Shaona Ghosh Adam Lord and Sharath Turuvekere Sreenivas and Shaun Kotek and Shaya Gharghabi and Shelby Thomas and Sheng-Chieh Lin and Shibani Likhite and Shiqing Fan and Shiyang Chen and Shreya Gopal and Shrimai Prabhumoye and Shubham Pachori and Shubham Toshniwal and Shuo Zhang and Shuoyang Ding and Shyam Renjith and Shyamala Prayaga and Siddhartha Jain and Simeng Sun and Sirisha Rella and Sirshak Das and Smita Ithape and Sneha Harishchandra S and Somshubra Majumdar and Soumye Singhal and Sri Harsha Singudasu and Sriharsha Niverty and Stas Sergienko and Stefana Gloginic and Stefania Alborghetti and Stephen Ge and Stephen McCullough and Sugam Dipak Devare and Suguna Varshini Velury and Sukrit Rao and Sumeet Kumar Barua and Sunny Gai and Suseella Panguluri and Sushil Koundinyan and Swathi Patnam and Sweta Priyadarshi and Swetha Bhendigeri and Syeda Nahida Akter and Sylendran Arunagiri and Tailling Yuan and Talor Abramovich and Tan Bui and Tan Yu and Terry Kong and Thanh Do and Thomas Gburek and Thorgane Marques and Tiffany Moore and Tijmen Blankevoort and Tim Moon and Timothy Ma and Tiyasa Mitra and Tomasz Grzegorzek and Tomer Asida and Tomer Bar Natan and Tomer Keren and Tomer Ronen and Traian Rebedea and Trenton Starkey and Tugrul Konuk and Twinkle Vashishth and Tyler Condensa and Udi Karpas and Ushnish De and Vahid Noorozi and Vahid Noroozi and Vanshil Atul Shah and Veena Vaidyanathan and Venkat Srinivasan and Venmugil Elango and Victor Cui and Vijay Korthikanti and Vikas Mehta and Virginia Adams and Virginia Wu and Vitaly Kurin and Vitaly Lavrukhin and Vladimir Anisimov and Wan Seo and Wanli Jiang and Wasi Uddin Ahmad and Wei Du and Wei Ping and Wei-Ming Chen and Wendy Quan and Wenliang Dai and Wenwen Gao and Will Jennings and William Zhang and Xiaowei Ren and Xiaowen Xin and Xin Li and Yang Yu and Yangyi Chen and Yaniv Galron and Yashaswi Karnati and Yejin Choi and Yev Meyer and Yi-Fu Wu and Yian Zhang and Ying Lin and Yonatan Geifman and Yonggan Fu and Yoshi Suhara and Youngeun Kwon and Yuan Zhang and Yuki Huang and Zach Moshe and Zhilin Wang and Zhiyu Cheng and Zhongbo Zhu and Zhuolin Yang and Zihan Liu and Zijia Chen and Zijie Yan and Zuhair Ahmed},
  year = {2026},
  abstract = {We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemotron 3 Super is the first model in the Nemotron 3 family to 1) be pre-trained in NVFP4, 2) leverage LatentMoE, a new Mixture-of-Experts architecture that optimizes for both accuracy per FLOP and accuracy per parameter, and 3) include MTP layers for inference acceleration through native speculative decoding. We pre-trai},
  url = {https://huggingface.co/papers/2604.12374},
  keywords = {hybrid Mamba-Attention Mixture-of-Experts, NVFP4, LatentMoE, Mixture-of-Experts, speculative decoding, supervised fine tuning, reinforcement learning, context length, inference throughput, huggingface daily},
  eprint = {2604.12374},
  archiveprefix = {arXiv},
}

Metadata

{}