Mixture of Experts: A Comprehensive Overview
Mathematical foundations of MoE, its applications in deep learning, and how it improves model scalability.
About
I am currently pursuing my PhD in Mathematics at the National University of Singapore under the co-supervision of Prof. Tan Nguyen and Prof. Zhang Lei. My interests lie in mathematics and its applications, particularly in optimization and deep learning. I aspire to an academic career, contributing to both the theoretical and practical aspects of mathematics in AI.
My research focuses on uncovering the mathematical intuition behind AI models and leveraging mathematical tools to optimize models and enhance computational efficiency. I seek to collaborate with researchers who share my enthusiasm for advancing mathematical knowledge and developing innovative solutions to complex problems.
Research Approach Levels
1
"LEGO Assembly" — Naively combining modules from different architectures via trial and error.
2
Methodological Abstraction — Integrating techniques to offset weaknesses and uncovering hidden connections.
3
Mathematical Modeling — Transforming problems into frameworks ensuring scalability and interpretability.
How I See Math
3
The Inventors — True pioneers who create new branches of math (Tao, Ngô Bảo Châu…).
2
The Builders — Use deep understanding to build impactful things: companies, AI, rockets.
1
The Teachers — Pass down knowledge and spark curiosity. Without them, the rest wouldn't exist.
Research Interests
Education
PhD in Mathematics
National University of Singapore (NUS)
Bachelor of Science in Mathematics and Informatics
Hanoi University of Science and Technology (HUST)
Selected Publications
-
ICLR 2026
Activation Steering with a Feedback Controller
-
ICLR 2026
Expert Merging in Sparse Mixture of Experts with Nash Bargaining
-
ICLR 2025
CAMEx: Curvature-aware Merging of Experts
-
Neural Networks 2024
A Hyper-Transformer model for Controllable Pareto Front Learning with Split Feasibility Constraints
-
Optim. Methods Softw. 2024
A neurodynamic approach for a class of pseudoconvex semivectorial bilevel optimization problems
-
Biomed. Signal Process. Control 2023
An EfficientNet-encoder U-Net Joint Residual Refinement Module with Tversky–Kahneman Baroni–Urbani–Buser loss for Biomedical Image Segmentation
Blog
Contact
Documents
Download my curriculum vitae for a full list of publications, awards, and experience.
Download CV