jie.ren

CURRICULUM VITAE

Jie Ren

Computer Science · King Abdullah University of Science and Technology

Homepage · Google Scholar

A print-friendly version: use your browser’s Print → Save as PDF.

Research Interests

My research focuses on the co-design of numerical algorithms and modern heterogeneous architectures. I study sparse factorizations, finite element methods, GPU scheduling, and architecture-aware kernel design, with an emphasis on reformulating classical numerical methods for extreme parallelism. My work spans high-performance scientific computing and machine learning systems, with publications at SC, NeurIPS, ICML, ICS, and ECCV, and has been recognized by the ACM/IEEE-CS George Michael Memorial High Performance Computing Fellowship and a Gordon Bell Prize Finalist distinction.

Education

Ph.D. in Computer Science, KAUST, Thuwal, Saudi Arabia

Sep. 2023 – Present

Advisor: Prof. David E. Keyes

B.Eng. in Electronics Engineering, Xidian University, Xi'an, China

Sep. 2017 – Jun. 2021

Research & Industry Experience

Baidu, Beijing, China

May 2025 – Aug. 2025

ERNIE Star Research Intern

Mentor: Prof. Haifeng Wang, CTO

Developed optimized INT2 GPU kernels based on Stream-K, combining pipelined memory loading, dequantization, and fused GEMM operations to achieve a 5x speedup over the Triton baseline. For an AI-for-Science project on aerodynamic optimization of high-speed trains, redesigned the CFD computational pipeline and implemented custom GPU kernels, achieving a 3x speedup while reducing memory usage by 50%.

NVIDIA Research (Networking Group), Santa Clara, CA, USA

Jul. 2024 – Oct. 2024

Research Intern

Mentor: Dr. Nic McDonald

Designed and implemented communication-computation-overlapped tensor-parallel GEMM kernels for efficient distributed inference on next-generation GPUs with high-bandwidth interconnects. The work was upstreamed into NVIDIA CUTLASS as part of its distributed GEMM capabilities.

ETH Zürich, Zürich, Switzerland

Feb. 2023 – Oct. 2023

Academic Guest (HiPEAC Grant)

Host: Prof. Torsten Hoefler

Contributed to a Transformer quantization project using INT4 precision and CUTLASS, achieving a 4x speedup. Also developed GPU-centric sparse linear system solvers using mixed-precision arithmetic.

University of Edinburgh, Edinburgh, UK

Jul. 2021 – Sep. 2023

Academic Guest

Supervisor: Prof. Luo Mai

Led the design and implementation of TorchOpt, a high-performance library for differentiable optimization supporting higher-order meta-learning algorithms, published in JMLR. Also contributed to GPU-centric sparse linear system research using mixed-precision techniques.

MegEngine, MEGVII Inc., Beijing, China

Feb. 2022 – Oct. 2022

Software Engineering Intern

Mentor: Biao Wang

Migrated depth-wise convolution kernels from NVIDIA Turing to Ampere by rewriting the CUTLASS global-memory access iterator, achieving a 5x+ speedup. Also designed and implemented a novel region-restricted convolution operator.

3D Vision, MEGVII Inc., Beijing, China

Apr. 2021 – Feb. 2022

Research Intern

Mentor: Mrs. Ran Yan

Designed and implemented MegBA, a GPU-based distributed library for large-scale bundle adjustment, published at ECCV 2022. Optimized the HF-Net inference pipeline using TensorRT and custom CUDA image-processing kernels, reducing latency from 20 ms to 5 ms on an RTX 2080 Ti.

Publications

  1. J. Ren, H. Ltaief, S. Zampini, and D. E. Keyes. “Cheetah: Optimizing Execution Pipelines for Matrix-Free Finite Element Operators on GPUs.”
    ACM International Conference on Supercomputing (ICS), 2026.
  2. L. Wang, J. Wang, J. Ren, Z. Xiang, D. E. Keyes, and D. Wang. “Private Training of Large-Scale Models with Efficient DP-SGD.”
    Advances in Neural Information Processing Systems (NeurIPS), 2025.
  3. L. Wang, J. Ren, H. Xu, J. Wang, D. E. Keyes, and D. Wang. “ZO-Offloading: Fine-Tuning LLMs with 100 Billion Parameters on a Single GPU.”
    Conference on Language Modeling (COLM), 2025.
  4. J. Ren, T. Zhong, Y. Hong, G. Feng, X. Wang, W. Jia, H. Ltaief, and D. E. Keyes. “Caracal: A GPU-Resident Sparse LU Solver with Lightweight Fine-Grained Scheduling.”
    International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), 2025.
  5. J. Ren, H. Ltaief, S. Abdullah, and D. E. Keyes. “Accelerating Mixed-Precision Out-of-Core Cholesky Factorization with Static Task Scheduling.”
    ISC High Performance, 2025.
  6. S. Ashkboos, I. Markov, E. Frantar, T. Zhong, X. Wang, J. Ren, T. Hoefler, and D. Alistarh. “Towards End-to-End 4-Bit Inference on Generative Large Language Models.”
    Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
  7. H. Ltaief, R. Alomairy, Q. Cao, J. Ren, L. Slim, T. Kurth, B. Dorschner, S. Bougouffa, R. Abdelkhalak, and D. E. Keyes. “Toward Capturing Genetic Epistasis from Multivariate Genome-Wide Association Studies Using Mixed-Precision Kernel Ridge Regression.”
    International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), 2024. Gordon Bell Prize Finalist.
  8. J. Ren, B. Liu, X. Feng, X. Pan, Y. Fu, Y. Yang, and L. Mai. “TorchOpt: An Efficient Library for Differentiable Optimization.”
    Journal of Machine Learning Research (JMLR), 2023.
  9. B. Liu, X. Feng, J. Ren, L. Mai, R. Zhu, H. Zhang, J. Wang, and Y. Yang. “A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning.”
    Advances in Neural Information Processing Systems (NeurIPS), 2022.
  10. X. Xia, W. Yang, J. Ren, Y. Li, Y. Zhan, B. Han, and T. Liu. “Pluralistic Image Completion with Probabilistic Mixture-of-Experts.”
    Advances in Neural Information Processing Systems (NeurIPS), 2022.
  11. J. Ren, W. Liang, R. Yan, L. Mai, S. Liu, and X. Liu. “MegBA: A High-Performance and Distributed Library for Large-Scale Bundle Adjustment.”
    European Conference on Computer Vision (ECCV), 2022.
  12. L. Tian, B. Chen, J. Ren, H. Zhang, Z. Wu, N. Han, Y. Chen, and H. Liu. “Multi-Scale Visual Attention for Attribute Disambiguation in Zero-Shot Learning.”
    Signal Processing: Image Communication, 2021.
  13. Z. Duan, D. Wang, B. Chen, C. Wang, W. Chen, Y. Li, J. Ren, and M. Zhou. “Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network.”
    International Conference on Machine Learning (ICML), 2021.

Preprints

  • J. Ren, Y. Li, Z. Ding, W. Pan, and H. Dong. “Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning.”
    arXiv:2104.09122, 2021.

Technical Skills

  • Programming: C++ (17/20), CUDA, Python, MATLAB.
  • GPU Systems: Architecture-aware algorithm design, memory hierarchy optimization, instruction-level parallelism, kernel fusion, and fine-grained task scheduling.
  • Frameworks & Libraries: CUTLASS, libCEED, MFEM, Eigen, PyTorch, and NumPy; experience with large-scale numerical and machine learning systems.

Selected Academic Activities

  • Invited Participant, Dagstuhl Seminar 26392 (September 2026)
  • Young Researcher, 13th Heidelberg Laureate Forum (September 2026)

Professional Service

Conference Reviewer

  • International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), 2026;
  • IEEE International Conference on Robotics and Automation (ICRA), 2025;
  • International Conference on Parallel Processing (ICPP), 2024;
  • International Conference on Learning Representations (ICLR), 2023 -- 2025;
  • Advances in Neural Information Processing Systems (NeurIPS), 2022 -- 2023;
  • International Conference on Machine Learning (ICML), 2022 -- 2024;
  • IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021 -- 2022.

Journal Reviewer

  • ACM Transactions on Mathematical Software (TOMS);
  • Parallel Computing;
  • International Journal of Computer Vision (IJCV);
  • IEEE Robotics and Automation Letters (RA-L).

Honours & Awards