We model the memorization of random, non-inferable facts as a membership testing problem, connecting Bloom-filter-style error metrics with the log-loss behavior of language models. In the sparse-fact regime, we prove a rate-distortion theorem showing that the optimal space-error tradeoff is governed by a KL-divergence frontier. The result gives an information-theoretic explanation for high-confidence hallucinations: in this regime under log-loss, the limiting space-optimal solution assigns the same confidence to all facts and a nonzero fraction of non-facts, rather than simply abstaining or forgetting.
About Me
I’m a fourth-year computer science PhD student in Northwestern University’s theory group. I’m very fortunate to be advised by Aravindan Vijayaraghavan. My research interests lie in theoretical computer science and the foundations of machine learning.
Before beginning my PhD studies at Northwestern University, I completed both my undergraduate and master’s degrees at the same institution. (Go Cats!)
Publications and Manuscripts
Publications
We give the first polynomial-time statistical query (SQ) algorithm for agnostic PAC learning of an arbitrarily biased ReLU neuron under Gaussian marginals, achieving a constant-factor approximation with any desired additive tolerance. Large negative biases are challenging because the neuron activates only in the Gaussian tail, where adversarial noise can overwhelm the informative gradient signal or even drive gradient descent in the wrong direction. Complementing our algorithm, we prove a hardness separation between general SQ and correlational SQ (CSQ), formalizing this limitation of gradient-based methods.
We study a graph-theoretic framework for dataset versioning that optimizes storage costs while controlling retrieval costs across versions. On the theory side, we prove hardness-of-approximation results and give provably near-optimal algorithms for version graphs of bounded treewidth. On one real GitHub version graph, our dynamic programming heuristic achieved nearly three orders of magnitude lower total retrieval cost than the prior heuristic at comparable storage budgets.
Manuscripts
A linear system can be sensitive to small errors when a combination of its columns almost cancels out. We study the same phenomenon for polynomials. After small random perturbations, we show that nearly canceling combinations are close to Koszul syzygies, simple exact cancellations that come from the fact that multiplication commutes. This answers an open question about efficiently decomposing a noisy sum of powers of polynomials in the smoothed setting [BHKX22]. We also establish robust uniqueness of symmetric third-order tensor decompositions at superlinear rank under small random perturbations, addressing an open question raised in [BCV14].
How much does an external hint sway a reasoning language model's prior judgment? On multiple-choice questions, we find that for a given source, a hint gives the suggested answer a roughly constant average boost in log-odds across a wide range of starting probabilities, as if it carried a fixed evidence weight. In our tests, the boost was not calibrated to the source's stated reliability, and explicit distrust warnings weakened it without eliminating it.
Language models that copy random text accurately can still stop too early or too late on a long run of identical characters. We prove a limit for a class of finite-precision transformers and construct a two-layer transformer that uses periodic position codes (with small prime periods) to copy any string up to a chosen length and stop exactly at the end. In synthetic copying experiments, baselines with standard positional encodings such as RoPE struggled on long repeated strings even within the training range, while our model trained on strings up to 2,000 tokens generalized to 5,000 tokens with near-perfect accuracy.
Selected Talks
Education
Awards
- Ph.D. student research award (Northwestern CS department).2024–2025
- Barris Award for outstanding TA, CS 335: Introduction to the Theory of Computing.Fall 2024
- Junior Career Award in Mathematics (Northwestern math department).2021–2022
Academic Activities
Research and Teaching
- IDEAL-funded summer research exchange, University of Chicago. Hosted by Prof. Chao Gao.Summer 2026
- IDEAL-funded summer research exchange, Toyota Technological Institute at Chicago (TTIC). Hosted by Prof. Zhiyuan Li.Summer 2025
- Teaching Assistant, CS 336: Design and Analysis of Algorithms.Spring 2026
- Teaching Assistant, CS 335: Introduction to the Theory of Computing. Guest lecture on busy beaver numbers.Fall 2024
Service and Programs
- Program Committee / Reviewer, Reliable ML from Unreliable Data, NeurIPS 2025 Workshop.2025
- Volunteer, FOCS 2024 (65th IEEE Symposium on Foundations of Computer Science), Chicago, IL.2024
- Directed Reading Program: read Linear Representations of Finite Groups with Wenyuan Li.Spring 2021
- Ross Mathematics Program (Asia): first-year student (2018), junior counselor (2019), counselor (2020).2018–2020
Miscellaneous
I have written three articles on 知乎 about theoretical computer science. Here is the link.
More personal notes
I grew up in Beijing and still have a soft spot for its subway system and shared bicycles.