Fengyuan Liu

Vector Institute

108 College St.

Toronto, Ontario, M5G 0C6

I am a fourth-year undergraduate student major in Engineering Science - Machine Intelligence at the University of Toronto. The goal of my research is to create efficient, adaptable AI systems that can seamlessly integrate into everyday life, ensuring accessibility and trust for all.

Currently, I am a research intern at Vector Institute and the University of Toronto under the supervision of Prof. Colin Raffel, where I work on large language model (LLM) attribution, modularity, and knowledge transfer. Previously, I have spent a year at Huawei Canada working on AI infrastructure, network protocols, network simulators, and AI for network. Additionally, I worked at the Advanced Electronics and Photonics Research Center at the National Research Council Canada under the guidance of Dr. Guocheng Liu, where I applied machine learning techniques to digital signal processing in optical and wireless communications.

Selected Publications

ICLR

AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution

Fengyuan Liu, Nikhil Kandpal, and Colin Raffel

In The Thirteenth International Conference on Learning Representations, 2025

Abs arXiv PDF Code

The influence of contextual input on the behavior of large language models (LLMs) has prompted the development of context attribution methods that aim to quantify each context span’s effect on an LLM’s generations. The leave-one-out (LOO) error, which measures the change in the likelihood of the LLM’s response when a given span of the context is removed, provides a principled way to perform context attribution, but can be prohibitively expensive to compute for large models. In this work, we introduce AttriBoT, a series of novel techniques for efficiently computing an approximation of the LOO error for context attribution. Specifically, AttriBoT uses cached activations to avoid redundant operations, performs hierarchical attribution to reduce computation, and emulates the behavior of large target models with smaller proxy models. Taken together, AttriBoT can provide a >300x speedup while remaining more faithful to a target model’s LOO error than prior context attribution methods. This stark increase in performance makes computing context attributions for a given response 30x faster than generating the response itself, empowering real-world applications that require computing attributions at scale. We release a user-friendly and efficient implementation of AttriBoT to enable efficient LLM interpretability as well as encourage future development of efficient context attribution methods.