Fangyu Lei 雷方雨

I am Fangyu Lei, a third-year PhD student at University of Chinese Academy of Sciences, advised by Prof. Kang Liu.

I was fortunate to collaborate with Dr. Shizhu He, Dr. Qian Liu and Prof. Tao Yu.

My research interest lies in NLP, especially on LLM-Agent, code generation, table reasoning and text-to-SQL.

Email  /  Google Scholar  /  Twitter  /  Twitter

profile photo
Publications
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
Fangyu Lei*, Jixuan Chen*, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, caiming xiong, Ruoxi Sun, Qian Liu, Sida Wang, Tao Yu
Preprint, 2024
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Yuchen Mao, Wenjing Hu, Tianbao Xie, Hongshen Xu, Danyang Zhang, Sida Wang, Ruoxi Sun, Pengcheng Yin, Caiming Xiong, Ansong Ni, Qian Liu, Victor Zhong, Lu Chen, Kai Yu, Tao Yu
NeurIPS, 2024
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu
NeurIPS, 2024
S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models
Fangyu Lei, Qian Liu, Yiming Huang, Shizhu He, Jun Zhao, Kang Liu
NAACL, 2024
S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering
Fangyu Lei, Shizhu He, Xiang Li, Yifan Wei, Yiming Huang, Jun Zhao, Kang Liu
ACL, 2023
Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decoderestion Answering
Fangyu Lei, Shizhu He, Xiang Li, Jun Zhao, Kang Liu
COLING, 2022
DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models
Yiming Huang*, Jianwen Luo*, Fangyu Lei*, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, Kang Liu
EMNLP, 2024
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
Xiaoyan Yu, Tongxu Luo, Yifan Wei, Fangyu Lei, Yiming Huang, Hao Peng, Liehuang Zhu
EMNLP, 2024
Competition-level problems are effective llm evaluators
Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Fangyu Lei, Chen Lin, Nan Duan, Weizhu Chen
ACL, 2024
Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering
Xiang Li, Shizhu He, Fangyu Lei, Jun Yang, Tianhuang Su, Kang Liu, Jun Zhao
ACL, 2024
MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Models
Yifan Wei, Yisong Su, Huanhuan Ma, Xiaoyan Yu, Fangyu Lei, Yuanzhe Zhang, Jun Zhao, Kang Liu
EMNLP, 2023
TableQAKit: A Comprehensive and Practical Toolkit for Table-based Question Answering
Fangyu Lei, Tongxu Luo, Pengqi Yang, Weihao Liu, Shizhu He, Jun Zhao, Kang Liu
Preprint, 2023