Fangyu Lei 雷方雨
I am Fangyu Lei, a third-year PhD student at University of Chinese Academy of Sciences, advised by Prof. Kang Liu.
I was fortunate to collaborate with Dr. Shizhu He, Dr. Qian Liu and Prof. Tao Yu.
My research interest lies in NLP, especially on LLM-Agent, code generation, table reasoning and text-to-SQL.
 / 
 / 
 / 
|
|
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
Fangyu Lei*, Jixuan Chen*, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, caiming xiong, Ruoxi Sun, Qian Liu, Sida Wang, Tao Yu
Preprint, 2024
|
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Yuchen Mao, Wenjing Hu, Tianbao Xie, Hongshen Xu, Danyang Zhang, Sida Wang, Ruoxi Sun, Pengcheng Yin, Caiming Xiong, Ansong Ni, Qian Liu, Victor Zhong, Lu Chen, Kai Yu, Tao Yu
NeurIPS, 2024
|
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu
NeurIPS, 2024
|
S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models
Fangyu Lei, Qian Liu, Yiming Huang, Shizhu He, Jun Zhao, Kang Liu
NAACL, 2024
|
S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering
Fangyu Lei, Shizhu He, Xiang Li, Yifan Wei, Yiming Huang, Jun Zhao, Kang Liu
ACL, 2023
|
Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decoderestion Answering
Fangyu Lei, Shizhu He, Xiang Li, Jun Zhao, Kang Liu
COLING, 2022
|
DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models
Yiming Huang*, Jianwen Luo*, Fangyu Lei*, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, Kang Liu
EMNLP, 2024
|
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
Xiaoyan Yu, Tongxu Luo, Yifan Wei, Fangyu Lei, Yiming Huang, Hao Peng, Liehuang Zhu
EMNLP, 2024
|
Competition-level problems are effective llm evaluators
Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Fangyu Lei, Chen Lin, Nan Duan, Weizhu Chen
ACL, 2024
|
Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering
Xiang Li, Shizhu He, Fangyu Lei, Jun Yang, Tianhuang Su, Kang Liu, Jun Zhao
ACL, 2024
|
MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Models
Yifan Wei, Yisong Su, Huanhuan Ma, Xiaoyan Yu, Fangyu Lei, Yuanzhe Zhang, Jun Zhao, Kang Liu
EMNLP, 2023
|
TableQAKit: A Comprehensive and Practical Toolkit for Table-based Question Answering
Fangyu Lei, Tongxu Luo, Pengqi Yang, Weihao Liu, Shizhu He, Jun Zhao, Kang Liu
Preprint, 2023
|
|