Tianpei Gu

Hey there, thanks for visiting my website. I am Tianpei Gu, currently a Research Scientist at ByteDance, based in San Jose, California. I like writing code and building things, and believe everyone should be full-stack in developing AI. Previously, I spent a couple years at GenAI startups in SF training foundation image/video models and shipping products.

My current research focuses on human-centric video generation and real-time interactive systems, or world models. I believe image and video generation models are far more than just a toy where you enter a prompt and get an image — they will have a great impact on our daily lives, just like LLMs. These models should have intelligence, not just act like render engines. For all AI products, I have my own “mom benchmark”: how can this product be used by ordinary people like my mom? To make video models more accessible, they have to be real-time and interactive. To make them viral, they have to generate content that’s not only visually appealing but also emotionally engaging. With that goal in mind, I’m mainly working on the following areas.

• Data Pipeline: The development of efficient and scalable data pipeline is still very under-explored, and the credits for data people are generally insifficient. I have developed data pipelines (image and video) at multiple companies that scale to thousands of GPUs and process trillions of data, serving large number of researchers and powering critical projects.
• Real-time Interactive Human Models: The final form of real-time human models consists of the following components:
  • Long-context memory and consistency
  • Intelligent modality interaction and alignment
  • Unified understanding and generation
  • Video model acceleration
  All of the above are very challenging and require systematic efforts, but I believe we will eventually get there. In [X-Streamer], we introduced the first human world model that can talk to users in real-time infinitely, with intelligence and memory.
• Motion and Expression Modeling: I study how to capture and model complex human motions, facial expressions, and emotional states in video content. This includes precise body motion modeling [X-UniMotion], human id perservation [Lynx], and facial expression modeling while syncing with audio [X-Actor].

Apart from research, I enjoy exploring new technologies and creative applications of AI. I also enjoy cooking and playing Dota2. I have a cute cat named "贝壳" (Bayker in English) and I love to play with him. The below is a gallery of Bayker. You are welcome to see more on Instagram and Xiaohongshu.