Kumara Kahatapitiya

I am a PhD candidate at Stony Brook University, working with Prof. Michael S. Ryoo, and an intern at Qualcomm AI Research, Amsterdam. My primary research focus is on video understanding. More recently, I also started working on video-language models, vision transformers and video diffusion models.

During my PhD, I was a student researcher at Google Brain, and an intern at Wormpex AI Research. Prior to this, I was a Research Assistant at University of Moratuwa, Sri-Lanka, advised by Dr. Ranga Rodrigo, where I also received my Bachelors in Electronic & Telecommunication Engineering.

[Google Scholar] [GitHub] [Twitter]

Recent News

[July 2023]	I joined Qualcomm AI Research, Amsterdam as a research intern.
[Apr 2023]	SWAT, a structure-aware family of token-based models was accepted at IJCAI 2023.
[Apr 2023]	Video-conditioned Text Representations for activity recognition is now on arXiv.
[Feb 2023]	Token Turing Machines for long-term memory in Transformers was accepted at CVPR 2023.
[Dec 2022]	Weakly-guided Self-supervised detection pretraining was accepted at AAAI 2023.
[Nov 2022]	Grafting Vision Transformers, an efficient add-on for multi-scale and global information sharing is now on arXiv.
[Sep 2022]	StARformer extended to real-world robot environments was accepted at T-PAMI.
[Jul 2022]	StARformer with an MDP-like inductive bias for RL was accepted at ECCV 2022.
[Mar 2022]	MS-TCT for temporal action detection with CNN+Transformer embeddings was accepted at CVPR 2022.
[Feb 2022]	I joined Robotics at Google as a student researcher.
[Dec 2021]	I was a finalist (1/30) for the Adobe Research Fellowship 2022. Congratulations to all the winners!
[Dec 2021]	Swift for real-time neural video decoding was accepted at NSDI 2022.
[Sep 2021]	I am officially a PhD candidate now!
[Mar 2021]	Coarse-Fine Networks for efficient temporal activity detection was accepted at CVPR 2021.
[Jan 2021]	Exploiting Redundancy in CNNs for parameter reduction was accepted at WACV 2021.

Selected Publications

	VicTR: Video-conditioned Text Representations for Activity Recognition Kumara Kahatapitiya, Anurag Arnab, Arsha Nagrani, Michael S. Ryoo arXiv 2023 [arxiv]
	SWAT: Spatial Structure Within and Among Tokens Kumara Kahatapitiya, Michael S. Ryoo IJCAI 2023 [paper] [code coming soon]
	Token Turing Machines Michael S. Ryoo, Keerthana Gopalakrishnan, Kumara Kahatapitiya, Ted Xiao, Kanishka Rao, Austin Stone, Yao Lu, Julian Ibarz, Anurag Arnab CVPR 2023 [paper] [code]
	Weakly-guided Self-supervised Pretraining for Temporal Activity Detection Kumara Kahatapitiya, Zhou Ren, Haoxiang Li, Zhenyu Wu, Michael S. Ryoo, Gang Hua AAAI 2023 [paper] [code] [talk] [poster]
	Grafting Vision Transformers Jongwoo Park, Kumara Kahatapitiya, Donghyun Kim, Shivchander Sudalairaj, Quanfu Fan, Michael S. Ryoo arXiv 2022 [arxiv]
	StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning Jinghuan Shang, Kumara Kahatapitiya, Xiang Li, Michael S. Ryoo ECCV 2022, TPAMI [paper] [journal] [code] [talk] [poster]
	MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detection Rui Dai, Srijan Das, Kumara Kahatapitiya, Michael S. Ryoo, Francois Bremond CVPR 2022 [paper] [code] [poster]
	Swift: Adaptive Video Streaming with Layered Neural Codecs Mallesham Dasari, Kumara Kahatapitiya, Samir Das, Aruna Balasubramanian, Dimitris Samaras NSDI 2022 [paper] [code] [slides]
	Coarse-Fine Networks for Temporal Activity Detection in Videos Kumara Kahatapitiya, Michael S. Ryoo CVPR 2021 [paper] [code] [talk] [poster]
	Exploiting the Redundancy in Convolutional Filters for Parameter Reduction Kumara Kahatapitiya, Ranga Rodrigo WACV 2021 [paper] [code] [talk]
	Feature-dependent Cross-Connections in Multi-Path Neural Networks Dumindu Tissera, Kasun Vithanage, Rukshan Wijesinghe, Kumara Kahatapitiya, Subha Fernando, Ranga Rodrigo ICPR 2020 [paper]
	Context-Aware Automatic Occlusion Removal Kumara Kahatapitiya, Dumindu Tissera, Ranga Rodrigo ICIP 2019 [paper] [code]

Open-source Projects

X3D-Multigrid [code]
A PyTorch implementation for "X3D: Expanding Architectures for Efficient Video Recognition models" [CVPR2020] with "A Multigrid Method for Efficiently Training Video Models" [CVPR2020]. In contrast to the original repository by FAIR, this repository provides a simpler, less modular and more familiar structure of implementation for faster and easier adoptation.
Optimal Transport in NumPy [code]
This repository contrains a few Optimal Transport Algorithms implemented using NumPy, including "A Direct O(1/epsilon) Iteration Parallel Algorithm for Optimal Transport" [NeurIPS2019], "Computational Optimal Transport: Complexity by Accelerated Gradient Descent is better than by Sinkhorn's Algorithm" [PMLR2018] and "Lightspeed Computation of Optimal Transport" [NeurIPS2013].

Teaching

CSE327: Computer Vision - TA (Spring 2020)
CSE215: Foundations of Computer Science - TA (Fall 2019)

Website template is here. Thanks!