Education

PhD, Machine and Language Learning Lab (MALL Lab), 2016 – 2022 (Defended April 2023)
Dept. of Computer Science & Automation (CSA), Indian Institute of Science, Bangalore

Masters in Computer Science, 2014 – 2016
Chennai Mathematical Institute (CMI), Chennai

B.Tech, 2008 – 2012
Dept. of Mechanical Engineering, Indian Institute of Technology (IIT) , Kharagpur

Work Experience

Amazon Search Science Team, Amazon, Bangalore, August 2022 – Present
Applied Scientist II

Amazon Lex Team, AWS AI, Seattle (Remote), July 2020 – Oct 2020
Applied Scientist Intern

Linear Algebra and its applications, Indian Institute of Science, Bangalore, Aug 2017 – Dec 2017
Teaching Assistant

Amazon Core Machine Learning Team, Amazon, Bangalore, June 2017 – Aug 2017
Applied Science Intern

Machine Learning Team, Real Image Media Tech., Chennai, March 2016 – June 2016
Intern

Product Engineering, eGain Communications Pvt. Ltd., Pune, 2012 – 2014
Software Engineer

PhD Thesis

Inducing Constraints in Paraphrase Generation and Consistency in Paraphrase Detection
Thesis slides

Publications

1. Striking a Balance: Alleviating Inconsistency in Pre-trained Models for Symmetric Classification Tasks
   Ashutosh Kumar, Aditya Joshi
   Accepted as a Short Paper in the Findings of Association for Computational Linguistics (ACL) 2022
   abstract paper code
   While fine-tuning pre-trained models for downstream classification is the conventional paradigm in NLP, often task-specific nuances may not get captured in the resultant models. Specifically, for tasks that take two inputs and require the output to be invariant of the order of the inputs, inconsistency is often observed in the predicted labels or confidence scores. We highlight this model shortcoming and apply a consistency loss function to alleviate inconsistency in symmetric classification. Our results show an improved consistency in predictions for three paraphrase detection datasets without a significant drop in the accuracy scores. We examine the classification performance of six datasets (both symmetric and non-symmetric) to showcase the strengths and limitations of our approach.
2. Syntax-guided Controlled Generation of Paraphrases
   Ashutosh Kumar, Kabir Ahuja, Raghuram Vadapalli, Partha Talukdar
   Transactions of the Association for Computational Linguistics (TACL) 2020
   Presented at ACL 2020
   abstract paper code slides
   Given a sentence (e.g., "I like mangoes") and a constraint (e.g., sentiment flip), the goal of controlled text generation is to produce a sentence that adapts the input sentence to meet the requirements of the constraint (e.g., "I hate mangoes"). Going beyond such simple constraints, recent works have started exploring the incorporation of complex syntactic-guidance as constraints in the task of controlled paraphrase generation. In these methods, syntactic-guidance is sourced from a separate exemplar sentence. However, these prior works have only utilized limited syntactic information available in the parse tree of the exemplar sentence. We address this limitation in the paper and propose Syntax Guided Controlled Paraphraser (SGCP), an end-to-end framework for syntactic paraphrase generation. We find that SGCP can generate syntax conforming sentences while not compromising on relevance. We perform extensive automated and human evaluations over multiple real-world English language datasets to demonstrate the efficacy of SGCP over state-of-the-art baselines. To drive future research, we have made SGCP’s source code available.
3. Submodular Optimization-based Diverse Paraphrasing and its Effectiveness in Data Augmentation
   Ashutosh Kumar*, Satwik Bhattamishra*, Manik Bhandari, Partha Talukdar
   Oral Presentation : Long paper in NAACL 2019, Minneapolis, USA
   *Equal Contributions
   abstract paper code slides
   Inducing diversity in the task of paraphrasing is an important problem in NLP with applications in data augmentation and conversational agents. Previous paraphrasing approaches have mainly focused on the issue of generating semantically similar paraphrases, while paying little attention towards diversity. In fact, most of the methods rely solely on top-k beam search sequences to obtain a set of paraphrases. The resulting set, however, contains many structurally similar sentences. In this work, we focus on the task of obtaining highly diverse paraphrases while not compromising on paraphrasing quality. We provide a novel formulation of the problem in terms of monotone submodular function maximization, specifically targeted towards the task of paraphrasing. Additionally, we demonstrate the effectiveness of our method for data augmentation on multiple tasks such as intent classification and paraphrase recognition. In order to drive further research, we have made the source code available.
4. eCommerceGAN : A Generative Adversarial Network for E-commerce
   Ashutosh Kumar, Arijit Biswas, Subhajit Sanyal
   Accepted as a Workshop paper in ICLR 2018, Vancouver, Canada
   abstract paper media
   E-commerce companies such as Amazon, Alibaba and Flipkart process billions of orders every year. However, these orders represent only a small fraction of all plausible orders. Exploring the space of all plausible orders could help us better understand the relationships between the various entities in an e-commerce ecosystem, namely the customers and the products they purchase. In this paper, we propose a Generative Adversarial Network (GAN) for orders made in e-commerce websites. Once trained, the generator in the GAN could generate any number of plausible orders. Our contributions include: (a) creating a dense and low-dimensional representation of e-commerce orders, (b) train an ecommerceGAN (ecGAN) with real orders to show the feasibility of the proposed paradigm, and (c) train an ecommerce-conditional-GAN (ec2GAN) to generate the plausible orders involving a particular product. We propose several qualitative methods to evaluate ecGAN and demonstrate its effectiveness. The ec2GAN is used for various kinds of characterization of possible orders involving a product that has just been introduced into the e-commerce system. The proposed approach ec2GAN performs significantly better than the baseline in most of the scenarios.
5. NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
   Kaustubh D. Dhole et. al, (includes: Ashutosh Kumar )
   Arxiv pre-print 2021
   abstract paper code
   Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on. In this paper, we present NL-Augmenter, a new participatory Python-based natural language augmentation framework which supports the creation of both transformations (modifications to the data) and filters (data splits according to specific features). We describe the framework and an initial set of 117 transformations and 23 filters for a variety of natural language tasks. We demonstrate the efficacy of NL-Augmenter by using several of its transformations to analyze the robustness of popular natural language models. The infrastructure, datacards and robustness analysis results are available publicly on the NL-Augmenter repository.
6. Discovering Non-Monotonic Autoregressive Ordering for Text Generation Models using Sinkhorn Distributions
   Ashutosh Kumar
   ICLR Blog Post 2022
   abstract blog
   In this post, we will be focusing on one of the lesser explored area - Non-Monotonic Autoregressive Order (NMAO) in Decoding. In particular, we will be discussing the ICLR 2021 paper “Discovering Non-monotonic Autoregressive Orderings with Variational Inference” [Li 2021], the background needed for understanding the work as well as the way-forward.

Service

Reviewer

2020: ICLR, ACL, EMNLP [Outstanding Reviewer]
2021: ICLR, EACL, NAACL, ACL-IJCNLP, EMNLP, NeurIPS
2022: ICLR, ACL ARR, NeurIPS
2023: ICLR, ACL ARR, EMNLP, NeurIPS

Judge

Initiative for Research & Innovation in STEM (IRIS) National Fair 2020

Contact

Address

Amazon Development Center India
Bagmane Constellation Business Park,
Bangalore - 560048

Email

firstname"k2401"[at]gmail[dot]com