publications

2021

1. HAMMER

   HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging

   Gupta, Nikunj, Srinivasaraghavan, G., Mohalik, Swarup Kumar, Kumar, Nishant, and Taylor, Matthew E.

   ALA 2021

   Abs PDF

   Cooperative multi-agent reinforcement learning (MARL) has achieved significant results, most notably by leveraging the representation learning abilities of deep neural networks. However, large centralized approaches quickly become infeasible as the number of agents scale, and fully decentralized approaches can miss important opportunities for information sharing and coordination. Furthermore, not all agents are equal - in some cases, individual agents may not even have the ability to send communication to other agents or explicitly model other agents. This paper considers the case where there is a single, powerful, central agent that can observe the entire observation space, and there are multiple, low powered, local agents that can only receive local observations and cannot communicate with each other. The job of the central agent is to learn what message to send to different local agents, based on the global observations, not by centrally solving the entire problem and sending action commands, but by determining what additional information an individual agent should receive so that it can make a better decision. After explaining our MARL algorithm, hammer, and where it would be most applicable, we implement it in the cooperative navigation and multi-agent walker domains. Empirical results show that 1) learned communication does indeed improve system performance, 2) results generalize to multiple numbers of agents, and 3) results generalize to different reward structures.

2. ESPA

   Evolutionary Adversarial Attacks on Payment Systems

   Kumar, Nishant, Vimal, Siddharth, Kayathwal, Kanishka, and Dhama, Gaurav

   In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA) 2021

   Abs PDF

   Credit card fraud detection is arguably the most critical use case of machine learning for any payment system. Deep neural networks and tree-based classifiers can provide state-of-the-art performance for fraud classification. However, we try to emphasize that these models have serious vulnerabilities that need to be addressed. Studies show that it is possible to fool machine learning models with curated input samples known as adversarial examples. Attackers can use these examples to deceive the fraud classifiers deployed by institutions, causing considerable financial harm. We feel that the literature on adversarial examples for fraud detection systems has been limited to simpler datasets. In this paper, we use two large publicly available datasets for credit card fraud detection to benchmark the performance of some conventional machine learning models and compare the effectiveness of different black-box attacks on the best-performing model. Lastly, we introduce a novel gradient-free approach to black-box attacks, which uses evolution-based specialized perturbations to create attacks (ESPA). We show that the new method requires far fewer queries than other black-box attack methods like Zeroth Order optimization, Boundary Attack, and HopSkipJump, and can leverage the information gained from previously successful attacks.