publications

2026

1. UIST EA ’26

   TractorBeam: Personalized AI Sensemaking Support via Collaborative Machine Annotation

   Sireesh Gururaja, Jordan Taylor, and Emma Strubell

   In The 39th Annual ACM Symposium on User Interface Software and Technology, Detroit, MI, USA, Nov 2026

   DOI
2. AIES ’26

   Locating Translation as a Craft in the Age of AI

   Daniel Chechelnitsky, Sireesh Gururaja, Seyi Olojo, and 4 more authors

   Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Oct 2026

   DOI
3. Big Data and Society

   Basic research, lethal effects: Military artificial intelligence research funding as enlistment

   David Gray Widder, Sireesh Gururaja, and Lucy Suchman

   Big Data & Society, Oct 2026

   Abs DOI

   In the context of unprecedented U.S. Department of Defense (DoD) budgets, this paper examines the recent history of DoD funding for academic research in algorithmically based warfighting. We draw from a corpus of DoD grant solicitations from 2007 to 2023, focusing on those addressed to researchers in the field of artificial intelligence (AI). Considering the implications of DoD funding for academic research, the paper proceeds through three analytic sections. In the first, we build on past scholarship to offer a critical examination of the distinction between basic and applied research, showing how funding calls framed as basic research nonetheless enlist researchers in a warfighting agenda. In the second, we offer a diachronic analysis of the corpus, showing how a ‘one small problem’ caveat, in which affirmation of progress in military technologies is qualified by acknowledgement of outstanding problems, becomes justification for additional investments in research. We close with an analysis of DoD aspirations based on a subset of Defense Advanced Research Projects Agency grant solicitations for the use of AI in battlefield applications. Taken together, we argue that grant solicitations work as a vehicle for the mutual enlistment of DoD funding agencies and the academic AI research community in setting research agendas. The trope of basic research in this context offers shelter from significant moral questions that military applications of one’s research would raise, by obscuring the connections that implicate researchers in U.S. militarism despite the pervasive presence of those connections in funding documents.

4. CHI-STAR Workshop

   Collaborative Annotation as an Interface Metaphor for Personalizable AI Reading Support

   Sireesh Gururaja, and Emma Strubell

   Oct 2026

   PDF

2025

1. NeurIPS Workshop

   Extracting Structured Policy Information from Climate Action Plans

   Tom Corringham, Nupoor Gandhi, Bryan Flores, and 4 more authors

   In NeurIPS 2025 Workshop on Tackling Climate Change with Machine Learning, Oct 2025

   Abs PDF

   Most of the world’s climate action policies are planned and implemented at the local level, through city and regional climate action plans (CAPs). To assess global progress in climate mitigation and adaptation, as in forthcoming assessments such as the 2027 IPCC Special Report on Climate Change and Cities, we need systematic ways to track and analyze these plans. However, CAPs are dispersed across thousands of jurisdictions, vary widely in structure and format, and are often difficult to access. We propose a standard CAP ontology, and a retrieval- and extraction-oriented pipeline that leverages recent advances in natural language processing (NLP) and information retrieval (IR) to transform CAPs into a structured, verifiable dataset of climate policies. As a case study, we focus on California, where more than 260 local governments have published one or more CAPs since 2006. We develop an annotated benchmark dataset of 17 San Diego County CAPs with over 1,800 extracted policies and associated attributes. Unlike prior efforts that rely on small annotated corpora or industry-specific disclosures, our system explicitly grounds every extracted element in its underlying PDF, ensuring transparency and reducing hallucination in the produced dataset. Addressing these challenges will enable large-scale comparative analyses of CAPs across jurisdictions world-wide, supporting policymakers, sustainability officers, and hazard managers, and accelerating climate adaptation and mitigation efforts.

2. Cambridge Forum

   AI interdisciplinarity as critical technical practice

   Lucy Suchman, Sireesh Gururaja, and David Gray Widder

   Cambridge Forum on AI: Culture and Society, Oct 2025

   DOI PDF
3. ACL ’25 Findings

   Beyond Text: Expert Needs in Document Research

   Sireesh Gururaja, Nupoor Gandhi, Jeremiah Milbauer, and 1 more author

   Oct 2025

   Abs PDF

   Working with documents is a key part of almost any knowledge work, from contextualizing research in a literature review to reviewing legal precedent. Recently, as their capabilities have expanded, primarily text-based NLP systems have often been billed as able to assist or even automate this kind of work. But to what extent are these systems able to model these tasks as experts conceptualize and perform them now? In this study, we interview sixteen domain experts across two domains to understand their processes of document research, and compare it to the current state of NLP systems. We find that our participants processes are idiosyncratic, iterative, and rely extensively on the social context of a document in addition its content; existing approaches in NLP and adjacent fields that explicitly center the document as an object, rather than as merely a container for text, tend to better reflect our participants’ priorities, though they are often less accessible outside their research communities. We call on the NLP community to more carefully consider the role of the document in building useful tools that are accessible, personalizable, iterative, and socially aware.

4. SDProc@ACL ’25

   Collage: Decomposable Rapid Prototyping for Information Extraction on Scientific PDFs

   Sireesh Gururaja, Yueheng Zhang, Guannan Tang, and 6 more authors

   Oct 2025

   Abs PDF

   Recent years in NLP have seen the continued development of domain-specific information extraction tools for scientific documents, alongside the release of increasingly multimodal pretrained transformer models. While the opportunity for scientists outside of NLP to evaluate and apply such systems to their own domains has never been clearer, these models are difficult to compare: they accept different input formats, are often black-box and give little insight into processing failures, and rarely handle PDF documents, the most common format of scientific publication. In this work, we present Collage, a tool designed for rapid prototyping, visualization, and evaluation of different information extraction models on scientific PDFs. Collage allows the use and evaluation of any HuggingFace token classifier, several LLMs, and multiple other task-specific models out of the box, and provides extensible software interfaces to accelerate experimentation with new models. Further, we enable both developers and users of NLP-based tools to inspect, debug, and better understand modeling pipelines by providing granular views of intermediate states of processing. We demonstrate our system in the context of information extraction to assist with literature review in materials science.

5. Non-archival

   Data-driven Design as a High-Impact, Ecologically Valid Benchmark for Document Understanding

   Sireesh Gururaja, Junwon Seo, Hung-Yi Lin, and 3 more authors

   Oct 2025

   Abs PDF

   In this work we present a challenge dataset for few- and zero-shot multimodal information extraction to support the data-driven design (DDD) of materials. The benchmark repurposes manually-verified tabular data from \citetjensen_machine_2019’s study of zeolite synthesis. The proposed dataset is intended to evaluate systems’ capabilities in information extraction, disambiguation, and normalization from tables and related text (e.g. captions), in both multimodal and text-only settings. We argue that data-driven design presents a promising task — data-rich, useful, and challenging — against which to benchmark next-generation information extraction systems.

6. CHI Case Study

   LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs

   Tongshuang Wu, Haiyi Zhu, Maya Albayrak, and 21 more authors

   In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, , Oct 2025

   Abs DOI

   LLMs have shown promise in replicating human-like behavior in crowdsourcing tasks that were previously thought to be exclusive to human abilities. However, current efforts focus mainly on simple atomic tasks. We explore whether LLMs can replicate more complex crowdsourcing pipelines. We find that modern LLMs can simulate some of crowdworkers’ abilities in these “human computation algorithms,” but the level of success is variable and influenced by requesters’ understanding of LLM capabilities, the specific skills required for sub-tasks, and the optimal interaction modality for performing these sub-tasks. We reflect on human and LLMs’ different sensitivities to instructions, stress the importance of enabling human-facing safeguards for LLMs, and discuss the potential of training humans and LLMs with complementary skill sets. Crucially, we show that replicating crowdsourcing pipelines offers a valuable platform to investigate 1) the relative LLM strengths on different tasks (by cross-comparing their performances on sub-tasks) and 2) LLMs’ potential in complex tasks, where they can complete part of the tasks while leaving others to humans.

2023

1. EMNLP ’23

   To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing

   Sireesh Gururaja, Amanda Bertsch, Clara Na, and 2 more authors

   In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Dec 2023

   Abs DOI

   NLP is in a period of disruptive change that is impacting our methodologies, funding sources, and public perception. In this work, we seek to understand how to shape our future by better understanding our past. We study factors that shape NLP as a field, including culture, incentives, and infrastructure by conducting long-form interviews with 26 NLP researchers of varying seniority, research area, institution, and social identity. Our interviewees identify cyclical patterns in the field, as well as new shifts without historical parallel, including changes in benchmark culture and software infrastructure. We complement this discussion with quantitative analysis of citation, authorship, and language use in the ACL Anthology over time. We conclude by discussing shared visions, concerns, and hopes for the future of NLP. We hope that this study of our field’s past and present can prompt informed discussion of our community’s implicit norms and more deliberate action to consciously shape the future.

2. ACL ’23

   Linguistic representations for fewer-shot relation extraction across domains

   Sireesh Gururaja, Ritam Dutt, Tinglong Liao, and 1 more author

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023

   Abs DOI

   Recent work has demonstrated the positive impact of incorporating linguistic representations as additional context and scaffolds on the in-domain performance of several NLP tasks. We extend this work by exploring the impact of linguistic representations on cross-domain performance in a few-shot transfer setting. An important question is whether linguistic representations enhance generalizability by providing features that function as cross-domain pivots. We focus on the task of relation extraction on three datasets of procedural text in two domains, cooking and materials science. Our approach augments a popular transformer-based architecture by alternately incorporating syntactic and semantic graphs constructed by freely available off-the-shelf tools. We examine their utility for enhancing generalization, and investigate whether earlier findings, e.g. that semantic representations can be more helpful than syntactic ones, extend to relation extraction in multiple domains. We find that while the inclusion of these graphs results in significantly higher performance in few-shot transfer, both types of graph exhibit roughly equivalent utility.

2022

1. DialDoc at ACL ’22

   R3 : Refined Retriever-Reader pipeline for Multidoc2dial

   Srijan Bansal, Suraj Tripathi, Sumit Agarwal, and 5 more authors

   In Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering, May 2022

   Abs DOI

   In this paper, we present our submission to the DialDoc shared task based on the MultiDoc2Dial dataset. MultiDoc2Dial is a conversational question answering dataset that grounds dialogues in multiple documents. The task involves grounding a user’s query in a document followed by generating an appropriate response. We propose several improvements over the baseline’s retriever-reader architecture to aid in modeling goal-oriented dialogues grounded in multiple documents. Our proposed approach employs sparse representations for passage retrieval, a passage re-ranker, the fusion-in-decoder architecture for generation, and a curriculum learning training paradigm. Our approach shows a 12 point improvement in BLEU score compared to the baseline RAG model.