Skip to content
Primary Menu

Education, Presentation, Publication

Funding & Recognition

Responsible Use of AI as a Research Method to Improve Public Engagement in Policy-Making

Summer 2026


Project Background

Can generative artificial intelligence (AI) be used as a research method to improve dialogue between publics and government organizations? The loss of personnel at organizations like the FDA [1] has resulted in reduced expertise and capacity to perform critical functions like reviewing scientific and public inputs for drug approval decisions. While the FDA now uses an AI tool to scour scientific studies [2], there is no plan to analyze the thousands of public comments received every month regarding novel drugs.
How, if at all, might the FDA analyze these comments to address public needs and input? We focus on an FDA guidance document about drug development for Amyotrophic Lateral Sclerosis (ALS), a terminal disorder that has no cure and limited treatments. Two trained graduate students have manually conducted thematic analysis on 162 public comments on this policy document. In these comments, people share intrinsic, contextual knowledge, which is valuable but difficult to interpret. Students' coded themes will be used as a benchmark for assessing the performance of AI as a research method.
We will address the following questions: To what extent, if at all, does AI streamline time-consuming research processes like qualitative data coding? What textual prompts and which models yield themes that approximate those found by trained human coders? Irresponsible use of AI as a research method can have severe consequences; e.g., results that are not valid/reproducible can undermine the scientific enterprise, reduce credibility of institutional experts, and further decrease societal trust in institutions and associated actors.

This project will also be mentored by co-PI Dr. Sara Yeo

Student Role

The SPUR student will build a novel foundational framework of effective approaches for AI-powered qualitative data coding under the joint guidance of Drs. Pandey and Yeo. The student will engage in ideation for the project methodology, will learn to make critical research decisions, and report on scientific data and findings. The student will perform three concrete activities:
  1. Understand the current state of the art and practice. The student will perform a focused software and literature review of current approaches regarding AI's role in the research process. Existing qualitative data analysis software with embedded AI tools do not generate useful themes. Simultaneously, scientific approaches to studying AI's efficacy are limited to evaluating how well AI performs compared to expert humans and do not propose new ways to combine manual and AI tools.
  2. Conduct empirical studies to assess the quality of work enabled by AI. The student will use a combination of qualitative and quantitative methods to study multiple AI tools' codes and themes compared to trained human coders. Questions include: Are the results generated by AI reliable, valid, and reproducible? To what extent do hallucinations occur? Can the AI tool extract emotional themes from the data?
  3. Design new techniques to improve the quality of human-AI collaboration. The student will design techniques that identify and rectify challenges when using AI as a research method. For example, how can the context of use be embedded in the output from such tools? Who will use such a tool? What are the various users' needs?

Student Learning Outcomes and Benefits

The SPUR student will benefit from experiential learning, where active engagement deepens their understanding of research and allows them to develop related skills. The undergraduate researcher will learn to explicate concepts, assess reliability and validity, and navigate the complexities of thematic analysis. This process fosters professional growth through writing, presentations, and collaboration in a supportive learning environment with two co-mentors.
Additionally, in the new era of AI tools, observing students' engagement with this tool can reveal how they evolve intellectually and adapt to new tools and technology. AI can assist with tasks like inductive coding, but overdependence may hinder novice researchers' ability to perform such analysis independently. A critical learning outcome, therefore, is understanding and learning how to use AI responsibly-not as a shortcut, but as an assistive tool. This interaction prompts reflection on how research methods are changing given this new frontier. Ultimately, they gain not only technical proficiency but also awareness to ask deeper questions about methodology, ethics, and the role of technology in research.
Overall, the SPUR student will take a significant step towards becoming an independent researcher by (i) performing interdisciplinary research (mentored by faculty in Communication and Computing) on a timely and exciting topic; (ii) developing qualitative and AI methods that can be applied to many areas; (iii) communicating their work to multiple audiences (e.g., in Computing (including HCI and AI) and Communication); and (iv) embedding themselves in an active, supportive, thoughtful community of practice that includes members of the co-mentors' research groups.

Vineet Pandey

Assistant Professor
Engineering
School of Computing

My mentoring philosophy is rooted in creative growth, scientific rigor, and the collaborative pursuit of knowledge that leads to impact. I am committed to fostering an environment that promotes socially responsible research, academic and professional development, and intellectual curiosity. I intend to help mentees grow to be independent thinkers who can design, engineer, and implement high-quality approaches to challenging problems. Mentees will be expected to be present, speak up, and to always be professional. Regular interaction, openness to feedback, and a willingness to learn quickly are essential. I strive to be accessible and supportive, tailoring guidance to mentee's unique background and goals. I seek early conversations over possibly challenging interpersonal aspects of research including authorship, data collection and analysis, and plans for submitting research that maximally benefits the mentee's growth. Specific mentoring activities include discussing and agreeing on a compact for the working relationship between mentees and mentors at the beginning of the SPUR experience, and regular (typically, weekly) meetings with the mentee. Mentees are expected to attend weekly lab meetings where they will present and answer questions about their project. During lab meetings, mentors will answer questions and guide the mentee through the project and engage in the mentee's professional development. Mentees will complete CITI IRB training.