NDSU assistant professor receives NSF CAREER Award to enhance software testing quality
NDSU assistant professor of computer science Ajay Jha has received a U.S. National Science Foundation CAREER Award for his project “Improving Software Test Quality and Developer Skills through Developer-in-the-Loop Test Optimization.”
NSF CAREER Awards support early-career faculty who exemplify the role of teacher-scholars through the integration of research and education.
“We are so proud of our NDSU NSF Career Awardees. These awards signify the highest honor and recognize the next generation of research leaders whose students benefit from their cutting-edge research and whose research is informed by classroom experiences,” said NDSU interim vice president for research and creative activity Heidi Grunwald. “Dr. Jha’s groundbreaking work in automated software testing and commitment to mentoring the next generation of computer scientists represent the very best of our academic community. As a society, we don’t spend a day without software. Dr. Jah’s research tackles fundamental challenges in software reliability and quality and paves the way for AI-assisted software testing thus bolstering our students’ competitive advantage in the workforce. Congratulations Dr. Jha!”
Jha joined NDSU in 2022, and his research primarily focuses on software engineering, specifically software testing and maintenance. “I study how we can help developers build and maintain high-quality software systems more effectively,” Jha said.
“We are thrilled to congratulate Dr. Jha on receiving this prestigious NSF CAREER Award,” said Alan Kallmeyer, NDSU College of Engineering Dean. “His pioneering work in AI-assisted software testing is a prime example of the groundbreaking research happening across the College of Engineering, where our faculty are driving real-world innovation and preparing students to lead in an AI-driven future.”
Software testing is a method for assessing software quality and minimizing the risk of failure during operation.
High test maintenance costs are a major challenge for software companies, and Jha pursued this award to help them reduce this time-consuming aspect of software development. “Many of these issues arise from poor testing practices, often referred to as “test smells,” which are tests that are too long or that depend on unnecessary external resources which makes them harder to maintain and less effective over time,” he said. “I aim to develop AI-assisted techniques that help developers improve test quality, strengthen their testing skills, and reduce test maintenance costs. Rather than replacing developers, my goal is to build tools that work alongside them and help them make better testing decisions.”
Software is used in everyday life, including transportation, health care, banking, agriculture and education. Failures in these systems can have economic, operational and even safety consequences, Jha said. The better the testing, the less chance of dealing with those types of problems.
“High-quality software depends not only on having many tests, but on having effective and maintainable tests,” Jha said. “Poor-quality tests can miss important defects, generate false alarms, or become expensive to maintain. Improving test quality ultimately improves software quality.”
Jha said the research for this program will combine AI-assisted tool development, empirical studies and human-centered evaluation. “My goal is to make software development more reliable, efficient, and accessible while preparing the next generation of software engineers to work effectively alongside AI,” Jha said.
“As software grows larger and more complex, testing becomes not just a final verification activity; it becomes a continuous engineering practice that influences how developers design, refactor, and evolve code,” said NDSU doctoral student Frank Kendemah, who has worked with Jha on the research. “By studying test utility methods, this research exposes an often-overlooked part of test suites that supports initialization, exercise, verification, cleanup, and reusable helper logic across many test cases in software test suites. Its impact lies in helping researchers and practitioners better understand how test code is organized, reused, and maintained over time.”
Jha’s team is working on a “developer in the loop” test framework that puts developers at the center of the testing process. Several current AI tools try to automatically generate or modify software with little involvement from software developers. Jha’s primary focus of this project are test smells, or the poor design and implementation choices in software tests, which make them hard to understand or maintain.
Jha’s AI-assisted framework takes a different approach by analyzing existing tests, identifying potential improvements, and explaining the reasoning behind its recommendations,” he said. “The developer can then decide whether to accept, modify, or reject those suggestions. This collaborative process combines the efficiency of AI with the expertise and judgment of human developers.”
One benefit of Jha’s research is the improvement of software testing practices across industries and businesses, which helps build more reliable and secure software while reducing maintenance costs. Jha said there will also be an educational component to the research, which may eventually help build the technology workforce in North Dakota.
“The research outcomes will also be integrated into NDSU courses, allowing students to learn about the latest advances in software testing and AI-assisted software engineering through hands-on learning experiences,” Jha said. “As North Dakota continues to grow its technology workforce, developing highly skilled software engineers will benefit local companies, strengthen the state's technology ecosystem, and contribute to its broader economy.”
Jha sees the research helping build collaboration between AI software testing tools and developers, instead of replacing them. “I hope this research will help establish best practices for developing AI-assisted tools that not only improve software quality but also help developers continuously strengthen their skills. Ultimately, my goal is to make software development more reliable, efficient, and accessible while preparing the next generation of software engineers to work effectively alongside AI.”
Funding for this research comes from NSF CAREER Award 2544248.