Despite 40+ years of extensive research, software maintenance and evolution have never been easy. Studies show that software maintenance claims up to 60% of the total software budget. Developers spend at least 50% of their time tackling various maintenance challenges (e.g., software bug resolution, new feature addition). With the rise of popular but highly complex computational frameworks (e.g., Large Language Models, Deep learning, Cloud Computing, Mobile Computing), software maintenance has become even more difficult and costlier. RAISE lab focuses on (a) a better understanding of software maintenance and evolution challenges (e.g., software bug resolution, new feature implementation) and (b) designing intelligent, automated, and cost-effective software solutions to overcome these challenges and thus make the developers' lives easier. RAISE Lab has three signature areas as follows.

Signature Research Areas

Automated software debugging

Software bugs and failures are inevitable! In 2017 alone, 606 software bugs cost the global economy $1.7 trillion+, with 3.7 billion people affected and 314 companies impacted! While these bugs are ubiquitous, new classes of bugs are emerging due to the adoption of popular but highly complex technologies such as Deep learning, Cloud computing, and Simulation models. Many of these bugs are hidden not only in the software code but also in other artifacts such as configuration files, training data, trained models, deployment modules, and even simulation models. Many of them are also non-deterministic and data-driven whereas the traditional software bugs are commonly logic-driven. Thus, the traditional debugging techniques that have been developed for the last 40+ years might not be enough to tackle these emerging, complex bugs. In the RAISE lab, we study the bugs from various complex software systems including machine learning software and simulation modelling software. We attempt to (a) better understand the challenges of software debugging with a particular focus on bug/fault localization, bug understanding, and bug reproduction that take up ~90% of the debugging time, and (b) design intelligent, explainable, cost-effective software solutions to overcome these challenges.

Automated code search

While finding and correcting bugs in the software is crucial, the majority of the maintenance budget (e.g., 60%) is spent incorporating new features in the existing software. Once a software product is released, requests for new features from the users are very common. To stay competitive in the market, the developers must add new, attractive features to their software at regular intervals. However, modifying an existing software is not always straightforward. A developer must know (a) how to implement a new feature and (b) where to add this feature within the software code. Thus, as a part of feature addition, developers often search for reusable code snippets not only in the local codebase but also in the web. Unfortunately, they often fail to make the right search queries. According to existing surveys, they might even fail up to 73%-88% of the time, which is a significant waste of their valuable development time. With the growing interest in prompt engineering in the context of large language models, search queries have been never more important. In RAISE Lab, we develop state-of-the-art, intelligent tools and techniques that can (a) automatically construct appropriate search queries and (b) find the desired code not only from the local codebase but from the Internet-scale code repositories (e.g., GitHub, SourceForge).

Automated code review

Once software code is modified as a part of bug resolution or feature enhancement, the modified code must be checked for errors or quality issues. Code review has been a vital quality-control practice for the last 30+ years where software code is manually checked by code reviewers (i.e., expert developers) for hidden errors or quality issues. Finding the right reviewers and ensuring the high quality of code reviews are a challenging task. Large software companies typically have large, geographically distributed development teams, which contain a tremendous pool of expertise but are too big and diversified for manual assignment of review tasks. Incorrect selection of code reviewers not only delays the reviewing process but also leads to poor reviews. Poorly written reviews often contain inaccurate suggestions and suffer from a lack of clarity or empathy when pointing out mistakes in the software code, which makes the reviews ineffective. As a result, poor-quality reviews become a waste of valuable development effort. Even at Microsoft Corporation, where 50,000+ professional developers spend six hours every week reviewing each other’s code, ~35% of their code reviews are ineffective in improving software quality. Thus, modern code review needs appropriate tools that can make it more productive and more efficient. In RAISE Lab, we design intelligent tools and techniques to (a) automatically find the right code reviewers, and (b) automatically improve poor-quality code reviews.

Active Research Topics

Bug Localization Bug Reproduction Bug Explanation Defect Prediction Duplicate Bug Detection Deep Learning Bugs Code Review Generation Patch Code Generation Search Query Reformulation Question-Answering

Concept Location Internet-scale Code Search Crowdsourced Knowledge Mining Code Comprehension Recommendation Systems in Software Engineering IDE-based Meta Search Engine Exception Handling Software Re-documentation

RAISE Team

	Masud Rahman Assistant Professor (Team Lead) Faculty of Computer Science Dalhousie University, Canada. Research Interets: Please click here!
Sigma Jahan PhD (Winter 2022 -- ) Research Interests: Bug localization, deep learning, transfer learning, and duplicate bug report detection. Usmi Mukherjee PhD (Winter 2024 -- ) Research Interests: Question-answering, information retrieval, generative AI models, and simulation models. Co-supervised By: Dr. Ahsan Habib, Dalhousie University. Asif Samir PhD (Fall 2022 -- ) Research Interests: Bug localization, question-answering, generative AI models, and information retrieval. Mehil Shah PhD (Winter 2023 -- ) Research Interests: Bug reproduction, bug localization, machine learning, and deep learning. Co-supervised By: Dr. Foutse Khomh, Polytechnique Montreal. Riasat Mahbub MCS (Fall 2023 -- ) Research Interests: Software debugging, simulation models. Co-supervised By: Dr. Ahsan Habib, Dalhousie University. Jitansh Arora BCS (Fall 2024 -- ) Research Interests: Code review automation and Generative AI.

RAISE Graduates

Parvez Mahbub MCS (Summer 2023) Thesis: Comprehending Software Bugs Leveraging Code Structures with Neural Language Models. Current Position: Software Engineer, Siemens Canada, Saskatoon Ohiduzzaman Shuvo MCS (Summer 2023) Thesis: Improving Modern Code Review Leveraging Contextual and Structural Information from Source Code. Current Position: IT Application System Analyst, City of Red Deer and Instructor, CIOT, Calgary Usmi Mukherjee MCS (Fall 2023) Thesis: Complementing Deficient Bug Reports with Missing Information Leveraging Neural Text Generation. Current Position: PhD Student, Dalhousie University. Callum MacNeil BCS (Fall 2023) Thesis: A Systematic Review of Automated Program Repair using Large Language Models. Current Position: MCS Student, Dalhousie University. Shihui Gao BCS (Fall 2023) Thesis: Code Search in the IDE with Query Reformulation. Current Position: Data Analyst, EMC, Halifax. Lareina Yang BCS (Winter 2023 -- ) Thesis: Search Term Identification for Concept Location Leveraging Word Relations. Current Position: MS Student, Cornell University, New York.

RAISE Interns & Visitors

Md Nizamuddin BCS (Spring/Summer 2024) Research Interests: Software bug reports, natural language processing, domain term classification, and duplicate bug detection. Current Position: BCS Student, SRM Institute of Science and Technology.

Join RAISE!

Recent Projects

Towards Enhancing the Reproducibility of Deep Learning Bugs: An Empirical Study (EMSE 2024)

---

Overview: Deep learning has seen significant progress across various fields. However, like any software system, deep learning models have bugs, some with serious consequences, such as crashes in autonomous vehicles. Reproducing deep learning bugs, a prerequisite for their resolution, has been a major challenge. According to an existing work, only 3% of deep learning bugs are reproducible. In this work, we investigate deep learning bug reproducibility by creating a dataset of 668 bugs from platforms like Stack Overflow and GitHub and by reproducing 165 bugs. While reproducing these bugs, we identify edit actions and useful information for their reproduction. Then we used the Apriori algorithm to identify useful information and edit actions required to reproduce specific types of bugs. In a study involving 22 developers, our recommended information from the Apriori algorithm increased bug reproducibility by 22.92% and reduced reproduction time by 24.35%. These findings provide valuable insights to help practitioners and researchers enhance reproducibility in deep learning systems. Explore more ...



deep-learning empirical-study bug-reproduction

Explaining Software Bugs Leveraging Code Structures in Neural Machine Translation (ICSE 2023, ICSME 2023)

---

Overview: Software bugs claim ≈ 50% of development time and cost the global economy billions of dollars. Over the last five decades, there has been significant research on automatically finding or correcting software bugs. However, there has been little research on automatically explaining the bugs to the developers, which is a crucial but highly challenging task. To fill this gap, we developed Bugsplainer, a transformer-based generative model, that generates natural language explanations for software bugs by learning from a large corpus of bug-fix commits. Bugsplainer can leverage structural information and buggy patterns from the source code to generate an explanation for a bug. A developer study involving 20 participants shows that the explanations from Bugsplainer are more accurate, more precise, more concise and more useful than the baselines. Explore more ...



bug-explanation deep-learning neural-text-generation transformer-based-model

A Systematic Review of Automated Query Reformulations in Source Code Search (TOSEM 2023)

---

Overview: In this systematic literature review, we carefully select 70 primary studies on query reformulations from 2,970 candidate studies, perform an in-depth qualitative analysis using the Grounded Theory approach, and then answer seven important research questions. Our investigation has reported several major findings. First, to date, eight major methodologies (e.g., term weighting, query-term co-occurrence analysis, thesaurus lookup) have been adopted in query reformulation. Second, the existing studies suffer from several major limitations (e.g., lack of generalizability, vocabulary mismatch problem, weak evaluation, the extra burden on the developers) that might prevent their wide adoption. Finally, we discuss several open issues in search query reformulations and suggest multiple future research opportunities. Explore more ...



query-reformulation bug-localization concept-location code-search empirical-study  grounded-theory

Towards Understanding the Impacts of Textual Dissimilarity on Duplicate Bug Report Detection (SANER 2023)

---

Overview: A large-scale empirical study using 92K bug reports from three open-source systems is done to understand the challenges of textual dissimilarity in duplicate bug report detection. First, empirical evidence is demonstrated using existing techniques that poorly detect textually dissimilar duplicate bug reports. Second, we found that textually dissimilar duplicates often miss essential components (e.g., steps to reproduce), which could lead to their textual dissimilarity within the same pair. Finally, inspired by the earlier findings, domain-specific embedding along with CNN is applied to duplicate bug report detection, which provides mixed results. Explore more ...



information-retrieval deep-learning duplicate-bug-detection empirical-study

Recommending Code Reviews Leveraging Code Changes with Structured Information Retrieval (ICSME 2023)

---

Overview: Review comments are one of the main building blocks of modern code reviews. Manually writing code review comments could be time-consuming and technically challenging. In this work, we propose a novel technique for relevant review comments recommendation -- RevCom -- that leverages various code-level changes using structured information retrieval. It uses different structured items from source code and can recommend relevant reviews for all types of changes (e.g., method-level and non-method-level). We find that RevCom can recommend review comments with an average BLEU score of ≈ 26.63%. According to Google's AutoML Translation documentation, such a BLEU score indicates that the review comments can capture the original intent of the reviewers. Our approach is lightweight compared to DL-based techniques and can recommend reviews for both method-level and non-method-level changes where the existing IR-based technique falls short. Explore more ...



structured-information-retrieval code-review-automation

The Forgotten Role of Search Queries in IR-based Bug Localization: An Empirical Study [EMSE 2021, ICSE 2022 (Journal First)]

---

Overview: We conduct an in-depth empirical study that critically examines the state-of-the-art query selection practices in IR-based bug localization. In particular, we use a dataset of 2,320 bug reports, employ ten existing approaches from the literature, exploit the Genetic Algorithm-based approach to construct optimal, near-optimal search queries from these bug reports, and then answer three research questions. We confirmed that the state-of-the-art query construction approaches are indeed not sufficient for constructing appropriate queries (for bug localization) from certain natural language-only bug reports although they contain such queries. We also demonstrate that optimal queries and non-optimal queries chosen from bug report texts are significantly different in terms of several keyword characteristics, which has led us to actionable insights. Furthermore, we demonstrate 27%--34% improvement in the performance of non-optimal queries through the application of our actionable insights to them. Explore more ...



query-reformulation bug-localization empirical-study  genetic-algorithm

Why Are Some Bugs Non-Reproducible? An Empirical Investigation using Data Fusion [ICSME 2020, EMSE 2022]

---

Overview: We conduct a multimodal study to better understand the non-reproducibility of software bugs. First, we perform an empirical study using 576 non-reproducible bug reports from two popular software systems (Firefox, Eclipse) and identify 11 key factors that might lead a reported bug to non-reproducibility. Second, we conduct a user study involving 13 professional developers where we investigate how the developers cope with non-reproducible bugs. We found that they either close these bugs or solicit for further information, which involves long deliberations and counter-productive manual searches. Third, we offer several actionable insights on how to avoid non-reproducibility (e.g., false-positive bug report detector) and improve reproducibility of the reported bugs (e.g., sandbox for bug reproduction) by combining our analyses from multiple studies (e.g., empirical study, developer study). Explore more ...



empirical-study   data-fusion   bug-reproduction  grounded-theory  

All Projects & Publications

Outreach, Milestones, & Social