On the Effectiveness of Interactive Detection of Code Anomalies: An Empirical Assessment

Background: Detection of code anomalies should be performed as early as possible in order to effectively reveal refactoring opportunities in due time. Refactoring aims at improving software maintainability, but their late application is counter-productive or even prohibitive. Detection of code anomalies is traditionally supported by non-interactive detection (NID) techniques, which encourage developers to reveal anomalies in later revisions or versions of a program. The reason is that this technique does not support progressive interaction of developers with anomalous code. In addition, it reveals anomalies in the entire source code upon an eventual developer request. More recently, the notion of interactive detection (ID) has emerged to address NID's limitations. This technique reveals anomalies when code fragments are still being edited and without an explicit developer request, thereby encouraging early anomaly detection. Problem Statement: Recent studies suggest the use of NID might lead to: (i) a low number of correctly identified anomalies, and (ii) ineffective refactoring actions. Although ID seems promising, there is no knowledge about its impact on anomaly detection and refactoring actions. Goal: Evaluate the effectiveness of an ID technique on early anomaly detection. In addition, we analyze the aid of an ID technique in performing effective refactoring actions. Method: We conducted a controlled experiment with 14 subjects that underwent tasks related to anomaly detection and judgments of required refactoring actions. Results: Our study reveals the use of ID improves anomaly detection as developers tend to early identify more anomalies when compared to the use of NID. Conclusions: Although ID contributes to detect more anomalies than NID, the former may induce to ineffective refactoring actions.