ChatGPT as a Debugging Tool

Introduction

No matter the technology or application debugging is a major part of software development.  Every developer who has ever written a program of any significant size knows that the application is going to have some kind of defect in it and probably won’t build the first few times it is run.  In short, a vast amount of time and energy is spent debugging software.  In many cases, debugging code can be more challenging than writing the code in the first place.  With the advent of systems like ChatGPT, spending hours debugging a piece of code may be a thing of the past, at least for relatively small code blocks.  This tutorial is going to explore prompts that we can use to have ChatGPT troubleshoot defective code for us.   

Expectations

Before we can explore troubleshooting with ChatGPT, we need to first set some realistic expectations.  To begin, ChatGPT works off a series of inputs known as prompts.  For ChatGPT to fix a code block, you’ll first have to submit the code and the issue as a prompt.  At first glance, this may not seem like a big deal; however, modern applications are conglomerates of many smaller components that rely on each other to function correctly.  On top of that, many of these “smaller” components may be, and usually are, composed of hundreds, if not thousands of lines of code or more.  This means that a defect may not stem from the current code block but from a service or line of code, some were no one may consider.  As such, if the root of the defect is not inputted into the prompt, ChatGPT may not be of much use.  This means that to properly use ChatGPT as a troubleshooting tool, it is important to at least have a clue as to where the offending code is; this is because pasting in thousands, if not millions, of lines of code, is impractical.  Ultimately, ChatGPT, at least the web version, cannot be used as a troubleshooting tool without a person who is knowledgeable in both the codebase, debugging, and coding guiding it.   With that, let’s start troubleshooting!

Troubleshooting with ChatGPT

Before we begin troubleshooting, let’s define the bugs we’re going to look at in this example.  Though there are many different types of software defects, this tutorial is going to use ChatGPT to troubleshoot compilation and logical errors as examples.  As such, we’re going to define these two terms with the following:

Compilation Error: Any error that will prevent the source code from compiling.

Logic Error: A software defect that gives an unanticipated result. 

For these examples, we’re going to use C and Python as the programming languages.  However, due to the nature of ChatGPT, the prompts can be modified to accommodate other languages.  We’re also going to assume that the program can fit in a single file; that is, we’re going to have ChatGPT analyze code snippets.  For now, let’s start by exploring some compilation errors!

Compilation Error

To begin with, let’s take a look at a very simple Hello World program in C.  Consider the following code,

#include <stdio.h>
int main(){
     printf("Hello World")
}

If you look at the code, you will see that there is a missing semicolon on the print line.  This is an obvious issue that the compiler will pick up, but for grins, let’s see what ChatGPT has to say.  For this example, we’re going to use the following prompt:

Why will this C code not compile? #include <stdio.h> int main(){ printf("Hello World") }

This prompt will output the following response:

The C code you provided will not compile because there is a missing semicolon (;) at the end of the printf statement in the main function.
Here's the corrected code:
#include <stdio.h>
int main() {
    printf("Hello World");
    return 0;
}
In C, semicolons are used to terminate statements, including function calls like printf(). The absence of a semicolon after printf("Hello World") causes a compilation error. Adding the semicolon at the end resolves this issue. 

As can be seen, ChatGPT instantly caught the error.  Now, this is a very simple error that basic tools such as the compiler would normally catch. 

Now, let’s explore how ChatGPT can troubleshoot a program in Python.  As many developers know, Python has a unique syntax compared to C and other C-like languages such as C++, C#, or Java.  Instead of using curly braces, Python uses indentation to denote hierarchy which can oftentimes lead to problems that are hard to diagnose, especially for the inexperienced.  As such, consider the following code block:

some_number = 2
if some_number <= 2:
print("some number is less than 2")

An experienced Python developer would probably catch this issue quickly.  However, for this example, we’re going to let ChatGPT troubleshoot the code.  For this code block, we’re going to use the following prompt:

What is the syntax error with this Python code block?

some_number = 2

if some_number <= 2:

print("some number is less than 2")

This prompt will output the following response:

The syntax error in the given Python code block is that the print statement is not indented properly. In Python, indentation is crucial to define the structure and hierarchy of code blocks. Here's the corrected version:
some_number = 2
if some_number <= 2:
    print("some number is less than or equal to 2")
In this corrected code, the print statement is indented with four spaces (or a tab) to align it with the if statement. This ensures that it is part of the code block executed when the condition is true.

In this output, ChatGPT gave us the correct syntax as well as an explanation of the problem and solution.  Similar to the C code, these are problems that are usually picked up by the IDE or the compilation system.  However, it is not that uncommon to have to make code modifications in a plain text editor or to get tracebacks that aren’t very useful.  In these cases, ChatGPT could be an awesome tool to use to quickly fix issues.  Now that we’ve explored compilation issues let’s take a look at logical errors.

Logical Errors

Logical errors are oftentimes the hardest to troubleshoot, and many stem from very simple errors, such as using the wrong variable in a control statement, using the wrong math equation, or anything of the sort.  Often these are the hardest issues to troubleshoot because unless they are an oversight, the developer will think their logic is sound.  As such, let’s explore how ChatGPT can be used to clarify a developer’s logic.  Consider the following C code:

#include <stdio.h>
int main(){
     int a = 2;
     int b = 3;
if (b > a){
          printf("b < a");
     }else{
          printf("a > b");
     }
     return 0;
}

The purpose of this code is to show which value is greater, a or b.  However, when the code is run with the current values, it will return the following output:

Figure 1 – C Program Output

Obviously, there is a logic error in the code.  To troubleshoot it, let’s use the following prompt in ChatGPT:

This code is producing the wrong results, it is saying that b is less than a when the inverse is true.  What is wrong with this code

 #include <stdio.h>

int main(){

            int a = 2;

            int b = 3;

            if (b > a){

                        printf("b < a");

            }else{

                        printf("a > b");

            }

            return 0;

}

When the prompt is executed, it will return the following:

The issue with the given C code is the incorrect comparison operator in the printf statements. The condition inside the if statement checks if b is greater than a, but the printf statements incorrectly state the opposite.
To fix this, you need to update the printf statements to reflect the correct condition. Here's the corrected code:
#include <stdio.h>
 
int main() {
    int a = 2;
    int b = 3;
    if (b > a) {
        printf("b > a");
    } else {
        printf("a >= b");
    }
    return 0;
}
Now the code will output "b > a" when b is indeed greater than a, and "a >= b" otherwise.

Again, ChatGPT gave a very clear and concise diagnosis and solution.  In short, the conditional statement in the if statement was wrong.  Now, let’s look at a snippet in Python. Consider the following code:

def testFunction():
    sum = 0
    for i in range(10):
        sum += i
        print(i)
    testFunction()

In short, if you tried to run this code, nothing would happen. A Python developer would probably catch this error quickly, but let’s see what ChatGPT has to say. To start, let's use the following prompt:

This function will not execute

def testFunction():

sum = 0 for i in range(10):

sum += i print(i)

testFunction()

After running this prompt, you should get something akin to the following:

The Python function you provided has a recursive call to itself (testFunction()) within its own body. This recursive call causes an infinite recursion, which results in a "RecursionError: maximum recursion depth exceeded" error.

If you intended to have a recursive function, you should define the base case that stops the recursion. However, in this case, it seems like you want to call the testFunction() only once, without recursion. To fix the code, you can remove the testFunction() call from within the function body. Here's the corrected version:

def testFunction():    
	sum = 0    
	for i in range(10):        
		sum += i        
		print(i) 
testFunction()

Now the function will execute correctly, iterating through the numbers 0 to 9 and printing each number on a separate line, without causing any recursion errors.

This time the diagnosis isn’t as clear because there are a couple of errors that could be the issue. The first is that ChatGPT thinks this might be a recursive function, in which case, ChatGPT suggests putting in an exit for the function. However, if you read the output a little closer, it suggests that the developer is trying to call the function. In this case, it says that the function call is embedded in the function itself and provides the correct working code. As such, it diagnosed and troubleshot the code.

Key Takeaways

ChatGPT is an excellent way to troubleshoot code. It should be noted that the code in this tutorial was relatively simple and short.  With that, ChatGPT is excellent at troubleshooting small snippets, for example, methods or maybe even whole classes. However, for extremely complex problems, that is, problems that require many lines of code to be examined, ChatGPT may not be the optimal tool because all those lines have to be inputted into the prompt. Considering the problem, ChatGPT may get confused with the code, and complex prompts may have to be engineered to find the problem. However, if you have a rough idea of where the defect originates from, like which class file, it may be worthwhile to run the code through ChatGPT. If nothing else, it probably will give you a fresh perspective and, at the very least, point you in the right direction. 

The key to using ChatGPT as a troubleshooting tool is giving it the proper information. As we saw with the compilation and logic errors, a compilation error only needed the source code; however, that prompt could have been optimized with a description of the problem. On the other hand, to get the most out of logic errors, you’re going to want to include the following at a minimum:

•   The programming language
•   The code (At least the suspected offending code)
•    A description of the problem
•    Any other relevant information

So far, the more information you provide to ChatGPT, the better the results are, but as we saw, a short description of the problem took care of the logic errors. Now, you could get away without specifying the problem, but when it comes to logical errors, it is wise to at least give a short description of the problem. ChatGPT is not infallible, and as we saw with the Python function, ChatGPT wasn’t too sure if the function was meant to be recursive or not. This means, much like a human, it needs to know as much about the problem as it can to accurately diagnose it.

Summary

In all, ChatGPT is a great tool for troubleshooting code. This tool would be ideal for compilation errors when tracebacks are not useful or not available. In terms of it being a tool for troubleshooting logical errors, ChatGPT can also be very useful. However, more information will be required for ChatGPT to accurately diagnose the problems. 

Again, the examples in this tutorial are very simple and straightforward. The goal was to simply demonstrate what kind of prompts can be used and the results of those inputs.  However, as was seen with the Python function, a complex code block can and probably will confuse the AI. This means that as a user, you have to provide as detailed information as you can to ChatGPT. It is also important to remember that no matter how you use the system, you will still need to use critical thinking and detective work yourself to hunt down the problem. ChatGPT is by no means a replacement for human developers, at least not yet. This means it is important to think of ChatGPT as another set of eyes on a problem and not a one-stop solution for a problem.  

Author Bio

M.T. White has been programming since the age of 12. His fascination with robotics flourished when he was a child programming microcontrollers such as Arduino. M.T. currently holds an undergraduate degree in mathematics, and a master's degree in software engineering, and is currently working on an MBA in IT project management. M.T. is currently working as a software developer for a major US defense contractor and is an adjunct CIS instructor at ECPI University. His background mostly stems from the automation industry where he programmed PLCs and HMIs for many different types of applications. M.T. has programmed many different brands of PLCs over the years and has developed HMIs using many different tools.

Author of the book: Mastering PLC Programming