Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon

How-To Tutorials - ChatGPT

114 Articles
article-image-chatgpt-as-a-debugging-tool
M.T. White
22 Aug 2023
14 min read
Save for later

ChatGPT as a Debugging Tool

M.T. White
22 Aug 2023
14 min read
IntroductionNo matter the technology or application debugging is a major part of software development.  Every developer who has ever written a program of any significant size knows that the application is going to have some kind of defect in it and probably won’t build the first few times it is run.  In short, a vast amount of time and energy is spent debugging software.  In many cases, debugging code can be more challenging than writing the code in the first place.  With the advent of systems like ChatGPT, spending hours debugging a piece of code may be a thing of the past, at least for relatively small code blocks.  This tutorial is going to explore prompts that we can use to have ChatGPT troubleshoot defective code for us.   ExpectationsBefore we can explore troubleshooting with ChatGPT, we need to first set some realistic expectations.  To begin, ChatGPT works off a series of inputs known as prompts.  For ChatGPT to fix a code block, you’ll first have to submit the code and the issue as a prompt.  At first glance, this may not seem like a big deal; however, modern applications are conglomerates of many smaller components that rely on each other to function correctly.  On top of that, many of these “smaller” components may be, and usually are, composed of hundreds, if not thousands of lines of code or more.  This means that a defect may not stem from the current code block but from a service or line of code, some were no one may consider.  As such, if the root of the defect is not inputted into the prompt, ChatGPT may not be of much use.  This means that to properly use ChatGPT as a troubleshooting tool, it is important to at least have a clue as to where the offending code is; this is because pasting in thousands, if not millions, of lines of code, is impractical.  Ultimately, ChatGPT, at least the web version, cannot be used as a troubleshooting tool without a person who is knowledgeable in both the codebase, debugging, and coding guiding it.   With that, let’s start troubleshooting!Troubleshooting with ChatGPTBefore we begin troubleshooting, let’s define the bugs we’re going to look at in this example.  Though there are many different types of software defects, this tutorial is going to use ChatGPT to troubleshoot compilation and logical errors as examples.  As such, we’re going to define these two terms with the following:Compilation Error: Any error that will prevent the source code from compiling.Logic Error: A software defect that gives an unanticipated result. For these examples, we’re going to use C and Python as the programming languages.  However, due to the nature of ChatGPT, the prompts can be modified to accommodate other languages.  We’re also going to assume that the program can fit in a single file; that is, we’re going to have ChatGPT analyze code snippets.  For now, let’s start by exploring some compilation errors!Compilation ErrorTo begin with, let’s take a look at a very simple Hello World program in C.  Consider the following code,#include <stdio.h> int main(){     printf("Hello World") }If you look at the code, you will see that there is a missing semicolon on the print line.  This is an obvious issue that the compiler will pick up, but for grins, let’s see what ChatGPT has to say.  For this example, we’re going to use the following prompt:Why will this C code not compile? #include <stdio.h> int main(){ printf("Hello World") }This prompt will output the following response:The C code you provided will not compile because there is a missing semicolon (;) at the end of the printf statement in the main function. Here's the corrected code: #include <stdio.h> int main() {    printf("Hello World");    return 0; } In C, semicolons are used to terminate statements, including function calls like printf(). The absence of a semicolon after printf("Hello World") causes a compilation error. Adding the semicolon at the end resolves this issue. As can be seen, ChatGPT instantly caught the error.  Now, this is a very simple error that basic tools such as the compiler would normally catch. Now, let’s explore how ChatGPT can troubleshoot a program in Python.  As many developers know, Python has a unique syntax compared to C and other C-like languages such as C++, C#, or Java.  Instead of using curly braces, Python uses indentation to denote hierarchy which can oftentimes lead to problems that are hard to diagnose, especially for the inexperienced.  As such, consider the following code block:some_number = 2 if some_number <= 2: print("some number is less than 2")An experienced Python developer would probably catch this issue quickly.  However, for this example, we’re going to let ChatGPT troubleshoot the code.  For this code block, we’re going to use the following prompt:What is the syntax error with this Python code block?some_number = 2if some_number <= 2:print("some number is less than 2")This prompt will output the following response:The syntax error in the given Python code block is that the print statement is not indented properly. In Python, indentation is crucial to define the structure and hierarchy of code blocks. Here's the corrected version: some_number = 2 if some_number <= 2:    print("some number is less than or equal to 2") In this corrected code, the print statement is indented with four spaces (or a tab) to align it with the if statement. This ensures that it is part of the code block executed when the condition is true.In this output, ChatGPT gave us the correct syntax as well as an explanation of the problem and solution.  Similar to the C code, these are problems that are usually picked up by the IDE or the compilation system.  However, it is not that uncommon to have to make code modifications in a plain text editor or to get tracebacks that aren’t very useful.  In these cases, ChatGPT could be an awesome tool to use to quickly fix issues.  Now that we’ve explored compilation issues let’s take a look at logical errors.Logical ErrorsLogical errors are oftentimes the hardest to troubleshoot, and many stem from very simple errors, such as using the wrong variable in a control statement, using the wrong math equation, or anything of the sort.  Often these are the hardest issues to troubleshoot because unless they are an oversight, the developer will think their logic is sound.  As such, let’s explore how ChatGPT can be used to clarify a developer’s logic.  Consider the following C code:#include <stdio.h> int main(){     int a = 2;     int b = 3; if (b > a){          printf("b < a");     }else{          printf("a > b");     }     return 0; }The purpose of this code is to show which value is greater, a or b.  However, when the code is run with the current values, it will return the following output:Figure 1 – C Program OutputObviously, there is a logic error in the code.  To troubleshoot it, let’s use the following prompt in ChatGPT:This code is producing the wrong results, it is saying that b is less than a when the inverse is true.  What is wrong with this code #include <stdio.h>int main(){            int a = 2;            int b = 3;            if (b > a){                        printf("b < a");            }else{                        printf("a > b");            }            return 0;}When the prompt is executed, it will return the following:The issue with the given C code is the incorrect comparison operator in the printf statements. The condition inside the if statement checks if b is greater than a, but the printf statements incorrectly state the opposite. To fix this, you need to update the printf statements to reflect the correct condition. Here's the corrected code: #include <stdio.h> int main() {    int a = 2;    int b = 3;    if (b > a) {        printf("b > a");    } else {        printf("a >= b");    }    return 0; } Now the code will output "b > a" when b is indeed greater than a, and "a >= b" otherwise.Again, ChatGPT gave a very clear and concise diagnosis and solution.  In short, the conditional statement in the if statement was wrong.  Now, let’s look at a snippet in Python. Consider the following code:def testFunction():    sum = 0    for i in range(10):        sum += i        print(i)    testFunction()In short, if you tried to run this code, nothing would happen. A Python developer would probably catch this error quickly, but let’s see what ChatGPT has to say. To start, let's use the following prompt:This function will not executedef testFunction():sum = 0 for i in range(10):sum += i print(i)testFunction()After running this prompt, you should get something akin to the following:The Python function you provided has a recursive call to itself (testFunction()) within its own body. This recursive call causes an infinite recursion, which results in a "RecursionError: maximum recursion depth exceeded" error. If you intended to have a recursive function, you should define the base case that stops the recursion. However, in this case, it seems like you want to call the testFunction() only once, without recursion. To fix the code, you can remove the testFunction() call from within the function body. Here's the corrected version: def testFunction():    sum = 0    for i in range(10):        sum += i        print(i)  testFunction() Now the function will execute correctly, iterating through the numbers 0 to 9 and printing each number on a separate line, without causing any recursion errors.This time the diagnosis isn’t as clear because there are a couple of errors that could be the issue. The first is that ChatGPT thinks this might be a recursive function, in which case, ChatGPT suggests putting in an exit for the function. However, if you read the output a little closer, it suggests that the developer is trying to call the function. In this case, it says that the function call is embedded in the function itself and provides the correct working code. As such, it diagnosed and troubleshot the code.Key TakeawaysChatGPT is an excellent way to troubleshoot code. It should be noted that the code in this tutorial was relatively simple and short.  With that, ChatGPT is excellent at troubleshooting small snippets, for example, methods or maybe even whole classes. However, for extremely complex problems, that is, problems that require many lines of code to be examined, ChatGPT may not be the optimal tool because all those lines have to be inputted into the prompt. Considering the problem, ChatGPT may get confused with the code, and complex prompts may have to be engineered to find the problem. However, if you have a rough idea of where the defect originates from, like which class file, it may be worthwhile to run the code through ChatGPT. If nothing else, it probably will give you a fresh perspective and, at the very least, point you in the right direction. The key to using ChatGPT as a troubleshooting tool is giving it the proper information. As we saw with the compilation and logic errors, a compilation error only needed the source code; however, that prompt could have been optimized with a description of the problem. On the other hand, to get the most out of logic errors, you’re going to want to include the following at a minimum:  The programming language  The code (At least the suspected offending code)   A description of the problem   Any other relevant informationSo far, the more information you provide to ChatGPT, the better the results are, but as we saw, a short description of the problem took care of the logic errors. Now, you could get away without specifying the problem, but when it comes to logical errors, it is wise to at least give a short description of the problem. ChatGPT is not infallible, and as we saw with the Python function, ChatGPT wasn’t too sure if the function was meant to be recursive or not. This means, much like a human, it needs to know as much about the problem as it can to accurately diagnose it.SummaryIn all, ChatGPT is a great tool for troubleshooting code. This tool would be ideal for compilation errors when tracebacks are not useful or not available. In terms of it being a tool for troubleshooting logical errors, ChatGPT can also be very useful. However, more information will be required for ChatGPT to accurately diagnose the problems. Again, the examples in this tutorial are very simple and straightforward. The goal was to simply demonstrate what kind of prompts can be used and the results of those inputs.  However, as was seen with the Python function, a complex code block can and probably will confuse the AI. This means that as a user, you have to provide as detailed information as you can to ChatGPT. It is also important to remember that no matter how you use the system, you will still need to use critical thinking and detective work yourself to hunt down the problem. ChatGPT is by no means a replacement for human developers, at least not yet. This means it is important to think of ChatGPT as another set of eyes on a problem and not a one-stop solution for a problem.  Author BioM.T. White has been programming since the age of 12. His fascination with robotics flourished when he was a child programming microcontrollers such as Arduino. M.T. currently holds an undergraduate degree in mathematics, and a master's degree in software engineering, and is currently working on an MBA in IT project management. M.T. is currently working as a software developer for a major US defense contractor and is an adjunct CIS instructor at ECPI University. His background mostly stems from the automation industry where he programmed PLCs and HMIs for many different types of applications. M.T. has programmed many different brands of PLCs over the years and has developed HMIs using many different tools.Author of the book: Mastering PLC Programming  
Read more
  • 0
  • 0
  • 244

article-image-chatgpt-for-ladder-logic
M.T. White
22 Aug 2023
17 min read
Save for later

ChatGPT for Ladder Logic

M.T. White
22 Aug 2023
17 min read
IntroductionChatGPT is slowly becoming a pivotal player in software development.  It is being used by countless developers to help produce quality and robust code.  However, many of these developers are using ChatGPT for text-based programming languages like C++ or Java.  There are few, if any, tutorials on how ChatGPT can be utilized to write Ladder Logic code.  As such, this tutorial is going to be dedicated to exploring how and why ChatGPT can be used as a tool for traditional Ladder Logic programmers.Why use ChatGPT for Ladder Logic?The first step in learning how to leverage ChatGPT is to learn why to use the system.  First of all, ChatGPT is not a programmer, nor is it designed to replace programmers in any way, shape, or form.  However, it can be a handy tool for people that are not sure how to complete a task, need to produce some code in a crunch, and so on.  To effectively use ChatGPT, a person will have to know how to properly produce a statement, refine that statement, and, if necessary, write subsequent statements that have the right amount of information for ChatGPT to effectively produce a result.  In other words, a ChatGPT user still has to be competent, but when used correctly, the AI system can produce code much faster than a human can, especially if the human is inexperienced at a given task.In terms of industrial automation, ChatGPT can be an especially attractive tool.  It is no secret that many PLC programmers are not formally trained developers.  It is common for many PLC programmers to be maintenance technicians, electricians, or other types of engineers.  In any case, it is common for many people who are forced to write complex PLC software to have little more than previous experience guiding them.  As such, when faced with a complex situation with little to no resources available, the programmer can often be lost with no hope of finding a solution.  This is where ChatGPT can be utilized as a user can pose questions and task the system with finding solutions.  With that, how do we use ChatGPT at a basic level?  How to use ChatGPT?The key behind using ChatGPT is writing what are called prompts.  In a lay sense, a prompt is a query or command that ChatGPT runs.  In other words, a prompt can be thought of as a task you ask ChatGPT to do.  For example, a person could input the following prompt into ChatGPT:Tell me how to install a Beckhoff PLC.In this case, ChatGPT will give you a high-level overview of the basics of installing a Beckhoff PLC.  However, a prompt like this will usually not return any useful results unless you want general, high-level information.  Generally, to get a viable answer, you’ll need to hone the prompt with detailed information and exceptions.  Learning to write effective prompts is a skill that is in the early stages of development.  It is important to remember that ChatGPT is a novel tool. This means that the IT and automation industry as a whole is still learning how to use it effectively.  However, a general flow of writing and refining prompts can be summarized in Figure 1.Figure 1 – Prompt Refining FlowThis is just a general logic flow for massaging a prompt to get the desired result.  However, sometimes it is necessary and easier to run subsequent prompts instead of refining existing ones.  Either way, it may take a couple of tries to get what you need out of ChatGPT.  Regardless, to explore how to use ChatGPT, let’s start by creating a basic AND gate. AND ConfigurationAs everyone knows, an AND configuration is simply two contacts in a series.  For the coil to turn on, both contacts have to be on at the same time.  This a relatively simple Ladder Logic program to write and understand, so we’re going to use it as a test for ChatGPT. To begin, we’re going to use the following as a prompt:Write a ladder logic program for an AND gateWhen this prompt is run, it’ll generate the following network.  NETWORK--| |--[ ]---[ ]---( )                                                                                                                                  |          |     |----[ ]---( )Figure 2 – AND Gate: Attempt 1 With Description In this example, an and gate is produced, but it also generates an extra rung that wasn’t asked for or necessary for the task at hand.  In this case, it is not easy to determine what the extra run does or why it was included; as such, this is a prime example of needing to refine the prompts to only produce what is needed.  To do this, let’s refine the prompt a little more to match the following:Write only the necessary contacts and coils for an AND gate in Ladder LogicWhen this query is run, it will produce the network in Figure 2:NETWORK--| |--[ ]---[ ]---( ) Figure 3 – Refined AND Gate: Attempt 2 With DescriptionIn this refined attempt, ChatGPT produces a very simple network of contacts in series.  There are no extra rungs or ambiguous instructions. In short, the moral of this exercise is that we started with a general prompt that produced a working result but a result that needed to be refined. We then refined that prompt and produced a high-quality network with only the necessary instructions for a working AND gate.  With that, let’s look at something a little more complex, timer networks. Timer ProgramsAs every PLC programmer knows, most PLC programs will usually include a timer of some type.  So, with that in mind, let’s explore how ChatGPT handles timer programs.  To begin, let’s have ChatGPT generate a simple 10-minute timer.  To do this, we can use the following query:Write a ladder logic program for a 10 minute timer.Depending on your usage of ChatGPT, you may get a result without the variables labeled or even in Structured Text.  However, the prompt can be adjusted to produce the desired output by using subsequent prompts that are akin to the following:I need this in ladder logicor,assign the variables in the diagramAs stated before, you won’t have to re-write the initial prompt. All you have to do is run these statements as if they were your initial prompt.  However, depending on the prompts you run, your result should be something akin to the following:VAR    Timer: TON;         (* Timer for the 10-minute duration *)    StartButton: BOOL;  (* Input for starting the timer *)    Done: BOOL;         (* Output to indicate when the timer is done *) END_VAR NETWORK -----[StartButton]----[/Timer]-----   |                            |   |                            |   |                           | ---|----------------------------|---- NO                           |                               |                               |                              ---[Done]---In the above ladder logic program:StartButton represents the input contact that starts the timer./Timer represents the timer coil, indicating that the timer is active and counting.Done represents the output contact, which becomes closed (energized) when the timer reaches 10 minutes.When the StartButton input contact is activated, and the timer starts counting. Once the timer reaches 10 minutes, the Done output contact will close, indicating that the timer is done. When the StartButton input contact is deactivated, the timer stops counting, and the Done output contact opens.Please note that the specific syntax and representation of timers may vary depending on the PLC programming software or the manufacturer's conventions. Adjust the syntax and parameters as per the specific requirements of your PLC system.As anyone who has programmed in Ladder Logic before can see, this isn’t standard notation for a timer function.  However, you probably noticed that each prompt will generate a relatively short description of how the snippet is supposed to work.  If you read the subsequent output, you can piece together the symbolic nature of the snippet.  With that, let’s move on to something a little more advanced.  Let’s explore building a cascading timer network that will have one timer set for 10 minutes and another set for 15, and when the final timer turns on, it will power an LED.  To do this, we can input the following prompt,Write a cascading timer Ladder Logic Network that has one timer set for 10 minutes and the other set for 15 minutes. Turn on an LED when both timers turn on. Give the necessary variables as well.This prompt will produce the following:VAR    Timer1: TON;     // Timer 1 (10 minutes)    Timer2: TON;     // Timer 2 (15 minutes)    LED: BOOL;       // Output LED END_VAR NETWORK    // Timer 1 (10 minutes)    Timer1(IN:= TRUE, PT:= T#10M);  // Set Timer 1 to 10 minutes duration    // Timer 2 (15 minutes)    Timer2(IN:= Timer1.Q, PT:= T#15M);  // Start Timer 2 when Timer 1 completes    // LED turns on when both timers complete    LED := Timer1.Q AND Timer2.Q; END_NETWORKIn this ladder logic program:Timer1 is a timer variable of type TON set for 10 minutes duration.Timer2 is a timer variable of type TON set for 15 minutes duration.LED is a boolean variable representing the output LED.Here's a breakdown of the logic:Timer1 is triggered to start with the input IN:= TRUE and is set to a preset time of 10 minutes (PT:= T#10M).Timer2 is started when Timer1 completes its timing sequence (IN:= Timer1.Q), and it is set to a preset time of 15 minutes (PT:= T#15M).The LED is turned on (LED := Timer1.Q AND Timer2.Q) when both Timer1 and Timer2 have completed their timing sequences.You can adjust the timing durations of the timers according to your specific requirements in your PLC programming software. Additionally, you can connect the LED output variable to the appropriate LED indicator in your setup.As can be seen, this isn’t our desired output.  We want Ladder Logic, and this is Structured Text.  In short, ChatGPT got confused.  If you read the description, ChatGPT thinks its result is Ladder Logic.  In all, this isn’t what we were looking for, so we can add a subsequent prompt that reads the following:I need this program in Ladder LogicThis prompt will produce the following output:NETWORK --|----[ ]----[/]---[ ]----[/]---[ ]----[/]---[ ]--+ |              |              |              | +---[ ]----[/]---[ ]----[/]---[ ]---[/]---[ ]--+    Timer1         Timer2    +---[ ]---+    +---[ ]---+    |         |    |         |    +---[/]---+    +---[/]---+      LED            LEDIn this ladder logic program:Each [ ] represents a normally open (NO) contact./ represents a normally closed (NC) contact.The timers Timer1 and Timer2 are cascaded to achieve the desired timing sequence.The LED output is controlled based on the completion of both timers.Here's a breakdown of the logic:Timer1 is a timer with a preset time of 10 minutes.Timer2 is a timer with a preset time of 15 minutes.When Timer1 completes its timing sequence, it energizes the output coil [/], and the LED associated with Timer1 turns on.Timer2 is enabled when the output coil [/] of Timer1 is energized.When Timer2 completes its timing sequence, it energizes the output coil [/], and the LED associated with Timer2 turns on.The LED output is controlled by the parallel combination of Timer1's output coil [/] and Timer2's output coil [/]. The LED turns on only when both timers have been completed.Please note that the ladder logic diagram provided is a visual representation, and the actual implementation may vary depending on the PLC programming software or hardware you are using.This output is closer to what we’re looking for, but depending on a person's experience level, this probably won’t be much help.  This particular diagram isn’t very descriptive of how everything is supposed to fit together, so if a person had to translate this to a specific programming software, this diagram would probably take a while to fully digest and implement.  As such, we’re going to have to add another prompt that will be as follows,I need the variables assigned to the proper contacts.This subsequent prompt will produce the following:NETWORK --|----[StartButton]----[/Timer1.Enabled]---[Timer1.TimerDN]----[/Timer2.Enabled]---[Timer2.TimerDN]--+ |                                                                                             | +---[Timer1.Timer.Q]----[/LED1]---+---[Timer2.Timer.Q]----[/LED2]---+                                   |                                   |                                    +---[/LED]--------------------------+In this ladder logic program:StartButton is a boolean variable representing the input contact [StartButton] that starts the timer sequence.Timer1.Enabled and Timer2.Enabled are boolean variables representing the input contacts [/Timer1.Enabled] and [/Timer2.Enabled] that enable the respective timers.Timer1.TimerDN and Timer2.TimerDN are boolean variables representing the output contacts [Timer1.TimerDN] and [Timer2.TimerDN] that indicates the completion of the respective timers.Timer1.Timer.Q and Timer2.Timer.Q are boolean variables representing the output contacts [Timer1.Timer.Q] and [Timer2.Timer.Q] that indicates the running state of the respective timers.LED1 and LED2 are boolean variables representing the output contacts [/LED1] and [/LED2] that control the LEDs associated with Timer1 and Timer2.LED is a boolean variable representing the output contact [/LED] that controls the final LED indicating the completion of both timers.Please note that the actual variable names may vary depending on the PLC programming software or hardware you are using. You can assign these variables in your PLC programming software and connect them to the appropriate input and output terminals to implement the desired cascading timer logicThis result is much better as it shows how to wire the network together while the description gives a clear overview of how everything works.  With this, a person with a basic knowledge of Ladder Logic will be able to piece this together and port the rungs to other PLC programming programs. SummaryChatGPT can be leveraged as an excellent tool to help Ladder Logic developers with concocting specialized programs.  However, ChatGPT is NOT a replacement for programmers, and to effectively use the system, a person must be skilled enough to write descriptive prompts and interpret the results.  This means that though ChatGPT is an excellent tool, it does not have the intuition nor the skill to fully replace a programmer.A big part of using ChatGPT is learning to write and refine prompts as well as subsequent follow-up prompts.  These prompts are a developing art form that probably will be the next iteration of software development.  For now, the art of using ChatGPT and similar systems is novel, and there aren’t any definitive standards that govern how to effectively use these yet, especially when it comes to graphical programming such as Ladder Logic.  When used by a knowledgeable person that has a basic idea of PLC programming and ChatGPT, it can be a great way of getting over hurdles that could take hours or days to solve. Author BioM.T. White has been programming since the age of 12. His fascination with robotics flourished when he was a child programming microcontrollers such as Arduino. M.T. currently holds an undergraduate degree in mathematics, and a master's degree in software engineering, and is currently working on an MBA in IT project management. M.T. is currently working as a software developer for a major US defense contractor and is an adjunct CIS instructor at ECPI University. His background mostly stems from the automation industry where he programmed PLCs and HMIs for many different types of applications. M.T. has programmed many different brands of PLCs over the years and has developed HMIs using many different tools.Author of the book: Mastering PLC Programming
Read more
  • 0
  • 0
  • 2370

article-image-chatgpt-as-a-documentation-tool
M.T. White
22 Aug 2023
14 min read
Save for later

ChatGPT as a Documentation Tool

M.T. White
22 Aug 2023
14 min read
It comes as no surprise that most developers do not like writing documentation.  As a result, documentation is often pushed to the side and, more often than not, haphazardly put together.  This is a serious problem since written documentation is the primary way developers communicate how a software system should work and be utilized.  As such, a poorly documented system can render it useless and confine it to the cyber trash heap long before its time.  However, with the rise of new tools like ChatGPT, poor documentation may no longer be a problem.  This article is going to explore how ChatGPT can be leveraged as a code documentation tool and examine the results to see just how well it can document code.What is quality documentation?If you ask five developers what they consider to be good documentation, you’re probably going to get five different answers.  What’s considered good documentation will vary among languages, IDEs, organizational coding standards, and preferences.  As such, for this article, “good” documentation is going to have the following parameters:1.     The documentation will provide context on how a block of code works or how to utilize it. 2.     The documentation will be logical and consistent.  This means that the documentation will be formatted and worded logically to provide context across all code blocks that ChatGPT documents.3.     The autogenerated code documentation will not clutter the code or distract the reader from the task at hand.Again, it is important to remember that there are many different opinions on what quality documentation looks like, and the points above are just high-level metrics to hit.    What is the documentation?Documentation is a very broad term that can mean a lot of things.  It can mean simply the documentation that comes packaged with the codebases like JavaDocs, or it could be a whole ecosystem that ranges from low-level code comments to YouTube videos and Wikipedia articles.  In short, documentation can be anything that helps relay information from one developer to another.  As such, for this article, the term documentation is going to be limited to code comments and high-level documentation like JavaDocs.  Also, most languages have their own best practices for documentation, so for this article, ChatGPT will generate documentation for Java.  Though Java will be the example language, there should only be minor tweaks, if any, to generate documentation for any other language. JavaDocsJavaDocs should provide high-level documentation to relay information about how to use a piece of code to a developer that does not have access to the codebase.  This means that key features like what the methods do, arguments, and return types should be noted.  With that in mind, let’s generate some JavaDocs for the following method,public static void add(int a, int b) {      System.out.println(a + b); }We’re going to generate the JavaDocs for this method using the following prompt:Generate some javadocs for this method public static void add(int a, int b) { System.out.println(a + b); }When the prompt is executed, it will produce the following output:/** * Calculates the sum of two integers and prints the result. * * @param a The first integer operand. * @param b The second integer operand. */ public static void add(int a, int b) {    System.out.println(a + b); }In this case, ChatGPT clearly defined what the arguments are, the datatypes of the arguments, and what the method is meant to do with straightforward prompts.  It is important to note that this method is straightforward, so the next thing to explore is a more complex method.The next method to try to automatically document will be the following:public static double divide(int a, int b) { double div = 0;      if (a > b) {            div = a / b ;      }else {            div = b / a;      }      return div; }For this method, the following prompt will be run. Write only JavaDocs for this method public static double divide(int a, int b) { double div = 0; if (a > b) { div = a / b ; }else { div = b / a; } return div; }After the prompt is run, we should be met with something similar to the following:/** * Calculates the division of two numbers. * * @param a The numerator. * @param b The denominator. * @return The division result. */These JavaDocs look very similar to the JavaDocs for the simple method.  They provide the arguments, return type, and the purpose of the method.   The one thing that was kind of ambiguous is that ChatGPT attempted to identify the variables as always the numerator or denominator.  From the code, we can see that this isn’t always true; however, if a developer is checking, they should be able to correct it fairly easily. Now, at first glance, this will appear as more trouble than it is worth.  A codebase of significant size is going to have many classes that are comprised of many different methods.  As such, copying and pasting methods one by one is going to be a herculean task.  However, ChatGPT can still be leveraged to speed up high-level code documentation.  Consider the following prompt:Write JavaDocs for these methods public static void add(int a, int b) { System.out.println(a + b); } public static double divide(int a, int b) { double div = 0; if (a > b) { div = a / b ; }else { div = b / a; } return div; }This prompt will produce the following results:/** * Adds two numbers and prints the result. * * @param a The first number. * @param b The second number. */ /** * Calculates the division of two numbers and returns the result. * * @param a The numerator. * @param b The denominator. * @return The division result. */Again, even with multiple methods, ChatGPT returned JavaDocs that are quality and provided enough information for a developer to use, with the exception of the same hiccup for the division method that was previously explored.  In all, ChatGPT did a decent job with writing JavaDocs.Used in this manner, ChatGPT can greatly enhance the speed and quality of development.  When using ChatGPT, multiple methods can be pasted into ChatGPT, and all the necessary JavaDocs are created.  As anyone who has ever had to write this type of documentation knows, it can be very time-consuming and mistake-prone.  However, with ChatGPT, all one has to do is input the code and let the AI system do the rest.  Ultimately ChatGPT will free up developers for more pressing tasks, and the AI will produce quality JavaDocs without making rudimentary errors like forgetting to include a parameter or return type.  With that, another question is how well ChatGPT can be used to write code comments. Code CommentsIn many cases, it is difficult for a 3rd party to write effective code comments.  To provide the most context to the code, the entity that is writing the code should also write the comments.  As such, the first thing that needs to be explored is how ChatGPT writes code comments.  To do this, consider the following code snippet:public static void add(int a, int b) {      System.out.println(a + b); }This code is very basic; however, in a quality codebase, even a method this simple should be documented.  This method would probably take as much time to document as it would be to write; this means it is a perfect candidate for ChatGPT.  To have ChatGPT document the code snippet runs the following prompt:Write a few comments to provide context for the following method public static void add(int a, int b) { System.out.println(a + b); }After running this prompt, ChatGPT will produce the following:public static void add(int a, int b) {    // This method takes two integer parameters, 'a' and 'b'.    // It adds the values of 'a' and 'b' together and prints the result.    System.out.println(a + b);    // The sum of 'a' and 'b' is calculated and printed to the console using System.out.println().    // This method does not return any value (void). }At first glance, one may notice that ChatGPT generated several comments.  This can be a pro as much as it can be a con since it can bloat the codebase.  With that, one may notice that the output from ChatGPT does have a couple of unnecessary comments that can bloat the code.  For example, there is a comment that says that the method takes in two integers, and another says that the method returns void.  For code comments, those comments are not necessary as they don’t contribute to the overall context of the code.  However, if the prompt is modified to,Write a comment that describes what this method does public static void add(int a, int b) { System.out.println(a + b); }It will produce the following:public static void add(int a, int b) {    // This method calculates the sum of two integers, 'a' and 'b',    // and displays the result by printing it to the console. }This prompt produced two comments that, when combined, form one complete sentence that describes what the method does. Now, this was for a relatively simple method.  As any programmer knows, a method will probably be larger and more complex than the add method.  As such, let’s consider the division method again:public static double divide(int a, int b) {      double div = 0;      if (a > b) {            div = a / b ;      }else {            div = b / a;      }      return div; }This method is a little more complex than the add method.  In this case, this method will return the quotient of a and b if a is greater than b or the quotient of b and a if b is greater than a.  With that, if the following prompt is inputted,Write comments to add context to this method public static double divide(int a, int b) { double div = 0; if (a > b) { div = a / b ; }else { div = b / a; } return div; }it will produce the following,public static double divide(int a, int b) {    // This method calculates the division of two integers and returns the result.    double div = 0;    // Variable to store the division result.    if (a > b) {        div = a / b;        // If 'a' is greater than 'b', divide 'a' by 'b' and assign the result to 'div'.    } else {        div = b / a;        // If 'b' is greater than or equal to 'a', divide 'b' by 'a' and assign the result to 'div'.    }    return div;    // Return the division result. }In short, ChatGPT will document every line of code again.  The comments do provide code context, but the comments don’t provide anything that can’t simply be read in the code.  Also, if this prompt is modified, ChatGPT, at the time of writing this article, will still produce relatively useless comments.Finally, if a prompt like the following is used:Write only the necessary comments for a developer to use public static double divide(int a, int b) { double div = 0; if (a > b) { div = a / b ; }else { div = b / a; } return div; }it will yield:// Calculates the division of two numbers and returns the result. // Parameters: //   a: The numerator. //   b: The denominator. // Returns: //   The division result.Of all the queries explored thus far, this one arguably yielded the best results.  In this case, the comments are what was generated in the last queries, but the format is much cleaner and easier to modify.  This block could be easily pasted into the codebase, and the unnecessary lines deleted. In all, it took a very specific prompt to produce marginally acceptable code comments.  At best, the code comments that were produced didn’t provide any extra context that could not be deduced from reading the code.  It can also be said that the code comments that were produced bloated the code and, to some, may have made it more confusing.  As such, for code comments, ChatGPT probably isn’t the best tool to use.  At best, a developer will have to remove unnecessary lines of comments and probably have to re-write many of them as well. There is also the issue of having to produce a prompt that is specific enough to generate proper comments. In all, whether a person should use ChatGPT as a code comment generator is up to them.  In theory, the comment produced could be leveraged in places like education, where code examples need to be heavily commented on to provide context to those who may not have a background in the language.  However, in terms of production code, though it will ultimately depend on the organization’s coding standard, ChatGPT will not produce code comments that would be mergeable in many places. Keytake Aways  In terms of codebase comments, ChatGPT is hit-and-miss.  As was seen, the code comments that ChatGPT produced were reminiscent of a college-level developer.  That is, ChatGPT commented on every line of code and only stated the obvious.  Since ChatGPT commented on every line of code, it can be argued that it bloated the codebase to a degree.  However, when a very specific prompt was run, it produced comments similar to what would be found in JavaDocs and what is expected by many organizations.  However, in terms of JavaDocs, ChatGPT shined.  The JavaDocs that ChatGPT produced were all very well written and provided the correct amount of information for a developer to easily digest and apply. As such, a few things can be summarized with what was explored.1.     Queries have to be very specific when it comes to code comments.2.     ChatGPT tends to produce unnecessary code comments that can bloat the codebase. 3.     Depending on the type/quality of code comments, ChatGPT may not be the ideal tool for automatic code documentation.4.     ChatGPT produces documentation akin to JavaDocs better than comments in the codebase.SummaryIn summary, what constitutes quality code documentation is often up to a team.  However, by many standards, ChatGPT tends to produce unnecessary code comments that don’t add much context and can easily bloat the codebase.  However, for higher-level documentation like JavaDocs, ChatGPT is an excellent tool that provides the proper amount of information.  In all, it probably isn’t the best idea to use ChatGPT as a means to generate comments for software written by a human, but it can be used to quickly produce higher-level documentation such as JavaDocs. As was seen, multiple methods can easily be documented in a matter of seconds using ChatGPT.  As such, in terms of productivity, when it comes to higher-level documentation, ChatGPT can be a great productivity tool that could help speed up development. Author BioM.T. White has been programming since the age of 12. His fascination with robotics flourished when he was a child programming microcontrollers such as Arduino. M.T. currently holds an undergraduate degree in mathematics, and a master's degree in software engineering, and is currently working on an MBA in IT project management. M.T. is currently working as a software developer for a major US defense contractor and is an adjunct CIS instructor at ECPI University. His background mostly stems from the automation industry where he programmed PLCs and HMIs for many different types of applications. M.T. has programmed many different brands of PLCs over the years and has developed HMIs using many different tools.Author of the book: Mastering PLC Programming 
Read more
  • 0
  • 0
  • 742

article-image-analyzing-eurostat-data-using-openai-code-interpreter
Alan Bernardo Palacio
21 Aug 2023
17 min read
Save for later

Analyzing Eurostat Data Using OpenAI Code Interpreter

Alan Bernardo Palacio
21 Aug 2023
17 min read
OpenAI's recent release of the ChatGPT Code Interpreter plugin has introduced a groundbreaking addition to their language model, ChatGPT. This plugin combines the power of Large Language Models (LLMs) with traditional programming capabilities, revolutionizing programming workflows and enhancing data analysis processes. By eliminating the need to write code or set up separate environments, the Code Interpreter plugin simplifies the process of leveraging the capabilities of ChatGPT for data analysis. Let's explore how this plugin can be used to analyze Eurostat data and derive valuable insights.Introduction to the ChatGPT Code Interpreter PluginThe ChatGPT Code Interpreter plugin provides a Python interpreter within a secure execution environment. It supports file uploads and downloads, allowing seamless handling of data. The plugin enables persistent code execution within a chat conversation, allowing users to build on previous code executions. Its purpose is to solve mathematical problems, perform data analysis and visualization, and convert files between different formats.Simplifying Programming WorkflowsBefore the introduction of the Code Interpreter plugin, users had to generate code separately and execute it outside of the ChatGPT model. However, with the Code Interpreter, the entire process can be completed within ChatGPT itself. This eliminates the need for external runtime environments and offers an easy-to-use interface for both programmers and non-programmers to access programming capabilities.Analyzing Eurostat Data Using Code InterpreterTo demonstrate the capabilities of the Code Interpreter plugin, let's analyze Eurostat data. Eurostat provides various datasets related to agriculture, trade, energy, and more. We will focus on analyzing livestock production data in this example.Installing the Required LibrariesTo begin, we need to install the eurostat library, which allows us to access Eurostat data in Python. We can use the following command within the Code Interpreter plugin:!pip install eurostatAnd then we need to obtain the table of contents to obtain the necessary data for the analysis.Obtaining Eurostat DataOnce the library is installed, we can proceed to obtain the table of contents (TOC) of the available Eurostat datasets. This will help us identify the datasets relevant to our analysis. Here is the code to retrieve the TOC:import eurostat import pandas as pd toc = eurostat.get_toc() toc_df = pd.DataFrame(toc[1:],columns=toc[0]) toc_df.head().to_string()We obtain a table that contains the next information:title      code     type       last update of data last table structure change data start data end 0                           Balance of payments  MED_ECBP  dataset  2020-05-28T23:00:00+0200    2021-02-08T23:00:00+0100       2005     2018 1       International trade by SITC - 1 000 EUR  MED_ECET  dataset  2020-05-28T23:00:00+0200   2021-02-08T23:00:00+0100       2005     2019 2                     Primary energy production  MED_EG10  dataset  2020-05-28T23:00:00+0200    2021-02-08T23:00:00+0100       2005     2018 3  Secondary energy production (transformation)  MED_EG20  dataset  2016-10-28T23:00:00+0200    2021-02-08T23:00:00+0100       2000     2015 4                            Energy consumption  MED_EG30  dataset  2020-05-28T23:00:00+0200    2021-02-08T23:00:00+0100       2005     2018Let’s save this into a CSV file:toc_df.to_csv('toc_df',index=False)The resulting table will contain information such as the dataset title, code, type, last update of data, and data start/end years.Identifying Relevant DatasetsNext, we can use the uploaded table of contents file to find the dataset codes that are most relevant to our analysis. In this case, we are interested in datasets related to crop production, livestock numbers, fertilizer usage, and economic accounts for agriculture. We can use the following code within the Code Interpreter:I have uploaded a file with Eurostat datasets and I need to find the codes for the datasets that most resemble the next indicators: - Crops production in tonnes by NUTS 2 regions. - Livestock and meat numbers by NUTS 2 regions. - Poultry production or similar by NUTS 2 regions. - Mlik or derivates production or similar by NUTS 2 regions. - Economic Accounts for Agriculture (EAA) by NUTS 2 regions, where we will take the total intermediate consumption as a proxy for input energy. The assistant will now analyze the uploaded and provided this: The uploaded file contains the following columns: - `title`: The title of the dataset. - `code`: The code of the dataset. - `type`: The type of the entry (all seem to be datasets). - `last update of data`: The date of the last update of the dataset. - `last table structure change`: The date of the last structural change of the dataset. - `data start`: The start year of the data. - `data end`: The end year of the data. To find the codes for the datasets that most resemble the indicators you mentioned, I will search the dataset titles for relevant keywords. For each indicator, I'll look for the following keywords: 1. Crop production: "crop", "production", "NUTS 2". 2. Livestock numbers: "livestock", "NUTS 2". 3. Fertilizer usage: "fertilizer", "NUTS 2". 4. Economic Accounts for Agriculture (EAA): "economic accounts", "agriculture", "NUTS 2". Let's start this search. Here are the dataset codes that most resemble your indicators: 1. **Crop production in tonnes by NUTS 2 regions:**    - `APRO_CPNHR`: Crop production in national humidity by NUTS 2 regions (data available from 2000 to 2023)    - `APRO_CPNHR_H`: Crop production by NUTS 2 regions - historical data (data available from 1975 to 1999)    - `APRO_CPSHR`: Crop production in EU standard humidity by NUTS 2 regions (data available from 2000 to 2023) 2. **Livestock numbers by NUTS 2 regions:**    - `TAI06`: Share of main livestock types in total livestock (data available from 2005 to 2016)    - `EF_KVAAREG`: Key farm variables: area, livestock (LSU), labour force (data available from 2005 to 2013)    - `EF_KVECSLEG`: Key variables: area, livestock (LSU), labour force by economic size class (data available from 2005 to 2013)    - `EF_KVFTREG`: Key variables: area, livestock (LSU), labour force by type of farming (data available from 2005 to 2013)    - `EF_LSK_MAIN`: Main livestock indicators by NUTS 2 regions (data available from 2005 to 2020)    - `EF_LS_OVAAREG`: Livestock: number of farms and heads by size of the economic output (data available from 1990 to 2007)    - `EF_LS_OVESU`: Livestock: number of farms and heads by economic size class (data available from 1990 to 2007)    - `EF_LS_OVLSUREG`: Livestock: number of farms and heads by livestock size class (data available from 1990 to 2007)    - `EF_OLSAAREG`: Livestock: number of farms and heads of animals by size of the agricultural area (data available from 2005 to 2013)    - `EF_OLSECSREG`: Livestock: number of farms and heads of animals by economic size class (data available from 2005 to 2013)    - `EF_OLSLSUREG`: Livestock: number of farms and heads of animals by livestock size class (data available from 2005 to 2013)    - `AEI_EF_LS`: Share of main livestock types in total livestock (data available from 2005 to 2016) 3. **Fertilizer usage or similar by NUTS 2 regions:**    - Unfortunately, there were no datasets found that match this indicator. 4. **Economic Accounts for Agriculture (EAA) by NUTS 2 regions:**    - `AGR_R_ACCTS`: Economic accounts for agriculture by NUTS 2 regions (data available from 1980 to 2021)    - `AGR_R_ACCTS_H`: Economic accounts for agriculture by NUTS 2 regions - historical data (data available from 1973 to 2012) Please note that the data ranges for each dataset may not cover the exact period you are interested in. You might need to merge data from multiple datasets or look for additional datasets to fill in any gaps.Based on this, we can retrieve the next datasets:Crop production in tonnes by NUTS 2 regions:APRO_CPNHR: Crop production in national humidity by NUTS 2 regions (data available from 2000 to 2023)APRO_CPSHR: Crop production in EU standard humidity by NUTS 2 regions (data available from 2000 to 2023)Livestock numbers by NUTS 2 regions:EF_LSK_MAIN: Main livestock indicators by NUTS 2 regions (data available from 2005 to 2020)Economic Accounts for Agriculture (EAA) by NUTS 2 regions:AGR_R_ACCTS: Economic accounts for agriculture by NUTS 2 regions (data available from 1980 to 2021)As a next step, we can start downloading one of the datasets using the Eurostat API.Analyzing Livestock Production DataWe will focus on the "Livestock numbers by NUTS 2 regions" dataset (EF_LSK_MAIN) for our analysis. Let's proceed with downloading and analyzing this dataset:# Animal production livestock_prod_code = 'EF_LSK_MAIN' livestock_prod_pars = eurostat.get_pars(livestock_prod_code) print(livestock_prod_pars) par_values = eurostat.get_par_values(livestock_prod_code, 'geo') # filter the regions for germany de_par_values = {    'unit':'LSU',    'so_eur':'TOTAL',    'geo':[p for p in par_values if all([p.startswith('DE'),len(p)==4])]} # Download data for de filtered regions livestock_prod_data = eurostat.get_data_df(livestock_prod_code, filter_pars=de_par_values) print(livestock_prod_data.head().to_string())Which produces the following result:['freq', 'lsu', 'animals', 'farmtype', 'agrarea', 'so_eur', 'unit', 'geo'] freq   lsu animals farmtype  agrarea so_eur unit geo\\TIME_PERIOD  2005  2007  2010  2013  2016  2020 0    A  LSU0   A0010  FT15_SO  HA10-19  TOTAL  LSU            DE11   0.0   0.0   0.0   0.0  None   0.0 1    A  LSU0   A0010  FT15_SO    HA2-4  TOTAL  LSU            DE11   0.0   0.0   NaN   NaN  None   NaN 2    A  LSU0   A0010  FT15_SO  HA20-29  TOTAL  LSU            DE11   0.0   0.0   0.0   0.0  None   0.0 3    A  LSU0   A0010  FT15_SO  HA30-49  TOTAL  LSU            DE11   0.0   0.0   0.0   0.0  None   0.0 4    A  LSU0   A0010  FT15_SO    HA5-9  TOTAL  LSU            DE11   0.0   0.0   0.0   0.0  None   0.0The code above filters the dataset for Germany's NUTS 2 regions, specifically focusing on the unit of measurement (LSU) and the standard output in Euros (TOTAL). The resulting DataFrame provides a glimpse of the livestock production data for different regions and years.Some of the codes are still not meaningful, specifically, the ones related to animals, so let’s obtain some results about it:# Get the values in the dataset livestock_prod_animal_vals = eurostat.get_par_values(livestock_prod_code, 'animals') # Obtain their meaning livestock_prod_animal_codes = [p for p in eurostat.get_dic(livestock_prod_code, 'animals') if p[0] in livestock_prod_animal_vals] livestock_prod_animal_codes = pd.DataFrame(livestock_prod_animal_codes,columns= ['code','meaning']) print(livestock_prod_animal_codes.head().to_string())Which produces the next:code                                meaning 0  A0010                              Livestock 1  A1000  Live horses, asses, mules and hinnies 2  A2000                    Live bovine animals 3  A3100           Live swine, domestic species 4  A4100                             Live sheepNow let’s download both datasets and ask the ChatGPT code interpreter to run some analysis for us. We want to see some basic analysis of the data that use these to run some basic analysis.Visualizing and Analyzing the Livestock Production DataWe can leverage the Code Interpreter plugin to visualize and analyze the downloaded livestock production data. I’ve uploaded the file to the code interpreter and ask it to create an analysis of the data:The result is a very good initial evaluation of the data. The complete response its:The data seems to pertain to livestock production and has the following columns: - `freq`: The frequency of the data - `lsu`: LSU (Livestock Unit) - a reference unit which facilitates the aggregation of livestock from various species and age according to their nutritional or feed requirement. - `animals`: The type of animals - `farmtype`: Type of the farm - `agrarea`: The agricultural area - `so_eur`: Unknown, needs further clarification - `unit`: The unit of measurement - `geo\\TIME_PERIOD`: Geographical region and time period - `2005`, `2007`, `2010`, `2013`, `2016`, `2020`: These seem to be the years for which the data was collected, indicating the measure of the livestock production. To provide an insightful analysis, let's check the basic statistics of the numerical data and count the unique values of the categorical data. The basic statistics of the numerical data (years 2005, 2007, 2010, 2013, 2016, and 2020) are as follows: - For each year, we have a substantial number of entries (count). - The mean (average) varies across the years, with the highest being around 3,250 for the year 2007 and the lowest around 1,759 for the year 2020. - The standard deviation, which measures the dispersion of the data from the mean, is quite large for all years, suggesting a significant variation in the data. - The minimum value is 0 for all years, which indicates that in some cases, there was no livestock production. - The 25th, 50th (median), and 75th percentiles are 0 for many years, indicating that a substantial portion of the data has low or zero values. - The maximum value is quite large for all years, reaching up to around 3,062,740 in 2020, suggesting that some entities had very high livestock production. For the year 2016, there are no data points available. This might be due to missing data or no data collection in that year. Regarding the categorical data: - `freq` has only one unique value, which does not provide any variability. - `lsu`, `animals`, `farmtype`, `agrarea`, and `geo\\TIME_PERIOD` have multiple unique values, indicating different categories within these fields. - `so_eur` and `unit` also have only one unique value each. Next, we can further examine the distribution of the numerical data and the most common categories in the categorical data. We could also investigate the missing values in the dataset.This is a great starting point, now I’ll ask Code Interpreter to run some basic analysis on the data. I’ve uploaded the codes and ask it to join the data and provide some basic visualizations, which results in the next answer:The uploaded file contains codes for different types of livestock and their meanings. Now, let's join this data with the original dataset. After that, we will visualize the livestock production by year and by different categories.Up next it creates a visualization of the data:The image comes with the next description:The bar chart illustrates the total livestock production over the years. Some observations from the plot include: - The total livestock production appears to be highest in the years 2005 and 2007. - There's a noticeable drop in total production in 2010 and 2013. - The year 2016 does not have any data. - There's a slight increase in total production in 2020 compared to 2013. Next, let's analyze the livestock production by different categories, such as the type of animals and the type of farms.Up next it creates a visualization of the joined data by type of animal:It not only provides the visualization but also an analysis of the data:The bar chart shows the top 5 animal types with the highest total livestock production. Some observations from the plot include: - "Livestock" has the highest total production. However, this category may be a general category that includes various types of animals. - "Live bovine animals" and "Live swine, domestic species" have the next highest total production levels. - "Live poultry, domestic species" and "Live sheep" also contribute significantly to the total livestock production. Next, let's analyze the livestock production by different farm types.As we can see, the analysis continues with the next of step of the analysis. First by creating the visualization:And then by providing the resulting analysis:The bar chart shows the top 5 farm types with the highest total livestock production. Some observations from the plot include: - The farm type "FT15_SO" has the highest total livestock production. However, without a proper definition for this farm type, it's hard to interpret what this category entails. - The farm types "FT16_SO", "FT10_SO", "FT14_SO", and "FT08_SO" also have significant levels of livestock production. These visualizations provide a broad overview of the livestock production trends by year, animal type, and farm type. For a more detailed analysis, it would be beneficial to have more context or information about the different categories, especially the farm types.The Code Interpreter plugin is particularly useful for conducting data analysis and visualization tasks. It allows users to directly interact with large datasets within ChatGPT, posing questions and extracting valuable insights without the need for writing code. The plugin's ability to handle substantial amounts of data enables efficient exploration and analysis, enhancing decision-making processes. Furthermore, it provides visual representations of complex datasets, simplifying data comprehension and aiding in the communication of findings.The resulting analyses provide insights into the most common animal types and farm types contributing to livestock production.ConclusionThe Eurostat dataset, integrated with OpenAI's ChatGPT Code Interpreter plugin, streamlines data analysis. This fusion of programming and language models empowers users to extract insights from vast datasets, regardless of coding expertise. With simplified workflows, the plugin caters to programmers and non-programmers alike, enabling data retrieval, filtering, visualization, and basic analysis. This innovation has applications in marketing, finance, HR, and healthcare, democratizing coding and fostering collaboration. The technology's evolution holds the promise of revolutionizing programming and data analysis, empowering users to glean insights from intricate datasets for informed decision-making across industries.Author Bio:Alan Bernardo Palacio is a data scientist and an engineer with vast experience in different engineering fields. His focus has been the development and application of state-of-the-art data products and algorithms in several industries. He has worked for companies such as Ernst and Young, Globant, and now holds a data engineer position at Ebiquity Media helping the company to create a scalable data pipeline. Alan graduated with a Mechanical Engineering degree from the National University of Tucuman in 2015, participated as the founder in startups, and later on earned a Master's degree from the faculty of Mathematics in the Autonomous University of Barcelona in 2017. Originally from Argentina, he now works and resides in the Netherlands.LinkedIn
Read more
  • 0
  • 0
  • 193

article-image-ai-in-the-real-world-insurance
Julian Melanson
21 Jul 2023
5 min read
Save for later

AI in the Real World: Insurance

Julian Melanson
21 Jul 2023
5 min read
As the relentless tide of technological advancement swells, the insurance industry, among many others, is facing a pivotal transformation. The inception and evolution of insurance technology or "insurtech" mandates that insurance agents, brokers, and companies diligently adapt and assimilate novel tools and methodologies to augment their operational efficiency and competitiveness. Of the emerging technologies, the innovative language model ChatGPT, conceived and developed by OpenAI, is showing significant potential to redefine the landscape of the insurance industry.This powerful AI model offers a diverse suite of services, each capable of improving the lives of insurance agents in numerous ways. From perfecting customer service and streamlining underwriting to improving data analytics and fraud detection, ChatGPT opens a Pandora's box of possibilities. Yet, the efficacy and feasibility of these innovative solutions call for a judicious understanding of the technology's strengths and limitations.The advantages of AI in InsuranceFirstly, customer service, the linchpin of the insurance business, is an area that stands to gain substantially from the implementation of ChatGPT. Insurance products and processes, notoriously labyrinthine to the average consumer, are sources of frequent queries and uncertainties. By employing ChatGPT, insurance firms can automatically answer routine questions related to policy details, billing, claims statuses, and more, in an array of languages. In doing so, it significantly alleviates the burden on customer service agents and concurrently boosts customer engagement.Such automated systems also find favor among modern consumers, with reports suggesting a notable preference for chatbot interactions. ChatGPT, with its impressive capabilities in generating human-like text responses, can amplify the effectiveness of customer service chatbots. These enhancements invariably lead to increased customer satisfaction, freeing up human agents to tackle more complex customer concerns. Furthermore, ChatGPT's natural language processing prowess can be harnessed to guide customers on suitable insurance products and services, digitizing sales and distribution.The underwriting process, a traditionally time-consuming task characterized by risk evaluation, is another sector ripe for the automation that ChatGPT brings. While Artificial Intelligence (AI) and Machine Learning (ML) models have previously been employed to improve the accuracy of risk assessment, gaps in data and information remain problematic. ChatGPT addresses this issue by enhancing data collection and analysis, investigating digital resources for analogous cases, and speeding up the identification of risk patterns.Through this sophisticated data analysis, ChatGPT can evaluate factors like a customer's age, financial status, occupation, and lifestyle, thereby determining their risk profile. This information enables insurers to offer personalized coverage and pricing, improving customer experience, and streamlining underwriting. In addition, it can alert insurers about high-risk individuals or circumstances, proactively averting potential losses. This automatic evaluation brings with it many questions around AI and ethics (which you can read more about here) but the advantages of getting such a system working are clear.Claims processing, an insurance operation infamous for its high cost and low level of digitization, is another area primed for disruption by ChatGPT. The AI model can proficiently extract and categorize information from claims forms and other documents, drastically reducing the laborious and time-intensive task of manual data entry.A significant advantage arising from automating claims processing is its potential in fraud detection. With estimates by the FBI suggesting that insurance fraud costs American families hundreds of dollars each year in premium hikes, the value of early fraud detection cannot be overstated. ChatGPT can help in showing patterns of inconsistencies in claim forms, then flagging suspicious entries for human review. By alerting insurers to both overt and covert attempts at fraud, ChatGPT can help save billions of dollars annually.Reasons to be carefulAs a caveat, while the utility and advantages of ChatGPT in the insurance industry are substantial, one must consider the nascent stage of this technology. Its real-world impact will be contingent on its successful integration within existing processes and wide-ranging adoption. Moreover, while AI systems offer remarkable capabilities, they are not infallible and require human supervision. Thus, the technology's explainability, its transparency, and its limitations should be carefully considered and understood.In summary, the potential of ChatGPT to transform the insurance industry is vast, promising efficiency gains, cost reductions, and enhanced customer service. But realizing these advantages requires industry-wide receptiveness, careful integration, and judicious application, along with a respect for the limitations of the technology.Author BioJulian Melanson is one of the founders of Leap Year Learning. Leap Year Learning is a cutting-edge online school that specializes in teaching creative disciplines and integrating AI tools. We believe that creativity and AI are the keys to a successful future and our courses help equip students with the skills they need to succeed in a continuously evolving world. Our seasoned instructors bring real-world experience to the virtual classroom and our interactive lessons help students reinforce their learning with hands-on activities.No matter your background, from beginners to experts, hobbyists to professionals, Leap Year Learning is here to bring in the future of creativity, productivity, and learning!
Read more
  • 0
  • 0
  • 111

article-image-ai-in-the-real-world-real-estate
Julian Melanson
20 Jul 2023
4 min read
Save for later

AI in the Real World: Real Estate

Julian Melanson
20 Jul 2023
4 min read
The fast-paced development of Artificial Intelligence has already started reshaping various sectors, with the real estate industry standing out as a prominent beneficiary. Of particular interest is the potential AI presents in streamlining property valuation, a critical process that underlies a myriad of real estate activities, including setting sale prices, making investment decisions, and optimizing home insurance premiums. While the conventional means of property valuation have their merits, they are far from perfect. This article delves into the potential of AI, specifically OpenAI's ChatGPT, in transforming property valuation in the real estate sector, discussing the challenges inherent to traditional approaches and exploring the benefits offered by this AI-driven approach.The Current State of Property ValuationProperty valuation is a meticulous process that draws on a variety of data sources, both public and private. Depending on the valuation's purpose, the time and effort committed to research can differ significantly. For instance, real estate brokers might base their Broker Price Opinions on a limited set of comparable properties, while appraisers might undertake a thorough firsthand inspection to understand a property's condition, quality, and value comprehensively.Despite the evolution of valuation methodologies over the years, traditional approaches still grapple with certain obstacles. One of the primary issues is data inconsistency, mainly arising from the dispersed and scattered nature of relevant property data across various sources. While attempts have been made to centralize information on property features, ownership changes, and other key insights, consistency in data remains elusive. The result is disparities in the Automated Valuation Models (AVMs) currently used, which can lead to divergent valuations for the same property.Moreover, human bias forms a significant challenge in property appraisals. It's often difficult to find identical properties for comparison, leading to inevitable subjectivity in adjustments made to reconcile price differences. Studies show that appraised values fall below the agreed purchase price in just 10% of cases, suggesting a propensity towards price confirmation bias, a situation that calls for greater objectivity in home appraisals.Integrating AI into Property Valuation: The Role of ChatGPTIn response to these challenges, the integration of AI into the property valuation process presents a promising solution. The application of AI, especially advanced language models like ChatGPT, can offer consistent examinations of a property’s condition and quality, mitigating issues associated with data inconsistencies and human bias.ChatGPT, a generative pre-trained transformer, has been designed to understand and generate human-like text based on given input. In the context of real estate, it offers tremendous potential in data analysis and, consequently, in generating accurate property valuations. Traditionally, property valuations have been conducted by human appraisers who assess a property’s worth based on a range of factors such as location, size, and condition. However, this approach can be time-consuming, costly, and susceptible to human error.By incorporating ChatGPT into the valuation process, real estate professionals can input relevant data into the AI model, which can then analyze the data and supply a detailed valuation report. The implications of this are transformative for the industry: it offers considerable time savings, reduces the potential for errors, and enhances the transparency of the valuation process.A Practical Application of ChatGPT in Property ValuationHere’s a very simple prompt that illuminates how ChatGPT can be a great guide in the property valuation process:    The evolution of AI has unlocked numerous opportunities for innovation and efficiency across a variety of sectors, with the real estate industry being no exception. Particularly, the advent of AI models like ChatGPT has opened new avenues for enhancing the accuracy and efficiency of property valuations. By surmounting the obstacles inherent to traditional valuation methodologies, such as data inconsistencies and human bias, AI offers a more streamlined, transparent, and precise approach to property valuation. Author BioJulian Melanson is one of the founders of Leap Year Learning. Leap Year Learning is a cutting-edge online school that specializes in teaching creative disciplines and integrating AI tools. We believe that creativity and AI are the keys to a successful future and our courses help equip students with the skills they need to succeed in a continuously evolving world. Our seasoned instructors bring real-world experience to the virtual classroom and our interactive lessons help students reinforce their learning with hands-on activities.No matter your background, from beginners to experts, hobbyists to professionals, Leap Year Learning is here to bring in the future of creativity, productivity, and learning!
Read more
  • 0
  • 0
  • 102
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-mitigating-the-risks-of-chatgpt-in-finance
Julian Melanson
13 Jul 2023
5 min read
Save for later

Mitigating the Risks of ChatGPT in Finance

Julian Melanson
13 Jul 2023
5 min read
The application of advanced AI tools, such as ChatGPT, in various industries, particularly finance, has proven transformative due to its extensive language processing capabilities. ChatGPT's functions within the financial sector are diverse and impressive. It can understand financial market dynamics, suggest products, identify specific entities, and generate financial summaries, reports, and forecasts. Furthermore, the potential of training ChatGPT for fraud prevention and detection is an exciting prospect.However, as the integration of ChatGPT into the financial services realm becomes more prevalent, it brings to the fore several ethical challenges. Therefore, the onus is on both researchers and practitioners to ensure that the technology's use is responsible and advantageous to all parties involved. The solutions to these ethical challenges often require a multi-faceted approach, focusing on data exposure, misinformation, technology dependency, privacy concerns, and social engineering.The Ethical Challenges InvolvedOne of the paramount ethical challenges is data exposure. For example, ChatGPT users working with financial data might unintentionally disclose sensitive information. Additionally, during the AI model's training phase, there's a risk of exposing confidential elements such as proprietary code snippets, API keys, or login credentials.ChatGPT can sometimes generate biased or inaccurate responses, causing misinformation. The tool, at present, operates based on data sets that only run up to September 2021, which are sourced online and not always accurate. Therefore, financial professionals must exercise caution while using such advice to prevent the propagation of misinformation.Furthermore, while AI can be a powerful tool for financial decision-making, relying solely on technology can undermine human judgment and intuition. Financial professionals could fall into the trap of misinterpreting or overly depending on ChatGPT's advice, thereby overlooking the importance of human expertise in the financial sector. Therefore, it is crucial to strike a balance between utilizing AI's efficiency and maintaining human critical thinking.As ChatGPT requires an extensive amount of data for training, this raises significant privacy concerns. The information collected could pose serious risks to both individuals and organizations if exposed or used maliciously. In tandem with privacy concerns, social engineering issues arise as well. There is a potential for cybercriminals to misuse ChatGPT, impersonating individuals or organizations to conduct successful phishing attacks.Solving the ProblemAddressing these ethical challenges requires robust solutions. Firstly, the co-creation approach, which emphasizes public participation and stakeholder involvement in designing the AI algorithm. This strategy includes key choices in the algorithm, from the scope of its use to mitigating biases and tackling misinformation. It also ensures that humans keep a certain level of control over the AI tool, thus preventing total dependency on the technology.Secondly, the institutional approach can ensure the ethical use of ChatGPT in finance. This approach demands the establishment of concrete rules for managing ChatGPT, including training policy regulators to scrutinize and audit the AI algorithm and developing regulations. The focus is on creating transparent tools that ensure user privacy and constantly upgrade security measures to prevent breaches by cybercriminals.Lastly, it's vital to maintain a harmonious blend of AI-based decision-making and human intuition. While ChatGPT can crunch data and analyze trends with efficiency, human professionals have the experiential knowledge to make intuitive financial decisions. The amalgamation of both AI and human insight can lead to mutual learning and overall improvement in financial decision-making. It can also help address legal obstacles in financial domains that AI might overlook, thus ensuring the accuracy and reliability of financial decisions.The UK Finance paper on AI Fairness in Financial Services recommends a multi-disciplinary approach:Frontline business must be clear on the objective of the use of AI, the risks to individuals and to the business, and the extent to which risks of unfair treatment will be managed and explained to stakeholders.Data scientists are central to the technical aspects of the use, testing and monitoring of AI.Legal and Compliance need to be involved (including in any preliminary stages) to provide appropriate challenge, to oversee testing and to assist with fair process and related transparency principle.In addition, human application can mitigate the looming threat of job loss due to automation. While technology like ChatGPT can automate many functions, it is essential to preserve roles where human intuition, expertise, and judgment are irreplaceable.While the adoption of ChatGPT in finance is indeed a technological advancement, it comes with ethical challenges that require strategic and thoughtful solutions. Companies must adopt strategies such as co-creation and institutional approaches to ensure ethical usage. Furthermore, they need to strike a balance between AI and human insight to maintain the integrity of financial decisions. By addressing these challenges and implementing relevant strategies, we can ensure a future where AI not only augments the financial sector but also respects the values that we hold dear.Author BioJulian Melanson is one of the founders of Leap Year Learning. Leap Year Learning is a cutting-edge online school that specializes in teaching creative disciplines and integrating AI tools. We believe that creativity and AI are the keys to a successful future and our courses help equip students with the skills they need to succeed in a continuously evolving world. Our seasoned instructors bring real-world experience to the virtual classroom and our interactive lessons help students reinforce their learning with hands-on activities.No matter your background, from beginners to experts, hobbyists to professionals, Leap Year Learning is here to bring in the future of creativity, productivity, and learning!
Read more
  • 0
  • 0
  • 98

article-image-chatgpt-and-ai-in-the-crypto-market
Julian Melanson
12 Jul 2023
6 min read
Save for later

ChatGPT and AI in the Crypto Market

Julian Melanson
12 Jul 2023
6 min read
OpenAI’s ChatGPT has gained significant attention since it first launched, and with its versatile capabilities and high accuracy, it has the potential to make a substantial impact on the crypto market. It’s crucial that we explore how AI and Natural Language Processing (NLP) can assist in fraud detection and prevention, understand the capabilities and limitations of ChatGPT in the crypto industry and trading, highlight the importance of AI in safeguarding the crypto market, and discuss ChatGPT's role in crypto compliance, AML, security, and its future implications.AI and NLP for Fraud Detection and PreventionNLP is a branch of AI that enables machines to read, understand, and draw conclusions from human languages. By using computational linguistics and statistical models, NLP can show suspicious behavior patterns and uncover fraud in financial transactions. For instance, NLP can detect inconsistencies in credit applications or identify suspicious transactions on credit cards.Capabilities and Limitations of ChatGPT in the Crypto IndustryIn the crypto industry, ChatGPT has various applications, particularly in trading. It can supply a historical overview of a certain type of cryptocurrency, analyze market data, forecast price movements, create strategies, and find trading opportunities. By leveraging ChatGPT, traders can make better-informed decisions and capitalize on emerging possibilities.Here is a very simple example of how ChatGPT can aid in creating a strategy for identifying Chainlink bottoms using the relative strength index (RSI), support and resistance levels, and moving averages:While ChatGPT can elucidate various aspects of the crypto arena, it's imperative to recognize its potential limitations, particularly pertaining to the source reliability of its information. The internet is fraught with misinformation, and since the advent of GPT-4 that offers web-browsing capabilities, such misinformation could inadvertently affect AI tools like ChatGPT. Within the volatile crypto market, such unreliable information can lead to imprudent investments. It’s advisable to fact-check the data ChatGPT provides to mitigate the risk of utilizing information from dubious sources.The Importance of AI in Safeguarding the Crypto MarketThe adoption of blockchain technology has brought benefits such as increased openness, data consistency, and security. By integrating AI with blockchain, a more secure and intelligent system can be established. Blockchain ensures the integrity of shared information and models used by AI, while AI enhances fraud detection capabilities. The combination of AI and blockchain creates a more resilient system that is resistant to attacks and fraud.ChatGPT in Crypto Fraud DetectionChatGPT, with its NLP capabilities, can contribute to fraud detection in the crypto market in several ways:Identifying Suspicious Transactions and Activities: By analyzing emails for suspicious language patterns and detecting anomalies, ChatGPT can help identify potential fraud. It can compare email texts to earlier communications from the same individual, ensuring consistency and detecting deviations.Analyzing Patterns and Anomalies in Crypto Trading Data: ChatGPT can analyze market data and find significant patterns and trends that can aid traders in making informed decisions.Monitoring Social Media and External Sources: ChatGPT can help compliance teams in monitoring chat and social networking platforms for suspicious activities, such as market manipulation and insider trading.Utilizing Advanced Machine Learning Algorithms for Risk Assessment: Machine learning algorithms can predict the likelihood of default on loans or identify risky transactions. This information helps lenders make more informed decisions and manage risks effectively.ChatGPT in Crypto Compliance and AMLIdentifying and Verifying the Identity of Crypto Traders and Investors: ChatGPT excels in identifying and verifying the identity of traders and investors, ensuring the authenticity of individuals involved in crypto transactions.Monitoring for Money Laundering and Financial Crimes: By leveraging AI capabilities, compliance teams can monitor transactions and identify suspicious patterns indicative of money laundering or other financial crimes.Keeping Up with Regulatory Changes and Compliance Requirements: AI chatbots like ChatGPT can adapt to regulatory changes and comply with requirements set by authorities to ensure seamless operations within legal frameworks.Developing and Implementing Effective KYC and AML Procedures: NLP and monitored machine learning techniques play a vital role in streamlining Know Your Customer (KYC) procedures. These technologies facilitate efficient identity verification and analysis of unstructured content.ChatGPT in Crypto SecurityProtecting Crypto Assets and Digital Wallets: AI tools like ChatGPT enhance security measures in crypto exchanges and platforms, safeguarding digital assets and wallets from potential threats.Enhancing Security in Crypto Exchanges and Platforms: ChatGPT helps in verifying the identities of investors, bolstering the overall security mechanism of crypto exchanges and platforms.Identifying and Preventing Phishing and Hacking Attempts: AI algorithms can block unauthorized smart contracts and reduce the risk of phishing and hacking attacks, thereby enhancing the security of the crypto industry.Developing and Implementing Advanced Security Protocols: AI algorithms and machine learning techniques help organizations identify vulnerabilities in their security architecture and improve overall system security.Future Developments and Implications of ChatGPT in CryptoAdvancements in NLP and AI are expected to have a significant impact on fraud detection and prevention. As society moves toward a cashless economy, the role of AI in identifying and preventing digital fraud becomes increasingly critical. ChatGPT's ability to fine-tune popular themes enables traders to stay updated on crypto news, retrieve relevant data, and generate trading strategies based on historical information. However, it is crucial to consider the ethical implications and potential risks associated with using AI for fraud detection and in financial systems. Responsible and informed use of AI technologies can contribute to building trust and credibility in the crypto market.ChatGPT, with its advanced NLP capabilities, offers exciting possibilities for the crypto market. Its potential to enhance fraud detection, bolster security measures, and build trust and credibility is promising. However, it is essential to approach AI adoption in the crypto market cautiously, taking into account ethical considerations and potential implications. As technology continues to evolve, the responsible and informed use of AI can pave the way for a safer and more efficient crypto ecosystem.Author BioJulian Melanson is one of the founders of Leap Year Learning. Leap Year Learning is a cutting-edge online school that specializes in teaching creative disciplines and integrating AI tools. We believe that creativity and AI are the keys to a successful future and our courses help equip students with the skills they need to succeed in a continuously evolving world. Our seasoned instructors bring real-world experience to the virtual classroom and our interactive lessons help students reinforce their learning with hands-on activities.No matter your background, from beginners to experts, hobbyists to professionals, Leap Year Learning is here to bring in the future of creativity, productivity, and learning!
Read more
  • 0
  • 0
  • 108

article-image-data-analysis-made-easy-with-chatgpt
Sagar Lad
02 Jul 2023
5 min read
Save for later

Data Analysis Made Easy with ChatGPT

Sagar Lad
02 Jul 2023
5 min read
Are you weary of trawling through heaps of analysis data in search of meaningful insights? With ChatGPT, the rules will soon alter. ChatGPT may reveal hidden patterns and trends in your data that you never imagined were there because of its sophisticated natural language processing skills. In this blog article, we'll look at how exploratory data analysis with ChatGPT can revolutionize your data and change the way you conduct business.Data Analysis with ChatGPTFor data analysts, ChatGPT can be a useful tool for processing, exploring, communicating, and collaborating on their data-driven ideas. Large volumes of data can be analyzed and processed by ChatGPT fast and effectively. Written inquiries can be interpreted and understood by ChatGPT through its language processing skills, which also allow it to extract pertinent insights from the data. Here are a few benefits that ChatGPT can provide: Data analysts can use ChatGPT to study their data, spot trends, and even produce useful data visualizations. The data is clearly outlined in these graphics, which makes it simpler for analysts to spot trends and insights. Data analysts can utilize ChatGPT to explain their findings to non-technical stakeholders. The chatbot can assist data analysts in providing simple explanations of complicated data ideas and insights by using natural language. Data analysts might benefit from ChatGPT's help in coming up with fresh, insightful queries to pose to their data. Analysts can investigate novel lines of inquiry and unearth previously unconsidered hidden insights by using natural language queries. Let's look at how chatGPT can make data analysis easy and straightforward. As a data modeler, I want to investigate the data's dictionary and metadata first. Image 1: : Data Dictionary Using ChatGPT, Part 1Image 2 : Data Dictionary Using ChatGPT, Part 2ChatGPT gives us thorough details about the data dictionary for each column, including a complete description of each column. The final user will benefit from this guidance on when and how to use the data.Asking chatGPT about the dataset's number of rows and columns will help you better grasp the overall statistics.  Image 2 : Dataset Statistics  As seen in the image above, chatGPT gives us a precise estimate of the dataset's number of rows and columns. After getting a broad overview of the dataset, let's examine the data's quality: Image 3 : Exploratory Data Analysis - Null Value StatisticsHere, we've given the chatGPT an input containing the dataset and requested it to determine the percentage of null values therein in order to determine whether the data can be used for analytics. The dataset does not contain any null values, hence chanGPT responds that the given dataset contains no missing values.Now, we can observe that the data set's header information is absent. Before we can use the data, the columns must contain meaningful data. Image 4 : Dataset Column Naming ConventionLet's ask chatGPT how it can deliver valuable header data. As you can see, the output of chatGPT is a column header with a description and business-specific naming standards. The technical team's and business users' lives are made easier in terms of using this data.We now know that the data quality is good. As this will affect the results of the data analysis, let's look for any outliers in the dataset. Image 5 : Detect Outliers in the DatasetIn this case, chatGPT is carrying out an in-depth analysis at the column level to see whether any outliers are there. It's okay if it doesn't exist. If it does, it also offers advice on what kind of outlier is present and how it can affect the entire data analysis procedure.Let's now look at how to use chatGPT to eliminate those outliers.Image 7 : Remove Outliers from the dataset using python, Part 1 Image 8 : Remove Outliers from the dataset using python, Part 2Therefore, for a given sample dataset, ChatGPT offers a thorough Python code that can be used to automatically eliminate the observed outliers. The team may have business analysts who are unfamiliar with Python. Let's see how chatGPT can assist business analysts with their data analysis work.Image 7 : SQL Query to calculate monthly revenue, Part 1Image 8 : SQL Query to calculate monthly revenueIn this case, chatGPT offers a default query that the business analyst may utilize to figure out the monthly income for a particular dataset. Let's then ask chatGPT to take on the role of a data analyst and offer further insights for a certain dataset. Image 8 : Step by Step Data Analysis using chatGPT, Part 1  Image 9 : Step-by-Step Data Analysis using ChatGPT, Part 2As we can see from the chatGPT's results, it offers us step-by-step advice on various studies and results that may be applied on top of this particular dataset. The execution of each of these tasks using chatGPT is possible for each phase of the overall data analysis process.Let's ask chatGPT to undertake this data analysis work so that it may use Python to analyze prices for the given dataset:Image 9 : Price Analysis using python, Part 1Image 10 : Price Analysis using python, Part 2Image 11 : Price Analysis using Python, Part 2For the purpose of doing price analysis on a given dataset, ChatGPT has developed a Python code and sample output. We can draw a judgment about how the prices are changing over time based on the data points at hand from this output.ConclusionIn this article, we go into great detail on how to use chatGPT for a variety of exploratory data analysis tasks. Additionally, we looked closely at different approaches to carrying out data analysis tasks using Python and SQL. ChatGPT is, in a word, a very useful tool for performing exploratory data analysis tasks while working with massive volumes of data.Author BioSagar Lad is a Cloud Data Solution Architect with a leading organization and has deep expertise in designing and building Enterprise-grade Intelligent Azure Data and Analytics Solutions. He is a published author, content writer, Microsoft Certified Trainer, and C# Corner MVP. Link - Medium , Amazon , LinkedIn 
Read more
  • 0
  • 0
  • 150

article-image-making-the-most-of-chatgpt-code-interpreter-plugin
Julian Melanson
22 Jun 2023
5 min read
Save for later

Making the Most of ChatGPT Code Interpreter Plugin

Julian Melanson
22 Jun 2023
5 min read
As we stand on the threshold of a new age in Artificial Intelligence,, OpenAI recently announced the rollout of its ChatGPT Code Interpreter plugin for ChatGPT Plus subscribers, marking a significant evolution in the capabilities of AI. This groundbreaking technology is not merely another link in the AI chain but rather an exemplar of the transformative capabilities AI can bring to the table, particularly in programming and data analysis.The ChatGPT Code Interpreter plugin positions itself as an invaluable tool for developers, promising to significantly augment and streamline their workflows. Among its multiple functionalities, three stand out due to their potential impact on the programming landscape and data analysis, which work in tandem to extract valuable insights.Data VisualizationAt its core, the ChatGPT Code Interpreter excels in the domain of data visualization. In a world increasingly reliant on data, the ability to transform complex datasets into visually comprehensible and comprehensive formats is priceless. The plugin simplifies the arduous task of crunching through complex numbers and data sets, producing insightful visualizations without the need for prompt engineering. This proficiency in creatively rendering data echoes the power of platforms like Wolfram, introducing a new era of ease and efficiency in data comprehension.Here’s an example from Justin Fineberg’s TikTok: Click hereFile ConversionThe ChatGPT Code Interpreter extends its versatility into the realm of file conversion. This feature provides a simple solution to the often cumbersome task of converting files from one format to another. Its impressive functionality ranges from changing audio file formats, like MP3 to WAV, to converting an image into a text file. This capability paves the way for more accessible content transformation, such as easily convert PDF documents into editable text files.Here’s an example from Twitter user Riley Goodside: Click herePython Code ExecutionWhat sets the ChatGPT Code Interpreter plugin apart is its prowess in executing Python code within a sandboxed, firewalled execution environment. This essentially means that all the data visualizations are generated using Python, thereby lending the plugin an additional layer of power and versatility.As the plugin is still in its alpha stage, gaining access currently involves joining a waitlist, and  OpenAI has not publicly stated when a large-scale rollout will take place. However, those eager to explore its features have an alternative route via Discord's GPT Assistant bot, which already incorporates the Code Interpreter plugin to enhance its features and functionalities.This revolutionary plugin is not merely an advanced code interpreter; it's a complete tool that uses Python to generate code from natural language input and run it. The results are then presented within the dialogue box. The chatbot’s functionality extends to solving mathematical problems, data analysis and visualization, and file conversion, with an adeptness in these domains that rivals experienced coders.Beyond its immediate capabilities, the ChatGPT Code Interpreter plugin has broader implications for the programming and data analysis industry. It is reminiscent of Github Copilot X in its design, aimed at making workflows more creative and efficient. For instance, when asked to plot a function, the plugin not only generates the graph but also offers the option to 'show work', revealing the exact code it created and executed to generate the graph.The accessibility and user-friendliness of the plugin are expected to democratize the coding landscape, opening up the world of programming to a wider audience. This feature holds tremendous potential to accelerate collaborations, allowing technical and non-technical team members to work together more effectively on data analysis projects.Practical use cases for the ChatGPT Code Interpreter extend beyond the realm of programming, spanning various industries. Marketing teams, for instance, can leverage their capabilities to analyze customer data, segment audiences, and create targeted campaigns. Finance teams can utilize the plugin for tasks like financial modeling, forecasting, and risk analysis. Similarly, human resource teams can use it to analyze employee data, identify performance trends, and make data-driven hiring decisions. Even the healthcare sector stands to benefit, as the tool can analyze patient data, identify patterns in health outcomes, and thus enhance patient care.Accessing ChatGPT Code InterpreterIf you’re selected from the waitlist, here’s a step-by-step guide on how to install the plugin:Ensure you're a ChatGPT Plus subscriber, paying the $20 monthly fee.Log into ChatGPT on the OpenAI website.Click on 'Settings', then the three-dot menu next to your login name.In the 'Beta features' menu, enable 'Plug-ins'. For web browsing access, enable that too.Close the menu, find the language model selector, and choose 'Plugin Store' from the drop-down.Click 'All plug-ins', find 'Code Interpreter' in the list and install it.Now, you can interact with ChatGPT using the Code Interpreter plug-in.SummaryThe ChatGPT Code Interpreter plugin presents a transformative approach to programming and data analysis, automating code generation, facilitating data exploration, and improving code quality. This plugin empowers users to derive more value from their data, aiding in the formulation of strategic insights. As AI continues to evolve, tools like the ChatGPT Code Interpreter will undoubtedly play an instrumental role in shaping the future of data interaction and understanding, ultimately revolutionizing the landscape of data analysis.Author BioJulian Melanson is one of the founders of Leap Year Learning. Leap Year Learning is a cutting-edge online school that specializes in teaching creative disciplines and integrating AI tools. We believe that creativity and AI are the keys to a successful future and our courses help equip students with the skills they need to succeed in a continuously evolving world. Our seasoned instructors bring real-world experience to the virtual classroom and our interactive lessons help students reinforce their learning with hands-on activities.No matter your background, from beginners to experts, hobbyists to professionals, Leap Year Learning is here to bring in the future of creativity, productivity, and learning!
Read more
  • 0
  • 0
  • 157
article-image-creating-cicd-pipelines-effortlessly-using-chatgpt
Sagar Lad
22 Jun 2023
5 min read
Save for later

Creating CI/CD Pipelines Effortlessly using ChatGPT

Sagar Lad
22 Jun 2023
5 min read
 In the fast-paced world of software development, continuous integration and continuous delivery (CI/CD) pipelines have become crucial for efficient and reliable software deployment. However, building and maintaining these pipelines can be a daunting task, requiring extensive knowledge and expertise. But what if there was a way to simplify this process and make it more accessible to developers of all skill levels? Enter ChatGPT, the groundbreaking language model developed by OpenAI. In this article, we explore how ChatGPT can effortlessly guide developers in building robust CI/CD pipelines, revolutionizing the way software is delivered.What is DevOps?DevOps team is a combination of the Development team and Operations team working together in order to accelerate the time to market and quality of the software development. DevOps way of working is more a shift in the mindset which has a major effect on the team and organization’s way of working. With the DevOps mindset, there are no silos between the development and operations teams. DevOps team mainly focuses on the automation to increase the reliability by enabling continuous integration, and continuous deployment pipelines.Image 1 : DevOps Lifecycle DevOps Lifecycle mainly consists of setting up an automated and collaborative environment to discover, plan, build, and test the artefacts. Once the artefacts are downloaded, they can be deployed to their respective environments. Throughout the DevOps lifecycle, the complete team has to work closely to maintain the alignment, velocity, and quality of the deliverables.DevOps implementation mainly involves below activities :Discover productAn iterative approach for agile planningBuild pipelines for branching, merging, and workflows for the development processAutomated Testing using CI pipelines Deployment using release pipelinesOperationalize and manage end-to-end IT deliveryContinuous monitoring of the deployed softwareContinuous feedback and improvements for future releaseChatGPT for DevOpsAutomation: DevOps Tools like Jenkins, Ansible, Terraform, etc provide workflow automation. On the other hand, we can use chatGPT to automate the below DevOps activities that require manual intervention.Automation Testing: Testing scenarios and automated testing can be enabled using the chatGPT as a part of the continuous build pipeline.Release documentation: Maintaining documentation for each feature is a manual and tedious task. With the help of chatGPT, we can write code in any language to create an automated documentation for each release. With ChatGPT, we can generate code for bicep templates and YAML for Azure DevOps, Terraform , Jenkins, and Lambda code for the DevOps activities.  DevSecOps Implementation: Security is a critical factor in order to develop any software. ChatGPT can be used to monitor cloud resources, manage and detect security vulnerabilities , and scan for networking issues, open ports, database, and storage configurations as per the industry standards and requirements.Continuous Monitoring: Customised dashboard for monitoring is a key metric for proactive monitoring and taking data-driven decisions. We can create a dashboard using the chatGPT to generate code including a variety of components such as charts, graphs, and tables.Let’s now ask chatGPT for each of these steps to create a DevOps process flow for a software project. The first step is to set up the Azure DevOps repository structure including the branching policies, and pre, and post-deployment approval:Image 2: Azure Devop Repo structure & branching policiesAs you can see, a recommendation from ChatGPT is to create a new Azure DevOps repository with proper naming conventions. In order to set up the branching policies, we need to configure the build validations, set up the reviewers, status check, and work item linking in the Azure Boards.Image 3: Azure DevOps Continuous Integration PipelineHere, we have requested chatGPT to create a YAML continuous integration build pipeline in Azure DevOps including the code quality checks and testing. ChatGPT provides us with a YAML pipeline that has multiple stages - one for sonarqube, one for fortify code quality checks, one for automation testing, and one to download the artefacts. Once the CI pipeline is ready, let’s ask ChatGPT to build IaC(Infrastructure as a Code Pipeline) to deploy Azure services like Azure Data Factory and Azure Databricks.Image 4 : Azure DevOps Continuous Deployment PipelinesHere, we can see the step-by-step process to build the continuous deployment pipelines which are using shell script to deploy the Azure Data Factory and Azure CLI to deploy the Azure Databricks. This pipeline also has an integration with the branches to include and it is using variable groups to create a generic pipeline. Let’s see how we can build monitoring dashboards using chatGPT:ConclusionSo chatGPT is not a threat to the DevOps engineers but it will boost up productivity by embracing the technology to set up and implement the DevOps way of working.  In order to get the desired results, detailed prompt input should be provided to generate content from chatGPT to meet the expectations.Author BioSagar Lad is a Cloud Data Solution Architect with a leading organization and has deep expertise in designing and building Enterprise-grade Intelligent Azure Data and Analytics Solutions. He is a published author, content writer, Microsoft Certified Trainer, and C# Corner MVP.Link - Medium , Amazon , LinkedIn
Read more
  • 0
  • 0
  • 872

article-image-creating-essay-generation-methods-with-chatgpt-api
Martin Yanev
22 Jun 2023
11 min read
Save for later

Creating Essay Generation Methods with ChatGPT API

Martin Yanev
22 Jun 2023
11 min read
This article is an excerpt from the book, Building AI Applications with ChatGPT API, by Martin Yanev. This book will help you master ChatGPT, Whisper, and DALL-E APIs by building nine innovative AI projectsIn this section, we will dive into the implementation of the key functions within the essay generator application. These functions are responsible for generating the essay based on user input and saving the generated essay to a file. By understanding the code, you will be able to grasp the inner workings of the application and gain insight into how the essay generation and saving processes are accomplished.We will begin by exploring the generate_essay() function. This function will retrieve the topic entered by the user from the input field. It will then set the engine type for the OpenAI API, create a prompt using the topic, and make a request to the OpenAI API for essay generation. The response received from the API will contain the generated essay, which will be extracted and displayed in the essay output area of the application. To add that functionality, simply remove the pass placeholder and follow the code below.def generate_essay(self): topic = self.topic_input.text()    length = 500    engine = "text-davinci-003"    prompt = f"Write an {length/1.5} words essay on the following topic: {topic} \n\n"    response = openai.Completion.create(engine=engine, prompt=prompt, max_tokens=length)    essay = response.choices[0].text    self.essay_output.setText(essay)Here, we retrieve the topic entered by the user from the topic_input QLineEdit widget and assign it to the topic variable using the text() method. This captures the user's chosen topic for the essay. For now, we can define the length variable and set it to 500. This indicates the desired length of the generated essay. We will modify this value later by adding a dropdown menu with different token sizes to generate essays of different lengths.We can also specify the engine used for the OpenAI API to be text-davinci-003, which will generate the essay. You can adjust this value to utilize different language models or versions based on your requirements. We can also create the prompt variable, which is a string containing the prompt for the essay generation.It is constructed by concatenating the text Write a {length/1.5} essay on the following topic: where the length/1.5 variable specifies how many words our essay should be. We need to divide the tokens number by 1.5, as 1 word in English represents about 1.5 tokes. After specifying the instructions, we can pass the topic variable to the prompt. This prompt serves as the initial input for the essay generation process and provides context for the generated essay.Once all variables are defined, we make a request to the ChatGPT API with the specified engine, prompt, and the maximum number of tokens (in this case, 500). The API processes the prompt and generates a response, which is stored in the response variable. From the response, we extract the generated essay by accessing the text attribute of the first choice. This represents the generated text of the essay. Finally, we can pass the AI response to the essay_output, displaying it in the user interface for the user to read and interact with.Moving on, we will examine the save_essay() function. This function will retrieve the topic and the generated essay. It will utilize the docx library to create a new Word document and add the final essay to the document. The document will then be saved with the filename based on the provided topic, resulting in a Word document that contains the generated essay. After removing the pass keyword, you can implement the described functionality using the code snippet below.def save_essay(self):    topic = self.topic_input.text()    final_text = self.essay_output.toPlainText()    document = docx.Document()    document.add_paragraph(final_text)    document.save(topic + ".docx")Here we retrieve the text entered in the topic_input widget and assign it to the topic variable using the text() method. This captures the topic entered by the user, which will be used as the filename for the saved essay. Next, we use the toPlainText() method on the essay_output widget to retrieve the generated essay text and assign it to the final_text variable. This ensures that the user can edit the ChatGPT-generated essay before saving it. By capturing the topic and the final text, we are now equipped to proceed with the necessary steps to save the essay to a file.We can now use the docx library to create a new Word document by calling docx.Document(), which initializes an empty document. We then add a paragraph to the document by using the add_paragraph() method and passing in the final_text variable, which contains the generated essay text. This adds the generated essay as a paragraph to the document. We can now save the document by calling document.save() and providing a filename constructed by concatenating the topic variable, which represents the topic entered by the user. This saves the document as a Word file with the specified filename.You can now test your Essay Generator by running the code in PyCharm and generating an essay following the steps below (see Figure 8.3):Enter a topic: Write an essay topic of your choice in the Topic Input field. For this example, I have chosen the topic “Ancient Egypt”.Generate Essay: Perform a single click on the Generate Essay button. The app will reach ChatGPT API and within a few seconds, you will have your essay displayed inside the Essay Output field.Edit the Essay: You can edit the essay generated by the Artificial Intelligence before saving it.Save: Perform a single click on the Save button. This action will save the edited essay to a Word document utilizing the save_essay() method. The Word document will be saved in the root directory of your project.Figure 8.3: Essay Generator creating an “Ancient Egypt” essayOnce the essay has been saved to a Word document, you can reshare it with your peers, submit it as a school assignment or use any Word styling options on it.This section discussed the implementation of key functions in our essay generator application using the ChatGPT API. We built the generate_essay() method that retrieved the user’s topic input and sent a request to the ChatGPT API for generating an AI essay. We also developed the save_essay() method that saved the generated essay in a Word document.In the next section, we will introduce additional functionality to the essay generator application. Specifically, we will allow the user to change the number of AI tokens used for generating the essay.Controlling the ChatGPT API TokensIn this section, we will explore how to enhance the functionality of the essay generator application by allowing users to have control over the number of tokens used when communicating with ChatGPT. By enabling this feature, users will be able to generate essays of different lengths, tailored to their specific needs or preferences. Currently, our application has a fixed value of 500 tokens, but we will modify it to include a dropdown menu that provides different options for token sizes.To implement this functionality, we will make use of a dropdown menu that presents users with a selection of token length options. By selecting a specific value from the dropdown, users can indicate their desired length for the generated essay. We will integrate this feature seamlessly into the existing application, empowering users to customize their essay-generation experience.Let's delve into the code snippet that will enable users to control the token length. You can add that code inside the initUI() methods, just under the essay_output resizing:self.essay_output.resize(1100, 500) length_label = QLabel('Select Essay Length:', self) length_label.move(327, 40) self.length_dropdown = QComboBox(self) self.length_dropdown.move(320, 60) self.length_dropdown.addItems(["500", "1000", "2000", "3000", "4000"])The code above introduces a QLabel, length_label, which serves as a visual indication for the purpose of the dropdown menu. It displays the text Select Essay Length to inform users about the functionality.Next, we create a QcomboBox length_dropdown which provides users with a dropdown menu to choose the desired token length. It is positioned below the length_label using the move() method. The addItems() function is then used to populate the dropdown menu with a list of token length options, ranging from 500 to 4000 tokens. Users can select their preferred length from this list.The final step is to implement the functionality that allows users to control the number of tokens used when generating the essay, we need to modify the generate_essay() function. The modified code should be the following:def generate_essay(self):    topic = self.topic_input.text()    length = int(self.length_dropdown.currentText())    engine = "text-davinci-003"    prompt = f"Write an {length/1.5} words essay on the following topic: {topic} \n\n"    response = openai.Completion.create(engine=engine, prompt=prompt, max_tokens=length)    essay = response.choices[0].text   self.essay_output.setText(essay)In the modified code, the length variable is updated to retrieve the selected token length from the length_dropdown dropdown menu. The currentText() method is used to obtain the currently selected option as a string, which is then converted to an integer using the int() function. This allows the chosen token length to be assigned to the length variable dynamically.By making this modification, the generate_essay() the function will utilize the user-selected token length when making the request to the ChatGPT API for essay generation. This ensures that the generated essay will have the desired length specified by the user through the dropdown menu.We can now click on the Run button in PyCharm and verify that the Dropdown menu works properly. As shown in Figure 8.4, a click on the Dropdown menu will show users all options specified by the addItems() function.Figure 8.4: Controlling essay length.The user will be able to choose a token amount between 500 and 4000. Now you can select the 4000 tokens option, resulting in a longer length of the generated essay. We can follow the steps from our previous example and verify that the ChatGPT API generates a longer essay for “Ancient Egypt” when the number of tokens is increased from 500 to 4000.This is how you can enhance the functionality of an essay generator application by allowing users to control the number of tokens used when communicating with ChatGPT. By selecting a specific value from the dropdown menu, users can now indicate their desired length for the generated essay. We achieved that by using the QComboBox to create the dropdown menu itself. The modified generate_essay() method retrieved the selected token length from the dropdown menu and dynamically assigned it to the length variable.SummaryIn conclusion, leveraging the capabilities of ChatGPT API to enhance essay generation opens up a world of interactive creativity. By incorporating practical examples and step-by-step instructions, we have explored how to generate essay-generating elements and make them interact seamlessly with ChatGPT. This powerful combination allows for the production of compelling, coherent, and engaging essays effortlessly. With the ever-evolving potential of AI, the future of essay generation holds immense possibilities. By embracing these techniques, writers and researchers can unlock their full creative potential and revolutionize the way we generate written content.Author BioMartin Yanev is an experienced Software Engineer who has worked in the aerospace and medical industries for over 8 years. He specializes in developing and integrating software solutions for air traffic control and chromatography systems. Martin is a well-respected instructor with over 280,000 students worldwide, and he is skilled in using frameworks like Flask, Django, Pytest, and TensorFlow. He is an expert in building, training, and fine-tuning AI systems with the full range of OpenAI APIs. Martin has dual master's degrees in Aerospace Systems and Software Engineering, which demonstrates his commitment to both practical and theoretical aspects of the industry.LinkedInUdemy
Read more
  • 0
  • 0
  • 133

article-image-exploring-openai-chatgpt-as-a-researcher-a-writer-and-a-software-user
Urszula Witherell
22 Jun 2023
6 min read
Save for later

Exploring OpenAI ChatGPT as a Researcher, a Writer and a Software User

Urszula Witherell
22 Jun 2023
6 min read
Artificial Intelligence is now a hot topic. The concept is not that new. In translations from one language to another, it was used since the 1960s to prepare rough drafts with the idea that the text will be refined by human editors. We interviewed Urszula Witherell, the author of the book Adobe Acrobat Ninja for her perspectives on using AI, specifically freely available ChatGPT: as a Reader, a Writer, and a Software User. What is your experience using ChatGPT as a Reader/Researcher?I am a curious person, and my mind constantly questions where and why. I love exploring new ideas, learning about people living in different cultures, and exploring different perspectives on topics that matter to me. AI as a research tool is wonderful, although very quickly I learned about its limitations. A simple test was to search for my own name. It knew nothing. My ego was not bruised as I am not a celebrity or want to be one. On the other hand, a simple search on Google very quickly found my name, my address, my family members, and everything I did online going back at least a decade. ChatGPT insisted that it knew nothing about me since its training ended in September 2021, 2 years ago. Similarly, when asked about a company website that has been active for at least 20 years it gave this reply: “I apologize for the confusion, but as an AI language model, I don't have the capability to browse the internet or visit specific websites. My responses are based on pre-existing knowledge up until September 2021. Therefore, I'm unable to provide a real-time description or analysis of the website “ or “Without specific information, I can't provide details about the website's content, purpose, or any other features it may have. It's always a good idea to visit the website directly to gather accurate and up-to-date information.” This made me ask a question how is ChatGPT trained? Where does the data come from?Once I manually copied information from the website and asked ChatGPT to generate a new response. It expanded on dry data and formulated a nice description of the website content.Image 1: ChatGPT cannot figure out what is the content of the websiteFigure 2: ChatGPT expanded the descriptionWhen I asked for descriptions of cities, landmarks, or public figures ChatGPT did provide sufficient information in full sentences with rich vocabulary. When asked to regenerate responses it provided fresh and even better-formulated descriptions. Again, not knowing where the data for AI come from, I depended on what I already knew about the locations to evaluate the accuracy of the response. It made me think about the famous quote: “Trust but verify”. So, compare results from multiple other sources to get the results that you can trust. Asked about liking a city, this is what I got: “As an AI language model, I don't have personal opinions or emotions, so I don't have the ability to like or dislike anything. However, I can provide information.” Not quite a chatty, opinionated friend, at least not yet.What is your experience using ChatGPT as a Writer?I think this is where the AI can be very helpful as it can be a thesaurus, spell-checker, and grammar checker all at once. It can re-write sentences; from just a rough draft it can expand descriptions and literally be your personal assistant during the writing process. You can see this in the screenshot example. As a bilingual person I often wonder how to express ideas in Polish for my friends in Poland. I discovered that translations of English text to Polish are very good. Grammar is correct, vocabulary is rich enough and sentences are full and mostly sound natural. Much better than what I had experienced in the past using Google Translate, when translated text needed a real human translator to be understood by a Polish-only speaking person (my parents). Of course, there are some flaws, such as the assumption of the male gender as an author. In Polish all nouns are assigned one of three genders: feminine (yes, ladies first), masculine and neutral. This is where corrections were needed. Additionally, since an example to be translated was a chapter of the software guide, it used many English terms in Polish grammatical form. My English teacher would faint at such a transgression, as we were always taught to use native expressions in place of foreign adoptions. But since technology vocabulary originated in English, so the rest of the world must just deal with it. I can live with that. I will depend on ChatGPT as a writer from now on. Definitely thumbs up!What about the perspective of a software user?Well, this is where the results vary, and it simply depends on what you are looking for. I tested a question about authoring Adobe InDesign documents for PDF accessibility and compliance with Section 508 requirements in the USA. ChatGPT provided a very detailed, step-by-step description of how to proceed, but in responses regenerated multiple times it missed the most advanced method when automated alt-text tags can be generated from image metadata using Object Styles. So again, AI helped, but I needed to know already what I could possibly do. Great help from a writer’s perspective, but not so good for someone who is exploring more advanced software functions.It is exciting to explore new platforms and technology. Like any new frontier, it brings excitement but also fear. And again, I am reminded that we as humans have not changed at all. Our toys changed, and we became more sophisticated and powerful, but we continue to need guidance on how to use yet another newly found power and knowledge. History is not a strong enough deterrent from misuse.SummaryIn conclusion, the interview with Urszula sheds light on the versatile applications of ChatGPT in the roles of a researcher, writer, and software engineer. Urszula highlights the efficiency and creativity enabled by ChatGPT, allowing her to tackle complex research problems, generate high-quality content, and even assist in software development tasks. Through her experiences, we gain insights into the immense potential of ChatGPT as a powerful tool for augmenting productivity and innovation across various domains. As the technology continues to evolve, researchers and professionals can leverage ChatGPT to unlock new possibilities and achieve greater efficiency in their work.Author BioUrszula is an expert graphic designer, software instructor, and consultant. As an instructor, she taught thousands of Adobe software users in government agencies and private corporations. Her easy-going and effective presentation style of complex features earned her many loyal clients. Her consulting work related to PDF included creating templates, editing files, and providing recommendations on the document production process. The final goal was to ensure that each publication met the highest publishing and accessibility standards, which was achieved successfully. Urszula also designed training curriculum and reference materials on proper document construction prior to conversion to PDF in a relevant authoring software, such as MS Word, Adobe InDesign, and FrameMaker.Author of the book: Adobe Acrobat Ninja
Read more
  • 0
  • 0
  • 124
article-image-help-chatgpt-improve-with-knowledge-graphs
Maxime Labonne
17 Jun 2023
9 min read
Save for later

Help ChatGPT Improve with Knowledge Graphs

Maxime Labonne
17 Jun 2023
9 min read
ChatGPT has shown impressive capabilities in processing and generating human-like text. However, it is not without its imperfections. A primary concern is the model's propensity to produce either inaccurate or obsolete answers, often called "hallucinations."The New York Times recently highlighted this issue in their article, "Here's What Happens When Your Lawyer Uses ChatGPT." It presents a lawsuit where a lawyer leaned heavily on ChatGPT to assist in preparing a court filing for a client suing an airline. The model generated fictional court decisions to back its arguments, which didn't go unnoticed. This incident underscores the need for solutions to ground AI models like ChatGPT and improve their performance.To address this, we propose an approach that focuses on augmenting ChatGPT using a knowledge graph. This method aims to provide a structured context, ensuring the model outputs are accurate but also relevant and up-to-date. By bridging the gap between the unstructured textual world of ChatGPT and the structured clarity of knowledge graphs, we strive to enhance the effectiveness and reliability of AI language models.All the code used in this article is available on Google Colab and on GitHub.What is a knowledge graph?A knowledge graph is a structured format of knowledge representation, usually composed of entities and relationships. In a typical knowledge graph, entities are the nodes, and the relationships between them are the edges. The graph-based representation allows complex relationships to be modeled in a way that's intuitive and closer to human understanding. Here is a simple illustration of a knowledge graph: Source: Wikipedia. CC BY-SA 4.0 Google has been using knowledge graphs since 2012 to provide additional contextual information and sources. The structured representation of data offers a new dimension of context to the AI model, grounding it in validated knowledge.Applying Knowledge Graphs to Improve ChatGPTA crucial limitation of ChatGPT is its lack of real-time information updates. Since the model was last trained using data up until 2021, it doesn't have access to events, data, or context after that year. This leads to ChatGPT having outdated or incomplete information about events, technological advancements, or other critical happenings post-2021.Let's illustrate this limitation by asking ChatGPT about a recent event, When did Apple announce the Vision Pro?. Given the model's knowledge cutoff in 2021, we would expect it to be unaware of this announcement, which happened in 2023.!pip install -q openai langchain import os import openai os.environ['OPENAI_API_KEY'] = "your OpenAI key" openai.api_key = os.environ['OPENAI_API_KEY'] question = "When did apple announced the Vision Pro?" completion = openai.ChatCompletion.create(model="gpt-3.5-turbo", temperature=0, messages=[{"role": "user", "content": question}]) print(completion["choices"][0]["message"]["content"]) As an AI language model, I do not have access to current events or real-time information. However, as of my last training data, Apple has not announced any product called "Vision Pro." It is possible that this product does not exist or has not been announced yet. As expected, ChatGPT is unable to provide the correct answer due to its training data limitations. This clearly highlights the need for constant updates to the model's knowledge base, which can be addressed by integrating it with a continuously updated knowledge graph.By implementing such a knowledge graph, we can ensure that ChatGPT can provide accurate, current, and reliable information, effectively addressing the "hallucination" issues as well as the knowledge cutoff limitations.Sentence-Level Knowledge GraphsTo demonstrate this, we'll use the LangChain library, a powerful tool designed for building frameworks around large language models. The library includes a component called GraphIndexCreator, which can parse a sentence and create a knowledge graph. This component is currently limited and cannot process long corpus of text, but it serves as a perfect starting point for our experiment.Let's start with a straightforward sentence: "Apple announced the Vision Pro in 2023."from langchain.llms import OpenAI from langchain.indexes import GraphIndexCreator from langchain.chains import GraphQAChain from langchain.prompts import PromptTemplate text = "Apple announced the Vision Pro in 2023." index_creator = GraphIndexCreator(llm=OpenAI(temperature=0)) graph = index_creator.from_text(text) graph.get_triples()[('Apple', 'Vision Pro', 'announced'), ('Vision Pro', '2023', 'was announced in')] By feeding this sentence into the GraphIndexCreator, it creates a knowledge graph by identifying the sentence's entities and relationships, forming triplets of information in the format of (source node, relation, and target node). However, the GraphIndexCreator might get confused with the relations and target nodes due to the inherent complexity of natural language.Even though it's a tiny graph based on a single sentence, we can represent it visually using popular Python libraries such as matplotlib and networkx.import networkx as nx import matplotlib.pyplot as plt # Create graph G = nx.DiGraph() G.add_edges_from((source, target, {'relation': relation}) for source, relation, target in graph.get_triples()) # Plot the graph plt.figure(figsize=(8,5), dpi=300) pos = nx.spring_layout(G, k=3, seed=0) nx.draw_networkx_nodes(G, pos, node_size=2000) nx.draw_networkx_edges(G, pos, edge_color='gray') nx.draw_networkx_labels(G, pos, font_size=12) edge_labels = nx.get_edge_attributes(G, 'relation') nx.draw_networkx_edge_labels(G, pos, edge_labels=edge_labels, font_size=10) # Display the plot plt.axis('off') plt.show() Image 2: Visual Pro Graph Now, let's enhance ChatGPT using the knowledge graph. We will use another component of the LangChain library, GraphQAChain, to this end.Initializing the GraphQAChain, we input the same question we asked earlier, "When did Apple announce the Vision Pro?". This time, ChatGPT leverages the knowledge graph we've just built.chain = GraphQAChain.from_llm(OpenAI(temperature=0), graph=graph, verbose=True) chain.run(question)> Entering new GraphQAChain chain...Entities Extracted: Apple, Vision ProFull Context:Apple announced Vision ProVision Pro was announced in 2023> Finished chain. Apple announced Vision Pro in 2023.This time, ChatGPT was able to output the correct information! The good thing is that we don't need any parser to build our knowledge graphs and can use existing ones. In the next experiment, let's try to use a bigger graph and see if it's still as performant.Bigger Knowledge GraphsIn this experiment, we manually create this more complex graph by supplying a list of triplets to the GraphIndexCreator object using the add_triple() method. Each triplet represents a distinct piece of knowledge related to Apple, such as the products it has created or where it is located.from langchain.graphs.networkx_graph import KnowledgeTriple # Knowledge graph kg = [ ('Apple', 'is', 'Company'), ('Apple', 'created', 'iMac'), ('Apple', 'created', 'iPhone'), ('Apple', 'created', 'Apple Watch'), ('Apple', 'created', 'Vision Pro'), ('Apple', 'developed', 'macOS'), ('Apple', 'developed', 'iOS'), ('Apple', 'developed', 'watchOS'), ('Apple', 'is located in', 'USA'), ('Steve Jobs', 'co-founded', 'Apple'), ('Steve Wozniak', 'co-founded', 'Apple'), ('Tim Cook', 'is the CEO of', 'Apple'), ('iOS', 'runs on', 'iPhone'), ('macOS', 'runs on', 'iMac'), ('watchOS', 'runs on', 'Apple Watch'), ('Apple', 'was founded in', '1976'), ('Apple', 'owns', 'App Store'), ('App Store', 'sells', 'iOS apps'), ('iPhone', 'announced in', '2007'), ('iMac', 'announced in', '1998'), ('Apple Watch', 'announced in', '2014'), ('Vision Pro', 'announced in', '2023'), ] graph = index_creator.from_text('') for (node1, relation, node2) in kg: graph.add_triple(KnowledgeTriple(node1, relation, node2))Although we could include many more triplets (real-world knowledge graphs often encompass millions of nodes), the size of our graph for this demonstration is sufficient. When visualized, this more extensive knowledge graph exhibits greater complexity and a richer depiction of information.# Create directed graph G = nx.DiGraph() for node1, relation, node2 in kg: G.add_edge(node1, node2, label=relation) # Plot the graph plt.figure(figsize=(25, 25), dpi=300) pos = nx.spring_layout(G, k=2, iterations=50, seed=0) nx.draw_networkx_nodes(G, pos, node_size=5000) nx.draw_networkx_edges(G, pos, edge_color='gray', edgelist=G.edges(), width=2) nx.draw_networkx_labels(G, pos, font_size=12) edge_labels = nx.get_edge_attributes(G, 'label') nx.draw_networkx_edge_labels(G, pos, edge_labels=edge_labels, font_size=12) # Display the plot plt.axis('off') plt.show()Image 4: Larger GraphWith this larger graph, we once again ask ChatGPT the question: "When did Apple announce the Vision Pro?" Leveraging the GraphQAChain object, ChatGPT processes the information embedded in the knowledge graph.chain = GraphQAChain.from_llm(OpenAI(temperature=0), graph=graph, verbose=True) chain.run(question)> Entering new GraphQAChain chain...Entities Extracted: Apple, Vision ProFull Context:Apple is CompanyApple created iMacApple created iPhoneApple created Apple WatchApple created Vision ProApple developed macOSApple developed iOSApple developed watchOSApple is located in USAApple was founded in 1976Apple owns App StoreVision Pro announced in 2023> Finished chain. Apple announced the Vision Pro in 2023.ChatGPT successfully extracts the correct information from the more expansive knowledge graph. This result demonstrates that our model can not only scale to larger graphs but can also efficiently navigate a more extensive knowledge base.The possibilities for implementing larger and more diverse knowledge graphs are practically endless. They can be populated with data from various sources, such as legal documents, code documentation, scientific literature, and more, enhancing the AI's understanding and response accuracy across multiple domains. The integration of ChatGPT and knowledge graphs thus holds immense promise for future AI development.ConclusionAs seen in our experiments, knowledge graphs can significantly aid in grounding and improving ChatGPT's outputs. A key challenge with large knowledge graphs is finding connections between distant nodes, a problem often referred to as graph completion. Successfully addressing this issue would allow ChatGPT to make insightful connections and propose new ideas based on the information available in the knowledge graph.However, the process of integrating knowledge graphs into language models like ChatGPT is still an evolving field. To further explore the various applications and delve into the details of implementing knowledge graphs, consider the book "Hands-On Graph Neural Networks Using Python", which provides a comprehensive guide on this subject. Through this type of research and experimentation, we can continuously improve AI's ability to understand and generate text, moving us closer to more reliable and grounded AI models.Author BioMaxime Labonne is a senior applied researcher at J.P. Morgan with a Ph.D. in machine learning and cyber security from the Polytechnic Institute of Paris.During his Ph.D., Maxime worked on developing machine learning algorithms for anomaly detection in computer networks. He then joined the AI Connectivity Lab at Airbus, where he applied his expertise in machine learning to improve the security and performance of computer networks. He then joined J.P. Morgan, where he develops techniques to solve a variety of challenging problems in finance and other domains.In addition to his research work, Maxime is passionate about sharing his knowledge and experience with others through Twitter (@maximelabonne) and his personal blog.
Read more
  • 0
  • 0
  • 599

article-image-creating-a-data-model-with-chatgpt-is-easier-than-you-think
Sagar Lad
16 Jun 2023
6 min read
Save for later

Creating a Data Model with ChatGPT is Easier than you think

Sagar Lad
16 Jun 2023
6 min read
In today's data-driven world, the ability to build accurate and efficient data models is paramount for businesses and individuals alike. However, the process of constructing a data model can often be complex and daunting, requiring specialized knowledge and technical skills. But what if there was a way to simplify this process and make it accessible to a wider audience? Enter ChatGPT, a powerful language model developed by OpenAI. In this article, we will explore how ChatGPT can be leveraged to build data models easily, using a practical example. By harnessing the capabilities of ChatGPT, you'll discover how data modeling can become a more approachable and intuitive task for everyone, regardless of their technical background.Build Data Model with ChatGPTConsider data modeling as the process of drawing diagrams for software applications that provide an overview of all the data pieces they include. The data flow is depicted in the diagram using text and symbols. It serves as a model for creating a new database that will allow a company to utilize the data efficiently for its needs. The primary objective of the data model is to establish an overall picture of the types of data that are used, how they are kept in the system, the relationships between the data entities, and the various ways in which they can be arranged and grouped. The norms and procedures for gathering feedback from the business stakeholders are taken into consideration when building data models.The Data Model functions as a better understanding of what is designed, much like a roadmap or blueprint might. It offers a comprehensive review of the standardized methodologies and schema to define and manage data in a way that is common and uniform throughout the organization. According to the level of abstraction, there are three different types of data models.Conceptual Data Model: It provides a helicopter view of the system description, its organization, and business rules to be considered. Initial project requirements are captured using the conceptual model. It mainly consists of the business entities, their constraints and characteristics, and the relationship between them for data integrity and security requirements.Logical Data Model: The logical data model provides detailed insights into the concepts and relationship which consists of data attributes and the relationship between the entities. It is very much useful for data-driven projects or initiatives.Physical Data Model: It provides an overview of how the data should be stored physically within the database. It is a final design to implement the relational database including the relationship using the primary and foreign keys.                           Image 1 : Types of Data Modelling TechniquesThe data model was created using a variety of data modeling methodologies, as seen in the graphic above. The most popular data modeling technique utilized by any corporate organization is entity relationship modeling, also known as dimensional modeling. Erwin Data Modeler, ER/Studio, Archi, and other tools are available on the market to construct data models utilizing these data modeling methodologies. The data Modelling technique mainly involves below steps :  Identify the entitiesFind the entity propertiesCreate a relationship between the entitiesCorrelated attributes to the entityDefine the degree of normalization to improve the performanceValidate and Finalise the data modelLet’s start with creating a data model using chatGPT. The goal is to ask chatGPT to start with the data modeling activities for the anti-money laundering(AML) system of a banking domain: Image 1: The data model for the banking system, Part 1  Image 2: Data Modelling  for AML Process for Bank As you can see in the image, once we provide an input to the chatGPT, it provides a step-by-step process of building the data model. The first step is to understand the AML regulations and identify the stakeholders for the system to capture the requirements. Once the stakeholders are identified, the next step is to define the data modeling goals including the list of data sources, and perform the data profiling. Once data profiling steps are done, the next activity is to create a conceptual, logical, and physical data model.Now, Let’s check with chatGPT to create a conceptual model with all the entities. Image 3: Conceptual Data Model, Part 1                 Image 4 : AML Conceptual ModelAfter the input, chatGPT responds with the list of actors, entities, and relationships between the entities to define the conceptual model. With this information, we can have a high-level overview of the system by building the conceptual data model. Let’s ask chatGPT to build the logical data model once the conceptual data model is ready:Image 5: AML data model for ERwin Tool                  Image 6 : AML Logical Data Model, Part 2 As you can see in the above image, step by step process to create a logical data model is to open the Erwin Tool and create a new data model. In the new data model, add all entities, their attributes, and the relationship between the entities. Once entities are defined, set up primary and foreign keys for all entities and validate the data model. After the validation, adjust the review comments and finalize the logical data model and generate the documentation for the same.Next, Let’s ask chatGPT if it can add new customer information to the existing conceptual model.                  Image 5 : AML Logical Data Model with Customer Information As we can see in the above image, chatGPT asks to first identify the source information and create an entity and attributes for the same. Once it is done, we have to define the cardinality to understand how entities are related to each other. Then define primary and foreign key relationships, data model validation and generate documentation.ConclusionIn this article, we understood the importance of building the data model and step by step process to create the data model. Later in this article, we also checked how to use chatGPT to create conceptual and logical data models.Author BioSagar Lad is a Cloud Data Solution Architect with a leading organisation and has deep expertise in designing and building Enterprise-grade Intelligent Azure Data and Analytics Solutions. He is a published author, content writer, Microsoft Certified Trainer, and C# Corner MVP.Link - Medium , Amazon , LinkedIn 
Read more
  • 0
  • 0
  • 934