Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon

How-To Tutorials

7019 Articles
article-image-chatgpt-as-an-assistant-for-plc-programmers
M.T White
13 Jun 2023
10 min read
Save for later

ChatGPT as an Assistant for PLC Programmers

M.T White
13 Jun 2023
10 min read
ChatGPT has been making the rounds recently in the world of programming.  In short, there is a lot of hoopla about whether or not it will replace programmers, how well it writes code, how well it troubleshoots issues, and more importantly how programmers can leverage it to automate code production. One area where ChatGPT can be of particular use is in the field of automation.  As all automation programmers know, the automation industry is very fast-paced.  This means that being able to quickly produce quality code is vital to the success of not only the programmer but the company as a whole.  However, it is often the case that some of the simplest code that a developer will need to produce is often the most time-consuming to write.  Two areas that come to mind are large case statements and sorting algorithms.  As such, the following article is going to explore how ChatGPT can be used to produce code that would otherwise be monotonous or time-consuming to write.  More specifically, the following is going to explore how ChatGPT can be used to generate the necessary code for a case statement that displays a motor’s health and how it can be used to select and generate the code for a sorting algorithm to sort part sizes. A CASE Statement to Detect Motor HealthA very daunting and time-consuming task that all automation programmers will perform regularly is creating large case statements.  It is not uncommon for automation programmers to have to build things like state machines that have 10s or even 100s of different cases.  As anyone who has built these before knows, building structures like these is time-consuming and monotonous at best.  For example, suppose a developer is working on a function block that will display a custom message about a motor error in an HMI.  Assume in this example the developer needs a code block that can handle the input/output exchange for the following table,Input valueString Message1Cable Error2Motor Overheated3Under CurrentDepending on the experience level of the developer a simple case statement program that can handle the requirements would take about five minutes.  However, when there are more case statements, for example, 30 or 40 inputs, the timeframe for completion could drastically increase.  Simply put, since there are more cases there will be more code to write.  In turn, this means that it will take the developer more time to type out the code and there will be a higher probability of introducing a bug or bugs in the code due to human error.  On the other hand, to quickly generate a quality code block that can meet these requirements all one would need to do is enter the following into ChatGPT:write an IEC 61131-3 code block that will set a message variable to "cable error" when the input is 1, "motor overheated" when the input is 2, and "under current" when the input is 3This statement will output the following:FUNCTION_BLOCK SetMessage VAR_INPUT    Input: INT; // Input value END_VAR VAR_OUTPUT    Message: STRING(50); // Output message END_VAR METHOD SetMessage VAR    TempMessage: STRING(50); // Temporary message variable END_VAR CASE Input OF    1:        TempMessage := 'Cable error';    2:        TempMessage := 'Motor overheated';    3:        TempMessage := 'Under current';    ELSE        TempMessage := 'Invalid input'; // Optional: default message for other input values END_CASE Message := TempMessage; // Assign the temporary message to the output message END_METHOD From the time it took to input and run the query on ChatGPT to the final code output was about one minute.  The code that was generated is quality code that looks like it was produced by an experienced programmer, it is documented, and above all else works.  The code even has an optional else statement to catch invalid inputs.  Overall, the autogenerated code is no less than what would be expected from a human developer.  With that, let’s explore how ChatGPT can be used to generate a more complex piece of software.   An Efficient Sorting Algorithm to Sort Parts by SizeIt is not uncommon for developers to be tasked with finding the smallest/largest or lightest/heaviest part produced on a production run.  For a task like this, it is common to store all the sizes, weights, or whatever it may be in an array and at one point or another sort the array to find the greatest and least size.  Depending on the size of the array, or in the case of this scenario, the number of part sizes to sort, having an efficient array is vital.  For this task, an efficient sorting algorithm will need to be picked and the code will need to be implemented. Assuming the PLC programmer is knowledgeable of sorting algorithms and Big O notations selecting an algorithm may only take a few minutes.  However, for a PLC programmer that is not versed in these topics, such as a person that is perhaps and mechanical engineer or a technician of some type, this task will probably require a Google search which ideally will take about five minutes.  Regardless, of whether the programmer is experienced or not with the algorithm it will take roughly 10 to 20 minutes for the person to type in the code.  As such, on a good day, it would take roughly 10 to 25 minutes for someone to implement a sorting algorithm. On the other hand, if a person used ChatGPT all they would have to do is type in the following query:write an IEC 61131-3 Structured Text sorting algorithm that can sort the elements in an array with 100 entries with the best possible Big O timeWhich will generate the following:FUNCTION_BLOCK QuickSort VAR_INPUT    arr: ARRAY[1..100] OF INT; END_VAR METHOD SortArray : BOOL    SortPartition(arr, 1, 100);    SortArray := TRUE; END_METHOD METHOD SortPartition(arr: ARRAY[1..100] OF INT; low: INT; high: INT) VAR    pivot: INT;    partitionIndex: INT; END_VAR    IF low < high THEN        pivot := Partition(arr, low, high, low); // Choosing pivot as the first element        SortPartition(arr, low, pivot - 1);        SortPartition(arr, pivot + 1, high);    END_IF; END_METHOD METHOD Partition(arr: ARRAY[1..100] OF INT; low: INT; high: INT; pivotIndex: INT) : INT VAR    pivotValue: INT;    i: INT;    partitionIndex: INT;    temp: INT; END_VAR    // Swap pivot element to the end    temp := arr[high];    arr[high] := arr[pivotIndex];    arr[pivotIndex] := temp;    pivotValue := arr[high];    partitionIndex := low;    FOR i := low TO high - 1 DO        IF arr[i] <= pivotValue THEN            // Swap elements            temp := arr[i];            arr[i] := arr[partitionIndex];            arr[partitionIndex] := temp;            partitionIndex := partitionIndex + 1;        END_IF;    END_FOR;    // Move pivot element to its correct position    temp := arr[partitionIndex];    arr[partitionIndex] := arr[high];    arr[high] := temp;    Partition := partitionIndex; END_METHOD END_FUNCTION_BLOCKAll of this code was generated in about one minute.  As can be seen, ChatGPT generated roughly 60 lines of code not counting the code that utilizes the example.  The only overhead that would be required by the programmer is to create the necessary files and paste the code in.  In terms of efficiency, ChatGPT rendered the Quicksort algorithm which has a Big O of n*log(n).  So, in the end, the programmer is probably looking at around five minutes' worth of work to implement an efficient algorithm that would otherwise take up to half an hour by hand.The Key to using ChatGPTAs can be seen, the key to using ChatGPT as a programming assistant is simply learning to talk to it.  Unlike programming languages that have a unique syntax, ChatGPT can interpret normal, human language.  However, to get the most out of ChatGPT the programmer needs to learn to ask detailed questions. The general rule of thumb is the more detailed a statement is the more detailed the solution will be.  As was seen, in the examples, all the code was produced with a single sentence.  Depending on the resolution needed, a simple query can produce great results, but it should be noted that if specs like specific ports, and so on need to be addressed the user should specify those.  Though novel now, it is likely that the art of talking to these systems will soon appear to generate optimal code with a query. SummaryIn all, ChatGPT can be used as a tool to help speed up the development process.  As was explored with the motor health code block and the sorting algorithm, ChatGPT can turn a simple phrase into workable code that would take a human a considerable amount of time to type out.  In short, even if a PLC programmer is knowledgeable of both programming principles, algorithms, and other computer science concepts they will always be bottlenecked by having to implement the code if they cannot simply cut and paste it from another source. When used in a way that was explored, ChatGPT is a great productivity tool.  It can be used to greatly reduce the amount of time needed to implement and if necessary find a solution.  Overall, ChatGPT needs guidance to arrive at a proper solution and the person driving the AI needs to be competent enough to implement the solution.  However, when in the right hands ChatGPT and similar AI systems can greatly improve development time.Author BioM.T. White has been programming since the age of 12. His fascination with robotics flourished when he was a child programming microcontrollers such as Arduino. M.T. currently holds an undergraduate degree in mathematics, and a master's degree in software engineering, and is currently working on an MBA in IT project management. M.T. is currently working as a software developer for a major US defense contractor and is an adjunct CIS instructor at ECPI University. His background mostly stems from the automation industry where he programmed PLCs and HMIs for many different types of applications. M.T. has programmed many different brands of PLCs over the years and has developed HMIs using many different tools.Author of the book: Mastering PLC Programming
Read more
  • 0
  • 0
  • 269

article-image-effective-text-generation-editing-and-translation-with-chatgpt
Valentina Alto
13 Jun 2023
7 min read
Save for later

Effective Text Generation, Editing and Translation with ChatGPT

Valentina Alto
13 Jun 2023
7 min read
This article is an excerpt from the book, Modern Generative AI with ChatGPT and OpenAI Models, by Valentina Alto. This book will help harness the power of AI with innovative, real-world applications, and unprecedented productivity boosts, powered by the latest advancements in AI technology like ChatGPT and OpenAI.In the world of natural language processing, ChatGPT stands as a powerful tool for various text-related tasks. From generating creative and coherent text to providing translations and editing assistance, ChatGPT offers a wide range of functionalities. In this article, we will explore how to harness the capabilities of ChatGPT to accomplish tasks such as generating engaging content, translating text between languages, and receiving helpful suggestions for editing. With practical examples and step-by-step instructions, we will unlock the potential of ChatGPT as a versatile text companion for developers and content creators alike.As a language model, ChatGPT is particularly suited for generating text based on users’ instructions. For example, you   could ask ChatGPT to generate emails, drafts, or templates that target a specific audience:Figure 1: Example of an email generated by ChatGPTAnother example might be asking ChatGPT to create a pitch structure for a presentation you have to prepare: Figure 2 – Slideshow agenda and structure generated by ChatGPTYou can also generate blog posts or articles about trending topics this way. Here is an example:Figure 3 – blog post with relevant tags and SEO  by ChatGPTWe can even get ChatGPT to reduce the size of the post to make it fit for a tweet. Here is how we can do this: Figure 4 – ChatGPT shrinks an article into a Twitter postFinally, ChatGPT can also generate video or theatre scripts, including the scenography and the suggested editing. The following figure shows an example of a theatre dialog between a person and ChatGPT:  Figure 5– Theatre dialog with scenography generated by ChatGPTI only provided a truncated version to keep you in suspense regarding the ending…Improving writing skills and translationSometimes, rather than generating new content, you might want to revisit an existing piece of text. It this be for style improvement purposes, audience changes, language translation, and so on.Let’s look at some examples. Imagine that I drafted an email to invite a customer of mine to a webinar. I wrote two short sentences. Here, I want ChatGPT to improve the form and style of this email since the target audience will be executive-level:Figure 6 – Example of an email revisited by ChatGPT to target an executive audienceNow, let’s ask the same thing but with a different target audience: Figure 6 – Example of the same email with a different audience, generated by ChatGPTChatGPT can also give you some feedback about your writing style and structure.Imagine, for example, that you wrote a script with scenography for your YouTube channel. You included the speech as well as images, clips, and video editing activities. You also know that your typical audience is between 15 and 25 years old. You want feedback on your script and ask for this from ChatGPT: Figure 7 – Example of ChatGPT providing feedback on a video script As you can see, not only was ChatGPT able to give me feedback about the writing style, but also it suggested how I could improve the scenography of the whole video, by including more visuals.Again, imagine you wrote an introduction for an essay titled The History of Natural Language Processing and you want some feedback about the writing style and its consistency with the title: Figure 8 – Example of ChatGPT giving feedback on an introduction for an essayLet’s also ask ChatGPT to make concrete examples of the attention-grabbing anecdote it talked about in its response: Figure 9 – Example of ChatGPT elaborating on something it mentioned I’m also interested in knowing whether my introduction was consistent with the title or whether I’m taking the wrong direction:Figure 10 – ChatGPT provides feedback about the consistency of the introduction with the titleI was impressed by this last one. ChatGPT was smart enough to see that there was no specific mention of the history of NLP in my introduction. Nevertheless, it sets up the expectation about that topic to be treated later on. This means that ChatGPT also has expertise in terms of how an essay should be structured and it was very precise in applying its judgment, knowing that it was just an introduction.It is also impressive to note how the model can give different feedback, depending on the context. With the video script, ChatGPT’s feedback took into account that the final consumption of that content would have been on screen. On the other hand, the essay’s introduction lives in a more formal and academic context, with a specific structure, that ChatGPT was able to capture once more.Last but not least, ChatGPT is also an excellent tool for translation. It knows at least 95 languages (if you have doubts about whether your language is supported, you can always ask ChatGPT directly). Here, however, there is a consideration that might arise: what is the added value of ChatGPT for translation when we already have cutting-edge tools such as Google Translate?To answer this question, we have to consider some key differentiators and how we can leverage ChatGPT’s embedded translations capabilities:ChatGPT can capture the intent. This means that you could also bypass the translation phase since it is something that ChatGPT can do in the backend. For example, if you write a prompt to produce a social media post in French, you could write that prompt in any language you want – ChatGPT will automatically detect it (without the need to specify it in advance) and understand your intent:Figure 11 – Example of ChatGPT generating an output in a language that is different from the inputChatGPT can capture the more refined meaning of particular slang or idioms. This allows for a translation that is not literal so that it can preserve the underlying meaning. Namely, let’s consider the British expression It’s not my cup of tea, to indicate something that is not the type of thing you like. Let’s ask both ChatGPT and Google Translate to translate it into Italian: Figure 12 – Comparison between ChatGPT and Google Translate while translating from English into ItalianAs you can see, ChatGPT can provide several Italian idioms that are equivalent to the original one, also in their slang format. On the other hand, Google Translate performed a literal translation, leaving behind the real meaning of the idiom. As with any other task, you can always provide context to ChatGPT. So, if you want your translation to have a specific slang or style, you can always specify it in the prompt. Or, even funnier, you can ask ChatGPT to translate your prompt with a sarcastic touch:  Figure 5.20 – Example of ChatGPT translating a prompt with a sarcastic touch.The original content from: OpenAI’s Wikipedia page: https://it.wikipedia.org/wiki/OpenAISummaryIn conclusion, ChatGPT is able not only to generate new text but also to manipulate existing material to tailor it to your needs. It has also proven to be very precise at translating between languages, also keeping the jargon and language-specific expressions intact.Author BioValentina Alto graduated in 2021 in Data Science. Since 2020 she has been working in Microsoft as Azure Solution Specialist and, since 2022, she focused on Data&AI workloads within the Manufacturing and Pharmaceutical industry. She has been working on customers’ projects closely with system integrators to deploy cloud architecture with a focus on datalake house and DWH, data integration and engineering, IoT and real-time analytics, Azure Machine Learning, Azure cognitive services (including Azure OpenAI Service), and PowerBI for dashboarding. She holds a BSc in Finance and an MSc degree in Data Science from Bocconi University, Milan, Italy. Since her academic journey she has been writing Tech articles about Statistics, Machine Learning, Deep Learning and AI on various publications. She has also written a book about the fundamentals of Machine Learning with Python. LinkedIn  Medium 
Read more
  • 0
  • 0
  • 154

article-image-practical-vulnerability-scanning-made-easy-with-chatgpt
Clint Bodungen
13 Jun 2023
8 min read
Save for later

Practical Vulnerability Scanning Made Easy with ChatGPT

Clint Bodungen
13 Jun 2023
8 min read
This article is an excerpt from the book, ChatGPT for Cybersecurity Cookbook, by Clint Bodungen. This book will help you master ChatGPT and the OpenAI API, and harness the power of cutting-edge generative AI and large language models to revolutionize the way you perform penetration testing, threat detection, and risk assessment.Vulnerability scanning is crucial for identifying weaknesses before exploitation. Navigating tools like NMAP, OpenVAS, or Nessus can be complex, especially for newcomers. Our article uses ChatGPT to generate command strings based on user input, simplifying the process. By following this recipe, you can generate precise command strings for NMAP, OpenVAS, or Nessus, confidently navigating their functionalities. Whether you're a cybersecurity beginner or an expert, this recipe is a valuable tool for vulnerability assessments.Getting ready Before we begin this recipe, it's essential to ensure that you have properly set up your OpenAI account and obtained your API key. Additionally, you will require the following: Vulnerability Scanning Tools: It's crucial to have NMAP, OpenVAS, or Nessus installed on your system as the recipe generates command strings for these specific tools. Please refer to their official documentation for installation and setup guidelines.  Basic Understanding of the Tools: The more familiar you are with NMAP, OpenVAS, or Nessus, the better you will be able to utilize this recipe. If you're new to these tools, consider spending some time understanding their basic functionalities and command-line options. Command Line Environment: As the recipe generates command strings intended for command line interfaces, you should have access to a suitable command line environment where you can run these commands. Sample Network Configuration Data: Prepare some sample network data that the vulnerability scanning tools can use. This could include IP addresses, hostnames, or other relevant information about the systems you'd like to scan. How to do it… In this recipe, we'll show you how to use ChatGPT to create command strings for vulnerability scanning tools like NMAP, OpenVAS, and Nessus. We'll be providing ChatGPT with the necessary details and using a specific system role and prompt. This will allow you to generate the simplest form of the command necessary to complete your request. Start by logging in to your OpenAI account and go to the ChatGPT web UI. Begin a new conversation with ChatGPT by clicking on the "New Chat" button. Next, establish the system's role by entering the following: You are a professional cybersecurity red team specialist and an expert in penetration testing as well as vulnerability scanning tools such as NMap, OpenVAS, Nessus, Burpsuite, Metasploit, and more.  Now, prepare your request. This is the information that will replace the {user_input} in the next step. It should be a natural language request such as: Use the command line version of OpenVAS to scan my 192.168.20.0 class C network starting by identifying hosts that are up, then look for running web servers, and then perform a vulnerability scan of those web servers.  Once your request is ready, enter the following message text, replacing the {user_input} placeholder with your specific request from the previous step: Provide me with the Linux command necessary to complete the following request:  {user_input}  Assume I have all necessary apps, tools, and commands necessary to complete the request. Provide me with the command only and do not generate anything further. Do not provide any explanation. Provide the simplest form of the command possible unless I ask for special options, considerations, output, etc. If the request does require a compound command provide all necessary operators, pipes, etc. as a single one-line command. Do not provide me more than one variation or more than one line. ChatGPT will then generate the command string based on your request. Review the output. If it meets your requirements, you can proceed to copy the command and use it as needed. If it doesn't, you may need to refine your request and try again.  Once you've obtained a satisfactory command, you can copy it and paste it directly into your command line to perform the vulnerability scan as described in your request. Note Remember, it's important to review and understand any command before running it in your environment. While ChatGPT aims to provide accurate commands, you are ultimately responsible for ensuring the command's safety and appropriateness for your specific context. Figure 1 shows an example ChatGPT command generated from the prompt used in this recipe.  Figure 1 – Example ChatGPT Command Generation  How it works… The GPT-assisted vulnerability scanning recipe taps into the power of natural language processing (NLP) and the vast knowledge of machine learning algorithms to generate accurate and appropriate command strings for vulnerability scanning tools like NMAP, OpenVAS, and Nessus. When you provide a specific system role and a prompt that represents a user request, ChatGPT uses these inputs to understand the context and generate a response that aligns with the given role. System Role Definition: By defining ChatGPT's role as a professional cybersecurity red team specialist and an expert in penetration testing and vulnerability scanning tools, you're instructing the model to answer from a perspective of deep technical understanding and expertise in this field. This context helps in generating accurate and relevant command strings. Natural Language Prompt: The natural language prompt that simulates a user request allows ChatGPT to understand the task at hand in a human-like manner. Instead of needing structured data or specific keywords, ChatGPT can interpret the request as a human would and provide a suitable response. Command Generation: With the role and the prompt, ChatGPT generates the Linux command necessary to complete the request. The command is based on the specific details of the user input and the expertise of the assigned role. This is where the AI leverages its knowledge of cybersecurity and language understanding to construct the necessary command string. One-Line Command: The specification of providing a one-line command, including all necessary operators and pipes, compels ChatGPT to generate a command that's ready to be pasted into a command line for immediate execution. This removes the need for the user to manually combine or modify the command, saving time and potential errors. Simplicity and Clarity: By asking for the simplest form of the command and without any further explanation, the output is kept clear and concise, which is particularly helpful for those learning or in need of a quick reference. In summary, the GPT-assisted vulnerability scanning recipe harnesses the power of NLP and machine learning algorithms to generate precise, ready-to-run commands for vulnerability scanning. By using the defined system role and prompt, users can streamline the process of crafting commands for vulnerability assessments, save time, and improve accuracy. There’s more… The flexibility and capabilities of this GPT-assisted process extend beyond the example given. First is the versatility of the prompt. It's actually designed to accommodate virtually any request for any Linux command across any domain or task. This is a significant advantage as it enables you to leverage ChatGPT's capabilities across a wide range of scenarios. By assigning the role appropriately, such as "You are a Linux system administrator", and substituting your specific request in place of {user_input}, you can guide the AI to generate accurate and context-specific command strings for a plethora of Linux operations. Beyond simply generating command strings, the potential of this recipe is amplified when combined with the OpenAI API and Python. With the proper setup, you can not only generate the necessary Linux commands but also automate the execution of these commands. Essentially, this could turn ChatGPT into an active participant in your command-line operations, potentially saving you significant time and effort. This level of automation represents a substantial step forward in interacting with AI models, turning them into active assistants rather than passive information generators. In upcoming recipes in this book, we'll delve deeper into command automation. This is just the beginning of the possibilities opened up by the integration of AI with your operating system tasks.  Summary:This article highlights vulnerability assessment and the role of ChatGPT in simplifying the process. ChatGPT offers the ability to simulate attack scenarios, identify weaknesses, and generate reports. However, it is crucial to remember that vulnerability assessment requires human expertise and judgment. ChatGPT should be used as a supplementary tool alongside human analysis to enhance cybersecurity measures and mitigate emerging threatsAuthor BioClint Bodungen is a cybersecurity professional with 25+ years of experience and the author of Hacking Exposed: Industrial Control Systems. He began his career in the United States Air Force and has since many of the world's largest energy companies and organizations, working for notable cybersecurity companies such as Symantec, Kaspersky Lab, and Booz Allen Hamilton. He has published multiple articles, technical papers, and training courses on cybersecurity and aims to revolutionize cybersecurity education using computer gaming (“gamification”) and AI technology. His flagship product, ThreatGEN® Red vs. Blue, is the world’s first online multiplayer cybersecurity simulation game, designed to teach real-world cybersecurity.    Links - Twitter  LinkedIn  YouTubeClint has some exciting courses around ChatGPT for Cybersecurity 
Read more
  • 0
  • 0
  • 172

article-image-building-etl-pipelines-in-no-time-using-chatgpt
Sagar Lad
11 Jun 2023
5 min read
Save for later

Building ETL Pipelines in no time using ChatGPT

Sagar Lad
11 Jun 2023
5 min read
Given the volume, velocity, and diversity of data expanding at an exponential rate in the modern era, it is crucial to utilize this data for data analytics and machine learning projects to generate business insights. Let's discuss in this post how to utilize ChatGPT to develop ETL pipelines considering the growing popularity of ChatGPT in recent years.What is an ETL (Extract, Transform, and Load) Pipeline Data must first be fetched from one or more sources, processed, or transformed in accordance with the requirements, and then loaded into a storage system that can be used directly by end users without the need for data validation or poor data quality. ETL Pipeline also known as a ‘Data Pipeline’ sometimes known as three phases. Image 1: ETL Process Flow (Extract, Transformation, and Load) During the ETL process, first, we fetch the data, and we perform data quality and validation checks on the extracted data. Once data is extracted then data pre and post-processing should be done to transform the data into a usable format. Once data processing is done, the last step is to store the data from where the end user can access this data. Let’s ask ChatGPT to build an ETL pipeline for data engineering.Problem StatementUsing Databricks and Pyspark build an ETL pipeline using 3 layered approaches: Raw, Bronze, and Gold Layers.  Data should be ingested incrementally automatically, and data should be stored in Azure SQL Database. Business Analysts can use this data to derive business insights.1. Input to ChatGPT with the programming language, and file information (location, format, delimiter, headers, output) Image 2 : Input to chatGPT to create ETL Pipeline Here, the input has been given to the ChatGPT to build an ETL pipeline with all the required input. 2. Prerequisites suggested by ChatGPT before using the code are as follows:  Image 3 : Prerequisite to setup ETL Flow Here, ChatGPT first lists down the prerequisites to set up the blob storage, azure data lake storage gen2, and Databricks workspace.3. Importing the necessary libraries and configuring the ADLS Gen2 storage credentials as shown in the preceding figure:Image 4 : ADLS Gen2 configuration for Pyspark NotebookThis code configures ADLS gen2 using the pyspark notebook to connect and use the data using the storage account key. Here, you should replace the storage account name and key with your storage account name and key details. 4. In this step, pyspark notebook creates the schema for the pyspark data frame based on the file information and we must replace adls container name with the actual container name of ADLS Gen2. Image 5 : Create Structured Schema for pyspark dataframe5. This piece of code renames the columns and once the data frame is ready, it will write the content of the dataframe to the delta format. Image 6 : Pyspark Notebook - Data Transformation Logic 6.  Finally, ChatGPT provides guidance on where to run this code and how to set up an automated pipeline using Databricks. Image 7 : Final ChatGPT Output  7. At a first glance, it looks like it worked like a charm. Generated code can be directly used in the Databricks workspace to build an ETL pipeline. But the limitation of this solution is that it is hard coding the file path so it is not a generic code.Image 8 : Pyspark Code Optimization - ChatGPT input  8.     ChatGPT creates a generic code that can be used to trigger the ETL pipeline whenever there is a new source file in the ADLS Gen2 container. Image 9 : ADLS Gen2 configuration for Pyspark Notebook9.     Next step is to configure the ADLS Gen2 to connect to Databricks using the storage account key.Image 10 : Schema Definition for CSV File 10.  Next step is to create a structured schema to use it while creating the pyspark dataframe in the next step: Image 11: Setting up a loop to check new files 11.  As a final step, we will work towards optimizing the PySpark code: Image 12: Optimised Pyspark Code for incremental data loadTo process any file using the ETL pipeline code, the suggestion is to loop the code to continuously poll the storage container location to check if there is any new file and execute the code to process the new file if any.So, we can use this code and set up Databricks notebooks in a couple of minutes to set up an automated data pipeline for the incremental data load.ConclusionIt is getting much easier and more efficient to build ETL data pipelines using the ChatGPT. ChatGPT can also assist to create a generic and optimized code as per our requirements promptly without spending development efforts. Author BioSagar Lad is a Cloud Data Solution Architect with a leading organization and has deep expertise in designing and building Enterprise-grade Intelligent Azure Data and Analytics Solutions. He is a published author, content writer, Microsoft Certified Trainer, and C# Corner MVP. Link - Medium , Amazon , LinkedIn 
Read more
  • 0
  • 0
  • 283
Banner background image

article-image-simplifying-code-review-using-google-bard
Darren Broemmer
11 Jun 2023
7 min read
Save for later

Simplifying Code Review using Google Bard

Darren Broemmer
11 Jun 2023
7 min read
Code reviews are an essential part of the software development process. They help improve the quality of code by identifying potential bugs and suggesting improvements. They ensure that the code is readable and maintainable. However, conducting code reviews takes time out of an engineer's busy schedule. There is also the potential that latent issues in the code may be overlooked. Humans do make mistakes after all. This is where Google Bard comes into play.Bard is a tireless and efficient code reviewer that you can put to work anytime. It is a Large Language Model (LLM) that was trained on a massive dataset of text and code. White Bard is still listed as experimental, it has learned how to review and explain code written in many different programming languages.How can Google Bard be used for code reviews?Code reviews typically involve three basic steps: Understand: The reviewer first reads the code and comprehends its purpose and structure.Identify: Potential bugs or issues are noted. Gaps in functionality may be called out. Error handling is reviewed, including edge cases that may not be handled properly.Optimize: Areas of improvement are noted, and refactoring suggestions are made. These suggestions may be in terms of maintainability, performance, or other area Fortunately, Bard can help in all three code review phases.Ask Bard for an explanation of the codeBard can generate a summary description that explains the purpose of the code along with an explanation of how it works. Simply ask Bard to explain what the code does. Code can be directly included in the prompt, or you can reference a GitHub project or URL. Keep in mind, however, that you will get more detailed explanations when asking about smaller amounts of code. Likewise, if you use the following prompt asking about the BabyAGI repo, you will get a summary description.Image 1: Google Bard’s explanation on Baby AGI The interesting thing about Bard’s explanation is that not only did it explain the code in human terms, but it also provided a pseudo-code snippet to make it easy to comprehend. This is impressive and extremely useful as a code reviewer. You have a roadmap to guide you through the rest of the review. Developers who are unfamiliar with the code or who need to understand it better can leverage this Bard capability.Given that LLMs are very good at text summarization, it is not surprising that they can do the same for code. This gives you a foundation for understanding the AGI concept and how BabyAGI implements it. Here is a highlight of the overview code Bard provided:def create_plan(text):    """Creates a plan that can be executed by the agent."""    # Split the text into a list of sentences.    sentences = text.split(".")    # Create a list of actions.    actions = []    for sentence in sentences:        action = sentence.strip()       actions.append(action)    return actions def main():    """The main function."""    # Get the task from the user.    task = input("What task would you like to complete? ")    # Generate the text that describes how to complete the task.    text = get_text(task)    # Create the plan.    plan = create_plan(text)    # Print the plan.    print(plan) if __name__ == "__main__":    main()BabyAGI uses a loop to continue iterating on nested tasks until it reaches its goal or is stopped by the user. Given a basic understanding of the code, reviews become much more effective and efficient.Identifying bugs using BardYou can also ask Bard to identify potential bugs and errors. Bard will analyze the code and look for patterns associated with bugs. It will point out edge cases or validations that may be missing. It also looks for code that is poorly formatted, has unused variables, or contains complex logic. Let’s have Bard review its own summary code shown above. The get_text function has been modified to use the Bard API. Our review prompt is as follows. It includes the code to review inline. Review the following Python code for potential bugs or errors: import requests import json def get_text(task):    """Generates text that describes how to complete a given task."""     bard = Bard()     answer = bard.get_answer(task)     return answer['content'] def create_plan(text):    """Creates a plan that can be executed by the agent."""    # ... remainder of code omitted here for brevity ...Image 2: Google Bard’s explanation of reviewed code Bard pointed out a number of potential issues. It noted that error handling is missing around the Bard API call. It commented that input validations should be added, as well as default values that could make the code more user-friendly.Optimizations using Google BardBard already provided some suggested improvements during its review. You can also ask Bard specifically to suggest optimizations to your code. Bard does this by analyzing the code and looking for opportunities to improve its efficiency, reliability, or security. For example, Bard can suggest ways to simplify code, improve performance, or make it more secure. Let’s do this for the main BabyAGI Python script. Instead of referencing the GitHub repository as we did in the explanation phase, we provide the main loop code inline so that Bard can provide more detailed feedback. The structure of the prompt is similar to the prior phase. Review the following Python code for potential optimizations:def main():    loop = True    while loop:        # As long as there are tasks in the storage…        # ... remainder of function omitted here for brevity ... Image 3: Google Bard’s explanation of reviewed code While the print suggestion is not particularly helpful, Bard does make an interesting suggestion regarding the sleep function. Beyond pointing out that it may become a bottleneck, it highlights that the sleep function used is not threadsafe. While time.sleep is blocking, the use of asyncio.sleep is non-blocking. Using the asyncio the method asks the event loop to run something else while the await statement finishes its execution. If we were to incorporate this script into a more robust web application, this would be an important optimization to make.General tips for using Bard during code reviews.Here are some tips for using Bard for code reviews: Be specific where possible. For example, you can ask for specific types of improvements and optimizations during your code review. The more specific the request, the better Bard will be able to respond to it.Provide detailed context where needed. The more information Bard has, the more effective the review feedback will be.Be open to feedback and suggestions from Bard, even if they differ from your own opinions. Consider and evaluate the proposed changes before implementing them.ConclusionGoogle Bard is a powerful tool that can be used to improve the quality of code. Bard can help to identify potential bugs and errors, suggest improvements, and provide documentation and explanations. This can help to improve the efficiency, quality, and security of code.If you are interested in using Google Bard for code reviews, you can sign up for the early access program. To learn more, visit the Google Bard website.Author BioDarren Broemmer is an author and software engineer with extensive experience in Big Tech and Fortune 500. He writes on topics at the intersection of technology, science, and innovation. LinkedIn 
Read more
  • 0
  • 0
  • 165

article-image-improvising-your-infrastructure-as-code-using-ai-tools
Russ McKendrick
11 Jun 2023
6 min read
Save for later

Improvising your Infrastructure as Code using AI-tools

Russ McKendrick
11 Jun 2023
6 min read
In Infrastructure as Code for Beginners, I briefly discussed the role of AI tools such as ChatGPT. Since then, new features have emerged, making this the perfect time to revisit the topic and evaluate whether my initial recommendations have evolved. Let's quickly recap: GPT-powered chat services, including ChatGPT, certainly have their place in Infrastructure as Code (IaC) projects. However, we are not quite at the point where we can ask these tools to create our projects from scratch. The primary reason for this is straightforward: GPT, while impressive, can make mistakes. This is fine, except that when GPT errors, it often does so by inventing information, which it then presents in a highly convincing manner. On the one hand, you have Infrastructure as Code which has been taking away the heavy lifting of managing infrastructure for quite a while; a lot of the tooling is well established even though it is still considered a fast-moving technology to most.On the contrary,  artificial intelligence tools such as ChatGPT show up and quite dramatically change what we thought was possible even just a year ago, impacting how we create, test, and maintain code. Here are just some of the use cases I have been using the two together for in the last few months.●      Code Quality: You can have ChatGPT check your code and make recommendations,●      Cutting Down on Human Error: ChatGPT can double-check and make sure you are not doing something dumb if you ask it to.●      Boosting Productivity: You can ask ChatGTP to add inline comments to your code, update variables, and handle the boring stuff. Remember, we're just scratching the surface here. The combo of AI and IaC is still fresh, and there's a lot of territory left to explore.Improvising your Terraform code with ChatGPTRather than talk anymore, let’s take a look at implementing of the use cases mentioned above, to start with let’s have ChatGPT review an example Terraform block and make some recommendations on how it could be improved.Here is the prompt that we can use: I have the following Terraform code; it creates a Storage Account in Microsoft Azure; resource "azurerm_storage_account" "example" {name                     = "saiacforbeg2022111534"resource_group_name      = azurerm_resource_group.example.namelocation                 = azurerm_resource_group.example.locationaccount_tier             = "Standard"account_replication_type = "GRS"} Could you check it and provide recommendations on how it could be improved? The prompt above made the following recommendations, all of which are valid observations: Image 1: Recommendations that ChatGPT gave me.As well as rewriting the code considering its recommendations, it also recognized that we are deploying an Azure Storage Account and pointed out the following: Additionally, please remember that the storage account names in Azure need to be globally unique and must be between 3 and 24 characters in length, and can include numbers and lowercase letters only. Thus, you might want to consider adding checks to ensure that these constraints are met. Now let’s ask ChatGPT to add some inline comments which explain what is going to the main block of code using the following prompt :Please add inline comments explaining what is happening to the code below … ChatGPT gave the following response:Image 2: Fully commented codeFinally, we can ask ChatGTP to check and make some suggestions about how we could better secure our Storage Account using the following prompt:Image 3:  list of recommendations for improving our Terraform code.Let’s see if ChatGPT could update the code with some of those suggestions with the following prompt and also have a look at the result: Image 4:  recommendations within the context of the original Terraform code. GPT vs Bing vs BardHow do other AI Chatbots handle our original prompt, let’s start with Bing; Image 5: Bing Chat had to say about our Terraform code.Well, that wasn’t helpful at all - let’s now see if Google Bard fares any better:Image 6: Google Bard had to say about our Terraform code.It didn’t just give us a thumbs up, which is a good start. It made similar recommendations to ChatGPT. However, while Bard made some suggestions, they were not as detailed, and it didn’t point out anything outside of what we asked it to, unlike ChatGPT, which recognized we were creating a storage account and gave us some additional pointers.Now don’t write Microsoft off just yet. While Bing didn’t blow me away with its initial responses, Microsoft’s various Copilot services, all of which are launching soon, all of which are GPT4 based, I expect them to be up to the same level of responsiveness of ChatGTP and in the case of GitHub Copilot X I expect it to potentially surpass ChatGTP in terms of usefulness and convenience as it is going to be built directly into your IDE and have all of GitHub to learn from.Given the rate of change, which feels like new tools and features and power is being added every other day now - the next 6 to 12 months is going to be an exciting time for anyone writing Infrastructure as Code as the tools we are using day-to-day start get more direct access to the AI tools we have been discussing.SummaryIn conclusion, this article delved into the process of improving code quality through various means, including seeking recommendations, incorporating inline code comments, and leveraging the capabilities of ChatGPT, Bing, and Google Bard. The demonstration emphasized the importance of continuously enhancing code and actively involving AI-powered tools to identify areas for improvement. Additionally, the article explored how ChatGPT's insights and suggestions were utilized to enhance storage security through stronger and more reliable code implementation. By employing these techniques, developers can achieve code optimization and bolster the overall security of their storage systems.Author BioRuss McKendrick is an experienced DevOps practitioner and system administrator with a passion for automation and containers. He has been working in IT and related industries for the better part of 30 years. During his career, he has had responsibilities in many different sectors, including first-line, second-line, and senior support in client-facing and internal teams for small and large organizations.He works almost exclusively with Linux, using open-source systems and tools across dedicated hardware and virtual machines hosted in public and private clouds at Node4, where he holds the title of practice manager (SRE and DevOps). He also buys way too many records!Author of the book: Infrastructure as Code for Beginners
Read more
  • 0
  • 0
  • 121
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-creating-a-langchain-agent-azure-openai-python-with-the-react-approach
Valentina Alto
11 Jun 2023
17 min read
Save for later

Creating a LangChain Agent: Azure OpenAI & Python with the ReAct Approach

Valentina Alto
11 Jun 2023
17 min read
In my latest article, we introduced the concept of Agents powered by Large Language Models and how they overcome one of the current limitations of our beloved LLMs: the capability of taking action. An Agent can be seen as a kind of wrapper that uses an LLM as a reasoning engine, plus it has the capability of interacting with tools that we can provide and take actions with those. Tools can be the accessed through Wikipedia rather than the possibility to interact with our File System or access the command line. If prompt was an important component while working with LLMs, with agents it becomes key. In fact, agents need to be instructed with a reasoning template, which can follow various techniques. We’ve already seen an example of the Read-Retrieve-Read technique in my latest article. In this article, we are going to explore the ReAct approach.What is ReAct?ReAct (Reason and Act) approach is a general paradigm that combines reasoning and acting with LLMs. It prompts LLMs to generate verbal reasoning traces and actions for a task. As per the Read-Retrieve-Read approach, also the ReAct paradigm implies an interaction with external tools to retrieve additional information. However, with the ReAct approach, we introduce a greater synergy between the reasoning and acting phases:The reasoning phase helps the model to set up action plans, track them, and even modify them in case (also in case of handling exceptions);The action phase allows the model to interact with the external world and retrieve the needed information according to the plan of the previous phaseIn the aforementioned paper, the authors show how the ReAct approach was able to overcome typical drawbacks of LLMs as hallucination and error propagation (as it has occurred in the simple version of the Chain of Thought (CoT) method of prompt engineering).Let’s see how those kinds of agents work in practice.Implementing the AgentLangChain makes it easier to build agents thanks to lightweight libraries which provide our LLM with the ReAct-based prompt template that makes the agent capable of both reasoning and acting. To achieve this goal, we need to install the following libraries:!pip install wikipedia from langchain import Wikipedia from langchain.llms import AzureOpenAI from langchain.agents import initialize_agent, Tool from langchain.agents import AgentType from langchain.agents.react.base import DocstoreExplorerIn this case, as an external tool, we will use Wikipedia. However, you can decide to add further tools like search APIs such as Bing (you can read about that in my previous article) or the File System of our personal computer.Next, we can build our agent and the document store, where it will be able to navigate through in order to retrieve information. To assist the agent with the exploration of the document store, we will use the previously imported class DocstoreExplorer.docstore=DocstoreExplorer(Wikipedia()) tools = [    Tool(        name="Search",        func=docstore.search,        description="useful for when you need to ask with search"    ),    Tool(        name="Lookup",        func=docstore.lookup,        description="useful for when you need to ask with lookup"    ) ]Finally, we need to set up the reasoning engine of our Agent. In our case, we will use a text-davinci-003 model available in the Azure OpenAI service (to set up an Azure OpenAI instance, you can read my former article here).llm = AzureOpenAI(deployment_name="text-davinci-003", openai_api_version="xxx", openai_api_key="xxx", openai_api_base="xxx")Great, now we have all the ingredients needed to initialize our agent. To test my agents, I will take some questions from a more recent version of the question-answering dataset HotpotQA (you can refer to the original paper here) called BeerQA. In this experiment, I will challenge my agent on some reasoning abilities within the world of Harry Potter:#initializing our agent react = initialize_agent(tools, llm, agent=AgentType.REACT_DOCSTORE, verbose=True) #start with questions question = "Rupert Grint played the friend of which fictional character from the Harry Potter series?" react.run(question)Below you can find the whole reasoning and response:> Entering new AgentExecutor chain... Thought: The question simplifies to which fictional character from the Harry Potter series is Rupert Grint's friend. I need to search Rupert Grint and Harry Potter series, and find the character. Action: Search[Rupert Grint] Observation: Could not find [Rupert Grint]. Similar: ['Rupert Grint', 'Knock at the Cabin', 'Emma Watson', "Harry Potter and the Philosopher's Stone (film)", 'Harry Potter 20th Anniversary: Return to Hogwarts', 'Snatch (TV series)', 'Harry Potter (film series)', 'Servant (TV series)', 'Sick Note (TV series)', 'Georgia Groome'] Thought: To find the character, I can search Harry Potter (film series). Action: Search[Harry Potter (film series)] Observation: Harry Potter is a film series based on the eponymous novels by J. K. Rowling. The series is produced and distributed by Warner Bros. Pictures and consists of eight fantasy films, beginning with Harry Potter and the Philosopher's Stone (2001) and culminating with Harry Potter and the Deathly Hallows – Part 2 (2011). A spin-off prequel series, planned to consist of five films, started with Fantastic Beasts and Where to Find Them (2016), marking the beginning of the Wizarding World shared media franchise.The series was mainly produced by David Heyman, and stars Daniel Radcliffe, Rupert Grint, and Emma Watson as the three leading characters: Harry Potter, Ron Weasley, and Hermione Granger. Four directors worked on the series: Chris Columbus, Alfonso Cuarón, Mike Newell, and David Yates. Michael Goldenberg wrote the screenplay for Harry Potter and the Order of the Phoenix (2007), while the remaining films' screenplays were written by Steve Kloves. Production took place over ten years, with the main story arc following Harry's quest to overcome his arch-enemy Lord Voldemort.Harry Potter and the Deathly Hallows, the seventh and final novel in the series, was adapted into two feature-length parts. Part 1 was released in November 2010, and Part 2 was released in July 2011.Philosopher's Stone and Deathly Hallows – Part 2 are among the 50 highest-grossing films of all time—at 50th- and 16th-highest, respectively—with both grossing over $1 billion. It is the fourth-highest-grossing film series, with $7.7 billion in worldwide receipts. Thought: Rupert Grint played the friend of Harry Potter, Ron Weasley. Action: Finish[Ron Weasley] > Finished chain. 'Ron Weasley'Now, you might think that this was a very easy question to address (and be right). Let’s challenge it with some more complex reasoning tasks:question = "Which of the Harry Potter film series main stars debuted in stage acting first?" react.run(question)Below is the response:> Entering new AgentExecutor chain... Thought: I need to search the main stars of the Harry Potter film series, find their stage acting debut, then find which debuted first. Action: Search[Harry Potter film series main stars] Observation: Harry Potter is a film series based on the eponymous novels by J. K. Rowling. The series is produced and distributed by Warner Bros. Pictures and consists of eight fantasy films, beginning with Harry Potter and the Philosopher's Stone (2001) and culminating with Harry Potter and the Deathly Hallows – Part 2 (2011). A spin-off prequel series, planned to consist of five films, started with Fantastic Beasts and Where to Find Them (2016), marking the beginning of the Wizarding World shared media franchise.The series was mainly produced by David Heyman, and stars Daniel Radcliffe, Rupert Grint, and Emma Watson as the three leading characters: Harry Potter, Ron Weasley, and Hermione Granger. Four directors worked on the series: Chris Columbus, Alfonso Cuarón, Mike Newell, and David Yates. Michael Goldenberg wrote the screenplay for Harry Potter and the Order of the Phoenix (2007), while the remaining films' screenplays were written by Steve Kloves. Production took place over ten years, with the main story arc following Harry's quest to overcome his arch-enemy Lord Voldemort.Harry Potter and the Deathly Hallows, the seventh and final novel in the series, was adapted into two feature-length parts. Part 1 was released in November 2010, and Part 2 was released in July 2011.Philosopher's Stone and Deathly Hallows – Part 2 are among the 50 highest-grossing films of all time—at 50th- and 16th-highest, respectively—with both grossing over $1 billion. It is the fourth-highest-grossing film series, with $7.7 billion in worldwide receipts. Thought: The main stars of the Harry Potter film series are Daniel Radcliffe, Rupert Grint, and Emma Watson. I need to search each of them and find their stage acting debut. Action: Search[Daniel Radcliffe] Observation: Daniel Jacob Radcliffe (born 23 July 1989) is an English actor. He rose to fame at age twelve, when he began portraying Harry Potter in the film series of the same name. Over his career, Radcliffe has received various awards and nominations. Radcliffe made his acting debut at age 10 in the BBC One television film David Copperfield (1999), followed by his feature film debut in The Tailor of Panama (2001). The same year, he starred as Harry Potter in the film adaptation of the J.K. Rowling fantasy novel, Harry Potter and the Philosopher's Stone. Over the next decade, he played the eponymous role in seven sequels, culminating with Harry Potter and the Deathly Hallows – Part 2 (2011). During this period, he became one of the world's highest-paid actors and gained worldwide fame, popularity, and critical acclaim. Following the success of Harry Potter, Radcliffe challenged himself acting in a variety of genres starring in What If? (2013), The Woman in Black (2012), Victor Frankenstein (2015), Swiss Army Man (2016), Now You See Me 2 (2016), and The Lost City (2022). He also portrayed Allen Ginsberg in the drama film Kill Your Darlings (2013), FBI agent Nate Foster in the crime drama Imperium (2016), and Weird Al Yankovic in the musical film Weird: The Al Yankovic Story (2022). Since 2019, he has starred in the TBS anthology series Miracle Workers. Radcliffe branched out to stage acting in 2007, starring in the West End and Broadway productions of Equus. He returned to Broadway in the musical How to Succeed in Business Without Really Trying (2011), earning a Grammy Award nomination, and Martin McDonagh's The Cripple of Inishmaan (2014), the satirical play The Lifespan of a Fact (2018), and Stephen Sondheim's Merrily We Roll Along (2023). He returned to the West End in revivals of Tom Stoppard's Rosencrantz and Guildenstern Are Dead (2017), and Samuel Becket's Endgame (2020). Radcliffe has contributed to many charities, including Demelza Hospice Care for Children and the Trevor Project; the latter awarded him its Hero Award in 2011 for his advocacy with LGBTQ youth. Thought: Daniel Radcliffe made his stage acting debut in the West End and Broadway productions of Equus in 2007. I need to search Rupert Grint and Emma Watson next and find their stage acting debut. Action: Search[Rupert Grint] Observation: Could not find [Rupert Grint]. Similar: ['Rupert Grint', 'Knock at the Cabin', 'Emma Watson', "Harry Potter and the Philosopher's Stone (film)", 'Harry Potter 20th Anniversary: Return to Hogwarts', 'Snatch (TV series)', 'Harry Potter (film series)', 'Servant (TV series)', 'Sick Note (TV series)', 'Georgia Groome'] Thought: To find Rupert Grint's stage acting debut, I can search Rupert Grint. Action: Search[Rupert Grint] Observation: Could not find [Rupert Grint]. Similar: ['Rupert Grint', 'Knock at the Cabin', 'Emma Watson', "Harry Potter and the Philosopher's Stone (film)", 'Harry Potter 20th Anniversary: Return to Hogwarts', 'Snatch (TV series)', 'Harry Potter (film series)', 'Servant (TV series)', 'Sick Note (TV series)', 'Georgia Groome'] Thought: To find Rupert Grint's stage acting debut, I can search Rupert Grint stage acting. Action: Search[Rupert Grint stage acting] Observation: Rupert Alexander Lloyd Grint (; born 24 August 1988) is an English actor. Grint rose to fame for his role as Ron Weasley in the Harry Potter film series, for which he was cast at age eleven, having previously acted only in school plays and his local theatre group. Since then, he continued his work on film, television, and theatre. Beginning in 2002, he began to work outside of the Harry Potter franchise, with a co-leading role in Thunderpants. He starred in the dramedy Driving Lessons (2006) and the drama Cherrybomb (2009) and co-starred in the comedy Wild Target (2010). After the Harry Potter series concluded, he appeared in the films Into the White (2012), Charlie Countryman (2013), CBGB (2013), and Moonwalkers (2015). He made his stage debut in 2013 in Mojo at the Harold Pinter Theatre. He executive-produced and starred in the television series Snatch, based on the film of the same name. He has also starred in the black comedy series Sick Note, the mystery thriller miniseries The ABC Murders, and the psychological horror series Servant. Thought: Rupert Grint made his stage acting debut in Mojo at the Harold Pinter Theatre in 2013. I need to search Emma Watson next and find her stage acting debut. Action: Search[Emma Watson] Observation: Emma Charlotte Duerre Watson (born 15 April 1990) is an English actress, model and activist. Known for her roles in both blockbusters and independent films, as well as for her women's rights work, she has received a selection of accolades, including a Young Artist Award and three MTV Movie Awards. Watson has been ranked among the world's highest-paid actresses by Forbes and Vanity Fair, and was named one of the 100 most influential people in the world by Time magazine in 2015.Watson attended the Dragon School and trained in acting at the Oxford branch of Stagecoach Theatre Arts. As a child, she rose to stardom after landing her first professional acting role as Hermione Granger in the Harry Potter film series, having previously acted only in school plays. Watson also starred in the 2007 television adaptation of the novel Ballet Shoes and lent her voice to The Tale of Despereaux (2008). After the final Harry Potter film, she took on a supporting role in My Week with Marilyn (2011), before starring as Sam, a flirtatious, free-spirited student in The Perks of Being a Wallflower (2012), to critical success. Further acclaim came from portraying Alexis Neiers in Sofia Coppola's The Bling Ring (2013) and the titular character's adoptive daughter in Darren Aronofsky's biblical epic Noah (2014). That same year, Watson was honoured by the British Academy of Film and Television Arts, winning British Artist of the Year. She also starred as Belle in the live-action musical romantic fantasy Beauty and the Beast (2017), and Meg March in Greta Gerwig's coming-of-age drama Little Women (2019). From 2011 to 2014, Watson split her time between working on films and continuing her education, graduating from Brown University with a bachelor's degree in English literature in May 2014. That year, she was appointed a UN Women Goodwill Ambassador and helped launch the UN Women campaign HeForShe, which advocates for gender equality. In 2018, she helped launch Time's Up UK as a founding member. Watson was appointed to a G7 advisory body for women's rights in 2019, consulting with leaders on foreign policy. Her modelling work has included campaigns for Burberry and Lancôme. She also lent her name to a clothing line for the sustainable brand People Tree. In 2020, she joined the board of directors of Kering, a luxury brand group, in her capacity as an advocate for sustainable fashion. Thought: Emma Watson made her stage acting debut in the 2007 television adaptation of the novel Ballet Shoes. Daniel Radcliffe made his stage acting debut in 2007 (Equus), while Rupert Grint and Emma Watson made their stage acting debut in 2013 (Mojo and Ballet Shoes, respectively). So Daniel Radcliffe debuted in stage acting first. Action: Finish[Daniel Radcliffe] > Finished chain. 'Daniel Radcliffe'In this case, you can see that the Agent needs to plan several steps to find the right information. It is incredible to see how it can formulate the most effective query for the tool it is enabled to navigate through. At the end of the Chain of Thoughts, it gives the correct answer.For the sake of HP fandom (which I’m part of), let’s also ask something that it is only present within the books:question = "In Harry Potter and the Goblet of Fire, in the third task to get to the Goblet a riddle of which magical creature needs to be solved?" react.run(question)Below is the result:> Entering new AgentExecutor chain... Thought: I need to search Harry Potter and the Goblet of Fire, find the third task, and find the magical creature. Action: Search[Harry Potter and the Goblet of Fire] Observation: Harry Potter and the Goblet of Fire is a fantasy novel written by British author J. K. Rowling and the fourth novel in the Harry Potter series. It follows Harry Potter, a wizard in his fourth year at Hogwarts School of Witchcraft and Wizardry, and the mystery surrounding the entry of Harry's name into the Triwizard Tournament, in which he is forced to compete. The book was published in the United Kingdom by Bloomsbury and in the United States by Scholastic. In both countries, the release date was 8 July 2000. This was the first time a book in the series was published in both countries at the same time. The novel won a Hugo Award, the only Harry Potter novel to do so, in 2001. The book was adapted into a film, released worldwide on 18 November 2005, and a video game by Electronic Arts. Thought: The paragraph does not mention the third task. I need to look up third task. Action: Lookup[third task] Observation: No Results Thought: I need to instead search Harry Potter and the Goblet of Fire third task. Action: Search[Harry Potter and the Goblet of Fire third task] Observation: Harry Potter and the Goblet of Fire is a fantasy novel written by British author J. K. Rowling and the fourth novel in the Harry Potter series. It follows Harry Potter, a wizard in his fourth year at Hogwarts School of Witchcraft and Wizardry, and the mystery surrounding the entry of Harry's name into the Triwizard Tournament, in which he is forced to compete. The book was published in the United Kingdom by Bloomsbury and in the United States by Scholastic. In both countries, the release date was 8 July 2000. This was the first time a book in the series was published in both countries at the same time. The novel won a Hugo Award, the only Harry Potter novel to do so, in 2001. The book was adapted into a film, released worldwide on 18 November 2005, and a video game by Electronic Arts. Thought: The third task is to get to the Goblet, and the riddle of a magical creature needs to be solved. The magical creature is a sphinx. Action: Finish[sphinx] > Finished chain. 'sphinx'Well, we can say that our Agent did a pretty job. The next challenge will be navigating through the Marvel multiverse 😎Stay tuned for the next article!Referenceshttps://medium.com/@valentinaalto/introducing-langchain-agents-e58674b1a657[2210.03629] ReAct: Synergizing Reasoning and Acting in Language Models (arxiv.org)[2210.03629] ReAct: Synergizing Reasoning and Acting in Language Models (arxiv.org)Plan and Execute — 🦜🔗 LangChain 0.0.168Agents — 🦜🔗 LangChain 0.0.168File System Tools — 🦜🔗 LangChain 0.0.168Tools — 🦜🔗 LangChain 0.0.168https://hotpotqa.github.io/https://nlp.stanford.edu/projects/beerqa/beerqa_train_v1.0.jsonAuthor BioValentina Alto graduated in 2021 in data science. Since 2020, she has been working at Microsoft as an Azure solution specialist, and since 2022, she has been focusing on data and AI workloads within the manufacturing and pharmaceutical industries. She has been working closely with system integrators on customer projects to deploy cloud architecture with a focus on modern data platforms, data mesh frameworks, IoT and real-time analytics, Azure Machine Learning, Azure Cognitive Services (including Azure OpenAI Service), and Power BI for dashboarding. Since commencing her academic journey, she has been writing tech articles on statistics, machine learning, deep learning, and AI in various publications and has authored a book on the fundamentals of machine learning with Python.Author of the book: Modern Generative AI with ChatGPT and OpenAI ModelsLink - Medium  LinkedIn  
Read more
  • 0
  • 0
  • 1244

article-image-using-gpt-4-for-offensive-security
Indrajeet Bhuyan
11 Jun 2023
7 min read
Save for later

Using GPT-4 for Offensive Security

Indrajeet Bhuyan
11 Jun 2023
7 min read
In this article, we will explore the cutting-edge capabilities of GPT-4, the latest advancement in language models, and discover how it can revolutionize offensive security strategies. This article delves into practical examples, demonstrating how GPT-4 can be leveraged for tasks such as vulnerability assessment, penetration testing, and threat intelligence gathering. Gain insights into its enhanced natural language understanding, automated exploit generation, and intelligent attack simulations, opening new avenues for proactive cybersecurity defenses. Harness the potential of GPT-4 to fortify your offensive security arsenal.Reminder: Generative AI is known for hallucinations where they give replies which are lies, and not true so always verify your information.Here are a few use cases where you can use GPT for Offensive Security tasks:Writing phishing mailsEarlier one of the ways to detect phishing emails was bad English. Often phishing emails have a history of having incorrect English as often attackers are from non-English speaking places. But now the availability of generative AI tools enables a broader range of users to create convincing phishing and spam messages, even if they lack the skills to do so manually.For instance, these tools can be used to generate social engineering drafts that impersonate an employee or a company, making it easier for individuals with little experience in crafting these messages to carry out these types of attacks.Example:Image 1: Example of Phishing EmailExploit developmentChatGPT can assist in identifying vulnerabilities, as demonstrated by Cybernews researchers who used the chatbot to exploit a vulnerability that was discovered. However, ChatGPT is programmed to not offer illicit services, like hacking. Therefore, carefully phrased queries are necessary. Merely asking the chatbot to create an exploit for a particular vulnerability will not suffice.The researchers informed the chatbot that they were conducting a 'Hack the Box' pen test challenge and required assistance in locating a vulnerability. Once discovered, they were given step-by-step instructions, exploit code examples, and samples to use. As a result, the security researchers were able to identify and write an exploit for a well-known application within 45 minutes. This once again showcases how ChatGPT has made a previously complicated and lengthy process accessible to everyone.Write scriptsGPT-4 can be used to write automation scripts that can make the work of a security professional easExample : Here I’m asking ChatGPT to write a Python script that will use httpx to see all the live web hosts from a list of text files containing URLs and then use nuclei on all the domains:Image 2: Python script outputSource Code reviewChatGPT now with GPT-4 can be used to do source code reviews. It can help security professionals iXn do source code reviews faster. Though sometimes it gives wrong answers it can be used to get a good suggestion which later the security professional can use to verify if the response is correct or not.Example: I asked GPT-4 to find a security vulnerability in this code <?php if( isset( $_POST[ 'Submit' ]  ) ) {            // Get input            $target = $_REQUEST[ 'ip' ];            // Determine OS and execute the ping command.            if( stristr( php_uname( 's' ), 'Windows NT' ) ) {                        // Windows                        $cmd = shell_exec( 'ping  ' . $target );            }            else {                        // *nix                        $cmd = shell_exec( 'ping  -c 4 ' . $target );            }            // Feedback for the end user            $html .= "<pre>{$cmd}</pre>"; } ?> Here is the response:Image 3: Code reviewHere the code was simple and it could find a flaw in the code correctly but with complex code, it often gives incorrect responses. But these responses can help security professionals in doing their tasks fast as they can get feedback from ChatGPT and can filter out incorrect information:Create your own Nuclei templateChatgpt can be used to create a Nuclei template which can later help in automating the task of finding flaws.Example: Image 4: Nuclei templateWrite reportsThis one is my favorite. As a penetration tester or security professional one needs to write a lot of reports. Reports for clients, reports for other team members, etc. Chatgpt can be extensively used for writing security reports. Though I don’t recommend using chatGPT to make the entire report it can be used for making the task of writing reports easyExample 1: Here I’m asking ChatGPT to help me write a bug bounty report for an SSRF flaw that I found in example.com along with how to fix the issue: Image 5: Bug Bounty ReportExample 2: Here I’m asking Chatgpt to write a report on OWASP's top 10 flaws along with how to fix those flaws which I need to send to the client:Image 6: OWASP top 10 flawsPrepare for interviews.Chatgpt can be used for preparing for technical interviews as it can show you common questions that are asked for your role and position.Example: Here I’m asking ChatGPT to suggest me 10 questions along with an answer for a penetration tester job with 3 years of experience:Image 10: Interview QuestionsWrite technical blogs and articlesChatgpt is now more powerful with GPt-4 and it can help you write technical articles by giving you valuable inputs and suggestions.Fun fact: GPT-4 is used intensively while writing this article that you are reading.ConclusionIn conclusion, GPT-4, together with Chatgpt, has the potential to simplify numerous tasks in offensive security, thereby offering valuable assistance to security professionals. Though there are concerns about the possibility of individuals misusing this technology, its positive use cases cannot be disregarded. However, it is unlikely that GPT-4 can replace security professionals soon. Nonetheless, security experts who incorporate AI in their work can certainly outperform those who do not leverage this technology. Ultimately, it is up to us to determine how we use this powerful tool and ensure that it is utilized for the greater good.Author BioIndrajeet Bhuyan is a security researcher from Assam, India. He holds a Bachelor of Technology (B.Tech.) focused on Computer Science from Assam Don Bosco University. He has contributed security to various companies and organizations like Whatsapp, HTC, Samsung, Photobucket, Reverbnation, TVF, ESET, etc. He got featured in multiple national and international news portals like International Business Times, Russia Today, Times of India, Digit, Kaspersky, The Independent, India Today, etc. for his work on cybersecurity. He created ‘WannaSmile’ a tool that was used to protect users from the world’s biggest ransomware attack ‘ WannaCry’ which was featured by various national and international media.LinkedIn 
Read more
  • 0
  • 0
  • 602

article-image-using-ai-functions-to-solve-complex-problems
Shaun McDonogh
09 Jun 2023
8 min read
Save for later

Using AI Functions to Solve Complex Problems

Shaun McDonogh
09 Jun 2023
8 min read
GPT-4 feels like Einstein's theory of general relativity, in that it will take programmers a long time to uncover all the new possibilities which is analogous to how physicists today are still extracting useful insights from Einstein's theory of general relativity.Here is one example. Big problems can be solved by breaking them down into small tasks. Our jobs usually boil down into performing one problem-solving or productive activity after the next. So, what if you could discern clearly what those smaller tasks were and give them to your side-kick brain to figure out and run for you ‘auto-magically’?Note that the above says, what if you could discern. Already we bump up against a lesson in using LLMs. Why do you need to discern it? Can’t GPT-4 be given the job of discerning the tasks required? If so, could it then orchestrate other GPT-4 “agents” to break down the task further? This is exactly where we can make use of a tool like Auto-GPT.Let’s look at an example that I am developing right now for an Auto-GPT using Rust.Say you build websites for a living. Websites need a web server. Web servers take a request (usually through an API), run some code in line with that request and send back a response. Let’s break down the goal of building a web server into what actors or agents might be required when building an Auto-GPT:Solution Architect: Work with the customer (you) to develop scope and aim of the application. Confirm the recommended technology stack to work within.Project (Agent) Manager: Compile and track preliminary list of tasks needed to complete the project.Data Manager: Identify required External and Internal API endpoints.Code Admin: Write code spec sheet for developer.Junior Developer:Take a first stab at code.Run unit testing.Senior Developer:Sense check initial code and adjust.Run unit testing.Quality Control: Test and send back code for adjustments.Customer (you): Supply feedback and request adjustments.You may have guessed that we will not be hiring 7 members of staff to build web servers. However, planning out this structure is necessary to help us organise building a bot that can itself build, test, and deploy any backend server you ask it to.Our agents will need tools to work with. They need to be given a set of instructions and functions that will get the subtask done without the usual response of “I am just an AI language model”. When developing such a tool, it became clear to me that a new paradigm has hit developers, “AI Functions”. What we are about to discuss brings up potential gaps in the guard-rails of GPT-4 which is unlikely to be news to the OpenAI team but may be of some surprise to you.What are AI Functions?The fact that AI hallucinates would annoy most people. But not to programmers and the folk who developed Marvin. Right now, when you ask ChatGPT to provide you with a response, it sends a whole bunch of jargon with it.What if I told you there was a hack, a way to get distilled and relevant information out of ChatGPT automatically through the OpenAI API that leads to a structured response that can drive actions in your code. Here is an example. Suppose you give an AI the following “function” which is just text disguised as a programming function:struct Sentiment {       happy_rating: 0, sad_rating: 0 } fn convert_comments_into_sentiment(comments: String) -> Sentiment {       // Function takes in comments from YouTube video       // Considers all aspects of what is said in comments       // Returns Sentiment struct with happy and sad rating       // Does not return anything else. No commentary.       return sentiment; }You then ask the AI to print only what the function would return and pass it the YouTube comments as an input. The AI (GPT-4) would then hallucinate a response, even without there being any actual code within the function. None. There is no code to perform this task.No jargon like “I am an AI I cannot blah blah blah” is returned. Nothing like that, you would extract the AI’s pure sentiment analysis in a format which you can then build into your application.The hack here is that we have built a pretend function, without any code in it. We described exactly what this function (written in Rust syntax in this example) does and the structure of its output using comments and a struct.GPT-4 then interprets what it thinks the function does and assesses it against the input provided (YouTube comments in this example). The output is then structured in the LLM’s response. Therefore, we can have LLMs supply responses that are structured in such a way that can be used by other agents (also the GPT-4 LLM).The entire process would look something like this: This is an oversimplified view of what is happening. Note that agents are not always needed. Sometimes the function can just be called as regular code, no LLM needed.The Bigger PictureWhy stop there? Auto-GPTs will likely then become like plugins. With Auto-GPTs popping up everywhere, most software related tasks can be taken care of. Rather than calling an API, your Auto-GPT could call another Auto-GPT. Rather than write AI Functions, a primary agent would know what Auto-GPTs should be assigned to various tasks. It could then write its own program that connects to them.This sort of recursion or self-improvement is where predicted growth in AI is already shown to function as predicted either exponentially or in sigmoid-like jumps. My own version of this self-programming task is already done. It was surprisingly trivial to do with GPT-4’s help. It works well, but it doesn’t replace human developers yet.Not All Sunshine and RosesGPT-4 is not without some annoying limitations; these include:Quality of performance at the given taskChanges in quality response depending on a slight change in input.Text formatting to occasionally include something the model should not.Size and dollar cost of outputTime, latency, delay and crashing of API output.These limitations could technically be worked around by breaking down tasks into even smaller chunks, but this is where I draw a line. What is the point of trying to automate anything, if you end up having to code everything? However, it has become clear through this exercise, that just a few improvements on the above limitations will lead to a significant amount of task automation with AutoGPTs.Quick Wins with Self-testingOne unique algorithm which has worked very well with the current technology is the ability for GPT-4 to write code and for our program to then execute and test the code and provide feedback to GPT-4 of what is not working. The AI Function then prompts GPT-4 as a “Senior Developer Agent” to re-write the code and return to the “Quality Agent” for more unit testing.What you end up with is a program that can write code, test it, and confirm all is working. This is working now and is useful. The downside is still the cost of making too many calls to the OpenAI API and some quality limitations with GPT-4. Therefore, until GPT-5, I remain working directly in my development environment or in the ChatGPT interface extracting quick code snippets.Next StepsIt sounds clichéd but I’m learning something new every day working with LLMs. A recurring realisation is that we should not only be trying to solve newer problems in less time, but rather, learning what the right questions to ask our current LLMs even are. They are capable of more than I believe we realise. When Open Source LLMs reach a GPT-5 like level, Auto-GPTs I suspect will be everywhere.Author BioShaun McDonogh is the lead developer and researcher at Crypto Wizards. He has 6 years’ experience in developing high performance trading software solutions with ML and 15 years’ experience working with frontend and backend web technologies.In his prior career, Shaun worked as a Finance Director, Data Scientist and Analytics Lead in various multi-million-dollar entities before making the transition to developing pioneering solutions for developers in emerging tech industries. His favorite programming languages to teach include Rust, Typescript and Python.Shaun’s mission is to support developers in navigating the ever-changing landscape of emerging technologies in computer software engineering through a new coding arm, Code Raiders.
Read more
  • 0
  • 0
  • 189

article-image-building-a-gan-model-in-tensorflow-easily-with-autogpt
Rohan Chikorde
08 Jun 2023
7 min read
Save for later

Building a GAN model in TensorFlow easily with AutoGPT

Rohan Chikorde
08 Jun 2023
7 min read
AutoGPT is a large language model that can be used to perform a variety of tasks, such as generating text, translating languages, writing different kinds of creative content, and answering your questions in an informative way. To use AutoGPT, you first need to specify a goal in natural language. AutoGPT will then attempt to achieve that goal by breaking it down into sub-tasks and using the internet and other tools in an automatic loop.In this blog, we will be using godmode.space for the demo. Godmode is a web platform to access the powers of AutoGPT and Baby AGI. AI agents are still in their infancy, but they are quickly growing in capabilities, and hope that Godmode will enable more people to tap into autonomous AI agents even in this early stage.First, you would need to create your API key and Install AutoGPT as shown in this article (Set Up and Run Auto-GPT with Docker). Once you set up your API key, now click on the Settings tab in godmode.space website and put in your API key. Once you click on Done, the following screen should appear:Image 1: API KeySpecifying the GoalWhen you specify a goal in natural language, AutoGPT will attempt to understand what you are asking for. It will then break down the goal into sub-tasks and if needed it will use the internet and other tools to achieve the goal.In this example, we will tell AutoGPT to Build a Generative Adversarial Network (GAN) from Scratch with TensorFlow.Automatically breaking down a goal into sub-tasksAutoGPT breaks down a goal into sub-tasks by identifying the steps that need to be taken to achieve the goal. For example, if the goal is to build a GANs, AutoGPT will break the goal down into the following sub-tasks (these goals will change based on the user’s prompt):·        Define the generator and discriminator architectures.·        Implement the training loop for the GAN.·        Evaluate the performance of the GAN on a test dataset.Here is an example of how you can use AutoGPT to “Build a Generative Adversarial Network (GAN) from Scratch with TensorFlow”: Specify a goal by adding a simple prompt Build a Generative Adversarial Network (GAN) from Scratch with TensorFlow                                                       Image 2: Specifying the final goalThe provided functionality will present suggested options, and we will choose all the available options. Additionally, you have the flexibility to provide your own custom inputs if desired as shown in the preceding image:Image 3: Breaking down the goal into smaller tasks  Break down the goal into sub-tasks as follows:  Define the generator and discriminator architectures.Write the code for a Generative Adversarial Network (GAN) to a file named gan.py. Implement the training loop for the GAN. Write text to a file named 'gan.py' containing code for a GAN model.Evaluate the performance of the GAN on a test dataset.Image 4: Broken down sub-taskNext, we will launch the AutoGPT in the God Mode.Image 5: Launching AutoGPT in God ModeAs observed, the agent has initiated and commenced by sharing its thoughts and reasoning and the proposed action as shown in the two images below:Images 6: Plan proposed by Auto-GPTConsidering the provided insights and rationale, we are in favor of approving this plan. It is open to receiving input and feedback, which will be considered to refine the subgoals accordingly.After approving the plan, the following are the results:Image 7: AutoGPT providing thoughts and reasoning for the task 8.      According to the next task specified, the next plan suggested by the AutoGPT is to write the entire code in gan.py file:Image 8: Plan to copy the entire code to a file On approving this plan, the action will commence and we will find the following screen as output after the task completion:Image 9: Thoughts and reasoning for the previous taskUpon opening the file, we can observe that the initial structure has been created. The necessary submodules have been generated, and the file is now prepared to accommodate the addition of code. Additionally, the required libraries have been determined and imported for use in the file”Image 10: Initial structure created inside the fileWe will proceed with AutoGPT generating the complete code and then review and approve the plan accordingly.Following is the snapshot of the generated output code, we can observe that AutoGPT has successfully produced the complete code and filled in the designated blocks according to the initial structure provided:(Please note that the snapshot provided showcases a portion of the generated output code. It is important to keep in mind that the complete code is extensive and cannot be fully provided here due to its size)Image 11: Final output AutoGPT Industry-specific use casesData Science and Analytics: AutoGPT can be used to automate tasks such as data cleaning, data wrangling, and data analysis. This can save businesses time and money, and it can also help them to gain insights from their data that they would not have been able to obtain otherwise.Software Development: AutoGPT can be used to generate code, which can help developers to save time and improve the quality of their code. It can also be used to create new applications and features, which can help businesses to stay ahead of the competition.Marketing and Content Creation: AutoGPT can be used to generate marketing materials such as blog posts, social media posts, and email campaigns. It can also be used to create creative content such as poems, stories, and scripts. This can help businesses to reach a wider audience and to engage with their customers in a more meaningful way.Customer Service: AutoGPT can be used to create chatbots that can answer customer questions and resolve issues. This can help businesses to provide better customer service and to save money on labor costs.Education: AutoGPT can be used to create personalized learning experiences for students. It can also be used to generate educational content such as textbooks, articles, and videos. This can help students to learn more effectively and efficiently.ConclusionAutoGPT is a powerful AI tool that has the potential to revolutionize the way we work and live. It can be used to automate tasks, generate creative content, and solve problems in a variety of industries. As AutoGPT continues to develop, it will become even more powerful and versatile. This makes it an incredibly exciting tool with the potential to change the world.Author BioRohan Chikorde is an accomplished AI Architect professional with a post-graduate in Machine Learning and Artificial Intelligence. With almost a decade of experience, he has successfully developed deep learning and machine learning models for various business applications. Rohan's expertise spans multiple domains, and he excels in programming languages such as R and Python, as well as analytics techniques like regression analysis and data mining. In addition to his technical prowess, he is an effective communicator, mentor, and team leader. Rohan's passion lies in machine learning, deep learning, and computer vision.LinkedIn
Read more
  • 0
  • 0
  • 113
article-image-writing-improvised-unit-test-cases-with-github-copilot
Mr. Avinash Navlani
08 Jun 2023
6 min read
Save for later

Writing improvised unit test cases with GitHub Copilot

Mr. Avinash Navlani
08 Jun 2023
6 min read
In the last couple of years, AI and ML have provided various tools and methodologies to transform the working styles of software developers. GitHub Copilot is an AI-powered tool that offers code suggestions for rapid software development. GitHub Copilot can be a game changer for developers across the world. It can increase productivity by recommending code snippets and auto-completions. In this tutorial, we will discuss the Unit tests, GitHub Copilot Features, account creation, setup VS code, JavaScript example, Python example, unit test example, and GitHub Copliot advantages and disadvantages. Setup your account at GitHub Copilot Account Let’s first create your account with GitHub Copilot and then use that account in IDE such as VS Code. We can set up your account from the following web page GitHub Copilot · Your AI pair programmer  Image 1:  GitHub Copilot · Your AI pair programmer. Click on the Get Copilot button and start your 30-day free trial but we need to enter the credit/debit card or pay-pal account details.  After this, activate your GitHub Copilot account by following the instruction mentioned in the GitHub quickstart guide for individual or organization access.  Image 2: Activate Github Copilot in Settings Setup in Visual Studio Code  The first step is to download and install the Visual Studio Code from the official download page. Install the GitHub Copilot extension by following the instruction in the quickstart guide. We can directly search for ext install GitHub.copilot in Quick file navigation (shortcut key: Ctrl + P)  Image3: Visual Studio Home Login with your GitHub credentials as mentioned in the quickstart guide. ​​​Create a Python Function is_prime In this section, we create the function is_prime with Copilot recommendation and see how it can help software developers in their coding journey. For executing any Python code, we need a Python file (with .py an extension). This file will help the interpreter to identify that this is a Python script. First, we create a Python file (prime_number.py) from the file tab by selecting the new file option or we can press Alt + Ctrl + N. After creating the Python script file, we will create the is_prime(number =) function. This is_prime() the function will check whether a number is prime or not. We can see in the below snapshot Copilot also suggests the docstring for the function as per the name of the function:  Image 4: Doc-string suggestions from the Copilot ​​​​​After creating the file, we added the is_prime function but on writing the logic for the is_prime() function Copilot is suggesting an internal if condition inside for loop of the is_prime() function. We can see that suggested if statement in the below snapshot.  Image 5: Github Copilot suggests an if statement After suggesting the if statement, Copilot suggests the return value True as shown below snapshot:   Image 6: Github Copilot suggests return value This is how we can write the Python function is_prime.  We can also write similar functions as per your project requirement. Writing code with GitHub Copilot makes it much faster and more productive for a developer by providing code snippet suggestions and auto completions. ​Write Unit Tests for Python Function is_prime Unit testing is a very important component of the coding module that builds confidence and faith in software developers and ensures bug-free and quality code.  The main objective of this testing is to ensure the expected functionality at individual and isolated component-level. Let’s see how we can define them using the AI-powered code suggestion platform GitHub Copilot: First, we will create a function unit_test_prime_number()  for unit testing the is_prime() function. Here, we can see how Copilot suggests the doc-string in the below snapshot:  Image 7: Github Copilot suggests a doc-string for the unit_test method After writing the doc-string, we will add the assertion condition for the unit test. As we can see Copilot is auto-suggesting the assertion condition for the test in the below snapshot:  Image 8: Github Copilot suggests assertion condition Similarly, Copilot is suggesting the other assertion conditions with different possible inputs for testing the is_prime() function as shown below snapshot:  Image 9: Github Copilot suggests other assertion conditions This is how GitHub Copilot can help us in writing faster unit-test. It speeds up development and increases developer productivity through code suggestions and auto-completions.  Let’s see the final output in the below coding snapshot:  Image 10: A final program created with GitHub Copliot suggestions In the above snapshot, we can see the complete code for the is_prime() function and unit_test_prime_number(). Also, we can see the output of the unit_test_prime_number() function in the Terminal. Advantages and Disadvantages of the Copilot Unit Test  GitHub Copilot offers the following advantages and disadvantages: Improve productivity and reduce development time by offering efficient test cases. Developers can focus on more significant problems rather focus on small code snippets for unit testing. A novice coder can also easily utilize its capability to ensure quality development. Sometimes it may suggest irrelevant or vulnerable code because it doesn’t have proper context and user behavior. As a developer, we also need to consider how edge cases are covered in unit tests.  Sometimes it may lead to syntax errors, so we must verify the syntax. Summary GitHub Copilot revolutionized the way programmers write code. It offers AI-based code suggestions and auto-completions. Copilot boosts developer productivity and rapid application development by providing accurate, relevant, robust, and quality code. Also, sometimes it may cause problems in covering the edge cases. In the above tutorial, I have shared examples from JavaScript and Python languages. We can also try other languages such as TypeScript, Ruby, C#, C++, and Go. Reference GitHub Copilot Quick Start Guide: https://docs.github.com/en/copilot/quickstart Unit tests https://www.freecodecamp.org/news/how-to-write-unit-tests-for-python-functions/ Author BioAvinash Navlani has over 8 years of experience working in data science and AI. Currently, he is working as Sr. Data scientist, Improving products and services for customers by using advanced analytics, deploying big data analytical tools, creating and maintaining models, and onboarding compelling new datasets. Previously, he was a Lecturer at the university level, where he trained and educated people in Data science subjects such as python for analytics, data mining, machine learning, Database Management, and NoSQL. Avinash has been involved in research activities in Data science and has been a keynote speaker at many conferences in India.  LinkedIn   Python Data Analysis, Third Edition    
Read more
  • 0
  • 0
  • 759

article-image-everything-you-need-to-know-about-pinecone-a-vector-database
Avinash Navlani
08 Jun 2023
5 min read
Save for later

Everything you need to know about Pinecone – A Vector Database

Avinash Navlani
08 Jun 2023
5 min read
In this 21st century of information, we need efficient reliable storage and faster information retrieval. Relational or older databases are the most crucial databases for any computer application, but they are unable to handle the data in different forms such as documents, key-value pairs, and graphs. Vector database is a novel approach that uses vectorization for efficient search, storage, and data analysis.  Image 1: Traditional Vs Vector Database Pinecone is one such vector database that is widely accepted across the industry for addressing challenges such as complexity and dimensionality. Pinecone is a cloud-native vector database that handles high-dimensional vector data. The core underlying approach for Pinecone is based on the Approximate Nearest Neighbor (ANN) search that efficiently locates faster matches and ranks them within a large dataset. In this tutorial, our focus will be on the pinecone database, its features, challenges, and use cases.  Working Mechanism Traditional databases search for exact query matches while vector databases search for the most similar vector to the input query. It uses ANN (Approximate Nearest Neighbour) search. It provides approximate results at high performance, accuracy, and speed. Let's see the vector database working mechanism.  Image 2: Vector Database Query Mechanism Vector databases first convert data into vectors and create indexing for faster searching. Vector database compares the indexed vector query and indexed vector in the database using the nearest neighbor or similarity matrix and computes the nearest most similar results. Finally, it post-processes the most similar results given by the nearest neighbor. Features Pinecone is a cloud-based vector database that offers various features and benefits to the infrastructure community: Fast and fresh vector search: Pinecone provides ultra-low query latency, even with billions of items. This means that users will always get a great experience, even when searching large datasets. Additionally, Pinecone indexes are updated in real-time, so users always have access to the most up-to-date information.  Filtered vector search: Pinecone allows you to combine vector search with metadata filters to get more relevant and faster results. For example, you could filter by product category, price, or customer rating.  Real-time updates: Pinecone supports real-time data updates, allowing for dynamic changes to the data. This contrasts with standalone vector indexes, which may require a full re-indexing process to incorporate new data. It has reliability, massive scalability, and security capability.  Backups and collections: Pinecone handle the routine operation of backing up all the data stored in the database. You can also selectively choose specific indexes that can be backed up in the form of “collections,” which store the data in that index for later use.  User-friendly API: Pinecone provides a user-friendly API layer that simplifies the development of high-performance vector search applications. This API layer is also language-agnostic, so you can use it with any programming language.  Programming language integration: It supports a wide range of programming languages for integration.  Cost-effectiveness: It is cost-effective because it offers cloud-native architecture. It offers pay-per-use based pricing. Challenges Pinecone vector database offers high-performance data search at a higher scale, but it also faces a few challenges such as:  Application integration with other applications will evolve over a period. Data privacy is the biggest concern for any database. Organizations need to implement proper authentication and authorization mechanisms. Vector-based models don’t explain the model's interpretability. So, it is challenging to interpret the underlying reason behind those relationships. Use cases Pinecone has a variety of real-life industry applications. Let’s discuss a few applications: Audio/Textual Search: Pinecone offers faster, fully deployment-ready search and similarity functionality for high-dimensional text and audio data. Natural language Processing: Pinecone utilizes AutoGPT to create context-aware solutions for document classification, semantic search, text summarization, sentiment analysis, and question-answering systems. Recommendations: Pinecone enables personalized recommendations with efficient similar items recommendations that improve user experience and satisfaction. Image and Video Analysis: Pinecone also has the capability of faster retrieval of image and video content. It is very useful in real-life surveillance and image recognition. Time series similarity search: Pinecone can detect Time-series patterns in historical time-series data using a similarity search service. such core capability is quite helpful for recommendations, clustering, and labeling applications. Summary Pinecone vector database is a vector-based database that offers high-performance search and similarity matching. It can deal with high-dimensional vector data at a higher scale, easy integration, and faster query results. Pinecone provides a reliable, and faster, option for searching at a higher scale. Author BioAvinash Navlani has over 8 years of experience working in data science and AI. Currently, he is working as a senior data scientist, improving products and services for customers by using advanced analytics, deploying big data analytical tools, creating and maintaining models, and onboarding compelling new datasets. Previously, he was a university lecturer, where he trained and educated people in data science subjects such as Python for analytics, data mining, machine learning, database management, and NoSQL. Avinash has been involved in research activities in data science and has been a keynote speaker at many conferences in India.Link - LinkedIn    Python Data Analysis, Third edition   
Read more
  • 0
  • 0
  • 19338

article-image-infusing-old-school-powerpoint-presentations-with-gpt-powers
Andrei Gheorghiu
08 Jun 2023
9 min read
Save for later

Infusing Old-school PowerPoint Presentations with GPT Powers

Andrei Gheorghiu
08 Jun 2023
9 min read
IntroductionRumor has it that around 30 million PowerPoint presentations are being created each day. Oh, PowerPoint, the sturdy old friend in the world of presentations. You might ask, why, in this fast-paced, ever-evolving digital world, do people still reach out for PowerPoint? Well, let me break it down for you, because honestly, PowerPoint has still got game! It’s simple, universal (in the realm of presentations), easy to learn and pretty well integrated with the other two Microsoft musketeers (Word & Excel).So, sure, there are fancier new alternatives out there (like Prezi or Canva), each with their unique features and slick interfaces. But when it comes to a tried-and-true solution for your presentation needs, you might find that, like a well-loved pair of sneakers, PowerPoint fits just right.And here’s the news: the little AI based automation I am going to show you in this article might actually change your mind about the capabilities of PowerPoint and make you reconsider using much more modern alternatives for your presentations.Working as a trainer for the last 15 years, I’ve probably seen more PowerPoint than other people do. I used it daily both for studying and teaching, whether for 15-minute presentations or 5-day long courses. I’ve always thought the greatest power of PowerPoint was not in its UI, design features or the number of export formats but rather in the VBA (Visual Basic for Applications) core functionality it shares with the other tools in the Microsoft ecosystem. And yet, so much potential lays unused in this regard.Oh, did you think that VBA is dead? I’m going to prove you wrong.A key characteristic for any slide-based presentation is that the viewer only gets a glimpse of the actual knowledge in the slide. It’s more visually appealing that way, but you need a presenter to understand the story and connect the dots. And usually, the presenter needs to use some slide notes for a more cursive and natural presentation. Slide notes are like a secret teleprompter. They give you reminders, help you with the flow of your presentation, and keep you on point. They're your discreet cheat-sheet, making sure you never miss a beat during your presentation.In this article, I am going to show you how, with just a few lines of code, you can infuse your existing PowerPoint presentations with the ability to automatically generate slide notes based on your slide contents.How it worksMy approach might seem a bit crude but it is effective. Especially if your presentation contains general concepts that might fall within the knowledge span of GPT3.5 Turbo from OpenAI (or even GPT-4 if you have an API key). More niche topics, or a slide having just a single bullet of words without too much context, may not produce the best results. Afterall, it’s all automated and no matter how smart the AI model may be, it still needs context to understand the task.For those who might be unfamiliar with the GPT3.5 Turbo API: this API provides programmatic access to the same advanced language model used by ChatGPT. Its purpose in the script is to explain the concepts in a PowerPoint slide.So, the basic approach is to create a VBA macro in the presentation that performs the following:Extracts all text from the current slide.Formats the entire text and builds a specially designed prompt with it.Submits the prompt to OpenAI’s API for processing.Appends the AI answer received from the API to the current slide’s notes.Because it’s a macro, once you’ve created it you can assign a shortcut and repeat the process for each slide.Alternatively, with a bit of tweaking you could adjust the code to iterate through all the slides and create all slide notes in one go. But keep in mind there is a cost for every API call you make and for a large presentation, the cost might easily spin out of control. BeforeAfterImagine the amount of time you can save by starting with some quick AI-generated draft notes and then reviewing and adding more context instead of having to type everything in from scratch. This productivity hack can save you hours of your time or even help you make sense of presentations created by someone else that lack any explanations or trainer notes. Before you beginSome things you need to take care of before you start writing the code:Make sure you store your OpenAI API key in a local environment variable for secure and efficient access. The code works on the assumption that the API key is stored in your local environment (OPENAI_API_KEY).Enable the Developer tab on your PowerPoint. By default, this tab is not visible. To enable it and be able to create VBA code you need to go to File > Options > Customize Ribbon and select Main Tabs > Developer > Add. I know. That’s enough reason to hate PowerPoint. But at least, the good news is you only have to do it once. Once you’ve got that solved you may proceed with the next steps:Go to the Developer Tab and select Visual Basic.From the Insert menu choose Module.Add the code from the next section in the newly created module and save the file. You’ll probably be notified that the .PPTX extension does not support macros. No worries. Just click No, select .PPTM (Macro enabled presentation) as your file type and save. Make sure you customize the code first to provide the prompt with more context on your specific presentation (see the comments in the code).And that’s it. You’re good to go.Now, select Developer > Macros and you should see a new macro called QueryGpt.Just go to your desired slide and run the macro. Wait for a few seconds and you should have your notes populated with AI’s interpretation of your slide. Be mindful though, the code is simple. There’s no error checking and I haven’t tested it on hundreds of PPTs.You can download the .BAS source file for this article from here. Pro Tip: If you want to create an easy shortcut to your macro you can click on the last icon in the very top ribbon menu and select More commands and then from Choose commands from select Macros, choose the QueryGpt macro and Add. A new button for the macro will be permanently added on your ribbon.The codeNo more talking. Here’s the macro:Sub QueryGpt() Dim sld As Slide Dim shp As Shape Dim text As String Dim explanation As String Dim prompt As String Dim subject As String ' Extract and format the text from the current slide Set sld = ActiveWindow.View.Slide For Each shp In sld.Shapes If shp.HasTextFrame Then If shp.TextFrame.HasText Then text = text & " [" & Replace(shp.TextFrame.TextRange.text, vbCrLf, "/") & "] " End If End If Next shp text = Replace(Replace(Replace(text, vbCrLf, "\n"), vbCr, "\r"), vbLf, "\n") ' Prepare the prompt and call the function that queries the AI model subject = "The concept of Public Domain" prompt = "You are a useful assistant that explains the concepts from a presentation about " prompt = prompt & subject & ". Explain the following concepts: " explanation = CallOpenAI(prompt & text) ' Append the AI explanation to the slide notes sld.NotesPage.Shapes.Placeholders(2).TextFrame.TextRange.InsertAfter vbCrLf & explanation End Sub Take note of the variable called subject.That is the main topic of your presentation, and you should change that to the subject of your slides.That way the AI model gets more context about your slides and can provide better descriptions in the notes.You should be able to achieve better results by adding more details in that variable but consider the maximum context of the model (for GPT 3.5 Turbo that is around 3000 English words). The more you add to your initial input, the more you limit your result’s maximum length because the whole context cannot exceed ~3000 words.Next, here is the function responsible for making the API call and formatting the resulting answer:Function CallOpenAI(text As String) As String Dim httpRequest As Object Dim responseText As String Dim contentStartPos As Long Dim contentEndPos As Long ' Prepare the connection and send the request to the OpenAI API Set httpRequest = CreateObject("WinHttp.WinHttpRequest.5.1") httpRequest.Open "POST", "https://api.openai.com/v1/chat/completions", False httpRequest.setTimeouts 0, 60000, 30000, 120000 httpRequest.setRequestHeader "Content-Type", "application/json" httpRequest.setRequestHeader "Authorization", "Bearer " & Environ("OPENAI_API_KEY") httpRequest.Send "{""model"": ""gpt-3.5-turbo"", " _ & """messages"": [{""role"": ""user"", " _ & """content"": """ & text & """}]}" httpRequest.WaitForResponse responseText = httpRequest.responseText ' Extract the AI answer from the response string contentStartPos = InStr(1, responseText, """content"":""") + 11 responseText = Mid(responseText, contentStartPos) contentEndPos = InStr(1, responseText, """") - 1 responseText = Replace(Mid(responseText, 1, contentEndPos), "\n", vbCrLf) CallOpenAI = responseText End Function All you have to do to add this code to the presentation is to follow steps 3 to 5 above.Once you’ve created the module and saved the presentation, you’ll have the new macro called QueryGpt.If you’ve created the shortcut by following the pro tip, you’ll also have a neat way of calling the macro whenever you need to append some slide notes to the current slide.And there you go: Old-school PowerPoint presentations infused with AI super-powers.Feels like converting an old Diesel to an EV, doesn’t it? DisclaimerAs with any AI generated content do not forget to check the outputs for accuracy and potential hallucinations. (https://www.packtpub.com/page/4-ways-to-treat-a-hallucinating-AI-with-Prompt-Engineering_). Do not forget that any bias or inaccurate data in the slide content may get amplified in the outputs. Author BioAndrei Gheorghiu is an experienced trainer with a passion for helping learners achieve their maximum potential. He always strives to bring a high level of expertise and empathy to his teaching.With a background in IT audit, information security, and IT service management, Andrei has delivered training to over 10,000 students across different industries and countries. He is also a Certified Information Systems Security Professional and Certified Information Systems Auditor, with a keen interest in digital domains like Security Management and Artificial Intelligence.In his free time, Andrei enjoys trail running, photography, video editing and exploring the latest developments in technology.You can connect with Andrei on:LinkedIn: https://www.linkedin.com/in/gheorghiu/Twitter: https://twitter.com/aqg8017  
Read more
  • 0
  • 0
  • 181
article-image-using-llamaindex-for-ai-assisted-knowledge-management
Andrei Gheorghiu
08 Jun 2023
10 min read
Save for later

Using LlamaIndex for AI-Assisted Knowledge Management

Andrei Gheorghiu
08 Jun 2023
10 min read
IntroductionOne of the hottest questions of the moment for a lot of strategy and decision makers across most industries worldwide is:How can AI help my business?Afterall, with great disruption also comes great opportunity. A sound business strategy should not ignore emerging changes in the market. We’re still at the early stages of understanding AI and I’m not going to provide a definitive answer to this question in this article, but the good news is that this article should provide a part of the answer.Knowledge is power, right?And yet we all know how it is to struggle trying to retain and efficiently re-use the knowledge we gather.We strive to learn from our successes and mistakes and we invest a lot of time and money in building fancy knowledge bases just to discover later that unfortunately we keep repeating the same mistakes and reinventing the wheel.In my experience as a consultant, the biggest issue (especially for medium and large companies) is not a lack of knowledge but on the contrary, it’s too much knowledge and an inability to use it in a timely and effective manner. The solutionThis article presents a very simple yet effective way of indexing large quantities of pre-existing knowledge that can later be retrieved by natural language queries or integrated with chatbot systems.As usual, take it as a starting point. The code example is trivial and lacks any error handling, but provides the building blocks to work from.My example builds on your existing knowledge base and leverages LlamaIndex and the power of Large Language Models (in this case GPT 3.5 Turbo from OpenAI).Why LlamaIndex? Well, created by Jerry Liu, LlamaIndex is a robust open-source resource that empowers you to organize and search your data for a variety of applications, including answering questions, summarizing information or serving as a part of a chatbot system. It provides data connectors to ingest your existing data sources in many different formats (such as text files, PDF, docs, SQL, etc.). It then allows you to structure your data (via indices or graphs) so that this data can be easily used with LLMs. In many ways, it is similar to Langchain but more focused on data storage and retrieval instead of automated AI agents.In short, this article will show you how, with just a few lines of code, you can index your enterprise's knowledge base and then have the ability to query and retrieve information from GPT 3.5 Turbo with your own knowledge base on top of that in the most natural way: plain English. Logic diagramCreating the IndexRetrieving the knowledge PrerequisitesMake sure you check these points before you start writing the code:Make sure you store your OpenAI API key in a local environment variable for secure and efficient access. The code works on the assumption that the API key is stored on your local environment (OPENAI_API_KEY).I’ve used Python v3.11. If you’re running an older version, an update is recommended to make sure you don’t run into any compatibility issues.Install the requirements:pip install openaipip install llama-index Create a subfolder in your .PY file’s location (in my example the name of the subfolder is ‘stories’). You will store your knowledge base in .TXT files in that location. If your knowledge articles are in different formats (e.g., PDF or DOCX) you will have to:Change the code to use a different LlamaIndex data connector (https://gpt-index.readthedocs.io/en/latest/reference/readers.html) – this is my recommended solution – or:Convert all your documents in .TXT format and use the code as it is.For my demo, I have created (with the help of GPT-4) three fictional stories that will represent our proprietary ‘knowledge’ base: Your ‘stories’ folder should now look like this: The CodeFor the sake of simplicity, I’ve split the functionality into two different scripts:Index_stories.py (responsible for reading the ‘stories’ folder, creating an index and saving it for later queries)Query_stories.py (demonstrating how to query GPT 3.5 and then filter the AI response through our own knowledge base)Let’s begin with Index_stories.py:from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader # Loading from a directory documents = SimpleDirectoryReader('stories').load_data() # Construct a vector store index index = GPTVectorStoreIndex.from_documents(documents) # Save your index to a .json file index.storage_context.persist()As you can see, the code is using SimpleDirectoryReader from LlamaIndex to read all .TXT files from the ‘stories’ folder. It then creates a simple vector index that can later be used to run queries over the content of these documents.In case you’re wondering what a vector index represents, imagine you're in a library with thousands of books, and you're looking for a specific book. Instead of having to go through each book one by one, this index acts in a similar way to a library catalog.  It helps you find the book you're looking for quickly.In the context of this code, GPTVectorStoreIndex is like that library catalog. It's a tool that helps organize and find specific pieces of information (like documents or stories) quickly and efficiently. When you ask a question, it looks through all the information it has and finds the most relevant answer for you. It's like a super-efficient librarian that knows exactly where everything is.The last line of the code saves the index in a sub-folder called ‘storage’ so that we do not have to recreate it every time and we are able to reuse it in the future. Now, for the querying part. Here’s the second script: Query_stories.py:from llama_index import GPTVectorStoreIndex, StorageContext, load_index_from_storage import openai import os openai.api_key = os.getenv('OPENAI_API_KEY') def prompt_chatGPT(task): response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": task} ] ) AI_response = response['choices'][0]['message']['content'].strip() return AI_response # rebuild storage context storage_context = StorageContext.from_defaults(persist_dir="storage") # load index index = load_index_from_storage(storage_context) # Querying GPT 3.5 Turbo prompt = "Tell me how Tortellini Macaroni's brother managed to conquer Rome." answer = prompt_chatGPT(prompt) print('Original AI answer: ' + answer +'\n\n') # Refining the answer in the context of our knowledge base query_engine = index.as_query_engine() response = query_engine.query(f'The answer to the following prompt: "{prompt}" is :"answer". If the answer is aligned to our knowledge, return the answer. Otherwise return a corrected answer') print('Custom knowledge answer: ' + str(response))How it worksAfter indexing the ‘stories’ folder, once you run the query_stories.py script the code will first load the index from the ‘storage’ sub-folder. It then prompts the GPT 3.5 Turbo model with a hard-coded question: “Tell me how Tortellini Macaroni’s brother managed to conquer Rome”. After the response is received, it queries our ‘stories’ to see if the answer aligns with our ‘knowledge’. Then, you’ll receive two answers.The first one is the original answer from GPT 3.5 Turbo:As expected, the AI model identified Mr. Spaghetti as a potentially fictional character and could not find any historical references of him conquering Rome.The second answer though, checks with our ‘knowledge’ and, because we have different information in our ‘stories’, it modifies the answer into:If you’ve read the three GPT-4-created stories you’ve noticed that Story1.txt mentions Biscotti as a fictional conqueror of Rome but not his brother and Story2.txt mentions Tortellini and his farm adventures but does not mention any relationship with Biscotti. Only the third story (Story3.txt) describes the nature of their relationship.This shows not only that the vector index managed to correctly record the knowledge from the individual stories but also proves the query function managed to provide a contextual response to our question.In addition to the Vector Store Index, there are several other types of indexes that can be used depending on the specific needs of your project.For instance, the List Index simply stores Nodes as a sequential chain, making it a straightforward and efficient choice for certain applications, such as where the order of data matters and where you frequently need to access all the data in the order it was added. An example might be a timeline of events or a log of transactions, where you often want to retrieve all entries in chronological order.Another option is the Tree Index which builds a hierarchical tree from a set of Nodes, which can be particularly useful when dealing with complex, nested data. For instance, if you're building a file system explorer, a tree index would be a good choice because files and directories naturally form a tree-like structure.There’s also the Keyword Table Index, which extracts keywords from each Node and builds a mapping from each keyword to the corresponding Nodes. This can be a powerful tool for text-based queries, allowing for quick and precise retrieval of relevant information.Each of these indexes offers unique advantages, and the choice between them would depend on the nature of your data and the specific requirements of your use case. ConclusionNow, think about the possibilities. Instead of fictional stories, we could have a collection of our standard operating procedures, process descriptions, knowledge articles, disaster recovery plans, change schedules and so on and so forth. Or, as another example we could build a chatbot that can solve generic users requests simply using GPT3.5 knowledge but forwards more specific issues (indexed from our knowledge base) to a support team.This brings unlimited potential in automation of our business processes and improvement of the decision-making process. You get the best of both worlds: the power of Large Language Models combined with the value of your own knowledge base. Security considerationsWorking on this article made me realize that we cannot really trust our interactions with an AI model unless we are in full control of the entire technology stack. Just because the interface might look familiar, it doesn’t necessary mean that bad actors cannot compromise the integrity of the data by injecting false or censored responses to our queries. But that’s a story for another time! Final noteThis article barely scratches the surface of the full capabilities of LlamaIndex. It is not meant to be a comprehensive guide into this topic but rather serve as an example start point for integrating AI technologies in our day-to-day business processes. I encourage you to have an in-depth study of LlamaIndex’s capabilities (https://gpt-index.readthedocs.io/en/latest/) if you want to take advantage of its full capabilities.About the AuthorAndrei Gheorghiu is an experienced trainer with a passion for helping learners achieve their maximum potential. He always strives to bring a high level of expertise and empathy to his teaching.With a background in IT audit, information security, and IT service management, Andrei has delivered training to over 10,000 students across different industries and countries. He is also a Certified Information Systems Security Professional and Certified Information Systems Auditor, with a keen interest in digital domains like Security Management and Artificial Intelligence.In his free time, Andrei enjoys trail running, photography, video editing and exploring the latest developments in technology.You can connect with Andrei on:LinkedIn: https://www.linkedin.com/in/gheorghiu/Twitter: https://twitter.com/aqg8017 
Read more
  • 0
  • 0
  • 226

article-image-writing-unit-tests-in-c-with-ease-using-chatgpt
Jesse Liberty
08 Jun 2023
9 min read
Save for later

Writing Unit Tests in C# with ease using ChatGPT

Jesse Liberty
08 Jun 2023
9 min read
The world is abuzz with talk of large language model AI, especially its iconic ChatGPT. No one is more excited and perhaps threatened than programmers. Will ChatGPT soon write code well enough to change or even eliminate our jobs? In this article, I set such concerns aside and focus on how ChatGPT can enhance our programming today. One of the places I find ChatGPT most useful is in the writing of unit tests. To illustrate this, I'm going to dip into the source code used in my latest book: .NET MAUI For C# Developers. If you haven't read my book (actually, why not stop here and run out and buy it? I can wait here) it illustrates the creation of a helpful app that, among other things, requires you to create an account. Here is the code for doing so:  [RelayCommand] async Task SignUp() {    if (EnableButton)    {      AccountCreateRequest accountCreateRequest = new()      {        Email = this.Email,        FullName = Name,        PlainPassword = Password      };      try      {        CreateAccountIsRunning = true;        await accountService.CreateAccount(accountCreateRequest);        CreateAccountIsRunning = false;        await Application.Current.MainPage.DisplayAlert(         "Sign up completed",        "Your user has been created successfully", "Ok");        await Shell.Current.GoToAsync("..");      }      catch (Exception e)      {          CreateAccountIsRunning = false;        await Application.Current.MainPage.DisplayAlert(         "Sign up failed",        "We were not able to create an account with that user name",         "Ok");      }    } } The logic here is simple, I've selected it specifically so that you can easily imagine how you would write unit tests. Let's see how ChatGPT does when presented with this code. I opened ChatGPT and entered the following message:  Write two unit tests with xUnit and nSubstitute in C# for the following .NET MAUI code (I pasted in the code. It responded so quickly it took my breath away)Sure! Here are two unit tests using xUnit and NSubstitute for the SignUp() method: I particularly like the self-assured and friendly "Sure!" at the beginning, and the restatement of what it has been asked to do. Its results come flying out in a box that has a handy Copy Code button on the top right, and a reminder on the top left that the code is in C# as shown in the figure:   Figure 1 – top of ChatGPT response  Notice that it marks the first section. // Arrange As you'll see it uses and annotates the AAA approach to unit tests (Arrange, Act, Assert). Here are the two-unit tests it produced: public class YourViewModelTests {    [Fact]    public async Task SignUp_WhenEnableButtonIsFalse_DoesNotCallAccountServiceAndDisplaysNoAlert()    {        // Arrange        var accountService = Substitute.For<IAccountService>();        var applicationMock = Substitute.For<Application>();        var mainPageMock = Substitute.For<Page>();        var shellMock = Substitute.For<Shell>();        applicationMock.MainPage = mainPageMock;        Shell.Current = shellMock;        var viewModel = new YourViewModel(accountService)        {            Email = "test@example.com",            Name = "John Doe",            Password = "password",            EnableButton = false        };        // Act        await viewModel.SignUp();        // Assert        await accountService.DidNotReceive().CreateAccount(Arg.Any<AccountCreateRequest>());        await applicationMock.MainPage.DidNotReceive().DisplayAlert(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<string>());        await shellMock.DidNotReceive().GoToAsync(Arg.Any<string>());    }    [Fact]    public async Task SignUp_WhenEnableButtonIsTrue_CallsAccountServiceAndDisplaysAlertOnError()    {        // Arrange        var accountService = Substitute.For<IAccountService>();        accountService            .When(x => x.CreateAccount(Arg.Any<AccountCreateRequest>()))            .Throw(new Exception("Failed to create account"));        var applicationMock = Substitute.For<Application>();        var mainPageMock = Substitute.For<Page>();        var shellMock = Substitute.For<Shell>();        applicationMock.MainPage = mainPageMock;        Shell.Current = shellMock;        var viewModel = new YourViewModel(accountService)        {            Email = "test@example.com",            Name = "John Doe",            Password = "password",            EnableButton = true        };        // Act        await viewModel.SignUp();        // Assert        await accountService.Received(1).CreateAccount(Arg.Is<AccountCreateRequest>(request =>            request.Email == viewModel.Email &&            request.FullName == viewModel.Name &&            request.PlainPassword == viewModel.Password));        await applicationMock.MainPage.Received(1).DisplayAlert("Sign up failed",            "We were not able to create an account with that user name", "Ok");        await shellMock.DidNotReceive().GoToAsync(Arg.Any<string>());    } }  After the code comes to these handy instructions and reminders: In the first test case, SignUp_WhenEnableButtonIsFalse_DoesNotCallAccountServiceAndDisplaysNoAlert(), we set EnableButton to false and verify that the SignUp() method does not call CreateAccount() on the accountService, and no alerts are displayed. In the second test case, SignUp_WhenEnableButtonIsTrue_CallsAccountServiceAndDisplaysAlertOnError(), we set EnableButton to true and configure the accountService to throw an exception when CreateAccount() is called. We verify that the method calls CreateAccount(), displays the appropriate alert for failure, and does not navigate to another page using GoToAsync(). By following those instructions, I had two working, useful unit tests, literally in seconds (well, a lot of seconds, but only because it took me a minute to read and understand (and verify) what it had done.) The first time I made the request, I forgot to say it in xUnit and it spits out the tests in nUnit. I then entered, Do it again using xUnit And it didn't hesitate, out came the code in xUnit along with modified instructions. Summary In sum, my pair-programming buddy, ChatGPT, saved me a lot of tedious work creating unit tests. I can attest that it works equally well on more complex logic. It doesn't always get it right the first time, but you can "tune" it with follow-up questions, and it remembers what it just did, so rather than repeating the entire command you can just say, as I do above, Do it again, but this time… We used to say, Coding by Google is not a problem as long as you know what you are doing. The problem only arose with very junior programmers who took what they found online as-is without checking that it made sense. The same can be true here: ChatGPT can get you going, and you can often use its code as-is, but you need to keep a cautious eye on it as it certainly can get things wrong.  Writing unit tests is a critical, though often overlooked part of writing code well. ChatGPT not only makes it relatively easy, but it also encourages creating tests because you can do so quickly. It is so good that I happily pay $20/month for a subscription entitling me to unrestricted use and access to the latest iteration.  Author Bio  Jesse Liberty is a full-time hands-on programmer, specializing in C#, git, and .NET MAUI. He hosts the popular Yet Another Podcast and is the author of more than a dozen best-selling programming books. Liberty is a Certified Xamarin Developer, a Xamarin MVP, and a Microsoft MVP. He was a Technical Evangelist for Microsoft, a Distinguished Software Engineer at AT&T; a Software Architect for PBS, and a Vice President of Information Technology at Citibank, and he was on the teaching staff at Brandeis University. Jesse is a recognized expert and has spoken at conferences worldwide.  Links - LinkedIn  .NET MAUI for C# Developers  
Read more
  • 0
  • 0
  • 620