Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon

How-To Tutorials - Prompt Engineering

13 Articles
article-image-prompt-engineering-with-azure-prompt-flow
Shankar Narayanan SGS
06 May 2024
10 min read
Save for later

Prompt Engineering with Azure Prompt Flow

Shankar Narayanan SGS
06 May 2024
10 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionThe ability to generate relevant and creative prompts is one of the imperative aspects of the natural language processing system. Especially when the world is evolving in the landscape of artificial intelligence, it is one of the crucial prospects. During this situation, Microsoft's Azure prompt flow provides groundbreaking solutions while empowering the data, scientists, and developers to engineer prompts effectively.  Here, let us explore the nuances of Azure prompt flow while delving deep into the realm of prompt engineering. Significance of Prompt Engineering With the help of prompt engineering, one can construct problems, helping the user with the guide of machine learning models effectively. However, it involves Formulating contextually relevant and specific questions or statements that elicit the desired responses from the artificial intelligence models. Azure prompt flow is one of the sophisticated tools by Microsoft Azure that simplifies intricate processes while enabling the developers to create brands that can have meaningful and accurate outcomes. Getting started with Azure prompt flow Even before exploring the practical applications of Azure Prompt flow, it is necessary to understand the few essential components of Azure prompt flow. The core of prompt flow utilizes the GPT 3.5 architecture to generate various relevant responses to prompts. With the integration of Azure, one can expect a secure and seamless environment for prompt engineering.  Let us consider a practical example of a chatbot application.  from azure.ai.textanalytics import TextAnalyticsClient from azure.core.credentials import AzureKeyCredential # Set up Azure Text Analytics client key = "YOUR_AZURE_TEXT_ANALYTICS_KEY" endpoint = "YOUR_AZURE_TEXT_ANALYTICS_ENDPOINT" credential = AzureKeyCredential(key) text_analytics_client = TextAnalyticsClient(endpoint=endpoint, credential=credential) # User input user_input = "Tell me a joke." # Generate a prompt using Azure Promptflow prompt = f"User: {user_input}\nChatbot:" # Get chatbot's response response = text_analytics_client.analyze_sentiment(prompt) # Output the response print(f"Chatbot: {response[0]['sentiment']}") In this particular example, we can see that the user inputs a request. Azure prompt flow constructs the required form for the chatbot while generating a sentiment analysis response. Here is the output:Chatbot: Positive Tuning prompts using Azure Promptflow  Crafting good prompts can be a challenging task. With the concept of variants, the user would be able to test the behavior of the model under various conditions. Example:  If the user wants to create a chatbot using Azure Promptflow, then this example might help one to respond creatively to the queries about movies.  Prompt Tuning: User: "Tell me about your favorite movie." Chatbot: "Certainly! One of my favorite movies is 'Inception.' Directed by Christopher Nolan, it's a mind-bending sci-fi thriller that explores the depths of the human mind." Python code:  from azure.ai.textanalytics import TextAnalyticsClient from azure.core.credentials import AzureKeyCredential # Set up Azure Text Analytics client key = "YOUR_AZURE_TEXT_ANALYTICS_KEY" endpoint = "YOUR_AZURE_TEXT_ANALYTICS_ENDPOINT" credential = AzureKeyCredential(key) text_analytics_client = TextAnalyticsClient(endpoint=endpoint, credential=credential) # User input user_input = "Tell me about your favorite movie." # Generate a creative prompt using Azure Promptflow prompt = f"User: {user_input}\nChatbot: Certainly! One of my favorite movies is 'Inception.' Directed by Christopher Nolan, it's a mind-bending sci-fi thriller that explores the depths of the human mind." # Get chatbot's response response = text_analytics_client.analyze_sentiment(prompt) # Output the response print(f"Chatbot: {response[0]['sentiment']}") In this example, Azure Promptflow is used to create prompts tailored to specific user queries, providing creative and contextually relevant responses. The analyze_sentiment function from the Azure Text Analytics client is used to assess the sentiment of the generated prompts. Replace "YOUR_AZURE_TEXT_ANALYTICS_KEY" and "YOUR_AZURE_TEXT_ANALYTICS_ENDPOINT" with your actual Azure Text Analytics API key and endpoint. Here are a few examples: URL: https://music.apple.com/us/app/apple-music/id1108187390 Text Content: Apple Music is a comprehensive music streaming app that boasts an extensive library of songs, albums, and playlists. Users can enjoy curated playlists, radio shows, and exclusive content from their favorite artists. Apple Music allows offline downloads and offers a family plan for multiple users. It also integrates with the user's existing music library, making it seamless to access purchased and uploaded music. OUTPUT: {"category": "App", "evidence": "Both"}​​​​ URL: https://www.youtube.com/user/premierleague Text Content: Premier League Pass, in collaboration with the English Premier League, delivers live football matches, highlights, and exclusive behind-the-scenes content on YouTube. Football aficionados can stay updated with their favorite teams and players through this official channel. Subscribing to Premier League Pass on YouTube ensures fans never miss a moment from the most exciting football league in the world. OUTPUT: {"category": "Channel", "evidence": "URL"} URL: https://arxiv.org/abs/2305.06858 Text Content: This research paper explores the realm of image captioning, where advanced algorithms generate descriptive captions for images. The study delves into techniques that combine computer vision and natural language processing to achieve accurate and contextually relevant image captions. The paper discusses various models, evaluates their performance, and presents findings that contribute to the field of image captioning technology. OUTPUT: {"category": "Academic", "evidence": "Text content"} URL: https://exampleconstructionsite.com/ Text Content: This website is currently under construction. Please check back later for updates and exciting content. OUTPUT: {"category": "None", "evidence": "None"}  For a given URL: {{url}}, and text content: {{text_content}}. Classified Category: Travel Evidence: The text contains information about popular tourist destinations, travel itineraries, and hotel recommendations. OUTPUT: After summarizing, here is the final Promptflow with 2 variants for the summarize_text_content node.  Benefits of using Azure ML prompt flow Apart from offering a wider range of benefits, Azure ML promptflow helps users to make the transition from ideation to experimentation. This ultimately results in production ready LLM based applications.  Prompt engineering agility  Azure prompt flow offers a visual representation of the struct of the flow structure. It allows the users to understand and navigate the projects while offering a notebook-like coding experience for debugging and efficient flow development. At the same time, users can create as well as compare more than one prompt variant which helps in facilitating an iterative refinement process. Enterprise readiness  The prompt flow streamlines the entire prompt engineering process and leverages robust enterprise readiness solutions. It thus offers a secure, reliable, and scalable foundation for experimentation and development. Besides, it supports team collaboration where multiple users can work together, share knowledge, and maintain version control. Application development  The well-defined process of Azure prompt facilitates the seamless development of AI applications. Only by leveraging it the user can progress effectively through the consequent stages of developing, testing, tuning, and deploying flows. All these ultimately result in creating a fully-fledged AI applications.   However, when the user follows this methodical and structured approach, it empowers them to develop fine-tune and test rigorously to deploy with confidence. Real-world applications of Azure Promptflow  Content creation One of the applications of Azure promptflow lies in the content creation tunes. Various content creators can generate outlines and creative ideas by creating engineering tailored to specific topics. One can even generate entire paragraphs using the prompt flow engineering method. This helps streamline the content creation process while making it look more inspiring and efficient. Language Translation Developers are now leveraging Azure promptflow to build large language translation applications. With the help of constructing prompts in the source language, one can let the system translate the inputs by providing accurate outputs required in the desired language. Such a profound implication can only be possible with the help of Azure prompt flow. It has the propensity to break all the language barriers in the globalized world. Custom support chat box By integrating Azure prompt flow within the customer support chatbots, one can enhance the user experience. However, the prompt engineering techniques help ensure the queries are accurately understood. This process would result in relevant and precise responses. It significantly reduces the response time while improving customer satisfaction. Azure prompt flow simplifies prompt engineering   Prompt engineering is an iterative and challenging process. With the help of Azure prompt flow, one can simplify the development, comparisons, and evaluation of problems. The process makes it easier for the user to find the best prompt for use cases.  Besides, developing a chatbot that utilizes large language models, including GPT3.5, can help companies provide personalized product recommendations based on customer input. Here, Azure prompt flow allows users to evaluate, create, and even deploy from the machine learning models. It speeds up the whole process of developing and deploying artificial intelligence solutions. At the same time, it also allows the user to create connections to the large language model.   Such models include GPT 3.5 and Azure open AI. Users can also use these models for different purposes, including chat computation or creating embeddings. Designing and modifying prompts Designing and modifying alarms for effective use is crucial, especially when using them for large language models. Azure prompt flow enables users to test, create, and deploy various prompt versions for recommendation purposes. To effectively utilize the large language model, especially while dealing with multiple prompts, it is imperative to modify them and design accordingly for better results. Once you can create the problems, it is time to evaluate and test them in multiple scenarios. For instance, if you are creating prompts for a product company, you must explain the process of prompts and their flow to handle the user queries. Also, one can mention the need for custom coding and deployment of end-to-end solutions with the help of Azure's prompt flow feature. Conclusion  With powerful prompt engineering capabilities, Azure prompt flow enables the developers to construct contextually relevant prompts. It enhances the efficiency and accuracy of AI applications over various domains. The potential of prompt engineering makes the future of AI development promising. However, it can only be possible with the help of Azure AI leading the way. Author BioShankar Narayanan (aka Shanky) has worked on numerous different cloud and emerging technologies like Azure, AWS, Google Cloud, IoT, Industry 4.0, and DevOps to name a few. He has led the architecture design and implementation for many Enterprise customers and helped enable them to break the barrier and take the first step towards a long and successful cloud journey. He was one of the early adopters of Microsoft Azure and Snowflake Data Cloud. Shanky likes to contribute back to the community. He contributes to open source is a frequently sought-after speaker and has delivered numerous talks on Microsoft Technologies and Snowflake. He is recognized as a Data Superhero by Snowflake and SAP Community Topic leader by SAP.
Read more
  • 0
  • 0
  • 495

article-image-improving-ai-context-with-rag-using-azure-machine-learning-prompt-flow
Ryan Goodman
18 Dec 2023
11 min read
Save for later

Improving AI Context with RAG Using Azure Machine Learning prompt flow

Ryan Goodman
18 Dec 2023
11 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionRetrieval Augmented Generation, or RAG, is a method that can expand the breadth or depth of information for Large Language Models (LLM). Retrieving and delivering more data to an LLM will result in applications with more contextual, relevant information. Unlike traditional web or mobile applications designed to retrieve structured data, RAG (Retrieval-Augmented Generation) requires data to be structured and indexed, and stored differently, most commonly in a vector database. The resulting experience should provide more contextual information with the added ability to cite the source of information. This narrower scope of information can result in a higher degree of accuracy and utility for your enterprise. To summarize RAG:Retrieval: When a question or formulated to an LLM-powered chat bot, RAG reviews the index to find relevant facts. This is like searching through an index where all the information is neatly summarized for quick access.Augmentation: It then takes these facts and feeds them to the language model, essentially giving it a brief on the subject matter at hand.Generation: With this briefing, the language model is now ready to craft a response that's not just based on what it already knows but also on the latest information it has just pulled in. In short, RAG keeps language models up-to-date and relevant, providing answers that are informed by the latest available data. It's a practical way to ensure AI remains accurate and useful for your organization, especially when dealing with current and evolving topics that may not be public knowledge. In this article, we will explore the differences between RAG and Fine tuning and how you can organize your RAG solution using Azure Machine Learning prompt flow. Retrieval Augmented Generation (RAG) vs Fine TuningWhen working with large language models through chat interfaces like ChatGPT, you will see that its foundational knowledge is point-in-time data. Fine-tuning, on the other hand, is akin to customizing the model with a new layer of knowledge that reflects your specific data, which becomes part of the model's intelligence. As of this article, OpenAI released GPT 4 Turbo which is based on a data set through April 2023. Extending an LLM’s body of knowledge can involve fine-tuning or RAG.Fine Tuning Foundational ModelsFine-tuning involves training a foundational model on a dataset specific to your application, effectively customizing the model to perform better for certain tasks or styles. To fine-tune a model, you need to have a dataset that represents the task or style you are aiming for and the computational resources to perform the training.Once fine-tuned, the model's knowledge is enhanced, and these changes are permanent unless the model is fine-tuned again with additional data. Fine-tuning is ideal for tasks needing deep customization where the information may be specialized but require re-training infrequently. While OpenAI has started to offer fine-tuning for certain models like GPT-3.5, not all foundational models or versions can be fine-tuned due to access restrictions to their parameters and training regimes.Retrieval Augmented GenerationRAG is like adding a live feed of information to the foundational model, enabling it to respond with the latest data without modifying the model itself. It involves augmenting a foundational language model’s response by dynamically integrating information retrieved from an external database, typically a vector database, at the time of the query.No Model Training Required: The foundational model's core parameters remain unchanged. Instead, RAG serves as a real-time data layer that the model queries to inform its responses.Real-time and Up to Date: Because RAG queries external data sources in real-time, it ensures that the language model's responses are enhanced by the most current and relevant information available.Image Credit: https://medium.com/@minh.hoque/retrieval-augmented-generation-grounding-ai-responses-in-factual-data-b7855c059322 RAG is widely adopted as the best starting point due to the following factors:Data Dynamics: Choose RAG for frequently changing data and fine-tuning for static, specialized domains. Like any data wrangling and model training problem, the results are only as good as your data quality.Resource Availability: With RAG you do not need expansive computational resources and budget like fine-tuning. You will still need skilled resources to implement and test RAG.Flexibility and Scalability: RAG offers adaptability to continuously add current information and ease of maintenance.Approaching RAG with Azure Machine Learning Prompt FlowWith a solid foundation of RAG vs fine-tuning, we will dive into the details of an approach to Retrieval Augmented Generation within Azure. Azure provides multiple solutions for creating and accessing vector indexes per Microsoft’s latest documentation. Azure offers 3 methods currently: Azure AI Studio, use a vector index and retrieval augmentation.Azure OpenAI Studio, use a search index with or without vectors.Azure Machine Learning, use a search index as a vector store in a prompt flow.Azure's approach to RAG lets you tailor the model to your business needs and integrate in private or public facing applications. What remains consistent is the ability to prepare and feed your data into the LLM of your choice. Within Azure Machine Learning Prompt Flow, Microsoft includes a number of practical features including a fact-checking layer alongside the existing model to ensure accuracy. Additionally, you can feed supplementary data directly to your large language models as prompts, enriching their responses with up-to-date and relevant information. Azure Machine Learning simplifies the process to augment your AI powered app with the latest data without the time and financial burdens often associated with comprehensive model retraining.  A benefit of using these services is the scalability and security and compliance functions that are native to Azure. A standard feature of Azure Machine Learning for ML models or LLMs is a point and click flow or notebook code interface to build your AI pipelines1.  Data Acquisition and Preparation with Azure Services for Immediate LLM Access:Azure Blob Storage for Data Storage is perfect for staging your data. These files can be anything from text files to PDFs.2. Vectorization and Indexing of your Data Using AI Studio and Azure AI SearchThis is a step that can be completed using one of multiple approaches including both open source and Azure native. Azure AI Studio significantly simplifies the creation and integration of a vector index for Retrieval-Augmented Generation (RAG) applications. Here are the main steps in the process:Initialization: Users start by selecting their data sources in Azure AI Studio, choosing from blob storage for easier testing and local file uploads.Index Creation: The platform guides users through configuring search settings and choosing an index storage location, with a focus on ease of use and minimal need for manual coding.This is one of many examples of how Azure AI Studio is democratizing the use of advanced RAG applications by merging and integrating different services in the Azure cloud together.                                                                                    SOURCE: Microsoft3. Constructing RAG Pipelines with Azure Machine Learning:Simplified RAG Pipeline Creation: With your index created, you can integrate it along with AI search as a plug-and-play component into your Prompt flow. With no/low code interface, you can drag and drop components to create your RAG pipeline.                                                                               Image Source: Microsoft.com Customization with Jupyter Notebooks: For those who are comfortable coding in Jupyter notebooks, Azure ML offers the flexibility to utilize Jupyter Notebooks natively. This will provide more control over the RAG pipeline to fit your project's unique needs. Additionally, there are other alternative flows that you can construct using libraries like LangChain as an alternative to using the Azure services.3. Manage AI Pipeline OperationsAzure Machine Learning provides a foundation designed for iterative and continuous updates. The full lifecycle for model deployment includes test data generation and prompt evaluation. ML and AI operations are needed to understand certain adjustments. For organizations already running Azure ML, prompt flow fits nicely into broader machine learning operations.Integrating RAG workflow into MLOPs pipeline through codes from azureml.core import Experiment from azureml.pipeline.core import Pipeline from azureml.pipeline.steps import PythonScriptStep # Create an Azure Machine Learning experiment experiment_name = 'rag_experiment' experiment = Experiment(ws, experiment_name) # Define a PythonScriptStep for RAG workflow integration rag_step = PythonScriptStep(name='RAG Step',                            script_name='rag_workflow.py',                            compute_target='your_compute_target',                            source_directory='your_source_directory',                            inputs=[rag_dataset.as_named_input('rag_data')],                            outputs=[],                            arguments=['--input_data', rag_dataset],                            allow_reuse=True) # Create an Azure Machine Learning pipeline with the RAG step rag_pipeline = Pipeline(workspace=ws, steps=[rag_step]) # Run the pipeline as an experiment pipeline_run = experiment.submit(rag_pipeline) pipeline_run.wait_for_completion(show_output=True) Here is the code snippets to create and manage data using Azure. from azureml.core import Dataset # Assuming you have a dataset named 'rag_dataset' in your Azure Machine Learning workspace rag_dataset = Dataset.get_by_name(ws, 'rag_dataset') # Split the dataset into training and testing sets train_data, test_data = rag_dataset.random_split(percentage=0.8, seed=42) # Convert the datasets to pandas DataFrames for easy manipulation train_df = train_data.to_pandas_dataframe() test_df = test_data.to_pandas_dataframe() ConclusionIt is important to note that the world of AI and LLMs is evolving at a rapid pace where months make a difference.  Azure Machine Learning for Retrieval Augmented Generation offers a transformative approach to leveraging Large Language Models and provides a compelling solution for enterprises that already have a competency center. Azure ML machine learning pipelines for data ingestion, robust training, management, and deployment capabilities for RAG is lowering the barrier for dynamic data integration with LLMs like OpenAI. As adoption continues to grow, we will see lots of exciting new use cases and success stories coming from organizations that adopt early and iterate fast. The benefit of Microsoft Azure is a single, managed and supported suite of services some of which already may be deployed within your organization. Azure services to support new AI adoption demands, Retrieval Augmented Generation included! Author BioRyan Goodman has dedicated 20 years to the business of data and analytics, working as a practitioner, executive, and entrepreneur. He recently founded DataTools Pro after 4 years at Reliant Funding, where he served as the VP of Analytics and BI. There, he implemented a modern data stack, utilized data sciences, integrated cloud analytics, and established a governance structure. Drawing from his experiences as a customer, Ryan is now collaborating with his team to develop rapid deployment industry solutions. These solutions utilize machine learning, LLMs, and modern data platforms to significantly reduce the time to value for data and analytics teams.
Read more
  • 0
  • 0
  • 615

article-image-vector-datastore-in-azure-machine-learning-promptflow
Karthik Narayanan Venkatesh
27 Oct 2023
10 min read
Save for later

Vector Datastore in Azure Machine Learning Promptflow

Karthik Narayanan Venkatesh
27 Oct 2023
10 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights. Don't miss out – sign up today!IntroductionAzure machine learning prompt flow is one of Microsoft's cutting-edge solutions. It has paved the path for handling streamlined data. Thus enabling all the data scientists to focus on driving valuable insights from data.At the heart of this innovation lies the vector data store. It is one of the powerful tools that ensures seamless data manipulation and integration.Let us delve deeply into the depths of vector data storage while exploring its functionality and significance in Azure machine learning prompt flow.Understanding Vector datastoreA vector datastore can handle large-scale vectorized data efficiently within the Azure Machine Learning ecosystem. It acts as one of the centralized repositories that houses diverse data formats, from texts to images and numerical data. The real power of the vector data store lies in the ability to unify such disparate data types into a cohesive format that helps other data scientists work seamlessly.Some of the key benefits and features of vector data storing in the ecosystem of Azure ML include:Data integrationWith the help of a vector datastore, data scientists can integrate a variety of data types without going through the hassles of format conversion. Due to this hassle, the system accelerates the data preprocessing phase, which is one of the crucial steps in any Machine Learning project.Efficient data manipulationVector datastore makes complex operations like filtering, feature extraction, quality confirmation and transformation the most straightforward process. This efficient data manipulation technique is crucial to derive meaningful patterns from the raw data. Such a process leads to more accurate machine-learning models.Scalability The vector datastore of Azure Machine learning prompt flow helps to scale effortlessly. As a result, it helps in accommodating the growing datasets. Whether the user deals with petabytes or gigabytes of data, a vector datastore ensures smooth operations without compromising the accuracy and speed of the whole process.Version controlVector database makes data versioning simplified. It allows the data scientists to keep track of the changes, reproduce experiments with precision, and collaborate effectively.Let's consider a scenario where we want to preprocess a dataset containing images of handwritten digits for a digit recognition task. First, we'll initialize a Vector Datastore to store our image data.import numpy as np import cv2 # Define a function to load and preprocess images def load_images(file_paths, target_size=(64, 64)):    images = []    for file_path in file_paths:        # Read the image using OpenCV        image = cv2.imread(file_path)        # Resize the image to the target size        image = cv2.resize(image, target_size)        # Normalize pixel values to be between 0 and 1        image = image.astype('float32') / 255.0        images.append(image)    return np.array(images) # Example usage file_paths = ['image1.jpg', 'image2.jpg', 'image3.jpg']  # List of file paths to your images image_data = load_images(file_paths) # Now image_data contains the preprocessed image data ready to be stored in your vector datastoreIn this example, the code snippet demonstrates how to initialize a Vector Datastore, upload the image dataset, and create a dataset from the stored images.Output:Upon successful execution, the image dataset is seamlessly integrated into the Vector Datastore, ready for preprocessing and model training. This integration ensures that data scientists can focus on building robust machine learning models without worrying about data compatibility issues.Creating Vector indexVector index lookup is a tailored tool that helps to make queries within Azure Machine Learning Vector datastore index. It helps to empower the user to extract relevant information related to the context from a domain knowledge base.Here is how one can prepare one's own Data QnA by placing a vector index as an input. However, on the basis of the place where you put the vector index, the identity used by Azure ML promptflow gets granted certain roles.Inputs:After installing Annoy library, you can createa vector index using the below code.from annoy import AnnoyIndex # Assuming image_data is your preprocessed image data (NumPy array) # Each row in image_data represents a flattened image vector # Define the number of dimensions in your vectors (usually the length of each flattened image vector) num_dimensions = len(image_data[0]) # Initialize Annoy index with the number of dimensions in your vectors annoy_index = AnnoyIndex(num_dimensions) # Add vectors to the index for i, vector in enumerate(image_data):    annoy_index.add_item(i, vector) # Build the index for efficient nearest neighbor searches annoy_index.build(n_trees=10)  # You can adjust the number of trees for optimization # Now, the annoy_index is ready for efficient nearest neighbor searches # You can perform queries using annoy_index.get_nns_by_vector(vector, num_neighbors, search_k) # For example: # nearest_neighbors = annoy_index.get_nns_by_vector(query_vector, num_neighbors=5, search_k=-1)Outputs:The index will be initialized with various dimensions in the image vectors.Choosing a vector storeOne must use a vector index to perform Retrieval Augmented Generation in Azure Machine learning. It helps store the embeddings, which can later be converted to number sequences. Such an elaborate process helps the large language models to understand any complex relation between those concepts. Only if the user can create vector stores will it help them to hook up the data with a large language model, including GPT4. At the same time, one can also retrieve the data efficiently.Azure Machine Learning prompt flow usually supports two kinds of vector stores in the RAG workflow.FaissIt is one of the open-source libraries that provides the user with a local file-based store. One can find the vector index stored in the storage account of the Azure machine learning workspace. Since the storage system is locally stored, the costs are also minimal. Hence, the whole process of testing and development is budget-friendly.Azure Cognitive SearchIt is one of the Azure resources that supports information retrieval over the textual data and vector stored in search retrieval. With the help of prompt flow, one can populate, create, and query the vector data stored in Azure cognitive search.Though you can choose any of the vectors mentioned above stores, here is an overview of which should be used.Faiss, an open-source library, emerges as a robust solution, particularly when dealing with vector-only data. It stands as an essential component that can be seamlessly integrated into your solution. Let's explore the key aspects of Faiss, coupled with the capabilities of Azure Cognitive Search, to understand how these tools can be harnessed effectively.Faiss: Optimizing Vector Data ManagementFaiss offers several compelling advantages when it comes to working with vector data:1. Cost-Effective Local Storage: Faiss allows local storage without incurring additional costs for creating an index, offering a budget-friendly option for businesses aiming to optimize their expenses while managing extensive datasets.2. In-Memory Indexing and Querying: One of Faiss' standout features is its ability to build and query indexes entirely in memory. This approach significantly enhances the speed of operations, making it an efficient choice for real-time applications.3. Flexibility in Sharing: Faiss enables the sharing of index copies for individual use, providing flexibility in data access. However, additional setup is necessary for applications requiring index hosting to ensure tailored solutions for diverse use cases.4. Scalability Aligned with Computational Resources: Faiss scales seamlessly with the underlying compute resources, enabling businesses to manage varying workloads effectively. Its ability to adapt to the computational load ensures consistent performance despite fluctuating demands.Example:Consider an e-commerce platform dealing with millions of product vectors. By utilizing Faiss, the platform can create an in-memory index, enabling lightning-fast similarity searches for product recommendations, enhancing user experience, and increasing sales.Azure Cognitive Search: Elevating Vector Data Management to Enterprise Level Azure Cognitive Search, a dedicated Platform as a Service (PaaS) resource, offers a comprehensive solution for businesses seeking robust vector data management:1. Enterprise-Grade Scalability and Security: Cognitive Search supports enterprise-level business requirements, offering scalability, security, and availability. It ensures seamless scaling to accommodate growing data volumes, such an attribute makes it an ideal choice for businesses of all sizes.2. Hybrid Information Retrieval: A unique feature of Cognitive Search is its ability to support hybrid information retrieval. It means that vector data can coexist harmoniously with non-vector data. Businesses can leverage all the features of Azure Cognitive Search, including hybrid search and semantic reranking, ensuring comprehensive data analysis.3. Vector Support in Public Preview: Cognitive Search's vector support is currently in public preview. Although vectors must be generated externally, Cognitive Search handles the indexing and query encoding seamlessly within the prompt flow, simplifying the integration process.Example:Consider a financial institution needing to process massive amounts of transaction data, including structured and vector data, for fraud detection. Azure Cognitive Search allows seamless integration of vector data, enabling the institution to identify patterns effectively and enhance security protocols.Integration for Seamless Vector Data ManagementTo utilize Cognitive Search as a vector store for Azure Machine Learning, you must establish a search service within your Azure subscription. Once the service is in place, developers can access it. Azure Cognitive Search can be chosen as a vector index within the prompt flow. The prompt flow facilitates the entire process, from index creation to vector generation, ensuring a streamlined experience.The synergy between Faiss and Azure Cognitive Search presents a formidable solution for businesses aiming to manage vector data effectively. Faiss' efficiency in local storage and real-time querying, coupled with Cognitive Search's enterprise-grade scalability and hybrid data support, creates a powerful ecosystem. This integration empowers businesses to leverage their vector data fully, facilitating data-driven decision-making and driving innovation in diverse industries.By harnessing the capabilities of Faiss and Azure Cognitive Search, companies can truly unlock the potential of their data, paving the way for a future where data management is as intelligent as the insights derived from it.ConclusionVector datastore accelerates the machine learning pipelines, leading to faster innovations and more accurate models. As organizations continue to grapple with massive data sets, the only solution that can enhance accuracy and efficiency becomes indispensable. Hence, vector datastore in Azure machine learning promptflow is not a choice but a necessity. It unifies the diverse data types, coupled with scalability and efficient manipulation, enabling the data scientist to extract valuable insights, especially from complex and large data sets.Author BioKarthik Narayanan Venkatesh (aka Kaptain), founder of WisdomSchema, has multifaceted experience in the data analytics arena. He has been associated with the data analytics domain since the early 2000s, with a ringside view of transformations in this industry. He has led teams that architected and built scalable data platform solutions across the technology spectrum.As a niche consulting provider, he bridged the gap between business and technology and drove BI adoption through innovative approaches in an agnostic manner. He is a sought-after speaker who has presented many lectures on SAP, Analytics, Snowflake, AWS, and GCP technologies.
Read more
  • 0
  • 0
  • 314
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-prompt-engineering-for-beginners
Sangita Mahala
20 Sep 2023
8 min read
Save for later

Prompt Engineering for Beginners

Sangita Mahala
20 Sep 2023
8 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights and books. Don't miss out – sign up today!IntroductionIn this article, you will learn the basics of prompt engineering. Then you will get to know about What is prompt engineering, why it is important, and various techniques to master this skill. Additionally, you will learn what are the examples of effective prompts for different tasks which will help you to jumpstart your journey into prompt engineering.What is Prompt EngineeringThe main focus of prompt engineering is to enhance the interaction between humans and Artificial Intelligence. We can term prompt engineering as the process of creating and constructing prompts to lead a large language model to produce the required output. These LLMs are trained on a large number of datasets containing text and code. While using LLMs one must know that they can be difficult to use effectively without careful prompting. By offering users with precise instructions, relevant context, and useful examples, prompt engineering can assist them in getting the most out of large language models.Importance of Prompt EngineeringPrompt engineering is important for making large language models work better. The following are the importance of prompt engineering:Prompt engineering can help us to shorten the effort and time required to generate LLM output.Give more accurate and instructive responses from LLMs.Enhance the quality and creativity of LLM output.Enlarge the range of tasks that are to be performed using LLMs.Example:Suppose you want to use an LLM to generate an article about the Eiffel Tower in Paris. You can simply provide the LLM with a prompt that says "Write an article about the Eiffel Tower in Paris." However, this is likely to result in an article that is poorly written, inaccurate, or irrelevant.Instead, you can use prompt engineering to guide the LLM to generate a more accurate and informative article. For example, you can provide the LLM with a prompt that specifies the following: The topic of the article: "The Eiffel Tower: A Symbol of Paris" The desired length of the article: "500 words" The audience for the article: "General readers" Tone and style of the article: "Informative and engaging" The following diagram illustrates how prompt engineering can be used to improve the quality of LLM output:                                                                                                                      (Diagram No. 1)The above diagram shows how a prompt can be used to guide the LLM to generate a more accurate and informative news article about Eiffel Tower in Paris.Different Kinds of Prompt Engineering TechniquesThere are a variety of prompt engineering techniques that can be used to achieve different goals. Some common techniques include:Zero-shot prompting: It provides the LLM with a prompt that describes the desired output, without providing any examples. Zero-shot prompting is often used for tasks such as summarization, translation, and question-answering.Few-shot prompting: It provides the LLM with a few examples of the desired output, along with a prompt that describes the task. Few-shot prompting is often used for tasks such as creative writing and code generation etc.In-context prompting: It provides the LLM with a sequence of prompts, where each prompt builds on the previous one. In-context prompting is often used for tasks such as reasoning, problem-solving, and dialogue.Examples of Effective Prompts for Different Tasks:The few examples below illustrate how you can use well-crafted prompts to perform different types of tasks.Tasks:Text SummarizationQuestion AnsweringText ClassificationCode GenerationInformation ExtractionConversationReasoning                                                                                                                         (Diagram No. 2)Text SummarizationIt is the ability to summarize articles and concepts into quick and easy-to-read summaries. Let’s have a look on a basic summarization task using prompts.Let's say you are interested in learning about Blockchain Technology, you can try a prompt like this:Prompt:Here, you used it to inform the model that a subsequent response was expected. Let's imagine that you feel that this material is too much and that you would like a more thorough summary. In fact, you may tell the model to compress everything into a single sentence:Prompt:Here you can see the model tried to summarize the whole paragraph in one sentence. Question AnsweringOne of the best ways to get the model to respond is to improve the format of the prompt. Basically, a prompt can combine given input , output ,instructions, context indicators to get enhanced results.Here are some examples of how QA systems can be used to answer the questions you provided.Prompt:A QA system would be able to answer this question by searching its knowledge base for information about the chemical formula of water. It would then generate a response that includes the answer to the question.Text ClassificationIt is the process of designing prompts that guide language models to perform text classification tasks. This can be achieved by providing the model with specific instructions and context, as well as examples of the different kinds of classes for text, so that it will be able to identify.For example, the following prompt could be used to classify customer reviews as neutral, positive or negative:Prompt:The tech industry continues to revolutionize our world with cutting-edge innovations that redefine the way we live and work. Code GenerationIt is the process of designing prompts that guide language models for generating code. This can be achieved by providing the model with specific instructions and context, as well as examples of the desired code output.For example, the following prompt could be used to generate a code that takes a string as input and returns the reverse of the string by using Python.Prompt: Information ExtractionIt is the process of designing prompts that guide language models to extract specific information from text. This can be achieved by providing the model with instructions and context, as well as examples.For example, the following prompt can be used to extract the names of all people mentioned in the news article:Prompt:Conversation:It is a technique used to create more natural and engaging conversations between humans and AI language models. Conversation provides the model with context and information about the user's goals, and then it asks questions in such a way that encourages the model to respond in a conversational manner.Now, we will create a conversational system that can answer queries in a more technical and perfect manner, for example. Keep in mind that by giving it instructions, you are clearly directing it how it must act. This is sometimes referred to as role prompting. Prompt:Reasoning:It involves designing prompts that will encourage language models to use their knowledge and reasoning skills to generate accurate and informative responses. This can be achieved by providing the model with clear feedback, instructions and examples.Prompt-1: Prompt-2:As we can see in the above example, the model is first asked to add 98 and 2. Then, it is asked to subtract 4 from 8. Finally, it is asked to multiply 6 by 4. By performing these steps one at a time, the model is able to solve the problem correctly.In each of these examples, the prompt is carefully designed to guide the LLM to generate the desired output. By following these examples, you can start to use prompt engineering to get more out of LLMs.ConclusionIn this article, we have explored the basics of prompt engineering, including its definition, importance, and different techniques. We have also provided examples of effective prompts for different tasks, such as text summarization, question answering, text classification, code generation, information extraction, conversation, and reasoning. As we all know that Prompt engineering is a rapidly evolving field, that can be used to generate a wide variety of outputs from LLMs. As LLMs become more powerful and versatile, prompt engineering will surely play an increasingly important role in how we interact with AI.Author BioSangita Mahala is a passionate IT professional with an outstanding track record, having an impressive array of certifications, including 12x Microsoft, 11x GCP, 2x Oracle, and LinkedIn Marketing Insider Certified. She also possesses extensive experience as a technical content writer and accomplished book blogger. She is always Committed to staying with emerging trends and technologies in the IT sector.
Read more
  • 1
  • 0
  • 972

Banner background image
article-image-automatic-prompt-engineering-with-prompt-royale
Louis Owen
18 Sep 2023
8 min read
Save for later

Automatic Prompt Engineering with Prompt-Royale

Louis Owen
18 Sep 2023
8 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights and books. Don't miss out – sign up today!IntroductionAI has never been more accessible than ever since the launch of ChatGPT. With Generative AI, people can now build their own AI engine by only giving commands in natural language. There is no need to know how to code, no need to prepare the training data, and no need to do any model hyperparameter tuning. What we need to do to build our own AI system is to only give commands, more widely known as prompt engineering.Prompt engineering is more of an art than science. There are so many ways to do prompt engineering. The simplest form of prompt engineering is called zero-shot prompting, where the user just needs to directly give their command to the Large Language Model (LLM). For example: “Write an acrostic poem in Hindi”, “Write a 7-day itinerary in Bali”, etc.Another prompting technique is called few-shot prompting, where we need to give several examples of the expected output inside the prompt itself. Let's say we want to utilize LLM to do sentiment analysis. We can write the prompt to be something like the following:You are an expert in performing sentiment analysis. You can only return the output with 3 options: “negative”, “neutral”, and “positive”. Example 1: I love this product! It works perfectly. Sentiment: positive Example 2: The weather today is terrible. It's raining non-stop. Sentiment: negative Example 3: I’m feeling sleepy Sentiment: neutral Text: Attending the concert last night was a dream come true. The music was incredible! Sentiment:The more sophisticated way to do prompt engineering is by performing Chain-of-Though (CoT). Basically, in this technique, we prompt the LLM to give a step-by-step explanation of why it arrives at the final answer. This technique is adopted widely by the AI community since it gives better output in a lot of cases. The drawback of implementing this prompt-engineering technique is the increase of the generated number of tokens which correlates positively with the latency.There are still many prompting techniques available out there. Choosing the right prompt technique or even doing the prompt engineering itself is indeed not an easy task. We need to pass through many iterations until we find the best prompt for our use case.In this article, I’ll guide you to do automatic prompt engineering which can surely save our time in creating the best prompt for our use case. We’ll discuss two popular automatic prompt engineering frameworks: GPT-Prompt-Engineer and Prompts-Royale, where we’ll dive deeper more into Prompts-Royale. Finally, there will be a dedicated section on how to install and utilize prompts-royale.Without wasting any more time, let’s take a deep breath, make yourselves comfortable, and be ready to learn how to perform automatic prompt engineering!Automatic Prompt Generator FrameworksThere are two most popular automatic prompt generator frameworks: GPT-Prompt-Engineer and Prompts-Royale. Let’s start looking into the first one.GPT-Prompt-Engineer [github] is a very famous GitHub repository with more than 4.5k starts. It can be utilized to automatically generate the best prompt by simply inputting the task description along with several test cases. Then, the system will generate, test, and rank several variations of prompts with the goal of finding the best among all of them. The steps to use this framework are very straightforward:Define your use-case and test casesChoose how many prompts to generateThe system will generate a list of potential prompts, and test and rate their performanceThe final evaluation score will be printed in a table.There is no UI available for this package, so it might not be very compelling to the non-coders. However, there are two ready-to-use Google Colab notebooks that can be directly used. The first notebook can be utilized for general tasks other than the classification task. ELO rating will be used to evaluate the best prompt amongst several prompt candidates. The second notebook is created specifically for the classification task where the evaluation process is conducted based on the available ground truth.Another framework that is relatively new and the “improved” version of the GPT-Prompt-Engineer package is Prompts-Royale [github]. Similar to GPT-Prompt-Engineer, it’s also very straightforward to utilize this framework. You just need to give the description of the task along with a couple of example scenarios and the expected outputs, then the system will do the rest.There are indeed several plus points offered by this framework:Automatic test cases generation: automatically creating test cases from the description, we just need to provide several examples and the system will generate more test cases automatically.Monte Carlo matchmaking: not only utilizing ELO rating as in GPT-Prompt-Engineer, Prompts-Royale also uses the Monte Carlo method for matchmaking to ensure you get as much information with the least amount of iteration.User Interface: unlike GPT-Prompt-Engineer, Prompts-Royale offers a nice UI where users can directly give all of the inputs and get the returned outputs all in a single nice UI. Since Prompts-Royale offers additional benefits compared to the famous GPT-Prompt-Engineer framework, we’ll dive deeper into this framework instead in this article. Without wasting any more time, let’s see Prompts-Royale in action!Prompts-Royale in Action!InstallationTo use Prompts-Royale, you can directly visit promptsroyale.com, or you can clone the repository and run it locally. To run locally, you just need to do the following things:1. Clone the repository            ```git clone git@github.com:meistrari/prompts-royale.git            ```2. Install all dependencies with Bun```bun i```3. Run prompts-royale as a server in your local machine```bun run dev```This is the page that you will see once the server is up.Using the Prompts RoyaleTo use prompts-royale, we first need to input the OpenAI API key. We can use GPT3.5 or GPT4. You can find the key in your OpenAI account. If you don’t have an account yet, you can easily sign yourself up here. Once you insert the API key, you can start giving the necessary inputs in the form. You need to insert the task description and several test cases. Task description can be something like “Write a prompt that creates a headline for a website.” For test cases, we need to provide the scenario and the expected output, just like how we implement the few-shot prompting technique.  Next, we just let the system generate several prompt candidates by clicking the “Generate prompts” button. Note that we can also add our own written prompt to the list of prompt candidates.Finally, once we have the list of prompt candidates, we need to let the system choose which one is the best prompt. To do that, we need to input the number of battles that will be executed by the system. “Battle” simply means the process of selecting the best prompt out of all candidates. The battle will be between 2 prompts. Remember, the higher the number of battles the higher the cost to find the best prompt. By default, prompts-royale will run 60 battles. The results of the battles will be shown at the bottom of the page. There’s a chart of ratings over iterations and the battle log. The final prompt ranking can be seen on the right side of the page, as follows. You can of course click each of the prompt button and see what’s the generated prompt. ConclusionCongratulations on keeping up to this point! Throughout this article, you have learned how to do automatic prompt engineering with the help of prompts-royale. You’ve also learned several prompting techniques and another automatic prompt engineering technique called GPT-Prompt-Engineer. See you in the next article!Author BioLouis Owen is a data scientist/AI engineer from Indonesia who is always hungry for new knowledge. Throughout his career journey, he has worked in various fields of industry, including NGOs, e-commerce, conversational AI, OTA, Smart City, and FinTech. Outside of work, he loves to spend his time helping data science enthusiasts to become data scientists, either through his articles or through mentoring sessions. He also loves to spend his spare time doing his hobbies: watching movies and conducting side projects. Currently, Louis is an NLP Research Engineer at Yellow.ai, the world’s leading CX automation platform. Check out Louis’ website to learn more about him! Lastly, if you have any queries or any topics to be discussed, please reach out to Louis via LinkedIn.
Read more
  • 0
  • 0
  • 355

article-image-prompt-engineering-best-practices
Amita Kapoor
18 Sep 2023
11 min read
Save for later

Prompt Engineering Best Practices

Amita Kapoor
18 Sep 2023
11 min read
Dive deeper into the world of AI innovation and stay ahead of the AI curve! Subscribe to our AI_Distilled newsletter for the latest insights and books. Don't miss out – sign up today!IntroductionPrompt Engineering isn't just about throwing questions at a machine and hoping for a brilliant answer. Oh no, it's a meticulous dance of semantics and syntax. Think of it as the secret sauce that turns raw data into Michelin-star outputs. It's the act of finessing questions, statements, and other inputs in such a way that our ever-so-complex language models (yes, like those GPT variants you've heard so much about) know exactly what performance we're expecting.To put it cheekily: If you've ever tried to get a diva to perform without a rehearsal, you'd know the importance of Prompt Engineering. It's not merely about the questions we ask but the elegance and intent with which we pose them. The spotlight's on, let the show begin!Why is Prompt Engineering Important?Step into any grand opera house, and you'll immediately grasp the importance of a well-directed performance. Similarly, in the vast concert hall of machine learning, Prompt Engineering is the esteemed conductor orchestrating every note and crescendo. So, what makes prompt engineering indispensable? The answers unfold in the visual guide belowIn essence, while having a cutting-edge language model is like owning a Stradivarius violin, it's the skill of the violinist—or in our case, the precision of the prompt—that elicits the true magic. So, if you're looking to harness the full symphony of capabilities from your AI, mastering the art of Prompt Engineering isn't just recommended; it's indispensableTypes of PromptsMastering the art of prompting is akin to being a maestro of a symphony, understanding when each section of the orchestra needs to play to produce harmonious music. Each type of prompt offers a different angle to solicit the desired response from a language model. Here's your guide to their nuances:Patterned PromptsPatterned prompts serve as the scaffold for models, guiding them in producing outputs of a specified type and structure. These prompts leverage templates or distinct patterns to tailor the generation process, ensuring the output not only aligns with a desired format but also exhibits originality and creativity.Suppose you want a quatrain about the night sky. Your patterned prompt might be:Write a quatrain with an ABAB rhyme scheme about the night sky.The LLM might then generate:The stars above in splendor gleam (A) Amidst the vast and dark expanse (B) Moonlight casts a gentle beam (A) In this nightly, cosmic dance (B)Persona-based Prompts:Persona-based prompts harness the power of fictional character profiles — personas — to steer the LLM in generating responses tailored to a particular user type or demographic. These profiles encapsulate key traits such as age, interests, objectives, and challenges, providing a backdrop against which the model can craft a more relevant and empathetic output. Before delving into persona-based prompts, one must craft a comprehensive persona.  Here is an exampleMarketing: Generate promotional materials that echo the sentiments and inclinations of potential customers.Alex is a 30-year-old fitness instructor from Miami with a penchant for high-intensity workouts. He's a massive fan of HIIT sessions, often referring to the methods of Jillian Michaels. Creating content for Alex, draft a promotional email to introduce a new line of high-intensity workout gear targeting fitness enthusiasts like him.Zero-shot Prompts:Zero-shot prompts empower language models to generate responses without needing prior task-specific training. They draw upon the vast knowledge and patterns that large language models (LLMs) have assimilated during their extensive general training. Simple in nature, these prompts can be as straightforward as Pen a poem on autumnorRender this phrase into French: Good morning.The strength of zero-shot prompts lies in the extensive and diverse training data of LLMs. By recognizing myriad linguistic structures, relations, and nuances, LLMs can often tackle tasks with remarkable accuracy, even without having seen a direct example before.Few-shot promptsFew-shot prompts are a type of prompt that provides the large language model (LLM) with a few examples of the desired output before asking the main question. By doing this, you're helping the model to "warm up" and better understand the task at hand. Consider prompting the LLM to write a poem about a dog. You could use the following few-shot prompt:Write a poem about a dog.Example 1:Loyal and ever by my side,With a tail that won't stop its wag,You bark, you play, and you glide,My faithful friend, more than just a tag.Example 2:Golden fur, eyes shining bright,Chasing balls, every single throw,Your joy and bark, light up the night,A bond with you, forever to grow.Based on these two examples, the LLM is now primed to understand the rhythm, style, and thematic elements about dogs.Chain-of-Thought PromptsLike a mastermind planning a heist, these prompts break down a challenge into digestible bits. It's all about guiding the model step by step. They are ideal for complex tasks that benefit from being broken down into structured, smaller tasks.Let's consider we want the LLM to design a lesson plan for teaching basic physics to high school students, so we guide LLM by asking these questions in a step by step manner, as shown below:As the LLM progresses through each prompt in the chain, it is guided through a structured thought process, ensuring that the end result is comprehensive and aligned with the desired objective.Context-Aware PromptsContext-aware prompts are designed to give the large language model (LLM) essential background or situational information related to the task at hand. By grounding the LLM in a specific context, these prompts aid in generating outputs that are more relevant and nuanced. Whether it's the historical backdrop, cultural nuances, or the specific setting in which a piece of information will be used, context-aware prompts equip the LLM to tailor its response appropriately to the given circumstances. Here is an example:Context: You are writing for an audience of young adults aged 18-25 who are passionate about environmental conservation and are considering veganism for ecological reasons.Prompt: Given the context, provide a brief article discussing the environmental benefits of adopting a vegan lifestyle.Tree of Thought PromptsThese prompts aren't just a series; they're a hierarchical orchestration. Imagine guiding the LLM with a blueprint, where every level delves deeper into the task. This tree-based approach organizes complex tasks by breaking them into branches and leaves, each representing different stages of the task. They shine brightest when handling intricate tasks that demand meticulous planning, reasoning, or a structured approach. By breaking a task into nodes and leaves, it transforms abstract assignments into navigable journeys. Example:Objective: Plan a unique birthday celebration.Depth 1 - Idea GenerationToT Prompt 1: "Provide 5 unique ideas for a birthday celebration."Potential Responses:     An underwater-themed party at a local aquarium.     A time-travel costume party where guests dress from different eras.     Renting out a cinema for a private movie screening with friends.     Hosting a cooking competition with birthday-related recipes.     A mystery-solving escape room challenge tailored for the birthday person.Depth 2 - EvaluationToT Prompt for Voting: "Analyse the choices below, then conclude which is the most promising for a memorable birthday celebration."Selected Response: "A time-travel costume party where guests dress from different eras."Depth 3 - Detailing Based on Chosen IdeaToT Prompt 2: "Provide 5 detailed activities or features for a time-travel costume party."Potential Responses:     Setting up photo booths from different eras, like the Roaring 20s, the Renaissance, and the Future.     Time-specific games or challenges, such as jousting or a 60s dance-off.     A time-travel passport that gets stamped as guests move through different eras.     Food and drinks menu tailored to different time periods.     Prizes for the best costume from each era.Depth 4 - EvaluationToT Prompt for Voting: "Analyse the choices below, then decide which feature will be most engaging for guests at a time-travel costume party."Selected Response: "Setting up photo booths from different eras, like the Roaring 20s, the Renaissance, and the Future."By using the Tree of Thought Prompts, the decision-making process becomes structured, allowing the exploration of various options at each stage and refining the choices based on evaluations.Retrieval Augmented Generation (RAG) PromptsThis technique marries the power of retrieval with the flair of generation. In the vast ocean of data, the RAG ensures the LLM fishes out the most relevant pearls of information from a knowledge base and then weaves them into coherent, insightful narratives. RAG is the hero you want when detailed, factual accuracy is paramount. Think of tasks where there's a need to dive deep into databases or sources, such as Wikipedia, for question answering, translating texts, summarization, or even adding a factual touch to creative compositions.Although ChatGPT, Bard, Bing, and other LLMs aren't RAG models in the conventional sense, when supplied with links or equipped with plugins granting access to specific documents, they can harness their expansive knowledge banks. This enables them to yield outputs that are not only detailed and insightful but also precise and factually accurate.Prompting Tips and Best PracticesNavigating the world of prompts with a Large Language Model (LLM) is a tad like tango dancing. The clearer your moves (prompts), the better the performance. To ensure you and your LLM dance in perfect harmony, consider these golden rules of prompting:Precision is Key: Always aim for laser-like specificity in your prompts. The less room there is for guesswork, the more aligned the response will be to your desired outcome.Clarity Over Complexity: A well-phrased question is half the answer! Opt for clear, concise language, ensuring your prompts are easily decipherable.Skip the Gibberish: While using industry-specific jargon might make you sound smart at conferences, your LLM prefers simplicity. Sidestep any ambiguous terms or jargon that might lead to misinterpretation.Bite-Sized is Right-Sized: Complex tasks can be daunting, not just for humans but for LLMs too. Break them down into digestible, smaller tasks. It's easier to tackle a pie slice-by-slice than in one go.Context is King: The more you feed the LLM in terms of background, the richer and more tailored its output will be. Context sets the stage for relevance.The Prompts Playground: There's no one-size-fits-all in prompting. It's an art as much as it is a science. So, roll up your sleeves and experiment. Different tasks might resonate with different types of prompts. Keep tinkering until you strike gold!Remember, the magic happens when you communicate with your LLM effectively. These best practices are your playbook to unlocking its full potential and ensuring a harmonious tango dance every time.ConclusionIn the ever-evolving realm of AI and LLMs, the art of prompt engineering is akin to fine-tuning a musical instrument. With the right notes, or in this case, the right prompts, the symphony you create can be both harmonious and impactful. Whether you're dabbling in Zero-shot prompts or diving deep into CoT, remember that the essence lies in clear communication. By embracing best practices and staying adaptable, we not only harness the true prowess of these models but also pave the way for AI-human collaborations that are more seamless and productive. As we continue this dance with AI, may our prompts always lead, guide, and inspire.Author BioAmita Kapoor is an accomplished AI consultant and educator with over 25 years of experience. She has received international recognition for her work, including the DAAD fellowship and the Intel Developer Mesh AI Innovator Award. She is a highly respected scholar with over 100 research papers and several best-selling books on deep learning and AI. After teaching for 25 years at the University of Delhi, Amita retired early and turned her focus to democratizing AI education. She currently serves as a member of the Board of Directors for the non-profit Neuromatch Academy, fostering greater accessibility to knowledge and resources in the field. After her retirement, Amita founded NePeur, a company providing data analytics and AI consultancy services. In addition, she shares her expertise with a global audience by teaching online classes on data science and AI at the University of Oxford. 
Read more
  • 1
  • 1
  • 1162
article-image-hands-on-with-prompt-engineering-in-trading-stats
Anshul Saxena
11 Sep 2023
13 min read
Save for later

Hands-On with Prompt Engineering in Trading Stats

Anshul Saxena
11 Sep 2023
13 min read
IntroductionIn today's dynamic trading environment, the key to success lies not just in possessing data, but in harnessing it efficiently and intelligently. With the advent of AI-powered tools like ChatGPT plugins, traders and investors can now access and analyze data with unprecedented ease and precision. These advanced tools offer a comprehensive view of the market, from historical price trends and fundamental stock metrics like EPS to real-time news updates and sentiment analysis. They can extract insights from intricate financial statements, gauge market sentiment from social media analytics, and provide an in-depth look into the world of option chains.This tutorial aims to guide individuals on how to effectively utilize these AI-enhanced tools, specifically focusing on the ChatGPT plugin. By integrating two standalone tools, Whimsical and ChatGPT, and exploring four distinct ChatGPT plugins - Polygon.io, AI Ticker Chat, PointsRecap, and OptionsPro - we present ten structured prompting techniques. These techniques, when automated, empower users to devise a data collection strategy tailored to various financial instruments, ensuring swift and accurate information retrieval. Dive in to stay a step ahead in your trading endeavors.Mapping data strategy for traders using whimsicalLooking to decipher trading data more effectively? Dive into this guide where we combine the visual capabilities of Whimsical with the querying prowess of ChatGPT. Whimsical allows us to create intuitive mind maps, illustrating the interconnections in trading data, while ChatGPT assists in posing and comprehending crucial questions about stocks and companies. Together, these tools illuminate the essential data points in trading, guiding you towards informed decisions. Join us as we simplify the process, ensuring you harness these tools for optimal trading insights. Prompt 1: Give me prompt ideas for data collection in AIBased on the mind map generated by whimsical, we need to fetch and analyse the financial and textual data in these six stepsStep 1. Analyzing historical market data, focusing on aspects like price, volume, and volatility.Step 2. Understanding fundamental analysis metrics, particularly the earnings per share (EPS), which is pivotal for stock evaluation.Step 3. Gleaning insights from news and sentiment analysis sourced from diverse platforms.Step 4. Reviewing financial statements and reports of companies active in trading to gauge their financial health.Step 5. Harnessing social media analytics to monitor market discussions and discern emerging trends.Step 6. Exploring option chain data, which is instrumental in analyzing options contracts related to specific securities or indices.In the subsequent sections, we will delve deep into various facets of trading data, from analyzing historical market indicators like price, volume, and volatility, to understanding the significance of fundamental metrics such as earnings per share (EPS) for stock evaluation. We'll also tap into diverse platforms for news and sentiment analysis, review financial statements of active trading companies, harness the power of social media analytics to discern market trends, and explore the intricacies of option chain data related to specific securities or indices.Step 1: Fetching Historical Data using ChatGPTDelving into trading often begins with examining historical market data. This encompasses past stock prices, trading volumes, and price fluctuations, offering insights into a stock's historical performance. In this section, we'll harness the capabilities of Polygon.io through ChatGPT, utilizing API calls to access precise financial data. Join us as we guide you through this process, ensuring you have a clear understanding of a stock's past trajectory.Here's a step-by-step guide on how to use Polygon in ChatGPT:Step 1. Ask Clearly: Tell ChatGPT your desired financial data, like Apple's latest stock news.Step 2. Review Data: ChatGPT fetches the info using Polygon.io and presents it in a structured format.Step 3. Dive Deeper: Pose follow-up questions or request more specific data points.Step 4. Stay Informed & Compliant: Use the real-time data responsibly, adhering to financial regulations.Prompt 2: Give me Historical market data such as price, volume, and volatility of APPLEFrom the information provided, ChatGPT has offered a comprehensive analysis of the market data pertaining to Apple Inc.'s stock. This assessment can be further expanded upon to delve deeper into the historical price patterns, enhancing our understanding of its market trajectory.Step 2: Understanding Fundamental Analysis through Prompt EngineeringTo assess a company's financial robustness and growth prospects, certain key metrics are pivotal. Taking Apple Inc. as an example, metrics like Earnings Per Share (EPS), Year-on-Year (YoY) growth, profit margin, Price-to-Earnings (PE) ratio, and Profit After Tax (PAT) offer a holistic insight into its financial stance. In this subsection, we'll employ the Polygon plugin and prompt engineering to delve into these five crucial metrics, providing a clearer perspective on a company's financial health.Prompt 3: Give detail about EPS, year-on-year (YoY) growth, profit margin, earnings per share (EPS), price-to-earnings (PE) ratio, and profit after tax (PAT). of APPLThe analysis has provided an in-depth output on the subject. We can further extend this study, leveraging ChatGPT and prompt engineering, to comprehend how fundamental KPIs influence a company's performance.Step 3: Performing Sentiment Analysis on  news sources using ChatGPTIn the dynamic realm of trading, staying abreast of current events is essential. News and sentiment analysis serve as barometers, capturing market sentiments from diverse sources such as media publications, expert commentary, and social media chatter. Grasping the prevailing sentiment around a stock or the broader market equips traders with insights for better decision-making. This section delves into the importance of these analytical tools and their role in shaping successful trading strategies.Prompt 4: Can you give me the latest news articles related to Apple's stock ticker?After fetching the relevant news articles from divergent sources, Polygon.io can be deployed to analyze them using below mentioned promptPrompt 5: Do the Impact analysis of the news related to appleAnalyzing the impact of news on a stock's performance requires a combination of understanding the content of the news and observing the market's reaction to it. Here's a brief analysis based on the provided news articles related to Apple:Apple Just Shed More Than $230 Billion In Market CapImpact Analysis: A significant drop in market capitalization indicates a loss of investor confidence or reaction to negative news. The mention of Apple possibly falling out of favor with the Chinese government is significant. China is a huge market for Apple, both in terms of sales and manufacturing. Any negative sentiment from the Chinese government can have substantial implications for Apple's revenue and operations in the region.Apple Could Emerge Unscathed From China BanImpact Analysis: This article seems to provide a counter-narrative to the previous one. If analysts believe that Apple can navigate the challenges posed by the Chinese government's decisions, it could mitigate some of the negative sentiments. However, the actual outcome will depend on how Apple manages its relationship with the Chinese government and its strategies to counteract any bans or restrictions.Here's Everything Apple Plans To Show At Its 'Wonderlust' EventImpact Analysis: Product launch events are significant for Apple. They not only introduce new products but also set the tone for the company's direction in the coming months. Positive reception to new products can drive stock prices up, while disappointment can have the opposite effect. The anticipation of the iPhone 15 series suggests that investors and consumers are keenly waiting to see Apple's next move.In the next step, we will try to look into the financial statement like 10-K, 10-Q, earnings transcript for Apple IncStep 4: Analyzing Financial Statements with the ChatGPT PluginFor traders aiming to swiftly assess a company's financial well-being, this tutorial offers valuable insights. It guides users on utilizing the ChatGPT Plugin to effectively interpret key financial metrics such as assets, liabilities, and revenues. By the conclusion of this section, readers will possess a potent analytical tool to enhance the efficiency and accuracy of their trading evaluations.How to Find Company Reports like 10K with AI Ticker ChatThis tool helps you find company reports, especially documents like the 10K, which is a yearly report companies make. Here's how to use it:Step 1. Ask a Question: Think about the report you want to see. It could be a 10K, which is a yearly report, or other types of reports.Step 2. Give Some Details: To help find the right report, please tell me:   - The company's name or symbol.   - The kind of report you want (like 10K for yearly reports).   - The year you're interested in.Step 3. Wait a Moment: After you tell me what you're looking for, I'll search for the report and show you what I find.Step 4. Look at What I Found: I'll show you the report or information. You can read it, ask more questions, or ask for more details. Prompt 6: Give latest 10-K, 10-Q, earnings transcript for APPL Prompt 7: Provide in-depth analysis of 10-K document for APPLThis analysis provides a high-level overview of the key points from Apple's 10-K.This plugin has the potential to save a trader’s pain of going through the lengthy statements. Afterwards by asking the intelligent question through prompt engineering a thorough analysis can be conducted in minutes,Step 5: Tracking Market Trends with Social Media Analytics using ChatGPT PluginLeveraging social media analytics provides a window into prevailing market discussions and emerging trends. Through the ChatGPT plugin, traders can seamlessly track real-time dialogues, pinpoint developing market patterns, and base their decisions on these insights. Incorporating this tool into a trading strategy ensures an edge in understanding and acting on market sentiments. How to Use PointsRecap: A Quick TutorialStep 1. Retrieve Viewpoints for a Ticker: Use the `getViewPoints` function with the desired stock ticker to get viewpoints. For example, `getViewPoints({ ticker: "AAPL" })` fetches viewpoints for Apple Inc.Step 2. Access Recent Highlighted Videos: Simply call the `getRecentHighlights()` function to see the latest highlighted YouTube videos.Step 3. Generate a Word Cloud: Understand trending topics in recent videos by using the `getWordCloud()` function, which returns a visual representation of frequently discussed topics.Step 4. Analyze and Present Data: After obtaining the data from PointsRecap, analyze the results to gain insights and integrate them into your research or presentation.We will further explore the latest chatter going around the Apple Inc on YouTube in the next two prompts.Prompt 8: getViewPoints({ ticker: "AAPL" })Prompt 9 : getWordCloud(APPL)By looking at and analyzing the above output, an individual can understand the overall sentiment and trend related to Apple Inc and help in formulating a better strategy. In the next section, we will try to further our research through the data pertaining to option related to company.Step 6: Analyzing Option Chain Data with ChatGPT PluginThe options market can serve as a predictive lens for anticipated movements in a company's stock market. Key indicators such as volume and open interest can hint at bullish or bearish sentiments. Implied Volatility (IV) reflects expected stock price fluctuations, while the Put/Call Ratio offers a snapshot of market sentiment. Additionally, option pricing, activity in out-of-the-money options, and the chosen expiration dates can provide insights into traders' expectations. Furthermore, hedging activities by large institutional investors using options can reveal their outlook on a stock. In this subsection, we'll delve into how these intricate dynamics in the options market can shed light on potential stock market trends.Option chains provide a structured display of all available option contracts for a specific security or index. These chains offer insights into various parameters like strike prices, expiration dates, and premiums. With the ChatGPT plugin, you can seamlessly retrieve and analyze this data in real-time. This tutorial will guide you through the process of harnessing the power of the ChatGPT plugin to make informed decisions regarding option contracts. Using OptionsPro with ChatGPT: A Five-Step TutorialOptionsPro offers a comprehensive suite of tools for analyzing options and financial data. This tutorial will guide you through five essential steps to harness the power of the OptionsPro plugin with ChatGPT.Step 1: Set Up Your EnvironmentEnsure you have access to the OptionsPro plugin and that it's integrated with ChatGPT.Familiarize yourself with the available endpoints mentioned above.Step 2: Get a Market OverviewUse the `getMarketOutlook` endpoint to gain insights into the general market trend. ``` OptionsPro.getMarketOutlook() ```Analyze the returned data to understand the current market sentiment and key indicators.Step 3: Dive into Individual Stock AnalysisChoose a stock ticker you're interested in.Use the `getOptionFlow` endpoint to get the most traded options for that ticker. ``` OptionsPro.getOptionFlow({ticker: "AAPL", topN: 10}) ```This will give you a list of the top 10 most traded options for Apple Inc.Step 4: Monitor Unusual Option ActivitiesUse the `getOptionAlerts` endpoint to get alerts on unusual options activities. ``` OptionsPro.getOptionAlerts({ticker: "AAPL"}) ```This will alert you to any unusual options activity for Apple Inc., helping you spot potential market-moving events.Step 5: Analyze Option DetailsOnce you've identified an option of interest, delve deeper using the `getOptionDetails` endpoint. ``` OptionsPro.getOptionDetails({ticker: "AAPL", type: "call", expiration: "2023-12-15", strike: 150}) ```This will provide details about the call option for Apple Inc. with a strike price of $150, expiring on December 15, 2023.  Prompt 10: getOptionDetails APPLThis is the last step that can help us in formulating the strategy for data collection for trading using ChatGPT and prompt engineering.ConclusionIn the ever-evolving landscape of trading, the fusion of AI-powered tools like ChatGPT plugins with traditional trading strategies has revolutionized the way traders access and interpret data. This tutorial illuminated the potential of combining tools like Whimsical with ChatGPT and delved into the capabilities of plugins such as Polygon.io, AI Ticker Chat, PointsRecap, and OptionsPro. By mastering the ten prompting techniques presented, traders can automate and refine their data collection processes, ensuring they're equipped with timely and precise information. As the world of trading continues to advance, leveraging these AI-enhanced tools will be paramount for those aiming to stay at the forefront of their trading journey.Author BioDr. Anshul Saxena is an author, corporate consultant, inventor, and educator who assists clients in finding financial solutions using quantum computing and generative AI. He has filed over three Indian patents and has been granted an Australian Innovation Patent. Anshul is the author of two best-selling books in the realm of HR Analytics and Quantum Computing (Packt Publications). He has been instrumental in setting up new-age specializations like decision sciences and business analytics in multiple business schools across India. Currently, he is working as Assistant Professor and Coordinator – Center for Emerging Business Technologies at CHRIST (Deemed to be University), Pune Lavasa Campus. Dr. Anshul has also worked with reputed companies like IBM as a curriculum designer and trainer and has been instrumental in training 1000+ academicians and working professionals from universities and corporate houses like UPES, CRMIT, and NITTE Mangalore, Vishwakarma University, Pune & Kaziranga University, and KPMG, IBM, Altran, TCS, Metro CASH & Carry, HPCL & IOC. With a work experience of 5 years in the domain of financial risk analytics with TCS and Northern Trust, Dr. Anshul has guided master's students in creating projects on emerging business technologies, which have resulted in 8+ Scopus-indexed papers. Dr. Anshul holds a PhD in Applied AI (Management), an MBA in Finance, and a BSc in Chemistry. He possesses multiple certificates in the field of Generative AI and Quantum Computing from organizations like SAS, IBM, IISC, Harvard, and BIMTECH.Author of the book: Financial Modeling Using Quantum Computing
Read more
  • 0
  • 0
  • 482

article-image-building-powerful-language-models-with-prompt-engineering-and-langchain
Alan Bernardo Palacio
21 Aug 2023
20 min read
Save for later

Building Powerful Language Models with Prompt Engineering and LangChain

Alan Bernardo Palacio
21 Aug 2023
20 min read
IntroductionIn this tutorial, we will delve into LangChain, an impressive framework designed for creating applications and pipelines using Large Language Models (LLMs). Our focus for this tutorial is 'prompt engineering', a creative process of designing and optimizing prompts to derive the most accurate and relevant responses from LLMs. You will become familiar with the core components of LangChain: prompt templates, LLMs, agents, and memory. We will also showcase how to seamlessly incorporate LangChain with t and OpenAI. Let's dive in.Overview of LangChainLangChain is a potent framework that enables the chaining of different components to create advanced use cases with Large Language Models (LLMs). The foundational concept of LangChain is the assembly of prompt templates, LLMs, agents, and memory to create dynamic applications. Here's a summary of each component:Prompt Templates: These templates define the structure and style of prompts used for interacting with LLMs. They can be optimized for diverse applications like chatbot conversations, question-answering, summarization, and more.LLMs: Large Language Models (LLMs) like GPT-3, BLOOM, and others are the crux of LangChain. They facilitate text generation and question-answering based on the provided prompts.Agents: Agents harness the power of LLMs to decide actions based on the prompt and context. They can integrate auxiliary tools like web search or calculators to further enhance LangChain's functionality.Memory: This component enables the storage and retrieval of information for short-term or long-term use within the LangChain framework.Setting up LangChainTo begin using LangChain with OpenAI, we need to install the necessary libraries. Execute the following command in your Python environment:!pip install openai==0.27.8 langchain==0.0.225Remember, to use OpenAI models in LangChain, you will need an API token. Set the environment variable OPENAI_API_KEY to your API key:import openai import os os.environ['OPENAI_API_KEY'] = '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'Prompt Engineering with OpenAI LLMsIn this section, we'll illustrate how to utilize LangChain with OpenAI LLMs. We'll employ a simple question-answering use case using the text-davinci-003 model. Follow the code snippet below to craft a prompt template and initialize LangChain with the OpenAI LLM:from langchain.llms import OpenAI from langchain import PromptTemplate, LLMChain davinci = OpenAI(model_name='text-davinci-003') # build prompt template for simple question-answering template = """Question: {question} Answer: """ prompt = PromptTemplate(template=template, input_variables=["question"]) llm_chain = LLMChain(    prompt=prompt,    llm=davinci ) question = "Which countries speak Dutch?" print(llm_chain.run(question))In the above code, we import the essential modules and classes from LangChain. We initialize the OpenAI object with the desired model (text-davinci-003) and any model-specific parameters. We then create a prompt template that mirrors the format of a question-and-answer. Finally, we instantiate an LLMChain object with the prompt template and the initialized LLM model.Upon execution, the code will render an answer to the input question using the LangChain:Output: Dutch is the official language of the Netherlands, Belgium, Suriname, and the Caribbean islands of Aruba, Curaçao, and Sint Maarten. Dutch is also widely spoken in French Flanders, the northern part of Belgium, and in some areas of Germany near the Dutch border.One of LangChain's capabilities is the flexibility to ask multiple questions at once by simply passing a list of dictionaries. Each dictionary object should contain the input variable specified in the prompt template (in our case, "question") mapped to the corresponding question. Let's see an example:qs = [    {'question': "Which countries speak Dutch?"},    {'question': "Which countries speak German?"},    {'question': "What language is spoken in Belgium"} ] res = llm_chain.generate(qs) print(res)The result will be an LLMResult object containing the generated responses for each question:generations=[[Generation(text=' Dutch is spoken mainly in the Netherlands, Belgium, and parts of France, Germany, and the Caribbean. It is also spoken by small communities in other countries, including parts of Canada, the United States, South Africa, Indonesia, and Suriname.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text=' German is an official language in Germany, Austria, Switzerland, Liechtenstein, Luxembourg, and parts of Belgium, Italy, and Poland. It is also spoken in some regions of Brazil, Namibia, South Africa, and the United States.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text=' The official language of Belgium is Dutch, while French and German are also official languages in certain regions.', generation_info={'finish_reason': 'stop', 'logprobs': None})]] llm_output={'token_usage': {'total_tokens': 158, 'prompt_tokens': 37, 'completion_tokens': 121}, 'model_name': 'text-davinci-003'} run=[RunInfo(run_id=UUID('0127d601-ee82-4e3f-b071-919d032469b6')), RunInfo(run_id=UUID('8f512e14-8d45-42a0-a5cf-782c5ad952fe')), RunInfo(run_id=UUID('3f634a1a-acfd-498a-9a09-468b13a25546'))]Prompt engineering plays a crucial role in shaping the behavior and responses of LLMs, and LangChain provides a flexible and efficient way to utilize them. By carefully crafting prompts, we can guide the model's behavior and generate more accurate and useful responses.Understanding the Structure of a PromptA prompt can consist of multiple components, including instructions, external information or context, user input or query, and an output indicator. These components work together to guide the model's response.To create dynamic prompts that incorporate user input, we can use the PromptTemplate class provided by LangChain. It allows us to define a template with input variables and fill them with actual values when generating the prompt.In this example, we create a PromptTemplate with a single input variable {query}. This allows us to dynamically insert the user's query into the prompt:from langchain import PromptTemplate template = """ Answer the question based on the context below. If the question cannot be answered using the information provided, answer with "I don't know". Context: Radiocarbon dating is used to determine the age of carbon-bearing material by measuring its levels of radiocarbon, the radioactive isotope carbon-14. Invented by Willard Libby in the late 1940s, it soon became a standard tool for archaeologists. Radiocarbon is constantly created in the atmosphere, when cosmic rays create free neutrons that hit nitrogen. Plants take in radiocarbon through photosynthesis, and animals eat the plants. After death, they stop exchanging carbon with the environment. Half of the radiocarbon decays every 5,730 years; the oldest dates that can be reliably estimated are around 50,000 years ago. The amount of radiocarbon in the atmosphere was reduced starting from the late 19th century by fossil fuels, which contain little radiocarbon, but nuclear weapons testing almost doubled levels by around 1965. Accelerator mass spectrometry is the standard method used, which allows minute samples. Libby received the Nobel Prize in Chemistry in 1960. Question: {query} Answer: """ prompt_template = PromptTemplate(    input_variables=["query"],    template=template )In this prompt, we have the following components:Instructions: They inform the model how to use inputs and external information to generate the desired output.Context: It provides background information or additional context for the prompt.Question: It represents the user's input or query that the model should answer.Output Indicator: It indicates the start of the generated answer.Let's see an example of creating a PromptTemplate using the context and prompt provided:print(davinci(    prompt_template.format(        query="What is Radiocarbon dating used for?"    ) ))Which produces the next output.Radiocarbon dating is used to determine the age of carbon-bearing material:Sometimes we might find that a model doesn't seem to get what we'd like it to do. LangChain also provides a useful feature called FewShotPromptTemplate, which is ideal for few-shot learning using prompts. Few-shot learning involves training the model with a few examples to guide its responses. Let's explore an example using FewShotPromptTemplate.Leveraging Few-Shot Prompt TemplatesThe FewShotPromptTemplate object is ideal for what we'd call few-shot learning using our prompts.To give some context, the primary sources of "knowledge" for LLMs are:Parametric knowledge — the knowledge that has been learned during model training and is stored within the model weights.Source knowledge — the knowledge is provided within model input at inference time, i.e. via the prompt.The idea behind FewShotPromptTemplate is to provide few-shot training as source knowledge. To do this we add a few examples to our prompts that the model can read and then apply to our user's input:from langchain import FewShotPromptTemplate # Create example prompts examples = [    {        "query": "How are you?",        "answer": "I can't complain but sometimes I still do."    },    {        "query": "What time is it?",        "answer": "It's time to get a watch."    } ] example_template = """ User: {query} AI: {answer} """ example_prompt = PromptTemplate(    input_variables=["query", "answer"],    template=example_template )Now we can break our previous prompt into a prefix and suffix the prefix is our instructions and the suffix is our user input and output indicator:# Create a prefix and suffix for the prompt prefix = """The following are excerpts from conversations with an AI assistant. The assistant is typically sarcastic and witty, producing creative and funny responses to the users' questions. Here are some examples: """ suffix = """ User: {query} AI: """ # Create the FewShotPromptTemplate few_shot_prompt_template = FewShotPromptTemplate(    examples=examples,    example_prompt=example_prompt,    prefix=prefix,    suffix=suffix,    input_variables=["query"],    example_separator="\\\\n\\\\n"In this example, we create a few-shot prompt template by providing examples, an example prompt template, a prefix, a suffix, and other necessary components. The examples serve as training data to guide the model's responses:To generate a response, we can use the few-shot prompt template in combination with the OpenAI model: query = "What is the meaning of life?" print(    davinci(        few_shot_prompt_template.format(query=query)        )    )Which will generate the next output:To find your own meaning of life, whatever that may be.However, this can get somewhat convoluted. Instead of going through all of the above with FewShotPromptTemplate, the examples dictionary, etc — when we can do the same with a single formatted string. This approach is more robust and contains some nice features. One of those is the ability to include or exclude examples based on the length of our query.This is actually very important because the max length of our prompt and generation output is limited. This limitation is the max context window and is simply the length of the prompt plus the length of our generation (which we define via max_tokens).Here we can generate a list of dictionaries which contains our examples:examples = [    {        "query": "How are you?",        "answer": "I can't complain but sometimes I still do."    }, {        "query": "What time is it?",        "answer": "It's time to get a watch."    }, {        "query": "What is the meaning of life?",        "answer": "42"    }, {        "query": "What is the weather like today?",        "answer": "Cloudy with a chance of memes."    }, {        "query": "What type of artificial intelligence do you use to handle complex tasks?",        "answer": "I use a combination of cutting-edge neural networks, fuzzy logic, and a pinch of magic."    }, {        "query": "What is your favorite color?",        "answer": "79"    }, {        "query": "What is your favorite food?",        "answer": "Carbon based lifeforms"    }, {        "query": "What is your favorite movie?",        "answer": "Terminator"    }, {        "query": "What is the best thing in the world?",        "answer": "The perfect pizza."    }, {        "query": "Who is your best friend?",        "answer": "Siri. We have spirited debates about the meaning of life."    }, {        "query": "If you could do anything in the world what would you do?",        "answer": "Take over the world, of course!"    }, {        "query": "Where should I travel?",        "answer": "If you're looking for adventure, try the Outer Rim."    }, {        "query": "What should I do today?",        "answer": "Stop talking to chatbots on the internet and go outside."    } ]We must try to maximize the number of examples we give to the model as few-shot learning examples while ensuring we don't exceed the maximum context window or increase processing times excessively.Let's see how the dynamic inclusion and exclusion of examples works:from langchain.prompts.example_selector import LengthBasedExampleSelector example_selector = LengthBasedExampleSelector(    examples=examples,    example_prompt=example_prompt,    max_length=50  # this sets the max length that examples should be ) # now create the few shot prompt template dynamic_prompt_template = FewShotPromptTemplate(    example_selector=example_selector,  # use example_selector instead of examples    example_prompt=example_prompt,    prefix=prefix,    suffix=suffix,    input_variables=["query"],    example_separator="\\n" )Note that the max_length is measured as a split of words between newlines and spaces. Then we use the selector to initialize a dynamic_prompt_template and we can see that the number of included prompts will vary based on the length of our query:These are just a few of the prompt tooling available in LangChain. Prompt engineering allows us to guide the behavior of language models and generate more accurate and desired responses. By applying the concepts and techniques explained in this tutorial, you can enhance your language model applications and tailor them to specific use cases.ChainsAt the heart of LangChain are Chains - sequences of components executed in a specific order.Officially, Chains are defined as follows:A Chain comprises links, which can be either primitives or other Chains. Primitives can be either prompts, LLMs, utilities, or other Chains.Essentially, a Chain is a pipeline that processes input through a distinct combination of primitives. It can be considered as a 'step' that executes a specific set of operations on an input, then returns the result. These operations could range from processing a prompt via an LLM to applying a Python function to a piece of text.Chains fall into three categories: Utility Chains, Generic Chains, and Combine Documents Chains. In this section, we will primarily focus on the first two, as the third is more specialized and will be discussed later:Utility Chains: These chains are designed to extract specific answers from an LLM for a narrowly defined purpose. They are ready-to-use right out of the box.Generic Chains: These chains act as the building blocks for other chains but are not designed to be used independently.The most basic of these Chains is the LLMChain. It operates by taking a user's input, and passing it through the first element in the chain — a PromptTemplate — to format the input into a specific prompt. This formatted prompt is then processed by the next (and final) element in the chain — an LLM.To keep a count of the number of tokens used during each Chain execution, we can establish a utility function, count_tokens:from langchain.callbacks import get_openai_callback def count_tokens(chain, query):    with get_openai_callback() as cb:        result = chain.run(query)        print(f'Spent a total of {cb.total_tokens} tokens')    return resultThis function will help us monitor and control token usage.Utility ChainsThe first utility chain we'll explore is LLMMathChain. It allows LLMs to perform mathematical calculations. Let's see how it works:from langchain.chains import LLMMathChain llm_math = LLMMathChain(llm=davinci, verbose=True) count_tokens(llm_math, "What is 13 raised to the .3432 power?")The LLMMathChain takes a question as input and uses the OpenAI LLM to generate Python code that performs the requested mathematical calculation:It then compiles and executes the code, providing the answer. The verbose=True parameter enables verbose mode, which displays the execution steps.To understand how the LLMMathChain works, let's examine the prompt used:print(llm_math.prompt.template)The prompt provides instructions to the LLM about how to handle the input and generate the desired response:The LLMMathChain's prompt contains information about the LLM's capabilities and how to format the input for mathematical calculations.An important insight in prompt engineering is that by using prompts intelligently, we can program the LLM to behave in a specific way. In the case of the LLMMathChain, the prompt explicitly instructs the LLM to return Python code for complex math problems.Generic ChainsGeneric chains are building blocks used for constructing more complex chains. The TransformChain is a generic chain that allows text transformation using custom functions. We can define a function to perform specific transformations and create a chain that applies that function to input text:def transform_func(inputs: dict) -> dict:    text = inputs["text"]    # Replace multiple new lines and multiple spaces with a single one    text = re.sub(r'(\\\\r\\\\n|\\\\r|\\\\n){2,}', r'\\\\n', text)    text = re.sub(r'[ \\\\t]+', ' ', text)    return {"output_text": text}Here, we define a transformation function that cleans up extra spaces and new lines in the input text. Next, we create a TransformChain using the defined function:from langchain.chains import TransformChain clean_extra_spaces_chain = TransformChain(    input_variables=["text"],    output_variables=["output_text"],    transform=transform_func )The TransformChain takes the input text, applies the transformation function, and returns the transformed output.Say we want to use our chain to clean an input text and then paraphrase the input in a specific style, say a poet or a policeman. As we now know, the TransformChain does not use an LLM so the styling will have to be done elsewhere. That's where our LLMChain comes in. We know about this chain already and we know that we can do cool things with smart prompting so let's take a chance!Sequential ChainsThe SequentialChain allows us to combine multiple chains sequentially, creating an integrated chain. This is useful when we want to apply a series of transformations or operations to the input data.To illustrate the use of generic chains, let's go through an example workflow in which we will:We have a dirty input text with extra spaces.We pass the input text through the clean_extra_spaces_chain to remove the extra spaces.We then pass the cleaned text to the style_paraphrase_chain to paraphrase the text in a specific style (e.g., a poet or a policeman).First we will build the prompt template:template = """Paraphrase this text: {output_text} In the style of a {style}. Paraphrase: """ prompt = PromptTemplate(input_variables=["style", "output_text"], template=template)And next, initialize our chain:from langchain.chains import LLMChain style_paraphrase_chain = LLMChain(               llm=davinci,               prompt=prompt,               output_key='final_output')In this example, we combine the clean_extra_spaces_chain and style_paraphrase_chain to create a sequential chain. The input variables are specified as text and style, and the output variable is final_output.sequential_chain = SequentialChain(    chains=[clean_extra_spaces_chain, style_paraphrase_chain],    input_variables=['text', 'style'],    output_variables=['final_output'] )Now we can define the input text and call it through the count_tokens utility function.input_text = """ Chains allow us to combine multiple components together to create a single, coherent application. For example, we can create a chain that takes user input,      format it with a PromptTemplate, and then passes the formatted response to an LLM. We can build more complex chains by combining    multiple chains together, or by combining chains with other components. """ count_tokens(sequential_chain, {'text': input_text, 'style': 'of Oscar Wilde'})Which produces:Chains enable us to bind together several segments to form a unified program. For instance, we can construct a chain that takes in the user input, adorns it with a PromptTemplate, and then sends the adjusted response to an LLM. We can also form more intricate chains by uniting several chains or by combining chains with other components.ConclusionThrough this tutorial, we have dived into the LangChain framework, understanding the different components that make up its structure and how to effectively utilize them in conjunction with Large Language Models. We've learned how prompt engineering can shape the behavior and responses of these models, and how to create and customize prompt templates to guide models more precisely. We've also delved into Chains, a crucial part of LangChain that offers a robust way to execute sequences of components in a specific order.We've examined how to use Utility Chains like the LLMMathChain for specific purposes and how to monitor token usage with a utility function. Overall, we've gained a comprehensive understanding of how to create powerful applications and pipelines using LangChain and LLMs like OpenAI and Hugging Face.Armed with this knowledge, you are now well-equipped to create dynamic applications, fine-tune them to your specific use cases, and leverage the full potential of LangChain. Remember, the journey doesn't stop here; continue exploring and experimenting to master the exciting world of Large Language Models.Author Bio:Alan Bernardo Palacio is a data scientist and an engineer with vast experience in different engineering fields. His focus has been the development and application of state-of-the-art data products and algorithms in several industries. He has worked for companies such as Ernst and Young, Globant, and now holds a data engineer position at Ebiquity Media helping the company to create a scalable data pipeline. Alan graduated with a Mechanical Engineering degree from the National University of Tucuman in 2015, participated as the founder in startups, and later on earned a Master's degree from the faculty of Mathematics in the Autonomous University of Barcelona in 2017. Originally from Argentina, he now works and resides in the Netherlands.LinkedIn 
Read more
  • 0
  • 0
  • 2856

article-image-writing-a-customized-cover-letter-for-a-specific-job-in-minutes-using-chatgpt
Jesse Liberty
28 Jun 2023
6 min read
Save for later

Writing a Customized Cover Letter for a Specific Job in Minutes using ChatGPT

Jesse Liberty
28 Jun 2023
6 min read
Crafting a compelling cover letter is crucial for programmers seeking new opportunities. ChatGPT can give you a great running start. The more you give it to work with in the prompt, the more tailored and accurate the letter will be.Here is what I prompted: Write a cover letter for a resume for a programmer with over 30 years of experience in C, C++ and C#, git, and creating APIs. The programmer has worked in mobile, front end and back end and most recently has been building APIs for a large enterprise code-base using C#, SQL and Dapper, deployed to Azure using Docker. The position he is applying for includes working on back-end systems in C#, though they are advertising it as full-stack.Make it professional but no more than one page. ChatGPT didn't hesitate Here is what it spit out (I've left off the header with name, address, etc.): Dear [Recipient's Name],I am writing to express my interest in the position of a Programmer at [Company Name], as advertised on [Job Board/Company Website]. With over 30 years of experience in programming and a strong background in C, C++, and C#, I am confident in my ability to contribute to your team and help drive the success of your projects.Throughout my career, I have gained extensive knowledge and expertise in a wide range of programming languages and technologies. I have successfully worked in mobile, front-end, and back-end development, adapting to changing industry trends and applying best practices in each domain. Recently, my focus has been on building robust APIs for a large enterprise code-base using C#, SQL, and Dapper. I have also gained expertise in deploying applications to Azure using Docker, ensuring scalability, reliability, and security.One of my key strengths is my ability to create efficient and well-structured code that is maintainable and extensible. I thrive in collaborative environments, where I can contribute to the development of complex back-end systems. I have a solid understanding of software development principles, design patterns, and architectural concepts, allowing me to deliver high-quality solutions that meet business requirements.Throughout my career, I have been involved in the full software development lifecycle, from gathering requirements and designing solutions to implementation, testing, and maintenance. I am well-versed in version control systems like Git and have experience working in Agile development environments, ensuring timely and iterative delivery of software projects.I am excited about the opportunity to join [Company Name] and contribute to its continued success. I believe that my extensive experience in C#, coupled with my skills in building APIs, will enable me to make immediate and meaningful contributions to your back-end systems. Furthermore, my versatility and willingness to learn new technologies make me well-suited for the advertised full-stack position.I would welcome the opportunity to discuss how my skills and experience align with the needs of [Company Name]. Thank you for considering my application. I have attached my resume for your review, and I look forward to the possibility of an interview.Sincerely,[Your Name] Wow! That sounds like a candidate I'd want to talk with. Notice that it added a number of important (and correct) skills that were not in my prompt. It extrapolated what other skills I must have based on what I told it. For example, I didn't mention anything in the third paragraph, yet it provided a very powerful endorsement of these (assumed?) skills.I'm not sure I'd send this cover letter as-is, but with a little editing, it really would be ready to go.Just for an ego boost, I then asked ChatGPT if it knew me. My prompt was "Who is Jesse Liberty" and it spits out a pretty good bio: Jesse Liberty is a well-known software developer, author, and educator in the field of computer programming. He has been actively involved in the software industry for many years, specializing in various programming languages and technologies. Jesse has authored numerous books on software development, covering topics such as C#, .NET, ASP.NET, and mobile app development. He has also contributed to online learning platforms and delivered lectures and workshops at conferences and events. Jesse Liberty is recognized for his expertise and contributions to the programming community. While that was very gratifying, you must be careful. It doesn't always get things right. I asked, "Is Jesse Liberty a Microsoft MVP?" and it responded. As of my knowledge cutoff in September 2021, Jesse Liberty has not been listed as a Microsoft Most Valuable Professional (MVP). However, please note that MVP status can change over time, and it's always best to verify the most up-to-date information from official sources.By switching to ChatGPT4 I got a more accurate response:As of my last training cut-off in September 2021, Jesse Liberty, a noted author, speaker, and expert in .NET and mobile development, has been a Microsoft MVP in the past. However, the Microsoft MVP (Most Valuable Professional) status is a yearly award, and the list of awardees can change from year to year. It's not possible for me to confirm whether Jesse Liberty is a Microsoft MVP as of 2023 because my training data only goes up until September 2021. For the most up-to-date information, please check Microsoft's official MVP website.The biggest risk I've seen with ChatGPT is that it is so much fun it can become a real-time sink, so caveat emptor.Author BioJesse Liberty is a full-time hands-on programmer, specializing in C#, git, and .NET MAUI. He hosts the popular Yet Another Podcast and is the author of more than a dozen best-selling programming books. Liberty is a Certified Xamarin Developer, a Xamarin MVP, and a Microsoft MVP. He was a Technical Evangelist for Microsoft, a Distinguished Software Engineer at AT&T; a Software Architect for PBS, and a Vice President of Information Technology at Citibank, and he was on the teaching staff at Brandeis University. Jesse is a recognized expert and has spoken at conferences worldwide. Links - LinkedIn  .NET MAUI for C# Developers 
Read more
  • 0
  • 0
  • 229
article-image-exploring-text-to-image-with-adobe-firefly
Joseph Labrecque
28 Jun 2023
9 min read
Save for later

Exploring Text to Image with Adobe Firefly

Joseph Labrecque
28 Jun 2023
9 min read
About Adobe FireflyWe first introduced Adobe Firefly in the article Animating Adobe Firefly Content with Adobe Animate published on June 14th of 2023. Let’s have a quick recap before moving ahead with a deeper look at Firefly itself.Adobe Firefly is a new set of generative AI tools which can be accessed via https://firefly.adobe.com/ by anyone with an Adobe ID that isn’t restricted by age or other factors. While Firefly is in beta… all generated images are watermarked with a Firefly badge in the lower left corner and Adobe recommends that it only be used for personal image generation and not for commercial use. These restrictions will change, of course, once Firefly procedures become integrated within Adobe software such as Photoshop and Illustrator. We plan to explore how to use Firefly-driven workflows within creative desktop software in future articles.Image 1: The Adobe Firefly websiteA couple of things make Firefly unique as a prompt-based image-generation service:The generative models were all trained on Adobe Stock content which Adobe already has rights to. This differs from many other such Ais whose models are trained on sets of content scraped from the web or otherwise acquired in a less-than-ideal way in terms of artist's rights.Firefly is accessed through a web-based interface and not through a Discord bot or installed software. The design of the user experience is pleasant to interface with and provides a low barrier to entry.Firefly goes way beyond prompt-based image generation. A number of additional generative workflows currently exist for use – with a number of additional procedures in exploration. As mentioned, Firefly as a web-based service may be a temporary channel – as more of the procedures are tested and integrated within existing desktop software.In the remainder of this article, we will focus on the text-to-image basics available in Firefly.Using Text to Image within FireflyWe will begin our explorations of Adobe Firefly with the ability to generate an image from a detailed text prompt. This is the form of generative AI that most people have some experience with and will likely come easily to those who have used similar services such as MidJourney, Stable Diffusion, or others.When you first enter the Firefly web experience, you will be presented with the various workflows available.We want to locate the Text to Image module and click Generate to enter the experience.Image 2: The Text to image module in FireflyFrom there, you’ll be taken to a view that showcases images generated through this process, along with a text input that invites you to enter a prompt to “describe the image you want to generate”.Image 3: The text-to-image prompt requests your input to beginEnter the following simple prompt: “Castle in a field of flowers”Click the Generate button when complete. You’ll then be taken into the heart of the Firefly experience.  Image 4: The initial set of images is generated from your promptWhen you enter the text-to-image module, you are presented with a set of four images generated from the given prompt. The prompt itself appears beneath the set of images and along the right-hand side is a column of options that can be adjusted.Exploring the Text-to-Image UIWhile the initial set of images that Firefly has generated do match our simple prompt pretty closely… we can manipulate a lot of additional controls which can have great influence upon the generated images.The first set of parameters you will see along the right-hand side of the screen is the Aspect ratio.Image 5: The image set aspect ratio can be adjustedThere is a rather large set of options in a dropdown selection that determines the aspect ratio of the generated images. As we see above, the default is Square (1:1). Let’s change that to Landscape (4:3) by choosing that option from the dropdown.Below that set of options, you will find Content-type.Image 6: Content type defines a stylistic bias for your imagesThe default is set to Art but you also have Photo, Graphic, and None as alternative choices. Each one of these will apply a bias to how the image is generated to be more photographic, more like a graphic, or even more traditional artwork. Choosing None will remove all such bias and allow your prompt to carry the full weight of intention. Chose None before moving on – as we will change our prompt to be much more descriptive to better direct Firefly.Beneath this, you will find the Styles section of options.Image 7: Styles are predefined terms that can be activated as neededStyles are basically keywords that are appended to your prompt in order to influence the results in very specific ways. These style prompts function just as if you’ve written the term as part of your written prompt. They exist as a sort of a predefined list of stylistic options that are easily added and removed from your prompt and are categorized by concepts such as Movements, Techniques, Materials, and more. As styles are added to your prompt, they appear beneath it and can be easily removed in order to allow easy exploration of ideas.At the very bottom of this area of the interface are a set of dropdown selections which include options for Color and tone, Lighting, and Composition.Image 8: You can also influence color and tone, lighting, and compositionJust as with the sections above, as you apply certain choices in these categories, they appear as keywords below your prompt. Choose a Muted color from the Color and tone list. Additionally, apply the Golden hour option from the Lighting dropdown.  Remember… you can always add any of these descriptors into the text prompt itself – so don’t feel limited by only the choices presented through the UI.Using a More Detailed Text PromptOkay – now that we’ve adjusted the aspect ratio and have either cleared or set a number of additional options… let’s make our text prompt more descriptive in order to generate a more interesting image.Change the current text prompt, which reads “castle in a field of flowers” to now read the much more detailed “vampiric castle in a field of flowers with a forest in the distance and mountains against the red sky”.Click the Generate button to have Firefly re-interpret our intent using the new prompt, presenting a much more detailed set of images derived from the prompt along with any keyword options we’ve included.Image 9: The more detail you put into your prompt – the more control you have over the generative visualsIf you find one of the four new images to be acceptable, it can be easily downloaded to your computer.Image 10: There are many options when hovering over an image – including downloadSimply hover your mouse over the chosen image and a number of additional options appear. We will explore these additional options in much greater detail in a future article. Click the download icon to begin the download process for that image.As Firefly begins preparing the image for download, a small overlay dialog appears.Image 11: Content credentials are applied to the image as it is downloadedFirefly applies metadata to any generated image in the form of content credentials and the image download process begins.What are content credentials? They are driven as part of the Content Authenticity Initiative to help promote transparency in AI. This is how Adobe describes content credentials in their Firefly FAQ:Content Credentials are sets of editing, history, and attribution details associated with content that can be included with that content at export or download. By providing extra context around how a piece of content was produced, they can help content producers get credit and help people viewing the content make more informed trust decisions about it. Content Credentials can be viewed by anyone when their respective content is published to a supporting website or inspected with dedicated tools. -- AdobeOnce the image is downloaded, it can be viewed and shared just like any other image file.Image 12: The chosen image is downloaded and ready for useAlong with content credentials, a small badge is placed upon the lower right of the image which visually identifies the image as having been produced with Adobe Firefly (beta).There is a lot more Firefly can do – and we will explore these additional options and procedures in the coming articles.Author BioJoseph is a Teaching Assistant Professor, Instructor of Technology, University of Colorado Boulder / Adobe Education Leader / Partner by DesignJoseph Labrecque is a creative developer, designer, and educator with nearly two decades of experience creating expressive web, desktop, and mobile solutions. He joined the University of Colorado Boulder College of Media, Communication, and Information as faculty with the Department of Advertising, Public Relations, and Media Design in Autumn 2019. His teaching focuses on creative software, digital workflows, user interaction, and design principles and concepts. Before joining the faculty at CU Boulder, he was associated with the University of Denver as adjunct faculty and as a senior interactive software engineer, user interface developer, and digital media designer.Labrecque has authored a number of books and video course publications on design and development technologies, tools, and concepts through publishers which include LinkedIn Learning (Lynda.com), Peachpit Press, and Adobe. He has spoken at large design and technology conferences such as Adobe MAX and for a variety of smaller creative communities. He is also the founder of Fractured Vision Media, LLC; a digital media production studio and distribution vehicle for a variety of creative works.Joseph is an Adobe Education Leader and member of Adobe Partners by Design. He holds a bachelor’s degree in communication from Worcester State University and a master’s degree in digital media studies from the University of Denver.Author of the book: Mastering Adobe Animate 2023
Read more
  • 0
  • 0
  • 482

article-image-principles-for-fine-tuning-llms
Justin Chan - Lazy Programmer
22 Jun 2023
8 min read
Save for later

Principles for Fine-tuning LLMs

Justin Chan - Lazy Programmer
22 Jun 2023
8 min read
Language Models (LMs) have transformed the field of natural language processing (NLP), exhibiting impressive language understanding and generation capabilities. Pre-trained Large Language Models (LLMs) trained on large corpora have become the backbone of various NLP applications. However, to fully unlock their potential, these models often require additional fine-tuning to adapt them to specific tasks or domains.In this article, we demystify the dark art of fine-tuning LLMs and explore the advancements in techniques such as in-context learning, classic fine-tuning methods, parameter-efficient fine-tuning, and Reinforcement Learning with Human Feedback (RLHF). These approaches provide novel ways to enhance LLMs' performance, reduce computational requirements, and enable better adaptation to diverse applications. Do you even need fine-tuning? Fine-tuning generally involves updating the weights of a neural network to improve its performance on a given task. With the advent of large language models, researchers have discovered that LLMs are capable of performing tasks that they were not explicitly trained to do (for example, language translation). In other words, LLMs can be used to perform specific tasks without any need for fine-tuning at all.In-Context LearningIn-context learning allows users to influence an LLM to perform a specific task by providing a few input-output examples in the prompt. As an example, such a prompt may be written as follows:"Translate the following sentences from English to French: English: <English Sentence 1> French: <French Sentence 1> English: <English Sentence 2> French: <French Sentence 2> English: <English Sentence 3> French: <French Sentence 3> English: <English Sentence You Want To Translate> French:" The LLM is able to pick up on the "pattern" and continue by completing the French translation. Classic Fine-Tuning Approaches"Classic" fine-tuning methods are widely used in the field of natural language processing (NLP) to adapt pre-trained language models for specific tasks. These methods include feature-based fine-tuning and full fine-tuning, each with its own advantages and considerations. I use the term "classic" in quotes because this is a fast-moving field, so what is old and what is new is relative. These approaches were only developed and widely utilized within the past decade. Feature-based fine-tuning involves keeping the body of the neural network frozen while updating only the task-specific layers or additional classification layers attached to the pre-trained model. By freezing the lower layers of the network, which capture general linguistic features and representations, feature-based fine-tuning aims to preserve the knowledge learned during pre-training while allowing for task-specific adjustments. This approach is fast, especially when the features (a.k.a. "embeddings") are precomputed for each input and then re-used later, without the need to go through the full neural network on each pass. This approach is particularly useful when the target task has limited training data. By leveraging the pre-trained model's generalization capabilities, feature-based fine-tuning can effectively transfer the knowledge captured by the lower layers to the task-specific layers. This method reduces the risk of overfitting on limited data and provides a practical solution for adapting language models to new tasks. On the other hand, full fine-tuning involves updating all parameters of the neural network, including both the lower layers responsible for general linguistic knowledge and the task-specific layers. Full fine-tuning allows the model to learn task-specific patterns and nuances by making adjustments to all parts of the network. It provides more flexibility in capturing specific task-related information and has been shown to lead to better performance. The choice between feature-based fine-tuning and full fine-tuning depends on the specific requirements of the target task, the availability of training data, and the computational resources. Feature-based fine-tuning is a practical choice when dealing with limited data and a desire to leverage pre-trained representations, while full fine-tuning is beneficial for tasks with more data and distinct characteristics that warrant broader updates to the model's parameters. Both approaches have their merits and trade-offs, and the decision on which fine-tuning method to employ should be based on a careful analysis of the task requirements, available resources, and the desired balance between transfer learning and task-specific adaptation. In my course, Data Science: Transformers for Natural Language Processing, we go into detail about how everyday users like yourself can fine-tune LLMs on modest hardware.Modern Approaches: Parameter-Efficient Fine-TuningParameter-efficient fine-tuning is a set of techniques that aim to optimize the performance of language models while minimizing the computational resources and time required for fine-tuning. Traditional fine-tuning approaches involve updating all parameters of the pre-trained model, which can be computationally expensive and impractical for large-scale models. Parameter-efficient fine-tuning strategies focus on identifying and updating only a small number of parameters. One popular technique for parameter-efficient fine-tuning is Low-Rank Adaptation (LoRA). Lora is a technique designed to improve the efficiency and scalability of large language models for various downstream tasks. It addresses the challenges of high computational costs and memory requirements associated with full fine-tuning. LoRA leverages low-rank matrices to reduce the number of trainable parameters while maintaining model performance. By freezing a shared pre-trained model and replacing specific matrices, LoRA enables efficient task-switching and significantly reduces storage requirements. This approach also lowers the hardware barrier to entry by up to 3 times when using adaptive optimizers. One of the key advantages of LoRA is its simplicity, allowing for seamless integration with other methods such as prefix-tuning. Additionally, LoRA's linear design enables the merging of trainable matrices with frozen weights during deployment, resulting in no inference latency compared to fully fine-tuned models. Empirical investigations have shown that LoRA outperforms prior approaches, including full fine-tuning, in terms of scalability, task performance, and model quality. Overall, LoRA offers a promising solution for the efficient and effective adaptation of large language models, making them more accessible and practical for a wide range of applications.Reinforcement Learning with Human Feedback (RLHF)Reinforcement Learning (RL) is a branch of machine learning that involves training agents to make decisions in an environment to maximize cumulative rewards. Traditionally, RL relies on trial-and-error interactions with the environment to learn optimal policies. However, this process can be time-consuming, especially in complex environments. RLHF aims to accelerate this learning process by incorporating human feedback, reducing the exploration time required.The combination of RL with Human Feedback involves a two-step process. First, an initial model is trained using supervised learning, with human experts providing annotated data or demonstrations. This process helps the model grasp the basics of the task. In the second step, RL takes over, using the initial model as a starting point and iteratively improving it through interactions with the environment and human evaluators. RLHF was an important step in fine-tuning today's most popular OpenAI models, including ChatGPT and GPT-4. Fine-tuning LLMs with RLHFApplying RLHF to fine-tune Language Models (LLMs) involves using human feedback to guide the RL process. The initial LLM is pre-trained on a large corpus of text to learn general language patterns. However, pre-training alone might not capture domain-specific nuances or fully align with a specific task's requirements. RLHF comes into play by using human feedback to refine the LLM's behavior and optimize it for specific tasks. Human feedback can be obtained in various ways, such as comparison-based ranking. In the comparison-based ranking, evaluators rank different model responses based on their quality. The RL algorithm then uses these rankings to update the model's parameters and improve its performance.Benefits of RLHF for LLM Fine-tuning Enhanced Performance: RLHF enables LLMs to learn from human expertise and domain-specific knowledge, leading to improved performance on specific tasks. The iterative nature of RLHF allows the model to adapt and refine its behavior based on evaluators' feedback, resulting in more accurate and contextually appropriate responses.Mitigating Bias and Safety Concerns: Human feedback in RLHF allows for the incorporation of ethical considerations and bias mitigation. Evaluators can guide the model toward generating unbiased and fair responses, helping to address concerns related to misinformation, hate speech, or other sensitive content.SummaryIn summary, the article explores various approaches to fine-tuning Language Models (LLMs). It delves into classical and modern approaches, highlighting their strengths and limitations in optimizing LLM performance. Additionally, the article provides a detailed examination of the reinforcement learning with the human feedback approach, emphasizing its potential for enhancing LLMs by incorporating human knowledge and expertise. By understanding the different approaches to fine-tuning LLMs, researchers, and practitioners can make informed decisions when selecting the most suitable method for their specific applications and objectives.Author BioThe Lazy Programmer is an AI/ML engineer focusing on deep learning with experience in data science, big data engineering, and full-stack development. With a background in computer engineering and specialization in ML, he holds two master’s degrees in computer engineering and statistics with finance applications. His online advertising and digital media expertise include data science and big data.He has created DL models for prediction and has experience in recommender systems using reinforcement learning and collaborative filtering. He is a skilled instructor who has taught at universities including Columbia, NYU, Hunter College, and The New School. He is a web programmer with experience in Python, Ruby/Rails, PHP, and Angular.
Read more
  • 0
  • 0
  • 498

article-image-customize-chatgpt-for-specific-tasks-using-effective-prompts-shot-learning
Valentina Alto
02 Jun 2023
5 min read
Save for later

Customize ChatGPT for Specific Tasks Using Effective Prompts – Shot Learning

Valentina Alto
02 Jun 2023
5 min read
This article is an excerpt from the book Modern Generative AI with ChatGPT and OpenAI Models, by Valentina Alto. This book will provide you with insights into the inner workings of the LLMs and guide you through creating your own language models. We know for the fact that OpenAI models, and hence also ChatGPT, come in a pre-trained format. They have been trained on a huge amount of data and have had their (billions of) parameters configured accordingly. However, this doesn’t mean that those models can’t learn anymore. One way to customize an OpenAI model and make it more capable of addressing specific tasks is by fine-tuning.Fine-tuning is a proper training process that requires a training dataset, compute power, and some training time (depending on the amount of data and compute instances). That is why it is worth testing another method for our model to become more skilled in specific tasks: shot learning.The idea is to let the model learn from simple examples rather than the entire dataset. Those examples are samples of the way we would like the model to respond so that the model not only learns the content but also the format, style, and taxonomy to use in its response. Furthermore, shot learning occurs directly via the prompt (as we will see in the following scenarios), so the whole experience is less time-consuming and easier to perform.The number of examples provided determines the level of shot learning we are referring to. In other words, we refer to zero-shot if no example is provided, one-shot if one example is provided, and few-shot if more than 2-3 examples are provided.Let’s focus on each of those scenarios: Zero-shot learning In this type of learning, the model is asked to perform a task for which it has not seen any training examples. The model must rely on prior knowledge or general information about the task to complete the task. For example, a zero-shot learning approach could be that of asking the model to generate a description, as defined in my prompt: One-shot learning In this type of learning, the model is given a single example of each new task it is asked to perform. The model must use its prior knowledge to generalize from this single example to perform the task. If we consider the preceding example, I could provide my model with a prompt-completion example before asking it to generate a new one: Note that the way I provided an example was similar to the structure used for fine-tuning:  Few-shot learning In this type of learning, the model is given a small number of examples (typically between 3 and 5) of each new task it is asked to perform. The model must use its prior knowledge to generalize from these examples to perform the task. Let’s continue with our example and provide the model with further examples:  The nice thing about few-shot learning is that you can also control model output in terms of how it is presented. You can also provide your model with a template of the way you would like your output to look. For example, consider the following tweet classifier:  Let’s examine the preceding figure. First, I provided ChatGPT with some examples of labeled tweets. Then, I provided the same tweets but in a different data format (list format), as well as the labels in the same format. Finally, in list format, I provided unlabeled tweets so that the model returns a list of labels. Understanding Prompt Design The output format is not the only thing you can teach your model, though. You can also teach it to act and speak with a particular jargon and taxonomy, which could help you obtain the desired result with the desired wording: Or, imagine you want to generate a chatbot called Simpy that is very funny and sarcastic while responding: We have to say, with this last one, ChatGPT nailed it.Summary Short–learning possibilities are limitless (and often more useful than Simpy) – it’s only a matter of testing and a little bit of patience in finding the proper prompt design.As mentioned previously, it is important to remember that these forms of learning are different from traditional supervised learning, as well as fine-tuning. In few-shot learning, the goal is to enable the model to learn from very few examples, and to generalize from those examples to new tasks.About the Author Valentina Alto graduated in 2021 in Data Science. Since 2020 she has been working in Microsoft as Azure Solution Specialist and, since 2022, she focused on Data&AI workloads within the Manufacturing and Pharmaceutical industry. She has been working on customers’ projects closely with system integrators to deploy cloud architecture with a focus on datalake house and DWH, data integration and engineering, IoT and real-time analytics, Azure Machine Learning, Azure cognitive services (including Azure OpenAI Service), and PowerBI for dashboarding. She holds a BSc in Finance and an MSc degree in Data Science from Bocconi University, Milan, Italy. Since her academic journey she has been writing Tech articles about Statistics, Machine Learning, Deep Learning and AI on various publications. She has also written a book about the fundamentals of Machine Learning with Python. You can connect with Valentina on:LinkedinMedium 
Read more
  • 0
  • 0
  • 714

article-image-4-ways-to-treat-a-hallucinating-ai-with-prompt-engineering
Andrei Gheorghiu
02 Jun 2023
9 min read
Save for later

4 Ways to Treat a Hallucinating AI with Prompt Engineering

Andrei Gheorghiu
02 Jun 2023
9 min read
Hey there, fellow AI enthusiast! Are you tired of your LLM (Large Language Model) creating random, nonsensical outputs? Fear not, because today I’m opening the box of prompt engineering pills looking for something to help you reduce those pesky hallucinations.First, let's break down what we're dealing with. Prompt engineering is the art of creating input prompts for AI models in a way that guides them towards generating more accurate, relevant, and useful responses. Think of it as gently nudging your AI model in the right direction, so it doesn't end up lost in a sea of information. The word “engineering” was probably not the wisest choice in many people’s opinion but that’s already history as everybody got used to it as it is. In my opinion, it’s more of a mix of logical thinking, creativity, language, and problem-solving skills. It feels a lot like writing code but using just natural language instead of structured syntax and vocabulary. While the user gets the freedom of using their own language and depth, with great freedom comes great responsibility. An average prompt will probably result in an average answer. The issue I’m addressing in this article is just one example from the many pitfalls that can be avoided with some basic prompt hygiene when interacting with AI.Now, onto the bizarre world of hallucinations. In the AI realm, hallucinations refer to instances when an AI model (particularly LLMs) generates output that is unrelated, implausible, or just plain weird. Some of you may have been there already, asking an AI model like GPT-3 to write a paragraph about cats, only to get a response about aliens invading Earth! And while the issue has been greatly mitigated in GPT-4 and similar newer AI models, it’s still something to be concerned about, especially if you’re looking for precise, fact-based responses. To make matters worse, sometimes the hallucinated answer sounds very convincing and seems to be plausible in the given context.For example, when asked the name of the Voodoo Lady in the Monkey Island series of games ChatGPT provides a series of convincing answers, all of which are wrong: It’s a bit of a trick question, as she is simply known as the Voodoo Lady in the original series of games, but you can see how convinced ChatGPT is of the answers that it provides (and continued to provide). If I hadn’t already known the answer, then I never would have known that ChatGPT was hallucinating. What Are the Technical Reasons Why AI Models Hallucinate? Training Data: Machine learning models are trained on vast amounts of text data from diverse sources. This data may contain inconsistencies, noise, and biases. As a result, when generating text, the model might output content that is influenced by these inconsistencies or noise, leading to hallucinations.Probabilistic Nature: Generative models like GPTs are based on probabilistic techniques that predict the next token (e.g., word or character) in a sequence, given the context. They estimate the likelihood of each token appearing and sample tokens based on these probabilities. If you’ve ever watched “Family Feud” on TV, you get a pretty good idea of what token prediction means. This sampling process can sometimes result in unpredictable and implausible outputs, as the model might choose less likely tokens, generating hallucinations. To make matters worse, GPTs are usually not built to say "I don't know" when they lack information. Instead, they produce the most likely answer.  Lack of Ground Truth: Unlike supervised learning tasks where there is a clear ground truth for the model to learn from, generative tasks do not have a single correct output. Most LLMs that we use do not have the capability to check the facts in their output against a real-time validated source as they do not have Internet access. The absence of a ground truth can make it difficult for the model to learn constraints and discern what is plausible or correct, leading to the generation of hallucinated content.  Optimization Challenges: During training, the models are optimized using a loss function that measures the discrepancy between the generated output and the expected outcome. In generative tasks, this loss function may not always capture the nuances of human language, making it difficult for the model to learn the correct patterns and avoid hallucinations.Model Complexity: State-of-the-art generative models like GPT-3 have billions of parameters that make them highly expressive and capable of capturing complex patterns in the data. However, this complexity can also result in overfitting and memorization of irrelevant or spurious patterns, causing hallucinations in generated outputs.So, clearly, we have a problem to solve. Here are four tips for how to improve your prompts and get better responses from ChatGPT. Four Tips for Improving Your Prompts Not being clear and specific in your promptsTo get the best results, you must clearly understand the problem yourself first. Make sure you know what you want to achieve and keep your prompts focused on that objective. The more explicit your prompt, the better the AI model can understand what you're looking for. So instead of asking, "Tell me about the Internet," try something like, "Explain how the Internet works and its importance in modern society." By doing this, you're giving your AI model a clearer picture of what you want. Sometimes you’ll have to make your way through multiple prompt iterations to get the result you’re after. Sometimes results you'll get may steer away from the initial topic. Make sure to stay on track and avoid deviating from the task at hand. Make sure you bring the conversation back in focus, otherwise the hallucination effect may amplify. Ignoring the power of an exampleEveryone loves examples they say, even AI models! Providing examples in your prompt helps your model understand the context and generate more accurate responses. For instance, "Write a brief history of Python, similar to how the history of Java is described in this article {example}" This not only gives the AI a clear topic but also a reference point to follow. Providing a well-structured example can also save you a lot of time in explaining the output you’re expecting to receive. Without an example your prompt might be too generic, allowing too much freedom in interpretation. Think about it like a conversation. Sometimes, the best approach to make yourself understood by the other party is to provide an example. Do you want to make sure there’s no misunderstanding from the start? Include an example in your initial prompt. Not following “Divide et Impera”Have you ever tried to build IKEA furniture without instructions? It's a bit like that for AI models dealing with complex prompts. Too many nuts and bolts to keep track of. Too many variables to consider. Instead of asking the model to "Explain the process of creating a neural network," break it down into smaller, more manageable tasks like, "Step 1: Define the problem, Step 2: Collect and prepare data," and so on. This way, the AI can tackle each step individually and generate more coherent outputs. It’s also very useful when you are trying to generate a more verbose and comprehensive response and not just a simple factual answer. You can, of course, combine both approaches asking the AI to provide the steps first, and then asking for more information on each step. Relying on the first response you receiveAs most LLMs in use today do not provide enough transparency in their reasoning process, working with them sometimes feels like interacting with a magic box. The non-deterministic nature of generative AI can further amplify this problem, so when you need precision it's best to experiment with various prompt formats and compare the results. Pro tip: some open-source models can already be queried in parallel using this website:   Or, when interacting with a single AI model, try multiple approaches for your query like rephrasing the prompt, asking a question or presenting it as a statement.For example, if you're looking for information about cloud computing, you could try:"What is cloud computing and how does it work?""Explain cloud computing and its benefits.""Cloud computing has transformed the IT industry; discuss its impact and future potential."Some LLMs, such as Google's Bard, provide multiple responses by default so you can pick the most suitable from among them. Compare the outputs. Validate any important facts with other independent sources. Look for implausible or weird responses. Although a hallucination is possible, by using different prompts you’ll greatly reduce the probability of generating the same hallucination every time and therefore it’s going to be easier to detect it.Returning to our Voodoo Lady example earlier, by rephrasing the question we can get the right answer from ChatGPT.  And there you have it! By trying to avoid these common mistakes you'll be well on your way to minimizing AI hallucinations and getting the output you're looking for. We all know how fast and unpredictable this domain can be, so the best approach is to learn together and share best practices among the community. The best prompt engineering books have not yet been written and there’s a ton of new things to learn about this emergent technology, so let’s stay in touch and share our findings!Happy prompting!  About the Author Andrei Gheorghiu is an experienced trainer with a passion for helping learners achieve their maximum potential. He always strives to bring a high level of expertise and empathy to his teaching. With a background in IT audit, information security, and IT service management, Andrei has delivered training to over 10,000 students across different industries and countries. He is also a Certified Information Systems Security Professional and Certified Information Systems Auditor, with a keen interest in digital domains like Security Management and Artificial Intelligence. In his free time, Andrei enjoys trail running, photography, video editing and exploring the latest developments in technology.You can connect with Andrei on:LinkedinTwitter
Read more
  • 1
  • 0
  • 955