The remarkable power of Generative AI lies in its ability to create realistic and diverse outputs that exhibit human-like creativity. For instance, in the area of visual arts, Generative AI can produce lifelike paintings, generate photorealistic images, or even assist in designing new products with particular aesthetics. In music composition, Generative AI algorithms can compose original melodies and harmonies, imitating the style of different composers or creating entirely new musical genres. Similarly, in the realm of storytelling, Generative AI can develop compelling narratives, write poetry, or generate realistic dialogue.

The impact of Generative AI extends far beyond artistic and creative pursuits. It has already found applications in various fields, including data augmentation, synthetic data generation, video game design, and drug discovery. Generative AI techniques can also aid in enhancing existing content, enabling the creation of high-quality image upscaling, text summarization, and even voice synthesis that closely resembles human speech! The future of what AI can do is only limited by the creativity of the people building and training the models.

However, Generative AI is not without its challenges. Ethical concerns, such as the potential for misuse or the propagation of biased content, need to be addressed. Striking the right balance between the creativity of the AI system and its alignment with societal values is a crucial aspect that requires careful consideration.

Ethical and responsible AI

The responsible use of AI is critical to ensuring its success in the marketplace. Responsible AI includes concepts such as fairness (treating every user equitably), inclusiveness (making sure people of all races, genders, and abilities are empowered to use the systems), transparency (understanding how the AI models reached their conclusions), and accountability (ensuring people can be held responsible for an AI system’s decisions).

Responsible AI design and use covers all aspects of all AI systems, from machine learning models having representative data samples that don’t hold biases toward nationalities or genders to ensuring Generative AI doesn’t get used for creating likenesses of people (sometimes called deepfakes) without their consent.

Microsoft, in part, has been working on establishing principles for ethical and responsible AI. You can read more about their commitments here: https://www.microsoft.com/en-us/ai/responsible-ai.

Overall, though, Generative AI represents a significant breakthrough in working toward AI. By harnessing the power of machine learning and deep neural networks, Generative AI systems can unlock new realms of creativity and innovation, business automation, and human-to-computer interaction.

Not to mention, it’s also unleashing many ninja cat dinosaur mashups:

Figure 1.1 – Bing Image Creator, powered by DALL-E

Let’s shift gears to understanding how various types of AI might be used in the context of business – specifically, with Microsoft’s Power Platform.

Power Platform is a comprehensive suite of low-code and no-code tools that’s designed to empower individuals and organizations to create custom business applications, automate processes, analyze data, and develop virtual agents. It consists of four main components:

• Power Apps: This component enables users to build web and mobile applications with drag-and-drop functionality, all while connecting to various data sources.
• Power Automate: Formerly known as Microsoft Flow, it allows the creation of automated workflows, integrating and synchronizing data and processes across multiple applications and services. Power Automate has expanded from being a cloud-only process automation platform to including process mining and robotic process automation (RPA).
• Power BI: This component provides robust data analytics and visualization capabilities, transforming raw data into meaningful insights and interactive reports.
• Copilots: This component allows intelligent chatbots to be created without the need to code, enabling organizations to provide instant support and engagement with customers.

Together, these tools empower users of all skill levels to drive digital transformation, improve productivity, and innovate within their organizations. This book will focus primarily on integrating AI services and models with the Power Automate, Power App, and Copilots components of Power Platform.

What is no-code or low-code software?

Microsoft bills the Power Platform tools as a development environment that encourages no-code and low-code solutions. So, what are those? No-code is just like it sounds – a way for creators to assemble a solution from widgets, components, or modules in a what-you-see-is-what-you-get (WYSIWYG) fashion. No formal development experience is necessary to generate a working solution.

Low-code software, which is one step up, involves using a simplified development language in conjunction with the available connectors or modules. Power Platform leverages a language called Power Fx, which is similar in structure to the syntax that’s used by popular Office macros or spreadsheet formulas.

Power Platform tools can also support pro-code or code-first authoring (which is decidedly the opposite of the low-code or no-code methodology), meaning you can interface with REST API interfaces or use traditional development environments such as Visual Studio.

Learning about Power Automate

Power Automate is a workflow and process automation tool. As a no-code/low-code solution, Power Automate relies on various components or building blocks to create automation.

Here’s a quick list of some of the terminology you’ll see in this book as it relates to Power Automate:

• Flow: The basic unit of Power Automate, flows are logical groupings of connectors, conditions, and tasks that are used to perform an automation
• Connector: Connectors are configuration components that are used to define the necessary parameters to communicate with services and apps
• Trigger: A trigger is an event or activity that causes a flow to begin, such as When a new file is created or When a row is added to a SQL database table
• Actions: Actions are the logical steps or units that describe the actions and evaluations being performed, such as Copy a file, Check if the value is greater than or equal to, or Add a row to a SQL database table

We’ll use Power Automate in many of the exercises and examples throughout this book.

Further reading

To learn more about Power Automate, check out Workflow Automation with Microsoft Power Automate, Second Edition (https://www.packtpub.com/product/workflow-automation-with-microsoft-power-automate-second-edition/9781803237671).

Learning about Copilots

Copilot technology (not to be confused with Copilot for something) empowers users to design and deploy chatbots to interact with people, providing instant support and engagement. With a visual interface and pre-built templates, creators can easily define conversation flows, connect and integrate with various systems, and train the chatbot using natural language understanding (NLU).

So many copilots

We’ll try to keep things straightforward, but Microsoft has infused its products with the copilot nomenclature. There’s Microsoft 365 Copilot (a Generative AI assistant that’s built into the Microsoft 365 experience), Copliot in Viva Sales (a Generative AI assistant that connects Outlook and other collaboration workloads with Dynamics CRM), and Copilot for Security (an AI assistant for threat hunting and management).

Power Platform has its own set of copilot features, including AI Copilot (an AI-enabled generative assistant for creating Power Apps and Flows) and Copilot Studio, the web interface that’s used to create – you guessed it – copilots. Copilots (in Copilot Studio) are the revamped Power Virtual Agents – chatbots that can be enabled to provide answers and initiate workflows in other applications.

Since we’re going to be using copilots in some of the exercises in this book, you’ll want to be familiar with their terminology, too:

• Topics: Topics represent the main areas or subjects that your chatbot can handle. Each topic consists of triggers, actions, and responses to guide the conversation flow.
• Triggers: Triggers are conditions or user inputs that initiate a conversation or direct it to a specific topic. They can be based on keywords, phrases, or system events.
• Actions: Actions are the steps or operations that the chatbot performs in response to user inputs or triggers. They can include sending messages, asking questions, calling APIs, or performing calculations.
• Entities: Entities are pieces of information that the chatbot can extract from user inputs.
• Responses: Responses are the messages or content that the chatbot generates to communicate with users.

Copilots can be deployed to a variety of locations and interfaces, including websites and Microsoft Teams.

Further reading

For more information on Copilots, please read Empowering Organizations with Power Virtual Agents (https://www.packtpub.com/product/empowering-organizations-with-power-virtual-agents/9781801074742).

Learning about Power Apps

The Power Apps component of Power Platform enables users to create custom web and mobile applications without extensive coding knowledge. It offers a low-code development environment where users can visually design app interfaces, define data sources, and add functionality through a wide range of pre-built connectors.

Like Power Automate and Copilots, Power Apps has terminology that you should know about:

• Screens: Screens are the building blocks of an app and represent different views or pages. Each screen can contain various controls and components.
• Controls: Controls are the interactive elements that are used to display data, capture user input, or trigger actions. Examples include text boxes, buttons, galleries, forms, and charts.
• Data sources: Data sources are the places where app data is stored, such as SharePoint, Microsoft Dataverse (formerly Common Data Service), Excel, SQL databases, or external systems via connectors.
• Formulas: Power Apps uses formulas (the Power Fx language) to perform calculations, manipulate data, and control app behavior. Formulas can be used in properties, events, and actions.
• Data cards: Data cards are containers for data entry controls within forms. They represent fields from a data source and enable users to view and edit data.
• Galleries: Galleries are controls that are used to display lists or collections of data. They can be customized to show data in different layouts, such as a table or a gallery with cards.

One of the most exciting new features of Power Platform is the Describe it to design it feature, which allows you to use natural language to build a framework for a Power App. We’ll explore that a little bit in Chapter 5, Bootstrapping a Power App with Copilot.

Further reading

For more information on creating apps with Microsoft Power Apps, check out Learn Microsoft Power Apps, Second Edition: https://www.packtpub.com/product/learn-microsoft-power-apps-second-edition/9781801070645.

Learning about AI Builder technologies

AI Builder is the original AI component of Power Platform and allows users to incorporate AI capabilities into their Power Apps and Power Automate workflows. It enables users, even those without extensive AI expertise, to build and deploy AI models for common business scenarios.

Working with prebuilt models

AI Builder offers a set of prebuilt models and samples that can be customized to meet specific needs. These models cover various AI capabilities, such as form processing, object detection, prediction, text classification, and sentiment analysis. Users can train and refine these models using their data or leverage existing data connectors.

With AI Builder, users can automate data extraction from forms, classify and predict outcomes, analyze sentiment in text, and identify objects in images. These AI capabilities enhance the power and functionality of Power Apps and Power Automate – and you’ll get hands-on experience with them in this book!

We’ll be using the sentiment analysis feature in Chapter 6, Processing Data with Sentiment Analysis.

We’ll work with the AI Builder document reader in Chapter 8, Building an Event Registration App with Identity Verification.

Finally, in Chapter 9, Implementing an AI-Enabled Resume Screener, you’ll learn how to train an AI Builder model to extract key pieces of data from a resume.

Working with custom models

Custom models, in contrast to prebuilt models, allow users to create and train their custom AI models tailored to their specific business needs. Where prebuilt models are already trained for common scenarios, custom models allow you to train models with your data to meet specialized requirements.

We’ll use a custom model to work with unstructured data in Chapter 9, Building a Resume Screener Using Copilot and Power Apps.

Understanding Power Platform licensing

The licensing structure of Power Platform can be a bit complex, but here’s an overview of the main licensing options:

• Power Apps and Power Automate Free: There is a free version available for Power Apps and Power Automate that provides basic functionality and limited access to connectors and features. This licensing plan is sometimes referred to as the seeded offering.
• Power Apps and Power Automate per-user plans: These are paid plans that offer enhanced capabilities and more extensive access to connectors and features on a per-user basis. These plans are suitable for individual users who need advanced functionality and are licensed on a monthly or annual basis.
• Power Apps and Power Automate per-app or per-flow plans: These plans allow organizations to license specific apps or flows, rather than licensing individual users. They are useful when an organization has a large user base but only a subset of users need access to specific apps or flows.
• Power Apps and Power Automate licensing with Microsoft 365 and Dynamics 365 plans: Power Apps and Power Automate are also included in various Microsoft 365 and Dynamics 365 plans. These plans provide broader access to Power Platform capabilities alongside other Microsoft productivity and business applications.

It’s important to note that certain premium features, add-ons, connectors, and capacity-based usage may require additional licensing or higher-tier plans. Additionally, Microsoft regularly updates and refines its licensing options, so it’s recommended to consult the official Microsoft licensing documentation or contact Microsoft directly for the most up-to-date information on licensing options for Power Platform.

AI Builder is a feature that is billed on a capacity model – separate from the Power Apps and Power Automate per-user, per-app, or per-flow plans. Here’s an overview of the key aspects:

• Free usage: AI Builder offers limited usage for free, allowing users to explore and experience its basic features without additional cost.
• Consumption-based pricing: When usage surpasses the free limit or requires more advanced capabilities, AI Builder capacity licensing comes into play. It follows a consumption-based pricing model, where organizations purchase capacity to enable AI Builder’s features.
• Capacity types: There are two types of capacity available for AI Builder licensing – AI Builder standalone capacity and AI Builder capacity add-on:
  • AI Builder standalone capacity: This type of capacity is dedicated solely to AI Builder and covers the consumption of AI Builder models and services.
  • AI Builder capacity add-on: This capacity is an add-on for existing Power Apps and Power Automate capacity. It allows organizations to extend their existing capacity so that it includes AI Builder capabilities.
• Capacity units: AI Builder capacity is measured in capacity units, with each unit providing a certain level of compute resources and performance. The number of capacity units required depends on the volume of AI Builder usage and the complexity of the models being deployed.

It’s important to consider the total number of users, expected usage, and the specific AI Builder features required when determining the appropriate capacity licensing for an organization. Organizations may need to allocate sufficient capacity units to ensure optimal performance and scalability.

For precise details on AI Builder capacity licensing, including pricing, specific features, and licensing agreements, it is recommended to refer to the official Microsoft documentation or consult with a Microsoft licensing specialist to ensure compliance and appropriate licensing for your organization’s needs.

Further reading

For a deeper dive into Power Platform licensing, see https://learn.microsoft.com/en-us/power-platform/admin/pricing-billing-skus and https://learn.microsoft.com/en-us/ai-builder/administer-licensing. If all that is as clear as mud, you can contact a Microsoft Partner (https://partner.microsoft.com/en-us/marketing).

Now that we’ve spent a little bit of time learning about some of the features of Generative AI and Power Platform, let’s look at a few of the AI services currently in the marketplace.

Working with Azure AI Services

Microsoft offers a wide array of AI and machine learning capabilities under the Azure AI Services (formerly Azure Cognitive Services) umbrella. These services enable developers to add intelligent capabilities to their applications without needing to build complex AI algorithms from scratch. With Azure Cognitive Services, developers can tap into pre-built models and application programming interfaces (APIs) to incorporate functionalities such as vision, speech, language, and decision-making into their applications.

The vision-capable services include the following:

• Computer Vision: This service enables image analysis and object detection. Computer Vision can do things such as identify objects in a scene (dog, frisbee, and tree) as well as perform contextual image analysis (There is a dog catching a frisbee by the tree) and optical character recognition.
• Face: The Face service is used to identify the presence of faces in image data, as well as identify and analyze faces in images. You might use the Face service to verify someone’s identity against a government-issued identification card or to blur out face content in a picture or video.
• Custom Vision: The Custom Vision service, now part of Image Analysis 4.0, enables developers to create custom image recognition models for tagging and applying labels to content based on visual characteristics.

Azure Cognitive Services includes several language and speech APIs, which provide the following features:

• Speech: The Speech service provides a broad array of speech-to-text, text-to-speech, content translation, and speaker recognition capabilities.
• Language: This service has several sub-services and capabilities based on natural language understanding, including the ability to perform key phrase extraction, named entity recognition, and the ability to detect personally identifiable information (PII). The Language service can also provide text translation between two languages, classify bodies of text and determine their sentiment, and perform content summarization.
• Translator: While the Language service handles text-to-text content conversion, the Translator service can provide machine-based translation in real time.
• Language Understanding (LUIS): The LUIS service is a machine learning model that can predict overall meaning and content extraction from natural language conversation.
• QnA Maker: You can think of the QnA Maker service as a bot or answering service that can reason over semi-structured content and provide answers to customers.

The decision models that are available alongside Azure AI Services also provide unique capabilities:

• Content Moderator: The Content Moderator service provides analysis for potentially offensive, undesirable, unsafe, or otherwise risky content
• Personalizer: Using behavior and habits analysis, the Personalizer service helps select which content experiences to choose for your customers
• Anomaly Detector: The Anomaly Detector service allows you to monitor and detect inconsistencies in time-based datasets

Azure AI Services also includes the Azure OpenAI service, which has several language models, including GPT-3 and Codex, to support content generation, summarization, and natural language-to-code translation.

Change is a-comin’

If there’s anything constant, it’s change. As the AI landscape evolves, newer features, services, or capabilities replace older counterparts. Over the next few years, you’ll need to say “goodbye” to the LUIS and QnA Maker services, as they’ll be retired. Microsoft recommends transitioning LUIS-enabled applications and services to Conversational Language Understanding and retooling services by using QnA Maker with the question-and-answer capability in the Azure AI Language service.

Together, this family of services provides developers with the mechanisms to programmatically add AI-based capabilities to their applications without having lots of experience in AI modeling, improving user experiences, and driving innovation in areas such as healthcare and customer service.

Further reading

For more information on the complete lineup of Azure Cognitive Services, go to https://learn.microsoft.com/en-us/azure/cognitive-services/what-are-cognitive-services.

Working with OpenAI models

OpenAI is the organization behind popular models such as GPT-3, GPT-4, and the DALL-E image generation platform.

OpenAI offers a range of advanced AI technologies and services that aim to empower individuals and organizations to leverage the power of AI. Let’s look at some of their offerings:

• Generative Pre-trained Transformer (GPT): This is OpenAI’s flagship language model. It can generate coherent and contextually relevant text, making it useful for tasks such as natural language processing, text completion, and chatbot development.
• OpenAI API: This offering provides developers with easy access to OpenAI’s models, enabling them to integrate language generation capabilities into their applications or services.
• Images API: The Images API provides methods for creating or editing new and existing images based on text prompts.
• OpenAI Gym: This is a toolkit for developing and comparing reinforcement learning algorithms. It provides a wide range of pre-built environments and tools to train and evaluate AI agents.
• OpenAI Platform: OpenAI Platform offers a suite of tools and resources for researching, developing, and deploying AI models. It includes model training infrastructure, collaboration features, and model management tools.
• OpenAI Scholars Program: This is a research internship program that supports aspiring AI researchers from underrepresented backgrounds. It provides mentorship, stipends, and resources to help participants advance their AI knowledge and contribute to the field.

OpenAI, like Microsoft, emphasizes ethical considerations and responsible AI development.

Further reading

In this book, we’ll mainly be focusing on the OpenAI services that are exposed through Power Platform directly or via Azure AI Services. OpenAI’s products are available separately, as well. For more information on developing with the OpenAI platform, go to https://platform.openai.com/overview.

Working with services from Google, Anthropic, and more

While we’re mainly going to focus on services offered in the Microsoft cloud (Power Platform AI Builder, Azure AI Services, and OpenAI, to be more specific), they aren’t the only big names in AI right now. There are several other AI platforms – both general and specific – that are currently being developed. Some are even open for you to experiment with now!

Here’s a list of a few services that you might come across:

• Google Bard: Based on Google’s Language Model for Dialogue Applications (LaMDA), Bard is an advanced language model that’s developed by Google’s research team. LaMDA’s training involves dialogues instead of isolated prompts, allowing it to grasp the underlying meaning and engage in more conversational exchanges. You can start a conversation with Bard at https://bard.google.com.
• Anthropic Claude: Claude is another type of conversational AI model. Anthropic approaches their AI models a bit differently, however, training with a method they call Constitutional AI. Constitutional AI involves supervised learning and review so that the AI learns from the feedback to generate harmless outputs. Talk to Claude at https://claude.ai.
• Midjourney: Whereas ChatGPT, Bard, and Claude use text for both input and output, Midjourney is an AI-based art generation platform. Midjourney can be used to create high-quality art using text prompts. Experience Midjourney at https://www.midjourney.com/.

AI has also found its way into countless plugins and extensions. A quick internet search or two will quickly reveal new ad hoc tools you can try out.

Hopefully, having a high-level understanding of some of the types of computing, algorithms, and frameworks that go into Generative AI makes it a little less scary to approach. While the machines may not be coming for us (yet), they certainly are going to be influencing how businesses and customers interact for years to come.

Now, it’s time to begin your AI journey by getting your development environment ready!