Chatbots, or simply bots for the context of this book, are not something new. They are almost as old as the internet. Of course, they evolved at different stages, taking various shapes and forms.

At a high level, the definition of a bot is a piece of software, or an application, that performs an automated task or set of tasks.

Way back in the beginning, chatbots were doing this by running a script or a set of scripts. This is nothing more than automating a set of commands.

If you think about it, it is pretty obvious how this would be valuable. It has been recognized that when dealing with repetitive tasks, leveraging a bot can not only take some of the workload from humans but also perform these tasks much faster and much more accurately.

This might sound familiar to those of us who have looked at how the personal computer was born. Not only have chatbots been around for almost as long as the personal computer, but they are now as important and prevalent as the personal computer, as we will see when we look at internet traffic consumption later in this chapter. As a matter of fact, the Encyclopedia Britannica defines a computer as a device for processing, storing, and displaying information (https://www.britannica.com/technology/computer).

But let's take a step back in time and look at some history. Bear with me here; this will set the necessary context for where we are going.

A condensed history of bots

The internet took shape in the 1970s. But it only caught the attention of the general public in the early 1990s. So, when I mentioned in the introduction that bots are almost as old as the internet, I was not lying.

Some of the first appearances of bots can be traced back to 1988. Yes, you read that right. Their preferred cradle at the time was a network called Internet Relay Chat (IRC). For those of us with gray hair, or no hair left at all, this will be familiar. We used to spend entire nights exchanging information and finding things out, reading documentation and other materials shared, among other things. Various servers were powering different networks with multiple channels, some more friendly or interesting than others. Funnily enough, IRC is still around; you can always poke around and find out more about it.

Those early bots provided all sorts of automation in a channel. From keeping a channel active, recognizing users, and providing them with moderator or administrator status, to responding to specific commands and even returning automated messages or documents, they were ubiquitous in that space.

As a matter of fact, they were extremely important. Due to the nature of IRC, a bot was always used by an established channel in order to keep the channel open and to prevent malicious users from taking over that channel. At the time, these kinds of bots were run from machines with long uptimes, typically running some version of BSD or Linux.

As they started to show true value, they started to become more refined and to escape the confines of IRC. They also started to separate functionality. Some bots are designed for the repetitive execution of tasks, while others are more inclined toward conversation. And that's how chatbots were born.

Some of the first incarnations of bots outside of IRC though were in fact web crawlers. To be more specific, the first such bot was called WebCrawler, and it was created in 1994. It went from AOL to Excite. But the most famous web crawler was created in 1996. It was named BackRub and was later renamed Googlebot.

Important note

To find out more about BackRub, take a look at the following link: https://en.wikipedia.org/wiki/History_of_Google#BackRub.

But bots have not always been used for a good cause, as we will see in the next section.

Malicious use of chatbots

As bots became recognized for their power and usefulness, they started to catch the attention of various malicious groups. Just like everything on the internet, they started to take on a life of their own, varying based on the group that adopted them.

Besides the obvious valid and good use cases, bots started to be adopted and used for malicious use cases. Having the ability to perform a large set of actions in an automated way was appealing to all.

Between the years 1999 and 2000, several incarnations of malicious bots and botnets (groups of bots working in conjunction) started to appear, for which people were unprepared. Some of these activities started through IRC but then expanded into the wild.

The year 2007 brought us one of the largest botnets at the time, called Storm. It was estimated that the botnet infected 50 million computers. There were various programmed use cases and scenarios leveraged in the attack. While negative in its intended use, it did show that bots can really scale.

Some of these use cases include actions such as sending large numbers of spam emails, identity theft, unauthorized distribution of malware, DDoS attacks, bots for artificially increasing traffic and revenue on advertising, game cheating bots, and many more.

But it is not all bad, so let's shift gears and fast forward a little.

Fast forward to today

What we know and use today on the internet was shaped in great part by bots. Starting with the assistance provided on various IRC channels, or the web crawlers that put information at the fingertips of users, bots evolved into indispensable tools for many business use cases. The ability to automate processes, as well as the power to interact with bots through normal conversation, became an indispensable tool. As technology evolved, so did the capacity of bots. We can now create smarter, faster, and better bots. Bots assist us in doing our day-to-day activities, assist our customers, and provide a differentiator for businesses that adopt and use them wisely. The current business landscape has evolved to leverage bots at scale.

In fact, bots are currently adopted so much that some statistics show that out of today's web traffic, roughly half is bot-generated traffic. As technology evolves, exponential bot traffic growth is expected. This will be driven by technologies such as Artificial Intelligence (AI) and Artificial Intelligence of Things (AIoT) and the ability to implement conversational scenarios through natural language processing.

Let's just look at the Turing test, developed in the 1950s. It was meant to test the ability of a machine to have a conversation with a human in an indistinguishable manner from a normal conversation between two humans.

As AI-infused conversational chatbots have taken shape, their uses have extended into various scenarios, including the following:

• Messaging applications, either as part of websites or baked into various applications.
• Marketing platforms with a focus on external customers and potential customers.
• Company internal platforms focused on serving internal users and employees.
• Customer service scenarios targeted at helping existing customers.
• Healthcare scenarios for scheduling appointments, locating services, or providing basic medical information.
• Toys are getting smarter, with educational scenarios for various ages.

Chatbots are used in many other scenarios too.

In today's business world, chatbots will have an essential role to play. Let's look at the expected impact next.

Impact on organizations

As organizations of all sizes strive to evolve and compete with one another, part of their digital transformation strategy is to look at the use of bots.

Also known as enterprise chatbots, bots can serve various use cases in business, including the handling of customer-facing functions such as managing order status updates, cancellations and returns, account balance and payment processing, and customer service through automated support, all of which lessen the load on customer support and other teams.

Through integration with various automation processes, including Robotic Process Automation (RPA), enterprise bots can not only increase customer satisfaction and provide better and faster services but also greatly reduce expenses, reduce workload on support staff, and serve various other purposes.

We do have to recognize, though, that while bots have gotten smarter and can do more, they are not in a position to replace humans. Hence, their position is ideal as a first line of contact, but processes will be escalated to a human team for more complex scenarios or scenarios that are not pre-determined.

Furthermore, the technological evolution over the last several years has brought us the democratization of bots and the bot creation experience. A multitude of platforms now allow non-developers to start creating basic bots for various scenarios.

To somewhat differentiate them from the old, negative connotation associated with the bot nomenclature, but also to more closely describe a specific category of services offered by bots nowadays, we find bots being referred to as virtual agents. Organizations have offerings of great value, such as the following:

• ServiceNow Virtual Agent: https://www.servicenow.com/products/virtual-agent.html
• IBM's Watson Assistant: https://www.ibm.com/cloud/watson-assistant
• Microsoft Power Virtual Agents: https://powervirtualagents.microsoft.com/en-us/
• Google Cloud Dialogflow: https://cloud.google.com/dialogflow/
• Amazon Lex: https://aws.amazon.com/lex/

And these are only some of the larger offerings. There is a whole range of solutions available from smaller vendors.

The differentiating factors between these offerings are their ease of use and creation, their ease of integration into various applications and services, their capability to leverage AI, their capability to resolve real use case scenarios in the most efficient manner possible, their capability to monitor their evolution and performance, and their ability to manage any associated risks. As AI is becoming more embedded in technology, ethics is a new topic on the table as well. The ethical consideration is at the forefront of a lot of new technologies, as we strive to make sure that the use of these platforms serves a real purpose and does not have a negative impact on people.

Some of the most common scenarios for enterprise virtual agents include the following:

• Support bots aimed at providing real-time help to customers
• Informational bots providing additional context in specific scenarios
• Application bots providing specific application access
• Enterprise productivity bots enabling organizations to streamline and integrate systems and processes

From a visual perspective, these chatbots are presented in various ways. It is often up to the design team to define the look and feel of chatbots on their specific target locations.

From a web presence perspective, the visual presentation often takes a cue from the overall web design, keeping with the tone of the branding of the respective site or pages where the chatbot is present. Let's have a quick look at some of the formats for presentation, as seen with a few of the larger providers of this service.

ServiceNow presents on their platform page a minimalistic chatbot at the bottom right of their page; it looks like this:

Figure 1.1 – ServiceNow chatbot

To make it obvious that a user browsing this page can engage with the chatbot, a starting message is also presented in a conversation bubble.

IBM, on the other hand, takes a more simplistic approach, presenting a simple icon:

Figure 1.2 – IBM chatbot

When clicked, a conversation window is presented to the user, as presented in the following screenshot:

Figure 1.3 – IBM chatbot conversation window

Other organizations take a less conservative approach, by presenting the chatbot trigger with a graphic representing a robot, as we see in the following example from ShareGate:

Figure 1.4 – ShareGate chatbot

Besides the custom-designed representations, there are many options for free graphics-depicting chatbots available on various sites that serve stock images. These can be used to put a face to your virtual agent.

It is not uncommon for these chatbots to receive a name and even a person's image or avatar in order to make them a little more personal. You might see messages such as the following, presented by the Amtrak virtual assistant:

Figure 1.5 – Amtrak chatbot

Typically, you will see these chatbots presented at the bottom right of a web page. Note that not all elements positioned in that area of the presented web page are chatbots. Chatbots will not always be presented in that particular page area. It is simply a design decision rather than a standard. Historically, the bottom-right area of a web page was the space used by organizations for mailing list subscription buttons or social network links. A little bit of healthy competition for that screen real estate takes place sometimes.

Now that we understand more about chatbots and have seen how other organizations create and present chatbots, let's next focus our attention on the Microsoft offering.

Power Virtual Agents is Microsoft's offering in the chatbot space. As organizations start to realize the importance of such a service, Microsoft joins the ranks of other vendors providing solutions in this space.

As we have just seen, there is quite the competition going on in the chatbot space, with offerings from all the big players in the industry, including Google, IBM, Amazon, ServiceNow, and others. Furthermore, there are offerings from small, specialized players with solutions tailored for various industries and various degrees of complexity or simplicity.

Around 2016, Microsoft recognized the potential of the bot offering for the masses and began its foray into this space with the introduction of Bot Framework. This was, at the time, a very developer-focused offering, with the ability to build complex bots using languages such as C# and Node.js as the core.

This robust framework was presented in conjunction with the Microsoft Cognitive Services offering, as well as the expansion of Cortana and chatbots into Skype. It is acknowledged that Cognitive Services played a role in the success of conversational chatbots.

The main components of Bot Framework included the following:

• Bot Framework Service (BFS)
• Bot Framework SDK (Software Development Kit)
• Bot Framework tools for development
• Bot deployment and channel configuration

As the resource needs for such a comprehensive offering are great, some Azure supporting services are leveraged, including the following:

• Azure Cognitive Services
• Azure Storage

The adoption of cloud services is now essential for such solutions.

Value to these offerings is provided through services such as the following:

• LUIS – natural language processing
• QnA Maker – answering questions in a natural way
• The Dispatch tool – allows for the selection of the correct tool for each scenario type
• Cards – complex presentational elements that include graphics, menus, and other artifacts to augment standard text presentation and provide a better user experience

The next evolutionary concept revolves around the ability of the bot to provide a REST endpoint to be called by the Bot Connector service. Having the ability to bypass the SDK, you can now create bots with other languages, such as Python. With version 4 of the SDK, support has extended to languages such as Python, Java, and JavaScript, along with the existing C# capabilities. All this was happening around 2018.

Important note

The Bot Framework SDK is in fact an open source SDK. This is a testament to Microsoft's renewed commitment to the community, and a stark contradiction to other organizations' walled gardens of proprietary solutions.

The most common channels supported by connectors include Facebook, Messenger, Slack, Teams, Telegram, Twilio, and many others.

Improvements and new features are continuously being added. For example, in 2020, a new implementation of Azure Blob Storage for better performance occurred, as well as better integration with Azure Queue Storage. Bots talking to other bots is possible through API communication or even through RPA.

Fast forward to the end of 2019, and Microsoft introduced the public preview of Power Virtual Agents for the first time. Wrapped in the Power Platform, it now provides users the ability to create chatbots in a no-code manner. Leveraging Azure AI and a specific toolset for building bots, the aim is to expand creation capabilities to allow citizen developers (your typical business users that do not have a development background), along with IT pros and pro-developers, to create and deploy chatbots with minimal effort and custom coding.

The new offering relies heavily on AI functionality to understand intention from existing conversations without the need to define all possible key phrases. It also relies on other features such as Power Automate to trigger specific actions on other platforms.

Another key aspect is the ability to generate and capture advanced analytics. You can easily track where the bot fails to provide the right answers and easily make adjustments to improve communication. A robust analytics dashboard provides deep insights into engagement over time, outcomes over time, resolution, escalation and abandon rates, as well as, when configured as such, customer satisfaction.

A lot of the analytics we were looking for in a regular customer service agent interaction are now bubbled up and presented to the administrator for the bot interactions. And this being a scripted process, you can easily go in and adjust the process as needed for better results.

The process to create new bots is now streamlined to the point where basic bots can be created by users with no development knowledge. Simple conversational bots can be created, monitored, and optimized by business users. This is part of a trend identifying these users as citizen developers. The bulk of the experience is built around a streamlined user interface that allows for configurable processes.

The developer role in the chatbot space

That is not to say that the role of IT pros or pro-developers becomes obsolete. Developers will always play an important role, especially when dealing with more complex or custom scenarios. The expectation is that developers will start to understand and leverage the tools available in their tool belts and use a no-code approach where possible. The advantages are obvious, with the ability to create processes and scenarios that require less effort to produce, less overhead in future maintenance, and a more streamlined product that protects the assets created from the unnecessary complexity that comes with custom development. This applies across the entire Power Platform and is not something specific to Power Virtual Agents. A configuration-first approach provides the ability to support the created assets in the future with fewer overheads and reduces the need to call upon a technical partner or internal pro-developer resource to support these assets in the future. It generally puts the power in the hands of IT pros, or even less technical and more business-focused people.

An important aspect is the understanding of the technology, best practices, and overall governance. While now we can allow non-developer users the ability to create business value through a no-code approach, we still need to enforce certain rules for better operation of the business assets created.

For those of us coming from the development world, this might sound a lot like the start of a typical Dynamics 365 (CRM) implementation, where a business analyst would start to mock up the requirements through simple configurations. As we've all seen so far, in some scenarios we could continue to build on top of those configurations, while in many other scenarios, we wind up leveraging those configurations only for the initial discussions with our customers and end up starting with a configuration from scratch. It all depends on the level of understanding of the platform capabilities from those resources that were involved in the initial stages of a project.

Overall, as those business analysts become more familiar with the platform, best practices, and overall design, more of what they produce in those initial stages will be reusable down the road.

From a user experience perspective, the bot creation and editing interface is simplified, as we can see in the following screenshot, which shows a brand-new bot I just created:

Figure 1.6 – Power Virtual Agents bot authoring

The left navigational area presents the various category options available, including the topics, the entities our bot will interact with, standard analytics on the performance of our newly created bot, and the ability to publish our bot to a selected channel. The Manage section provides detailed bot configuration options. We will be looking at each of those areas in more detail throughout this book, as we begin to build specific bots to serve various purposes.

The next area to the right of the navigation is the Test bot panel. This is where we can run and interact with our bot as we are building it, allowing us to validate the logic and the expected behavior. We can always show or hide this area as needed. Typically, we will keep this section closed to maximize the edit area while we are building our bot logic.

Finally, the right-most area covering the majority of the screen is the editing area. While on the Home area, this will present various details on how to get more information about building bots. This is a great place to start when learning about bot creation, as it provides links to the most relevant learning materials available from Microsoft.

As you move through to any of the other categories, you will find you are able to configure the details of the bot behavior. Again, don't worry if you are unfamiliar with these options at this time; we will inspect and analyze each of these options throughout this book.

Note that at the time of writing, there are two distinct scenarios envisioned for Power Virtual Agents. They are presented as the following versions:

• Power Virtual Agents web app: This is meant for creating bots that will integrate into existing or new web applications or mobile applications. This is what we saw in the previous screenshot and is aimed at users creating bots that will interact and engage with customers. These are, for the most part, publicly available bots. Of course, there is nothing stopping you from creating and presenting these bots on an intranet portal to internal users and taking advantage of internal authentication, as we will see in Chapter 6, Handling Authentication and Personalization.
• Power Virtual Agents Microsoft Teams app: This is meant for supporting existing employees of your organization. By definition, they are not exposed publicly and are only aimed at supporting your staff, team members, or any other internal resources. In many cases, you can leverage existing Microsoft 365 licensing, which includes the standard Office functionality along with several cloud services such as SharePoint, OneDrive for Business, Exchange, Teams, and others.

We will see throughout the book how to build bots for both these scenarios and will gain a better understanding of the differences.

Now we know what Power Virtual Agents is, let's look at its value and benefits.

Power Virtual Agents is an integral part of the Power Platform, a robust offering from Microsoft targeting the no-code marketplace. As such, for organizations invested heavily in Microsoft technologies, it simply makes perfect business sense to obtain services that would keep them in the same technology realm. This provides easier maintenance, less of a need to reskill existing support and administration staff, and a better integration story across the board.

In addition, Power Virtual Agents leverages some of the same technologies that other platforms use, for example, Dataverse. Dataverse is the data model formerly known as Common Data Service (CDS) – it powers a few other solutions in the business applications space and can also be leveraged with other components of the Power Platform. This allows the creation of unified solutions, sharing data, and creating robust business processes unbound by standard application limitations.

Also, from a licensing perspective, the ability to leverage Power Virtual Agents with your Microsoft 365 license allows organizations to start taking advantage of these features without incurring any additional costs.

From a usability perspective, with Power Virtual Agents being an integral part of the Power Platform, you have the ability to leverage a large number of pre-built connectors and services to increase the capability of the services you provide through chatbots.

Finally, from a user experience perspective, the ability to use Power Virtual Agents through a common interface, similar to other Microsoft services, provides a level of familiarity to users. This greatly reduces the need for training and adjustments from one solution vendor offering to another.

These are only some of the highlights; there are a lot more benefits to implementing chatbot functionality using Power Virtual Agents, as we will continue to see throughout the following chapters.

In the next section, we will tackle the licensing aspect of Power Virtual Agents at a very high level.

While this is not a comprehensive licensing guide, I want to start touching on this topic right from the first chapter. Licensing is extremely important to understand when designing solutions, in particular solutions spanning multiple technology stacks. Licensing Power Virtual Agents is tightly integrated with the Power Platform general licensing. We will be looking at more licensing details and how to estimate licensing requirements in Chapter 2, Licensing for Power Virtual Agents.

Microsoft provides a comprehensive Power Platform licensing guide that you should be reviewing to understand the intricacies and details related to licensing various components of the Power Platform. Power Virtual Agents being one of the Power Platform service offerings, it is also covered in this licensing guide.

Microsoft revises this guide with every release, and you should always refer to the latest version. At the time of writing, the current version is the November 2020 guide, which is available to download from the Power Virtual Agents pricing page at https://powervirtualagents.microsoft.com/en-us/pricing/.

Keeping in line with the separate offering for Power Virtual Agents, the licensing provides two distinct options for Power Virtual Agents for the web versus Power Virtual Agents for Teams.

We have a tenant license available for Power Virtual Agents, which covers the functionality for the respective tenant it is associated with. The user licenses are licenses assigned to bot authors. They are available at no additional cost on a tenant with a Power Virtual Agents license and can be assigned by an administrator through the admin portal. This is done through the Microsoft 365 admin center, as shown in the following screenshot:

Figure 1.7 – Licenses

To view the available licenses in your tenant, you can navigate in the admin center to the Billing area and look at the Licenses tab, as shown in the following screenshot:

Figure 1.8 – Licenses tab

In addition to the standard Power Virtual Agents tenant license, an organization can purchase a capacity add-on if needed. This allows organizations to extend usage beyond the standard license limitations for sessions and storage.

Note that the default license allows up to 2,000 sessions per month and the following capacities for storage:

• Database capacity up to 10 GB
• File capacity up to 20 GB
• Log capacity up to 2 GB

These capacities are calculated on a monthly basis and remaining unused sessions do not carry over to the next monthly cycle. Storage capacity is fixed, though capacity add-ons can be purchased in increments of 1 GB as needed.

At this time, while organizations adopt and implement this functionality, some reasonable overages might be deemed acceptable, but organizations should plan accordingly for a possible future date when enforcement could be enabled automatically, resulting in potential business disruption. It is always a good idea to monitor usage and stay within the permitted licensing limits.

Since the Teams bot functionality is slightly different, the licensing is also separate. The Power Virtual Agents for Teams functionality is included with the standard Microsoft 365 licensing. Typically, the Enterprise and Business licenses will also include user rights for Power Virtual Agents for Teams.

With the Teams license, currently there is no restriction on the number of sessions or capacity. Capacity is actually calculated based on the Office 365 licensing capabilities. This might change as the platform evolves and Microsoft revises the licensing model.

Note that only some Microsoft 365 licenses allow chatbots to leverage Dataverse. In this scenario, once the capacity limits are reached, organizations will need to upgrade to a dedicated Power Virtual Agents plan to increase capacity.

In addition, another limitation of chatbots built for Teams is the ability to leverage only standard connectors. Premium and custom connectors are not available for these scenarios at this time.

Seeing how chatbots have evolved over time should now put this technology into perspective. We now have an understanding of how this technology was born, how it evolved to where it is today, and what the true business value is for current and future organizations.

Throughout this chapter, we looked at how the current chatbot offering was born, how it evolved through time to where it is now, as well as some of the good and bad uses of this technology. We continued our journey to the present day, looking at the importance of leveraging chatbots in the current business landscape and the value differentiator between Power Virtual Agents and Power Virtual Agents for Teams.

After we looked at the various providers of similar chatbot solutions, we narrowed our focus to Microsoft. Power Virtual Agents, being an integral part of the Power Platform offering, provides distinct advantages for various organizations and business scenarios. We looked at the different approaches to the creation of chatbots, based on the target audience. We now should have a clear picture of the difference between creating public chatbots aimed at customers and the larger audience and chatbots for internal organization users.

While we had a quick look at the licensing options and limitations, in the next chapter we will delve deeper into licensing concepts and look at some of the important aspects to consider when choosing your licensing model.