The Architectural capabilities of MuleSoft

The MuleSoft Anypoint platform can be used to deliver integration solutions architected on premises, in the cloud, or a combination of on premise and cloud. The latter is referred to as hybrid IPaaS. Note the individual services and capabilities available in the Anypoint platform depend on the subscription purchased.

In this section, we will look at how the Anypoint Platform is organized. We will look at the options available in the platform on where to run integration applications and where to manage the platform. We will take a brief look into some of the reasons why organizations will choose one option over another.

Planes of operations

There are two logical “planes”, Control Plane and Runtime Plane, within which the components and services of the MuleSoft Anypoint Platform exist and operate. In the Anypoint Platform high level diagram shown in Figure 1.5 we can see the services included in the Control plane in the top half and the Runtime plane in the lower half.

Figure 1.5 - Anypoint Platform High Level view

The control plane refers to all the components used to:

• Manage the platform
• Develop code
• Design and publish API specifications
• Collaborate with other developers
• Configure and manage runtime settings
• View logs
• Manage APIs

Essentially the control plane is where the platform itself is managed along with every API or Mule application you developed and are running somewhere. The “running somewhere” is the domain of the runtime plane.

The runtime plane is made of many components or services responsible for running the MuleSoft applications but at its heart is a Java Virtual Machine (JVM). This includes the Mule runtime engine, but it also includes the services where the runtime engine can be hosted including Runtime Fabric and CloudHub. The runtime plane also includes services used by a MuleSoft application while it is running. This includes:

• Virtual Private Cloud (VPC)
• Virtual Private Network (VPN)
• Object Store V2
• Anypoint MQ
• Connectors and any associated drivers
• DataGraph
• Dedicated Load Balancers (DLB)

When you open the Design Center in Anypoint, you are operating within the control plane. When you deploy and start your MuleSoft application either through the Runtime Manager interface in Anypoint or using an Anypoint CLI command, you are operating components in the control plane. The running applications and all the services they can use, along with the runtime engine are part of the runtime plane.

The screen shown in Figure 1.6 is running in the control plane and hosts all the services seen here. When you log in to your trial Anypoint platform you may see a few components are missing. Components such as Anypoint MQ and Partner Manager are items requiring additional licensing.

Figure 1.6 - Anypoint Control Plane and available components and services

The figure shows many of the services available in MuleSoft including:

• Anypoint Code Builder (ACB) – A new developer IDE offering based on Visual Code, able to operate fully in the cloud without any installation required.
• Design Center – A cloud based development tool for building API specifications in RAML or OAS and AsyncAPI
• Exchange – A catalog of artifacts including API’s, Connectors, Templates, Examples, Snippets, Dataweave, all of which can be shared with other consumers in order to support reuse.
• Management Center (and all the controls therein) – All of the controls and management capabilities of the platform including:
  • Access management
  • API Management
  • Runtime Management
  • Governance
  • Anypoint MQ management
  • Visualizer for creating views of APIs
  • Monitoring
  • Secrets management

We will learn more about these different services and more about the runtime plane options in the next chapter. But first let’s consider the architectural advantages of having these two planes.

• Fault tolerance: When the control plane is down, whether because of an outage or upgrades, the runtime plane and all the running applications can continue to operate. Likewise, when the control plane identifies an application is not running for some reason, the control plane can restart the service or attempt to run it in a different availability zone.
• Hosting flexibility and regulation compliance: Organizations can choose to run the control plane in the cloud, with everything installed, hosted, and managed by MuleSoft. Some EU organizations may require all their assets, data, and metadata to be operated strictly within the EU. This is supported by with an EU hosted Control Plane. Organizations requiring FedRAMP compliance can run the Control Plan in the MuleSoft hosted AWS GovCloud. Some organizations may have requirements or regulations preventing them from having any assets, data, and/or metadata in the cloud. For these organizations, the Anypoint Platform supports running the control plane in your own data center.
• The Runtime Plane can be fully managed by MuleSoft on the AWS public cloud, on an AWS VPC, and on the AWS GovCloud. There are also options for run on Customer-hosted infrastructure running the Mule runtime engine either directly on servers specially provisioned for the Mule Runtime or as an appliance in container deployments. Depending on the deployment selection for the control panel and runtime panel, the availability of Anypoint service configurations may change.

In the next section we will explore where the Control plane can be hosted, where the Runtime plane can be hosted, and the impact the hosting decision on one plane impacts the options of the other plane.

Platform deployment options

One of the jobs of the architect is to examine and understand the current state of the organization’s systems landscape. Has the organization embraced a cloud managed services approach. Does the organization have critical systems still in their own data center. These next sections will describe the options available for hosting the planes and how they relate to each other.

Control plane hosting

Currently, there are 2 deployment options available from MuleSoft for running the control plane, the first of which has 3 deployment locations:

• Hosted and managed by MuleSoft
  • US AWS cloud
  • EU AWS cloud
  • AWS GovCloud
• Running control plane functionality locally on the organizations own hardware, hosted and managed by the organization.

Running the Control Plane on the organization’s own infrastructure uses a product called Anypoint Platform Private Cloud Edition (PCE). Installing this product requires working with MuleSoft professional services during the installation.

Runtime plane

The runtime plane as described earlier, is where your programs run. This is where the HTTP connector you added to a flow, listens to the port you told it to listen to. It’s where the Dataweave transformation logic sits waiting for a payload to reshape. It is also the domain of Anypoint Object Store V2 and Anypoint MQ.

In order to run these components, we must have compute resources, memory resources, storage and network resources. MuleSoft provides four options where the MuleSoft runtime engine can execute:

• CloudHub
• CloudHub 2.0
• Runtime Fabric
• Standalone

Later chapters will take a deeper look a each of these runtime hosting options. In Chapters 9 and 10 we look at the MuleSoft hosted options, CloudHub and CloudHub 2.0. In Chapter 11 we look at the nuances of containerizing the runtime plane with Runtime Fabric. And finally in Chapter 12 we will look at the self-hosted standalone option of deploying the MuleSoft runtime engine on infrastructure obtained and managed by the organization.

But just because each of these can be a host for the runtime engine, does not mean all the other runtime plane components are supported in each of these environments. Table 1.1 is a matrix showing the runtime components available for each hosting option for the runtime plane.

Table 1.1 - Hosting Options for Runtime Plane components

In this table we can see that DataGraph is not available in CloudHub 2.0, Runtime Fabric, or Standalone servers. We can also see that Object Store v2 is not available in Runtime Fabric or Standalone servers. We can also see the most flexible runtime hosting options is CloudHub and CloudHub 2.0.

In the next section we will see what combinations of these runtime hosting options can be used with the different control plane options.

Combining Control Plane hosts and Runtime Hosts

The control plane as we have said is responsible for managing all the things we have running in the runtime plane. Therefore, the deployment choices we make to host the control plane will impact where we can host the runtime plane.

Table 1.2 is a matrix showing the relationship between the Control Plane hosting option and the Runtime hosting option.

Table 1.2 - Control Plane deployments and runtime hosting options matrix

We can see in the table If we are hosting our control plane in the US or EU, we can host the runtime plane in any of the 4 hosting options identified earlier. However, if you are hosting the control plane in the GovCloud the only hosting options are CloudHub and Standalone servers. CloudHub 2.0 and Runtime Fabric are not available in GovCloud. If you must host the control plane on your own servers and infrastructure (PCE) then you can only host the runtime plane on your own servers and infrastructure.

Sometimes an organization has to make difficult choices in deciding where to deploy both the control plane and the runtime plane. Federal and State governments often need to follow regulations and may require software solutions to be FedRAMP compliant. MuleSoft has a solution for this called Government Cloud but it also comes with other limiting factors and can be more expensive. When using Government Cloud the Runtime plane must be CloudHub, standalone MuleSoft Runtimes, or a combination of the two (Hybrid).

Likewise, European organizations may need to keep all software entirely within EU datacenters. MuleSoft provides an option to use an EU hosted control plane. This control plane will also limit CloudHub deployments to be in the EU region and in EU Availability Zones.

For organizations with a mandate to retain all IT infrastructure and systems under their direct control and within their data center, there is the option to install Private Cloud Edition and run both the control plane and runtime plane on your own hardware.

The architect’s job in these situations is to understand how to navigate the differences. If the runtime host does not support Anypoint DataGraph, or Object Store v2, what options do we have instead? Do we even need DataGraph or Object Store v2? Moreover, if an organization is operating from the US or EU control plane and have every option open to them, what business criteria or IT policies or organizational resourcing constraints or any other of a host of environmental variables would prompt an architect to choose CloudHub over Runtime Framework. What would drive the architect to recommend combining CloudHub 2.0 with a standalone instance of the runtime engine.

To answer these questions, we need to understand each of these MuleSoft delivery approaches in a little more detail and see what they can do and what they can not. Where our applications will execute and what runtime plane capabilities are available is important. Part 3 of this book will examine these details so we as architects can be equipped to answer these questions and make our recommendations. In the next section we will take a first look at the specific capabilities and components availabel in the platform.

MuleSoft capabilities and components

As we have seen already, the Anypoint platform provides the components to design, deploy, and manage the APIs we build. The platform also provides the components, and in many cases, the infrastructure to run or execute these applications. Let’s review the different components that support each of these phases in the lifecycle of API first development.

Discover Capability

Anypoint Exchange can be thought of as a catalog or registry of all the different reusable components, or assets, available to use in your solution. This catalog can be searched for specific phrases or filtered based on pre-defined asset types or asset categories. You can list assets from your own organization or assets developed by MuleSoft. You can even filter based on the lifecycle stage the asset is in.

In Figure 1.7, We can see Exchange searching for all the assets provided by MuleSoft.

Figure 1.7 - Anypoint Exchange assets from MuleSoft

Looking at the different assets in this figure we can see the types of assets that can be registered or published to Anypoint Exchange has grown dramatically over the past 4 years. Initially Exchange was limited to Connectors and APIs. Anypoint Exchange is now home to 11 different types as of the time of this writing:

• Connectors: components, developed with the Mule SDK, which developers can use in MuleSoft flows to connect and interact natively with different systems.
• Dataweave Libraries: Developers can build Dataweave transformations which can be reused in other integrations.
• Examples: assets which provide an example application such as “How to Batch ETL with Snowflake”
• Policies: policies which govern the security of APIs, typically running in service mesh
• API Spec Fragments: reusable segments of RAML or OAS to be reused in the development of new API specifications.
• REST APIs: API specifciations for implementations which can used or consumed by other approved applications
• RPA Activity Templates: templates for the MuleSoft Robotic Process Automation (RPA) capability
• Rule sets: rules for API governance
• SOAP APIs: APIs which can used or consumed by other approved applications
• Templates: templates which can be copied and configured for your environment
• Custom (e.g. accelerators)

Design capability

API design begins with the API Specification. Whether the organization has standardized on the the Open API Specification (OAS, formerly known as Swagger) or RESTful API Modeling Language (RAML), the specification can be created in Anypoint Platform Design Center. Design Center allows you to design specifications, fragments, and AsyncAPI specs as shown in Figure 1.8 Design Center API Specification.

Figure 1.8 - Design Center API specification

In this screen, MuleSoft can provide a guide or “scaffolding”, making the development of the specification “low-code/no-code”. This capability is also on the cloud so designers can get started without needing to install anything on their laptop or desktop.

API design specifications can also be created inside the two primary development tools, Anypoint Studio and ACB. Both tools allow you to synchronize your API specification with Design Center. Likewise, for any API specifications started in Design Center, you can open and work on these in Studio or ACB.

Management capability

The Anypoint Management Center provides several Anypoint capabilities focused on the operations and administration of the platform as well as the API applications running.

The screen in Figure 1.9 shows the options in the Management Center.

Figure 1.9 - Anypoint Management Center

The list of management capabilities in Figure 1.9 are defined here.

• Access Management: provides the capability to create Anypoint Platform users or set up an external identity provider (IdP) to manage users of the platform and the permissions they have been granted to the platform. It also exposes the audit logs which capture anything done on the platform from deploying an application to publishing an API.
• API Manager: provides the capabilities of applying policies, setting up service level agreements (SLAs), and securing APIs. The API Manager is also used to manage two types of API alerts. Alerts for API request/response/policy and alerts for contract changes or violations. API Manager also manages and deploys any proxy API runtimes you need to associate with an API to manage it.
• Runtime Manager: provides capabilities for the management of Anypoint VPCs, Anypoint VPNs or Private Spaces, MuleSoft applications, load balancers, standalone servers, and flex gateways across different environments. Additionally, application deployment, setup, and management can all be controlled from the Runtime Manager.
• API Governance: gives architects the capability to define rule sets for API conformance and monitor compliance and send notifications to developers to resolve compliance issues.
• The Visualizer: provides a graphical view of the APIs deployed to a Runtime plane, as observed through three lenses: Architecture, Troubleshooting, and Policies. The application view shows the relationship of the applications, the nodes, in your application network. The troubleshooting view includes metrics information for all the nodes in the application network. The policy view allows you to see an overview of the policies applied to the MuleSoft applications in your network.
• Monitoring: provides the capability to view performance dashboards, review logs and cpu/memory/disk consumption, and look for trends across deployed APIs. Monitoring also allows operations to look for stability issues of any APIs based on performance and usage findings.
• Secrets Manager: provides a secure vault for keeping certificates, key stores and trust stores, and username/password key value pairs safe. Security Manager is part of a Security capability which supports edge policies and tokenization in the Runtime Framework runtime plane.

We have now looked at some of the primary capabilities of the Anypoint platform, many of which we will go into more detail later. With a historical backdrop of the different approaches to integration, and now with all these new iPaaS platform capabilities available, let’s take a look at some of the difficulties, new and old, organizations are facing today.