Implement an API Design-first approach for building APIs [Tutorial]

The Monster Records & Associates (MRA) –a fictional music records company, having realised that its biggest asset is in fact its data, embarked on a digital transformation with the aim to offer its product and offerings completely online and via APIs.

This article is an excerpt taken from the book Implementing Oracle API Platform Cloud Service, written by Andy Bell, Sander Rensen, Luis Weir, Phil Wilkins.

In this post we are is going to take you through an interesting MRA case study who adopted an API design-first approach for building its APIs. We will go through the process and steps performed by MRA for this implementation.

The Problem Scenario

MRA had embarked on a digital transformation journey with the objective to become a digital organisation capable of offering tailored (à la carte) offerings to artists such as handling of an artist’s online presence to on-demand distribution of an artist's digital media to Music Streaming Services such as Spotify, Apple Music, Google Play Music, Amazon Prime Music, Pandora, Deezer to name a few.

Having fully acknowledged that their most valuable asset is in fact their media data, MRA wanted to materialise in such assets and determined that the quickest and most effective way to achieve this was by exposing a public API capable of providing access, on-demand, to MRA's Media Catalogue assets such as artists, songs and albums.

implementing-an-api-design-first-approach-for-building-apis-img-0

Figure 1: MRA's Media Catalogue API

The idea being, once such assets became accessible via an API, streaming services could, on-demand and 24x7, explore MRA's repertoire, purchase rights-to-use and start streaming. In addition, the API could also open the door to a brand new global audience: millions of app developers constantly innovating. If only a fraction of such a huge audience leveraged MRA's Media Catalogue API, it would still represent a considerable success for MRA.

However, as with everything, there is a challenge to realise such vision.

MRA like many other organizations, had a level of experience with systems integrations and Service Oriented Architectures (SOA). One of the lessons learnt from SOA however was that the cycles for designing, building, prototyping, and testing SOAP-based Web Services could be quite lengthy and expensive.

An API differentiates from a service in that the former represents the RESTful interface a consumer application interacts with, whereas the latter is the actual implementation (the code) behind an API. A HTTP endpoint exposed by a service is defined as an unmanaged API.

When a service endpoint is accessed via an API Gateway where policies such as app-key validation, authentication/authorization and other policies are enforced, then it becomes a managed API.

The book, Implementing Oracle API Platform Cloud Service, refers to managed APIs as simply APIs and unmanaged APIs as simply service endpoints.

Especially when it came to capturing and accommodating the feedback from Client Application Developers (API consumers), MRA had very bad experiences as in the majority of occasions they came to realize very late in the software lifecycle that the Web Service developed did not meet the expectations of its consumers.

implementing-an-api-design-first-approach-for-building-apis-img-1

Figure 2: feedback-loops in traditional web service design

Refactoring web services in this approach wasn't just time consuming but also an expensive exercise as both the Service design (WSDL) and code had to be refactored and re-tested in order to accommodate the feedback received and before application developers could try a service again. Naturally service designers and developers avoided as much as possible making changes, thus challenging feedback received from application developers, which in turn created friction amongst both teams but in some occasions meant application developers finding alternative routes to solve their needs rather than using the web service. This was the worst possible scenario as it meant that the investments made in implementing a web service could've been wasted.

API design-first process

Learning from experiences and acknowledging the challenges that such waterfall like process imposed to a digital transformation initiative, MRA were quite keen to adopt a more agile, interactive but also quicker way to deliver modern RESTful based APIs.

The idea was clear. By engaging application developers (API consumers) in the initial stages of the design process, feedback would be captured and reflected back in the interface design (API) early as well. Not only this would shorten feedback loops, but ensure that once the underlying services are implemented, it would expose an interface already endorsed and tested by its consumers, as opposed to risk building a service that won't satisfy the client expectations and needs late in the process.

implementing-an-api-design-first-approach-for-building-apis-img-2

Figure 3: API design-first approach vs traditional service design

The implication of this approach though, is that the tooling and notation to define the API, had to be both simple, yet rich in capability such as the task of designing and mocking API endpoints is quick and easy, given that if the process becomes cumbersome it would defeat its purpose.

We elaborate on the different steps undertaken by MRA when designing its Media Catalogue API using Apiary and related tools in the book, Implementing Oracle API Platform Cloud Service. Here are the steps:

Defining the API type

Defining the API’s domain semantics

Creating the API definition with its main resources

Trying the API mock

Defining MSON Data Structures

Pushing the API Blueprint to GitHub

Publishing the API mock in Oracle API Platform CS

Setting up Dredd for continuous testing of API endpoints against the API blueprint

Defining the API type: A fundamental step when designing any API is to first define what the type is. This is important as it will determine the guiding principles to consider when doing the design. We have three types of APIs:

Single-Purpose APIs: These are APIs that serve a unique and specific purpose, typically derived from an unambiguous need associated with a user journey or use case.

Multi-Purpose APIs: These APIs are more generic in nature and are meant to satisfy not just one but multiple use cases and scenarios. They are not bound (coupled) to a specific user journey or system of engagement (for example, a mobile app) therefore ideal for reuse enterprise-wide.

MRA’s Media Catalogue API: MRA's Media Catalogue API was specifically targeted at two main audiences: Music Streaming Services and Application Developers in general. Therefore, the API had to be both Public and Multi-Purpose.

Defining the API’s domain semantics: This step elaborates proper understanding of the API's bounded context, Media Catalogue. To do so, entities, key attributes, and relationships within the bounded context itself were identified and also defined using semantics appropriate for the purpose of the API.

Creating the API definition with its main resources: This step shows how to create an API and define its main resources, parameters, and sample payloads.It involves steps followed by MRA when creating the Media Catalogue API definition and its associated API mock.

Trying the API mock: This part describes how Apiary's automatically generated API mocks can be used to satisfy one of the most important steps in API design-first: try an API early in the lifecycle, before the API is actually implemented. This is a critical step as collecting feedback from API consumers early can potentially save numerous hours in code refactoring later in the project.

Defining MSON Data Structures: The Markdown Syntax for Object Notation (MSON) is a plain-text syntax for the description and validation of Data Structures in API Blueprint. It provides a way to represent objects (for example, an artist) in a human-readable plain text form. This part involves steps to define the Artist, Album and Song objects using the MSON notation.

Pushing the API Blueprint to GitHub: API Blueprints can be pushed into GitHub repositories, so they can be version controlled but most importantly it can follow a similar GitHub cycle as any other code asset. This step is also required in order to configure Dredd to validate API endpoints against API blueprint definitions.

Publishing the API mock in Oracle API Platform CS: Although Apiary provides an API mock URL that can be can be accessed directly, it is recommended that instead, the API mock is published and accessed via the Oracle API Platform Cloud Service.

Setting up Dredd for continuous testing of API endpoints against the API blueprint: The last step of the API design-first process is to configure Dredd to continuously validate that an API endpoint exposed through the API Gateway is always compliant with its corresponding API Blueprint definition. The idea is to ensure that Client Application code is not broken once an API Policy is changed to point to a Backend Service once it has been built and deployed.

We discussed the API design-first approach for building its APIs. MRA's business scenario demanded the need for more efficient and leaner process for implementing APIs. We saw how an API design-first process could effectively help organizations such as MRA gain greater speed, agility, and efficiencies. Here’s a summary of the basic steps to realize such process.

Choose your API type: We introduced the conceptual concepts such as Single Purpose and Multi-Purpose APIs to decide on what type of API to adopt.

Define your APIs: The need for creating an API definition and an API mock in Apiary based on API Blueprints and the Markdown Syntax for Object Notation (MSON).

Create & publish API: Creation and publication of an API using the Oracle API Platform Cloud Service.

Continuously test: Finally, the configuration of Dredd to verify API endpoints compliance with the API definition.

You just enjoyed an excerpt from the book Implementing Oracle API Platform Cloud Services. Grab the latest edition of this book to work with the newest Oracle APIs, and interface with an increasingly complex array of services your clients want.

What are the best programming languages for building APIs?

Glancing at the Fintech growth story – Powered by ML, AI & APIs

What RESTful APIs can do for Cloud, IoT, social media and other emerging technologies