There are a few definitions. For example, Red Hat says that APIs are “a set of definitions and protocols for building and integrating application software.” whereas Amazon Web Services (AWS) states that “APIs are mechanisms that enable two software components to communicate with each other using a set of definitions and protocols.”. Well, APIs are not limited to two software components only, for sure, but both definitions share this part: “definitions and protocols”. Let’s craft our own definition by making a comparison with the analog world.

An API is a bridge (communication path) between two distinct parts (codes), belonging to the same city or not (the same program). By following a set of pre-established traffic rules (protocols) and conventions (definitions), vehicles (requests and responses) can freely flow between both sides. Sometimes, APIs may have speed controls (throttling gears) that are enforced as needed.

As it happens with all kinds of communication, definitions need to be established first. This rule is not limited to the digital world. I can’t ask you to sell me a car if you have no idea what selling is or if a car is a type of vehicle. Protocols also are paramount. Unless you are donating a product, a sale starts with me paying you for the product I want and you handing it over to me. It includes giving me change if necessary.

In terms of APIs, definitions are related to which types and lengths of data are acceptable and allowed between the communicating partners. A requester cannot send certain data as a string when the receiver is expecting to receive a number, for example. Negative numbers may also pose an additional challenge to badly written APIs. When dealing with data lengths, minimum and especially maximum sizes are applicable. You will learn later how important it is to block data chunks that are bigger than what your API is able to handle.

Protocols are the second component of an API. As their counterparts in the networking arena, they are responsible for guaranteeing that independently written software will be able to communicate in an effective manner. Even though you might be reading this book primarily because of web-bound APIs and ways to explore their security flaws, I need to tell you that even inside your computer, there are APIs working between your Operating System (OS) and your Wi-Fi card, with definitions and protocols like their more famous web cousins. If you are familiar with the Transmission Control Protocol/Internet Protocol (TCP/IP) stack, the following figure is not strange. The communication on TCP/IP can only happen because each small rectangle has their own lower-level protocols implemented in a way that allows the same Network Interface Card (NIC) to be used in different OSs and those different OSs can communicate with each other:

Figure 1.1 – Communication with TCP/IP

Every API should be well documented so anyone who wants to use it does not have to request information from its creators or maintainers. Can you imagine the avalanche of NIC manufacturers sending enquiries to Defense Advanced Research Project Agency (DARPA) scientists to understand how the data link layer should be developed, which data structures should be in place, and which sizes and types of data should be considered every time a new product was going to be released?

When documenting an API, at least the definitions of data types and protocol(s) adopted need to be made explicit. Well-documented APIs also usually have examples of their usage, along with exceptions that may be generated when something goes wrong, such as bad data manipulation or unexpected behavior.

A brief history of APIs

You will read a lot about web APIs in this book. However, as you saw in the TCP/IP example, APIs were not created along with the web. The idea was born many decades ago, in 1951, when Maurice Wilkes, David Wheeler and Stanley Gill, three British computer scientists, proposed this concept while they were building the Electronic Delay Storage Automatic Calculator (EDSAC), one of the very first computers ever. Their book, The Preparation of Programs for an Electronic Digital Computer, focused primarily on explaining the library they built, as well as its subroutines (should you need to develop a program to run on the EDSAC). Observe the concern in explaining how the computer could be used beginning with the book’s title. This book became the first API documentation we have records of.

Moving on to the 1960s and 70s, the usage of computers grew, leveraging the improvements in electric and electronic circuitry. Their sizes also started to reduce. Nonetheless, they were still the size of some rooms. The use of APIs was now attached to the need for developers to not have to worry about the details of how displays or other peripherals worked. We were in the era of mainframes, and the advent of new ways to interact with a computer, such as terminals and printers, was posing additional challenges to program developers. In 1975, Cristopher Date and Edgar Codd, a British mathematician and computer scientist respectively, released a paper titled The relational and network approaches: Comparison of the application programming interfaces. In this work, APIs were proposed to databases, something that is still in use today.

In the 1980s, we started seeing commercial explorations of consumer networks. In 1984, The Electronic Mall, an online shopping service sold by CompuServe, was offered to the company’s subscribers. They could buy products from other merchants through their Consumer Information Service network. You may ask yourself where there is an API in all of this. With the incremental usage of computer networks, developers needed to sophisticate their code, and requirements to access code and libraries located in remote computers began to show up. It was in 1981 that the term Remote Procedure Calls (RPCs) was coined by the American computer scientist Bruce Nelson. The concept is as simple as a client sending a request to a network server that then processes the request (executes some computation) and returns a result to the client. RPC is therefore what we know as a message passing mechanism, in which some channel (usually a computer network) is applied to allow communication between different elements through message exchanges.

In the 1990s, that is, more than 40 years after the idea of APIs was first used, the internet was generally used around the world (in the USA, this happened nearly one decade before). Previously restricted to research institutions and government agencies only, the commercial use of the network was then completely possible. This increased the adoption of APIs even more and they became the de facto way of exchanging information between programs. New websites came up, new consumer products and services became commercially accessible through the internet, and it was clear that software needed standards to communicate with each other. Java, a programming language created by Sun Microsystems (now part of Oracle Inc.), played a vital role. In 1984, John Gage, the #21 employee of Sun Microsystems, coined the phrase “The network is the computer”. In his own words, “We based our vision of an interconnected world on open and shared standards.” Eleven years after, James Gosling, another Sun Microsystems employee, created the Java programming language, which would evolve to Java 2 afterward and became the seed of notable APIs, released as part of its Java 2 Enterprise Edition (J2EE, now Jakarta EE) and Java 2 Micro Edition (J2ME).

In the 2000s, the internet had pretty much been consolidated. The always-growing number of companies joining the network among massive amounts of developers creating new web solutions demanded a quick and effective way to establish a communication path between clients (at this time, those were mostly browsers) and web servers. In 2000, a PhD thesis entitled Architectural Styles and the Design of Network-based Software Architectures by Roy Fielding proposed a structured way to allow clients and servers to exchange messages on the internet. Roy proposed Representation State Transfer (REST), which became one of the most popular API protocols in the world. This decade also saw the explosion of cloud computing offerings, both private and public, which mostly implemented REST. It also saw the creation of Web 2.0 in 2004, which states the new way that the internet should be used (with a greater focus on centering on the user), as well as the birth of applications such as Facebook, X (previously Twitter), Reddit, and many more.

Ten years later, in the 2010s, web protocols were even more evolved. We were in the decade of social media and apps, with millions of requests per minute. To give you an idea, in 2013, each minute on the Internet was occupied, among other traffic, with 461,805 Facebook logins, 38,000 photos uploaded to Instagram, and 347,000 tweets sent. This was also the decade when containers and microservice-based applications faced their most expressive adoption. The release of Kubernetes, an open source container orchestrator, augmented the possibilities for dynamic applications on the internet. It was in the 2010s that the term Web 3.0 was coined for the first time, with its focus primarily based on blockchain. APIs became fundamental for companies creating and delivering their products to the public.

As the Tears for Fears’ 1985 hit song Head Over Heels states, it’s “funny how time flies”. Time really flew and we arrived in the 2020s. Nowadays, applications keep modernizing themselves, but now we have the presence of systems running even more spread. The advent of concepts such as edge computing and the Internet of Things (IoT) increased the complexity of the whole scenario and demanded the evolution of APIs to encompass such changes. Web 3.0 was, in fact, only incorporated in 2021. We currently have applications being designed and developed around an API, and not the opposite, as it happened in the early stages of the technology.

Back to our web world, there are some notable API protocols:

• Simple Object Access Protocol (SOAP): This allows access to objects, maintains communication using HTTP, and is based on Extensible Markup Language (XML). It is simple and presents a good way to establish communications between web applications, as it is OS-independent and agnostic about technologies and programming languages.
• REST: Maybe one of the most famous web API protocols in use nowadays, REST is an architectural style to design web services. Therefore, the services that follow such a style are said to be RESTful. The predefined set of REST operations is stateless, and the services have access to constructs to manipulate text-based representations of the data.
• Google Remote Procedure Call (gRPC): Developed by the company behind the search engine, it is another HTTP-based architecture that happens to be open source. It applies buffers to allow data transmissions between pairs.
• JavaScript Object Notation – Remote Procedure Call (JSON-RPC): Just like REST, JSON-RPC is also stateless, uses objects (like SOAP), and can be applied instead of REST when higher performance is necessary.
• Graph Query Language (GraphQL): It was created by Meta (previously Facebook) and designed to be a database query language. GraphQL is open source and allows for complex responses by using simple data structures such as JSON.

Let’s analyze each one of them in more depth.

SOAP

Since SOAP is based on objects, for the sake of simplicity, both peers in a conversation must agree on which elements they would use to exchange information. SOAP messages are implemented by regular XML files containing at least the following elements:

• Body: It keeps information about the call and the response.
• Envelope: This identifies a file as a SOAP message.
• Fault: It carries information about errors and status.
• Header: As the name implies, holds header information.

Although SOAP messages must use XML as their structure, such documents cannot contain processing instructions or Document Type Definitions (DTDs). An XML document has its attributes defined inside a DTD. The SOAP 1.1 specification had three parts:

• The envelope, where the contents of the message are defined, the responsible structures that should handle it, and a specification if it is mandatory or optional.
• The encoding rules that define the mechanism to be used when serializing the datatype.
• The RPC representation that indicates how to represent remote calls and their responses.

The SOAP 1.2 specification has only two parts:

• The message envelope.
• The data model and protocol bindings.

In terms of organizational structure, SOAP messages are comprised of namespaces. The root element is the SOAP envelope. The Header, Body, and eventual Fault elements are all inside of it. All SOAP envelopes must specify the http://www.w3.org/2003/05/soap-envelope/ Universal Resource Identifier (URI) as their namespace indication attribute. The encodingStyle attribute may appear to indicate which encoding schema is used inside the message. The envelope declaration would look something like this:

<soap:Envelope
xmlns:soap="http://www.w3.org/2003/05/soap-envelope/"
soap:encodingStyle="http://www.w3.org/2003/05/soap-encoding">

A header in a SOAP message is optional, but if one is present, it must be at the beginning of the message, just after the Envelope declaration. Its purpose is to store data that is specific to the application, such as payment information or an Authentication (AuthN) mechanism. Inside the header, some attributes can be declared, such as env:role, env:mustUnderstand, and env:relay. The first one is used to define which role is associated with the header block. The second one is a Boolean variable. When true, it means that the recipient of the message must process the header. If some issue is raised while processing the header, a fault element is generated. Finally, the env:relay component is only checked or processed by relay (intermediary nodes). It is a new feature of the SOAP 1.2 specification. An example header with two blocks could look like this (the tags were wrapped in multiple lines to facilitate reading):

<env:Header>
  <BA:BlockA xmlns:BA="http://mysoap.com"
   env:role="http://mysoap.com/role/A" env:mustUnderstand="true">
   ...
  </BA:BlockA>
  <BB:BlockB xmlns:BB="http://mysoap.com"
   env:role="http://mysoap.com/role/B" env:relay="true">
    ...
  </BB:BlockB>
</env:Header>

In this example, the block A part has a mustUnderstand clause that is true, which means that the recipient must process it. Block B is meant to be parsed by intermediary nodes only, since the env:relay attribute is set to true. Both blocks have role specifications.

XML Protocol (XMLP) was another XML-based message-exchanging protocol that was on spot until 2009, two years after SOAP specification 1.2 was released. XMLP proposed an abstract model, whereas SOAP details the primitives to allow for the practical application of this model. SOAP and XMLP have the concept of binding that determines which other protocol XMLP and/or SOAP should connect to work. One of (if not the) most popular bindings for SOAP is HTTP. This means that SOAP messages can and are effectively employed to allow communication of peers through HTTP.

REST

The predefined set of REST operations is stateless (as is also the case with XMLP), and the services have access to constructs to manipulate text-based representations of the data. While SOAP and XMLP have bindings that allow both to connect to other application-layer protocols and even to the transport layer (TCP or UDP), REST is more related to HTTP (also stateless), and therefore, manipulating such constructs reduces the learning curve for developers and sysadmins that are already used to HTTP terms. While using HTTP, all the protocol’s methods are available with REST: CONNECT, DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT, and TRACE. REST was used to define the HTTP version 1.1 specification.

There may be the presence of intermediary nodes, which, in the case of REST, are translated as gateways such as cache or proxy servers, or even firewalls. Those nodes could allow scalability to the architecture since no state is held inside the messages, and some explicit cache information could be inserted into the responses. According to Roy Fielding’s specification, there are six constraints that rule whether a system can be categorized as RESTful. They are as follows:

• Client-server: Although there might be intermediary nodes, the communication usually happens between two peers only.
• Stateless: No state is stored in RESTful messages. The session state must be managed by the client. As the state is not controlled, this grants scalability to the architecture.
• Cache: Intermediary nodes can present themselves as cache servers. The server points to the content that can be cached, and this is respected by the client.
• Uniform interface: Using generality, the architecture becomes simpler, which improves the visibility of interactions.
• Layered system: Through the adoption of a hierarchy, each layer only has visibility to the layers it directly interacts with, which allows for the encapsulation of legacy services.
• Code-on-demand: Client functionality can be extended through the download and execution of additional codes from the server, which simplifies the client design.

The heart of any REST-based design is the state transfer operations. They are universal to any retrieval or storage system, and the acronym that encompasses them is Create, Read, Update, Delete (CRUD). There are direct associations between those operations and HTTP verbs (or commands). Create relates to POST, Read relates to GET, Update relates to PUT and Delete relates to DELETE (HTTP verbs are usually represented in technical literature with all capital letters).

Despite the similarities, some notable differences exist between REST and SOAP. They are specially related to how to do remote invocations (RPCs). On the other hand, with REST, a client locates a resource in a server and chooses what to do with it (change it, delete it, or get info about it – which could be mapped to the UPDATE, DELETE, and GET HTTP methods, respectively). With SOAP, there is no direct interaction with a resource. Instead, the client needs to call a service and the service, in turn, does all the required actions with related objects and resources.

To circumvent this way of work, SOAP leverages some frameworks that allow it to give additional capability to the clients. One of those frameworks is Web Services Description Language (WSDL), a World Wide Web Consortium (W3C) recommendation from 2007. With the inclusion of specific attributes, such as getTermRequest, and a type, such as string, WSDL grants one step beyond using SOAP with web services.

We need to understand why REST virtually took over SOAP in the modern web API landscape. One of the points that counted in favor of REST when compared to SOAP was that SOAP is based on XML. This language can produce quite complex and verbose documents that obviously need to be correctly crafted by the sender and parsed by the receiver. Parsing an XML document (or structure) means reading it and transforming its elements into some data structure that can be further handled by the application. One of the most well-known parsers is called Document Object Model (DOM). One drawback of using DOM is its high memory consumption, which might be many times bigger than the amount of memory originally described in the document.

In computer science, data serialization is the activity of transforming abstract objects (or elements) present in data structures into something that can be stored at or transferred between computers. Deserialization means the opposite. Data serialization becomes more complex as nesting is used in documents. XML allows element nesting. There is no formal limit for this in the XML specification, which essentially means that an infinite number of elements could be nested. Complexity may raise security threats. Through the parsing of an XML document, an application could store its elements in a Structured Query Language (SQL) database, translating them to tables, rows, and columns, or even as Key-Value (KV) pairs in a NoSQL database. When accepting serialized objects from unknown or untrusted sources, this might impose an unnecessary risk to the application.

Open Web Application Security Project (OWASP) is a global organization that regularly releases cyber security best practices, including secure code development, and maintains some notable security projects. One of them is Top Ten, which lists the top ten most dangerous threats to web applications. The most current version was published in 2021. Insecure data deserialization is in the A03-2021 Injection group, which means that it is considered the third-most dangerous threat for applications.

Under the same project but classified as the fifth-most dangerous threat to web security is the XML External Entities (XXE) attack, categorized under the A05-2021 Security Misconfiguration group. If an XML document makes use of DTDs, it can be incorrectly interpreted by the XML parser. A DTD was the first way to specify the structure of an XML document, and it can also be used to determine how XML data should be stored.

With the usage of DTDs, a vulnerable XML parser might be the victim of a Denial of Service (DoS) attack called an XML bomb (also known as a billion laughs attack). Through the specification of ten DTD entities, with each subsequent entity being ten times a reference of the previous entity, this would result in one billion copies of the first entity. As previously explained, to accommodate all entities in memory, the XML parser needs to allocate a considerable amount of memory, eventually crashing and making the application unavailable.

REST APIs, on the other hand, are primarily based on JSON data structures. Those are simpler documents organized as maps that leverage the concept of KV pairs. JSON files do not require a specific parser; they support different types of data, such as strings, Boolean, numbers, arrays, and objects. However, JSON files are usually smaller when compared to their equivalents on XML. JSON also does not support comments. JSON structures are therefore more compact, as well as easy to craft and process. The code block that follows contains an example of a JSON structure:

{
    "config_file": "apache.conf",
    "number_of_replicas": 2, "active": true,
    "host_names": [
        "server1.domain", "server2.domain"
    ]
}

gRPC

The core idea of gRPC is to let you, a developer, invoke a remote method (located on your colleague’s computer or on the other side of the world) as if it was in your codebase itself. In other words, a client (or stub, as it is referred to inside the specification) calls a function, with its expected parameters, but that function is not even inside its code. It is implemented somewhere else. To tackle this, you need to follow definitions established by the server side of the gRPC invocation. Such definitions include the acceptable data types and the methods to return after their invocations end. Everything is based on creating a service that will leverage such methods to provide data to clients.

Another interesting part of gRPC is the support of modern programming languages, which allows you to split the development efforts among your team, with, for example, the Go programmers being responsible for the server and the Python programmers being occupied with building the client. As the protocol was created by Google, a gRPC server can also be hosted on the company’s public cloud.

There is one major difference between gRPC and the other two protocols already covered: it uses protocol buffers, although it can also be configured to work with other data formats, such as JSON. Protocol buffers is a data serialization technology created by Google in which you define the data structures you are going to use in your applications and, by applying the protoc protocol buffer compiler, object classes are created in your code. The data structures are stored in text files with the .proto extension. In a .proto file, you create a service and define what makes the message that will flow between the client and server. When you run protoc, it creates or updates the corresponding classes. The code block that follows shows an example of a file like this:

service MyService {
    rpc ProcessFile (FileRequest) returns (ExitCode);
} // Comments are supported.
message FileRequest {
    string FileName = 1;
}
message ExitCode {
    int code = 1;
}

In the preceding code, you are creating a service called ProcessFile that is invoked by the client side of your application on a method called FileRequest that returns ExitCode as the output. This last method is implemented on the server portion of your application. Obviously, as per the definition of gRPC, client and server portions can be in separate machines. Services can be of four different types:

• Unary: The client sends a single request and waits for a single response.
• Server Streaming: The client sends a request, and the response is returned as a stream of messages. The messages are sent in sequence.
• Client Streaming: The client sends a sequence of messages and waits for a single response from the server.
• Bidirectional Streaming: Both parts send sequences of messages.

It is interesting to realize how gRPC also works as a Software Development Kit (SDK). This means that the package has some software development support foundations that can be leveraged to design and deploy applications. It is not only a protocol per se but also a toolbelt to help you create your applications, led by the protoc compiler. In Python, the compiler is implemented as a Package Installer for Python (PIP) module.

JSON-RPC

As we’ve introduced, JSON-RPC is a good replacement for REST when performance is an important factor. One characteristic of this protocol is that a client can send a request with no need to wait for a server response. Another feature allows clients to send multiple requests to the server and the server returning the responses out of the original requested order. In other words, the server’s responses follow asynchronously.

The current specification is 2.0 and it is not fully compatible with the previous one (1.0). JSON-RPC 2.0 request and response objects may not be correctly understood when the client and server are not running the same version of the protocol, although it is easy to identify the 2.0 specification, since it uses a jsonrpc key whose value is 2.0. All JSON primitives (strings, numbers, Booleans, null) and structures (arrays and objects) are fully supported.

There is a strict syntax (remember when we started talking about API definitions?) that must be respected when sending requests and receiving responses. The following are possible members of a request:

• jsonrpc: This contains 2.0 when this is the specification in use.
• method: String containing the name of the remote method to be invoked.
• params: Optional member that’s structured (either an array or object) and contains parameters to be passed to the invoked method.
• id: Optional member that can be a string, number, or null and contains the identification of the request.

Likewise, there is a definition for the response structure. Its members are as follows:

• jsonrpc: Same description as for the request.
• result: Exists only when the method was successfully invoked; the contents are provided by the invoked method.
• error: Only exists when the method is not successfully invoked; this is an object member, and its contents are provided by the invoked method.
• id: Same description as for the request, needs to carry the same value as the one specified in the request.

The error object has its own structure. You can easily realize another difference between REST and JSON-RPC. There are no HTTP methods, such as GET, PUT, or POST, to be called. Instead, a simple JSON structure is provided. Another difference lies in the response. Where REST can use JSON or XML formats, JSON-RPC only supports JSON. For error handling, you just saw that JSON-RPC has its own error member. REST provides HTTP status codes, such as 200 (OK), 404 (Not Found) or 500 (Server Error). Caching is supported by REST but not by JSON-RPC, and finally, JSON-RPC is simpler than REST simply because it only supports the request and response JSON structures. The code block that follows shows examples of requests and responses. A method called IsStudent is invoked to return True or False should a provided numeric enrollment id be a registered student. The first request succeeds, while the second request generates an error:

{"jsonrpc": "2.0", "method": "IsStudent", "params": [100], "id": 1}
{"jsonrpc": "2.0", "result": true, "id": 1}
{"jsonrpc": "2.0", "method": "IsStudent", "params": ["ABC"], "id": 2}
{"jsonrpc": "2.0", "error": {"code": -1, "message": "Invalid enrollment id format"}, "id": 2}

GraphQL

GraphQL, as the name implies, is a language to allow querying data served by an API. Wait a moment! This is inside a subsection on protocols. What is a language doing here? A generic definition of protocol could be “a set of rules that need to be properly followed to allow the successful establishment of communication between two or more peers.” GraphQL implements this as well.

It was created by Meta (then Facebook) in 2012 and released as an open source project in 2015. Later, in 2018, it was started to be hosted by the Linux Foundation and its ownership was taken by the GraphQL Foundation. One notorious feature is the fact that a single endpoint is exposed, making it easier for developers to request and receive the desired data. Other API protocols may eventually expose multiple endpoints to fulfill the needs of providing different types of data, or data spread in various databases or systems.

The data formats are also like JSON with some slight changes. There is a tremendous difference between GraphQL and REST. Rather than making requests, fetching the results, and adjusting the requests after analyzing the results to then submit new requests, with GraphQL, the application can interactively change the request until the received results are satisfactory. This is supported by WebSockets, a technology that allows continuous bidirectional communications between an HTTP client and a server where both sides send and receive data and any side can close the connection.

Since any side, client, or server can send data to each other at any time, WebSockets is also useful for sending notifications, especially from server to client, while the connection is still open. One possible application for this protocol is a currency exchange website. A client queries the server for the rate once. Every time the rate changes, the server notifies the client of the new rate. GraphQL also supports query parameters. You can filter results based on a criterion or ask the server to make data conversions or calculations all in the same query. The code block that follows shows an example of a request:

{
  student(id: 100) {
    name
    grade(average: True)
  }
}

The preceding code queries the server for a student whose id is 100. The client wants the student’s name and their grade, but only the average grade (calculated over the course modules), not the grade itself (average: True). A possible answer is in the code block that follows. Observe that responses in GraphQL follow the structure of the request:

{
  "data": {
    "student" {
      "name": "Mauricio Harley"
      "grade": 85.2128
    }
  }
}

GraphQL data structures have a schema. This way, when designing queries, a developer will know the possible types of data that could be returned in a response in advance. It is useful to know that a single query may generate a list of items as a response with not much effort, considering the schemas have been properly set.

Even with the simple code and template examples you have seen here, an attentive reader will realize that potential security flaws may arise from them since those simple data structures and queries can result in extensive resource consumption with a small number of lines. Secure software development is not a new buzzword but has gradually received more focus as new threats and attacks enter general awareness.

Some companies prefer investing more of their time and money into containment strategies, such as implementing an incident response team, bound to a business continuity plan. Albeit very important, we know that such teams are put into action when something has already happened. They can only do damage control, trying to reduce the impact some intrusion has on the company’s assets. Some other companies believe that their systems are secure simply because they are running on a public cloud. It is well known that public cloud players share this responsibility with their customers in a way that is called the shared responsibility model:

Figure 1.2 – The public cloud shared responsibility model

As it is valid with any substantial software being developed, APIs have their life cycles, and they belong to a pipeline. There is a general sense that security should be shifted left as much as possible, which means that concerns about potential flaws should be taken into consideration sooner rather than later. You need to think about security starting with the API design. However, this may not be exactly easy for companies with a small budget or without proper technical enablement. This is to say that not all companies adopt security countermeasures in the early stages because they simply cannot.

When developing an API, you should start by choosing the protocol your API will use. Consider the ones discussed here and select one that you believe will fulfill the application’s needs and user expectations. Look for the protocol’s drawbacks and verify the possibility of letting a public cloud player implement the API for you. All major players have API management or gateway offerings. They usually implement security best practices and are integrated with web firewalls.

APIs are frequently the only entrance doors, or at least the most used ones, for applications. This is why reinforcing them is paramount for any business segment. All parts of an API should get their corresponding protection. For example, how do you handle AuthN and Authorization (AuthZ)? Do you use tokens or only user/password credentials pairs? How are such tokens or credentials stored and how do they flow between your API endpoints and clients? Do you handle their life cycles? Do you record every time a token or credential is used and what parts of your system a user with such tokens or credentials tries to access? Do you frequently rotate tokens or credentials? Can you see how many questions were raised for a single point of attention? Badly handled AuthN and AuthZ may lead to potential intrusions and massive damage.

AuthN and AuthZ are just some of the topics that deserve strict care when designing and developing an API. Although they are two separate concepts, they are usually spoken of and discussed together because it does not make much sense to have one without the other. They are not only relevant when dealing with external users. When your application needs to interact with internal systems or partner applications, the same or other controls must be in place. Applications talk to applications, and impersonating an application or an external user is the first vulnerability I would like to talk about.

OWASP, the same organization mentioned earlier, also owns the Top 10 API 2023 security project. Its API Security Top 10 initiative positions API1:2023 – Broken Object Level AuthZ and API1:2023 – Broken AuthN as the two most dangerous threats. The first topic is about not correctly handling access to objects throughout the API execution. This can lead to inadvertent exposure of data, including sensible data, to unauthorized people. So, controls to verify and protect access at the level of objects need to be in place. The second topic is related to what we discussed in the last paragraph of the previous section. Incorrectly handling AuthN data or implementing weak AuthN mechanisms or with known security flaws becomes a very big headache on your API management.

Moving on, we have Broken Object Property Level AuthZ as the third-most problematic threat. APIs vulnerable to this either do not implement or only partially implement the security controls necessary to protect object-level access, which results in data being exposed more than necessary, especially to unauthorized people. It is like Broken Object Level AuthZ, but this vulnerability has to do with APIs displaying more data than necessary to carry out their activities. Next on the list is unrestricted resource consumption. Do you remember, back when we were talking about XML and XMLP, how we mentioned that the way the XML documents are created may lead to security exploitations? This is what it is about. By not correctly parsing the input, an API might suffer DoS, since more processing power or disk access will happen, leading to increases in costs. Imagine the API is running on a public cloud provider and the result of more processing being demanded is the launching of new instances (virtual machines) or storing random data on high-performance disk areas. This would augment the monthly bill exponentially or activate some throttling mechanism – managed by the cloud provider or by your company – that would put the API down or into a dormant state. In any case, the application stops running.

AuthZ issues come up one more time with the next threat. As your API grows in complexity and reach, especially if it touches other systems, you may hit Broken Function Level AuthZ, which means you need to pay close attention to roles and personas created to separate permissions inside the API. When they are not clearly defined and enforced, bad handling of the API hierarchy may lead to vulnerabilities in which a user belonging to a role can purposefully or accidentally (valid users may face this issue even when they do not mean to do so) assume permissions of a higher role. An API does not constitute the whole application. It is part of something bigger and sometimes, various business flows are running to sustain the system. When you have unrestricted access to business flows, a subsequent vulnerability that may arise is the API exposing how such flows are internally structured. Hence, an attacker exploiting this vulnerability could infer the business logic behind the API. This will be covered later.

Server-side request forgery is a very common threat to web applications and APIs, including in cloud environments. A vulnerable API would accept any URI, including running internal commands that could reveal the supporting system behind the API: OS, kernel or library versions, and additional components, among others. It is important to protect the API itself by securely designing and implementing it. There is a saying in Brazilian Portuguese that translates to something like this: “one swallow doesn’t make a summer”. I mean to say that only protecting the API itself is not enough. When a system is vulnerable to security misconfiguration, in other words, when the systems that help the API to work are not updated frequently enough or when they are not tuned to implement security best practices, this threat becomes a reality for the API.

It is quite important to manage the whole environment where the API runs, including endpoints, underlying systems, libraries, and so on. APIs, as is the case for any software, have versions that are eventually made obsolete. Endpoints running deprecated versions should either be decommissioned or made unavailable. When improper inventory management occurs, forgotten API endpoints or sustaining systems may still be participating in the API’s current implementation and expose additional exploitable vulnerabilities. The API you developed is meant to be consumed by valid users or third parties. However, investment in protecting APIs is usually dedicated more to external users than to partners. When an attacker discovers API integrations, they might try to exploit the third party to then intrude on the originally targeted endpoint. This is referred to as the unsafe consumption of APIs and can be avoided or at least reduced when you adopt a terminology called zero trust, which we will talk about later.

This chapter introduced the concepts behind APIs and included a brief account of their history, including explaining what data definitions are and disclosing the main protocols that implement APIs. We moved on and discussed how important API security is for modern applications and we finished the chapter by talking about the most common API vulnerabilities. I hope you have enjoyed the beginning of our journey toward pentesting APIs.

In the next chapter, we will set up our pentesting environment. Some tools will be introduced, examples of execution will be given, and we will have the chance to save some time by cloning the book’s repository, which will allow us to leverage some utilities.

• You can find a definition of API from Red Hat at https://www.redhat.com/en/topics/api/what-are-application-programming-interfaces
• You can find a definition of API from AWS at https://aws.amazon.com/what-is/api/
• See a scientific article comparing TCP/IP and OSI-RM models at https://ieeexplore.ieee.org/document/46812
• The Preparation of Programs for an Electronic Digital Computer: https://archive.org/details/programsforelect00wilk/mode/2up
• You’ll find a scientific article comparing APIs at https://dl.acm.org/doi/10.1145/800297.811532
• Read up on a brief history of e-commerce at https://web.archive.org/web/20160326123900/http://gsbrown.org/compuserve/electronic-mall-1984-04/
• DSs thesis Architectural Styles and the Design of Network-based Software Architectures: https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
• This infographic shows data generation during one minute on the internet, comparing 2013 and 2014: https://www.fourthsource.com/general/internet-minute-2013-vs-2014-infographic-18293
• OWASP Top Ten Project: https://owasp.org/www-project-top-ten/
• gRPC Official Documentation: https://grpc.io/docs/
• See the official JSON-RPC documentation at https://www.jsonrpc.org/specification
• You can learn more about the OWASP Topen Ten API Project at https://owasp.org/www-project-api-security/