Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Build Your Own Web Framework in Elixir
Build Your Own Web Framework in Elixir

Build Your Own Web Framework in Elixir: Develop lightning-fast web applications using Phoenix and metaprogramming

eBook
$9.99 $29.99
Paperback
$36.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

Build Your Own Web Framework in Elixir

Introducing the Cowboy Web Server

“Web servers are written in C, and if they’re not, they’re written in Java or C++, which are C derivatives, or Python or Ruby, which are implemented in C.”

– Rob Pike, co-creator of Go

The web server is a key component of any modern-day web framework. Expanding on the point made in the preceding quote by Rob Pike, the Cowboy web server, written in Erlang, is also in a way implemented in C. Cowboy is the default web server used by Phoenix, the ubiquitous web framework in Elixir.

In this chapter, we will not be learning C, unfortunately, but we will take a closer look at how a web server is designed. We will provide some background on how a web server is built and set up to communicate with a client using HyperText Markup Language (HTML).

We will also learn the fundamentals of how HTTP requests and responses work, including their anatomy. We will then learn how to construct an HTTP response and send it using a web server. Moreover, we will learn the fundamentals of web server architecture by examining the components of Cowboy. Lastly, we will learn ways to test a web server and measure its performance. Doing this will put us in a better position to build our own web server in the next chapter.

The following are the topics we will cover in this chapter:

  • What is a web server?
  • Fundamentals of client-server architecture
  • Fundamentals of HTTP
  • How an HTTP server works
  • Using Cowboy to build a web server
  • Using dynamic routes with Cowboy
  • Serving HTML
  • Testing the web server

Going through these topics and looking at Cowboy will allow us to build our own HTTP server in Chapter 2.

Technical requirements

The best way to work through this chapter is by following along with the code on your computer. So, having a computer with Elixir and Erlang ready to go would be ideal. I recommend using a version manager such as asdf to install Elixir 1.11.x and Erlang 23.2.x, to get similar results as the code written in the book. We will also be using an HTTP client such as cURL or Wget to make HTTP requests to our server, and a web browser to render HTML responses.

Although most of the code in this chapter is relatively simple, basic knowledge of Elixir and/or Erlang would also come in handy. It will allow you to get more out of this chapter while setting the foundation for other chapters.

Since most of this chapter isn’t coding, you can also choose to read without coding, but the same doesn’t apply to other chapters.

The code examples for this chapter can be found at https://github.com/PacktPublishing/Build-Your-Own-Web-Framework-in-Elixir/tree/main/chapter_01

What is a web server?

A web server is an entity that delivers the content of a site to the end user. A web server is typically a long-running process, listening for requests on a port, and upon receiving a request, the web server responds with a document. This way of communication is standardized by the Transmission Control Protocol/Internet Protocol (TCP/IP) model, which provides a set of communication protocols used by any communication network. There are other layers of standardization, such as the HyperText Transfer Protocol (HTTP) and File Transfer Protocol (FTP), which dictate standards of communication at the application layer based on different applications such as web applications in the case of HTTP, and file transfer applications in the case of FTP, while still using TCP/IP at the network layer. In this book, we will be primarily focusing on a web server using HTTP at the application layer.

Example HTTP server

If you have Python 3 installed on your machine (you likely do), you can quickly spin up a web server that serves a static HTML document by creating an index.html file in a new directory and running a simple HTTP Python server. Here are the commands:

$ mkdir test-server && cd test-server && touch index.html

$ echo "<h1>Hello World</h1>" > index.html

$ python -m http.server 8080

Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/) . . .

If you are on Python 2, replace http.server with SimpleHTTPServer.

Now, once you navigate to http://localhost:8080/ on your web browser, you should see "Hello World" as the response. You should also be able to see the server logs when you navigate back to the terminal.

To stop the HTTP server, press Ctrl + C.

The primary goal of web servers is to respond to a client’s request with documents in the form of HTML or JSON. These days, however, web servers do much more than that. Some have analytical features, such as an Admin UI, and some have the ability to generate dynamic documents. For example, Phoenix’s web server has both of those features. Now that we know what a web server is, let’s learn about how it is used with the client-server architecture.

Exploring the client-server architecture

In the context of HTTP servers, clients generally mean the web browsers that enable end users to read the information being served, whereas servers mean long-running processes that serve information in the form of documents to those clients. These documents are most commonly written in HTML and are used as a means of communication between the client and the server. Clients are responsible for enabling the end user to send a request to the server and display the response from the server. Browsers allow the users to retrieve and display information without requiring any knowledge of HTML or web servers, by just providing an address (the URL).

At a given time, many clients can access a server’s information. This puts the burden of scaling on the servers as they need to be designed with the ability to respond to multiple requests within an acceptable period of time. Now that we understand a web server’s primary goal, let’s move on to the protocol that enables communication between web servers: HTTP.

Understanding HTTP

HTTP is an application layer protocol that provides communication standards between clients (such as web browsers) and web servers. This standardization helps browsers and servers talk to each other as long as the request and the response follow a specific format.

An HTTP request is a text document with four elements:

  • Request line: This line contains the HTTP method, the resource requested (URI), and the HTTP version being used for the request. The HTTP method generally symbolizes the intended action being performed on the requested resource. For example, GET is used to retrieve resource information, whereas POST is used to send new resource information as a form.
  • Request headers: The next group of lines contains headers, which contain information about the request and the client. These headers are usually used for authorization, determining the type of request or resource, storing web browser information, and so on.
  • Line break: This indicates the end of the request headers.
  • Request body (optional): If present, this information contains data to be passed to the server. This is generally used to submit domain-specific forms.

Here’s an example of an HTTP request document:

GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.75.0
Accept: */*
Body of the request

As you can see, the preceding request was made with the GET method to localhost:8080 with the body, Body of the request.

Similarly, an HTTP response contains four elements:

  • Response status line: This line consists of the HTTP version, a status code, and a reason phrase, which generally corresponds to the status code. Status codes are three-digit integers that help us categorize responses. For example, 2XX status codes are used for a successful response, whereas 4XX status codes are used for errors due to the request.
  • Response headers: Just like request headers, this group of lines contains information about the response and the server’s state. These headers are usually used to show the size of the response body, server type, date/time of the response, and so on.
  • Line break: This indicates the end of response headers.
  • Response body (optional): If present, this section contains the information being relayed to the client.

The following is an example of an HTTP response document:

HTTP/1.1 404 Not Found
content-length: 13
content-type: text/html
server: Cowboy
404 Not found

The preceding response is an example of a 404 (Not found) response. Notice that content-length shows the number of characters present in the response body.

Now that we know how HTTP facilitates client-server communication, it is time to build a web server using Cowboy.

Understanding Cowboy’s architecture

Cowboy is a minimal and fast HTTP web server written in Erlang. It supports several modern standards, such as HTTP/2, HTTP/1.1, and WebSocket, for example. On top of that, it also has several introspective capabilities, thus enabling easier development and debugging. Cowboy has a very well-written and well-documented code base, with a highly extendable design, which is why it is the default web server for the Phoenix framework.

Cowboy uses Ranch, a TCP socket accepter, to create a new TCP connection, on top of which it uses its router to match a request to a handler. Routers and handlers are middleware that are part of Cowboy. Upon receiving a request, Cowboy creates a stream, which is further handled by a stream handler. Cowboy has a built-in configuration that handles a stream of requests using :cowboy_stream_h. This module spins up a new Erlang process for every request that is made to the router.

Cowboy also sets up one process per TCP connection. This also allows Cowboy to be compliant with HTTP/2, which requires concurrent requests. Once a request is served, the Erlang process is killed without any need for cleanup.

The following figure shows the Cowboy request/response cycle:

Figure 1.1 – Cowboy request/response cycle

As you can see in Figure 1.1, when a client makes a request, Ranch first converts it into a stream, which is further handled by the router and handler middleware in Cowboy. Traditionally, a response is sent either by the router or the handler. For example, a handler could handle a request and send a response or, if no handler is present for a route, the router could also send a 404 response.

Cowboy also generates a few response headers, as we will see in the next section, where we build and test a Cowboy-powered web application.

Building a web application using Cowboy

In this section, we will take a look at some of the individual components of the Cowboy web server and use them to build a simple web application in Elixir.

Creating a new Mix project

Let’s start by creating a new Mix project by entering the following in your terminal:

$ mix new cowboy_example --sup

What is Mix?

Mix is a build tool written in Elixir. Its purpose is to bundle all the dependencies required by a project and provide an interface to run tasks that rely on the application environment. If you’re familiar with the Ruby world, you can think of Mix as a combination of Rake and Bundler.

Passing the --sup option to the mix new command allows us to create a Mix project with a supervision tree. A supervision tree (or a supervisor) is a process that simply monitors other processes and is responsible for automatically restarting any process within the tree if it fails or crashes. We will be using the supervision tree in this application to start and supervise our web server process to make sure it is started when the application is started and to ensure that it keeps running.

Now, we will add Cowboy as a dependency to our project by adding it to the mix.exs file’s dependencies:

mix.exs

defmodule CowboyExample.MixProject do
  # ...
  defp deps do
    [
      {:cowboy, "~> 2.8"}
    ]
  end
end

Follow it up by running mix deps.get from the project’s root directory, which should fetch Cowboy as a dependency.

Adding a handler and router to Cowboy

Now that we have added Cowboy, it is time to configure it to listen on a port. We will be using two functions to accomplish that:

  • :cowboy_router.compile/1: This function is responsible for defining a set of requested hosts, paths, and parameters to a dedicated request handler. This function also generates a set of rules, known as dispatch rules, to use those handlers.
  • :cowboy.start_clear/3: This function is responsible for starting a listener process on a TCP channel. It takes a listener name (an atom), transport options such as the TCP port, and protocol options such as the dispatch rules generated using the :cowboy_router.compile/1 function.

Now, let us use these functions to write an HTTP server. We can start by creating a new module to house our new HTTP server:

lib/cowboy_example/server.ex

defmodule CowboyExample.Server do
  @moduledoc """
  This module defines a cowboy HTTP server and starts it
  on a port
  """
  @doc """
  This function starts a Cowboy server on the given port.
  Routes for the server are defined in CowboyExample.Router
  """
  def start(port) do
    routes = CowboyExample.Router.routes()
    dispatch_rules =
      :cowboy_router.compile(routes)
    {:ok, _pid} =
      :cowboy.start_clear(
        :listener,
        [{:port, port}],
        %{env: %{dispatch: dispatch_rules}}
      )
  end
end

The preceding function starts a Cowboy server that listens to HTTP requests at the given port. By pattern matching on {:ok, _pid}, we’re making sure that :cowboy.start_clear/3 doesn’t fail silently.

We can try to start our web server by calling the CowboyExample.Server.start/1 function with a port. But, as you can see, we will also need to define the CowboyExample.Router router module. This module’s responsibility is to define routes that can be used to generate dispatch rules for our HTTP server. This can be done by storing all the routes, parameters, and handler tuples in the router module and passing them to the :cowboy_router.compile/1 call.

Let’s define the router module with the route for the root URL of the host (/):

lib/cowboy_example/router.ex

defmodule CowboyExample.Router do
  @moduledoc """
  This module defines all the routes, params and handlers.
  This module is also the handler module for the root
  route.
  """
  require Logger
  @doc """
  Returns the list of routes configured by this web server
  """
  def routes do
    [
      # For now, this module itself will handle root
      # requests
      {:_, [{"/", __MODULE__, []}]}
    ]
  end
end

We will also be using CowboyExample.Router itself as the handler for that route, which means we have to define the init/2 function, which takes the request and its initial state.

So, let us define the init/2 function:

lib/cowboy_example/router.ex

defmodule CowboyExample.Router do
  # ..
  @doc """
  This function handles the root route, logs the requests
  and responds with Hello World as the body
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "Hello World",
        req0
      )
    {:ok, req1, state}
  end
end

As you can see in the preceding code, we have defined the routes/0 function, which returns the dispatch rules routes for our web application. For the handler module, we’re currently using the CowboyExample.Router module itself by defining the init/2 function, which responds with "Hello World" and a status of 200, whenever invoked. We have also added a call to the Logger module to log all requests to the handler. This will increase visibility while running it in the development environment.

In order for our web server to start up when the app is started, we need to add it to our application’s supervision tree.

Supervising the web server

Now that we have added a router and a handler to our web server, we can add it as a child to our supervision tree by updating the list of children in our application module. For now, I will use a hardcoded TCP port of 4040 for our server, but we will use application-level configurations to set it later in this chapter:

lib/cowboy_example/application.ex

defmodule CowboyExample.Application do
  @moduledoc false
  use Application
  @impl true
  def start(_type, _args) do
    children = [
      # Add this line
      {Task, fn -> CowboyExample.Server.start(4040) end}
    ]
    opts = [
      strategy: :one_on_one,
      name: CowboyExample.Supervisor
    ]
    Supervisor.start_link(children, opts)
  end
end

In the preceding code, we’re adding to the supervised children a Task with the function to start the Cowboy listener as an argument that eventually gets passed to Task.start_link/1. This makes sure that our web server process is part of the application’s supervision tree.

Now, we can run our web application by running the mix project with the --no-halt option:

$ mix run --no-halt

Note

Passing the --no-halt option to the mix run command makes sure that the application, along with the supervision tree, is still running even after the command has returned. This is generally used for long-running processes such as web servers.

Without stopping the previous command, in a separate terminal session, we can make a request to our web server using the cURL command-line utility with the –v option to get a verbose description of our requests and responses:

$ curl –v http://localhost:4040/
*   Trying ::1:4040...
* connect to ::1 port 4040 failed: Connection refused
*   Trying 127.0.0.1:4040...
* Connected to localhost (127.0.0.1) port 4040 (#0)
> GET / HTTP/1.1
> Host: localhost:4040
> User-Agent: curl/7.75.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 11
< content-type: text/html
< server: Cowboy
<
* Connection #0 to host localhost left intact
Hello world%

As we can see in the preceding code, we get the expected "Hello World" response along with the expected status code of 200. As mentioned in the previous section, Cowboy adds custom response headers to give us more information about how it was processed. We can also see headers for the type of server (Cowboy), content length, and content type.

We should also see an application-level log corresponding to the request in the terminal session running the mix project. The logs should look somewhat like this:

$ mix run --no-halt
20:39:43.061 [info]  Received request: %{
  bindings: %{},
  body_length: 0,
  cert: :undefined,
  has_body: false,
  headers: %{
    "accept" => "*/*",
    "host" => "localhost:4040",
    "user-agent" => "curl/7.75.0"
  },
  host: "localhost",
  host_info: :undefined,
  method: "GET",
  path: "/",
  path_info: :undefined,
  peer: {{127, 0, 0, 1}, 35260},
  pid: #PID<0.271.0>,
  port: 4040,
  qs: "",
  ref: :listener,
  scheme: "http",
  sock: {{127, 0, 0, 1}, 4040},
  streamid: 1,
  version: :"HTTP/1.1"
}

We can see that we’re logging all the details corresponding to the request including headers, the host, the URI, and the process ID of the process processing the request.

Congratulations, you have now successfully built a Hello World web server using Cowboy. Now, it’s time to add more routes to our web server.

Adding routes with bindings

Most web applications support the ability to serve not only a static route but also dynamic routes with a specific pattern. It’s time to see how we can leverage Cowboy to add dynamic routes to our router.

Say we want to add a new route to our application that responds with a custom greeting for a person whose name is dynamic. Let’s update our router to define a handler for a new dynamic route. We can also use this opportunity to move our Root handler (the init/2 function) to a different module. This makes our code more compliant with the single-responsibility principle, making it easier to follow:

lib/cowboy_example/router.exdefmodule

CowboyExample.Router do
  @moduledoc """
  This module defines all the routes, params and handlers.
  """
  alias CowboyExample.Router.Handlers.{Root, Greet}
  @doc """
  Returns the list of routes configured by this web server
  """
  def routes do
    [
      {:_, [
        {"/", Root, []},
        # Add this line
        {"/greet/:who", [who: :nonempty], Greet, []}
      ]}
    ]
  end
end

In the preceding code, we have added a new route that expects a non-empty value for the :who variable binding. This variable gets bound to a request based on the URL. For example, for a request with the URL "/greet/Luffy", the variable bound to :who will be "Luffy", and for a request with the URL "/greet/Zoro", it will be "Zoro".

Now, let’s define the Root handler and move the init/2 function from our router to the new handler module. This separates the concerns of defining routes and handling requests:

lib/cowboy_example/router/handlers/root.ex

defmodule CowboyExample.Router.Handlers.Root do
  @moduledoc """
  This module defines the handler for the root route.
  """
  require Logger
  @doc """
  This function handles the root route, logs the requests
  and responds with Hello World as the body
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "Hello World",
        req0
      )
    {:ok, req1, state}
  end
end

Similarly, let’s define the Greet handler for our new dynamic route. We know that the request has a variable binding corresponding to the:who key by the time it gets to this handler. Therefore, we can use the :cowboy_req.binding/2 function to access the value of :who bound to the request:

lib/cowboy_example/router/handlers/greet.ex

defmodule CowboyExample.Router.Handlers.Greet do
  @moduledoc """
  This module defines the handler for "/greet/:who" route.
  """
  require Logger
  @doc """
  This function handles the "/greet/:who", logs the
  requests and responds with Hello `who` as the body
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    who = :cowboy_req.binding(:who, req0)
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "Hello #{who}",
        req0
      )
    {:ok, req1, state}
  end
end

In the preceding code snippet, we get the value bound to :who for the request and use it with string interpolation to call "Hello :who". Now, we have two valid routes for our web server: the root and the dynamic greet route.

We can test our updates by restarting the Mix application. That can be done by stopping the HTTP server using Ctrl + C, followed by running mix run --no-halt again. Now, let’s make a request to test the new route with Elixir as :who:

$ curl http://localhost:4040/greet/Elixir
Hello Elixir%

Cowboy offers another way to add dynamic behavior to our routes, and that is by passing query parameters to our URL. Query parameters can be captured by using the :cowboy_req.parse_qs/1 function. This function takes a binding name (:who in this case) and the request itself. Let’s update our greet handler to now take a custom query parameter for greeting that overrides the default "Hello" greeting, which we can put in a module attribute for better code organization:

lib/cowboy_example/router/handlers/greet.ex

defmodule CowboyExample.Router.Handlers.Greet do
  # ..
  @default_greeting "Hello"
  # ..
  def init(req0, state) do
    greeting =
    # ..
      req0
      |> :cowboy_req.parse_qs()
      |> Enum.into(%{})
      |> Map.get("greeting", @default_greeting)
    req1 =
      :cowboy_req.reply(
        200,
        %{"content-type" => "text/html"},
        "#{greeting} #{who}",
        req0
      )
    {:ok, req1, state}
  end
end

We have now updated our greet handler to use :cowboy.parse_qs/1 to fetch query parameters from the request. We then put those matched parameters into a map and get the value in the map corresponding to the "greeting" key, with a default of "Hello". Now, the greet route should take a “greeting” query parameter to update the greeting used to greet someone in the response. We can test our updates again by restarting the application and making a request to test the route with a custom greeting parameter:

$ curl http://localhost:4040/greet/Elixir\?greeting=Hola
Hola Elixir%

We now have a web server with a fully functional dynamic route. You might have noticed that we didn’t specify any HTTP method while defining the routes. Let us see what happens when we try to make a request to the root with the POST method:

$ curl -X POST http://localhost:4040/
Hello World%

As you can see in the example, our web server still responded to the POST request in the same manner as GET. We don’t want that behavior so, in the next section, we will see how to validate the HTTP method of a request and restrict the root of our application to only respond to GET requests.

Validating HTTP methods

Most modern web applications have a way of restricting requests to a route based on the HTTP method. In this section, we will see how to restrict our handlers to work with a specific HTTP method in a Cowboy-based web application. The simplest way of accomplishing that in a Cowboy handler is by pattern matching on the first argument of the init/2 function, which is the request. A Cowboy request contains a lot of information, including the HTTP method used to make the request. So, by pattern matching on the request with a specific HTTP method, we can filter requests based on HTTP methods. However, we will also be needing a general clause for the init/2 function, which responds with a 404 error.

In the Greet handler, let us update init/2 to match only on requests with the GET method, along with another clause that responds with a 404 (Not Found) error:

lib/cowboy_example/router/handlers/greet.ex

defmodule CowboyExample.Router.Handlers.Greet do
  # ..
  def init(%{method: "GET"} = req0, state) do
  # ..
  end
  # General clause for init/2 which responds with 404
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    req1 =
      :cowboy_req.reply(
        404,
        %{"content-type" => "text/html"},
        "404 Not found",
        req0
      )
    {:ok, req1, state}
  end
end

Now, let’s make sure only GET requests are accepted by our server for the route. Let’s first make sure GET requests are working:

$ curl http://localhost:4040/greet/Elixir\?greeting=Hola
Hola Elixir%

It’s time to check that a POST request for the greet route returns a 404 error:

$ curl http://localhost:4040/greet/Elixir\?greeting=Hola -X POST
404 Not found%

This ensures that our route works only for GET requests. Another way of validating HTTP methods of our requests would be by using Cowboy middleware, but we will not be covering that in this chapter.

Cowboy middleware

In Cowboy, middleware is a way to process an incoming request. Every request has to go through two types of middleware (the router and the handler), but Cowboy allows us to define our own custom middleware module, which gets executed in the given order. A custom middleware module just needs to implement the execute/2 callback defined in the cowboy_middleware behavior.

Great! We have successfully enforced a method type for a route. Next, we will learn how to serve HTML files instead of raw strings.

Responding with HTML files

Generally, when we write web servers, we do not write our HTML as strings in handlers. We write our HTML in separate files that are served by our server. We will use our application’s priv directory to store these static files. So, let’s create a priv/static folder in the root of our project and add an index.html file in that folder. To add some HTML, we can use this command:

$ echo "<h1>Hello World</h1>" > priv/static/index.html

The priv directory in OTP

In OTP (Open Telecom Platform or Erlang) and Elixir, the priv directory is a directory specific to an application that is intended to store files needed by the application when it is running. Phoenix, for example, uses the priv/static directory to store processed JavaScript and CSS assets for runtime usage.

Let’s add an endpoint to our server that returns a static HTML file:

lib/cowboy_example/router.ex

defmodule CowboyExample.Router do
  @moduledoc """
  This module defines routes and handlers for the web
  server
  """
  alias CowboyExample.Router.Handlers.{Root, Greet, Static}
  @doc """
  Returns the list of routes configured by this web server
  """
   def routes do
    [
      {:_, [
        {"/", Root, []},
        {"/greet/:who", [who: :nonempty], Greet, []},
        # Add this line
        {"/static/:page", [page: :nonempty], Static, []}
      ]}
    ]
  end
end

Now, we need a static handler module, which will look for and respond with the given page in the /priv/static folder and, if not found, will return a 404 error:

lib/cowboy_example/router/handlers/static.ex

defmodule CowboyExample.Router.Handlers.Static do
  @moduledoc """
  This module defines the handler for "/static/:page"
  route.
  """
  require Logger
  @doc """
  This handles "/static/:page" route, logs the requests and
  responds with the requested static HTML page.
  Responds with 404 if the page isn't found in the
  priv/static folder.
  """
  def init(req0, state) do
    Logger.info("Received request: #{inspect req0}")
    page = :cowboy_req.binding(:page, req0)
    req1 =
      case html_for(page) do
        {:ok, static_html} ->
          :cowboy_req.reply(
            200,
            %{"content-type" => "text/html"},
            static_html,
            req0
          )
        _ ->
          :cowboy_req.reply(
            404,
            %{"content-type" => "text/html"},
            "404 Not found",
            req0
          )
             end
    {:ok, req1, state}
  end
  defp html_for(page) do
    priv_dir =
      :cowboy_example
      |> :code.priv_dir()
      |> to_string()
    page_path = priv_dir <> "/static/#{page}"
    File.read(page_path)
  end
end

In the preceding module, the html_for/1 function is responsible for fetching the HTML files from our application’s priv directory, for a given path. If the file is present, the function returns {:ok, <file_contents>}, >}; otherwise, it returns an error, upon which we will respond with a 404 message.

We can test the preceding route by restarting our server again and making a request to the /static/index.html path. But this time, let us use the web browser in order to render the HTML contents properly. Here’s what you should see:

Figure 1.2 – Successful HTML response

Also, to make sure our 404 handler is working correctly, let’s make a browser request to /static/bad.html, a file not present in our application’s priv directory. You should see a 404 message:

Figure 1.3 – Failed HTML response

Now, we have a web server that can respond with static HTML files. It’s time to see how we can go about testing it.

Testing the web server with ExUnit

Automated testing is a key part of any software, especially in a dynamic-typed language such as Elixir. It is one of the catalysts for writing deterministic software while documenting the expected behaviors of its components. Due to this reason, we will be making an effort to test everything we build in this book, including the Cowboy-powered web application we have built in this chapter.

In order to test our web application, we first need to be able to run our application on a different port in the test environment. This is to ensure that other /static/bad.html environments do not interfere with our tests. We also can use an application-level configuration to set a port on which the Cowboy server listens to all the requests. This will allow us to separate the test port from the development port.

So, let’s update our application to use the configured port or default it to 4040 using an @port module attribute:

lib/cowboy_example/application.ex

defmodule CowboyExample.Application do
  @moduledoc false
  use Application
  @port Application.compile_env(
          :cowboy_example,
          :port,
          4040
        )
  @impl true
  def start(_type, _args) do
    children = [
      # Add this line
      {Task, fn -> CowboyExample.Server.start(@port) end}
    ]
    opts = [
      strategy: :one_for_one,
      name: CowboyExample.Supervisor
    ]
    Supervisor.start_link(children, opts)
  end
end

We can make sure that the application configuration is different for different Mix environments by adding the config/config.exs file, and setting a different port in our config for the test environment. We will also be updating the logger to not log warnings. So, let’s add a config file with the following contents:

config/config.exs

import Config
if Mix.env() == :test do
  config :cowboy_example,
    port: 4041
  config :logger, warn: false
end

Note

Mix.Config has been deprecated in newer versions of Elixir. You might have to use the Config module instead.

Now, let’s add tests for our server endpoints. In order to test our web server, we need to make HTTP requests to it and test the responses. To make HTTP requests in Elixir, we will be using Finch, a lightweight and high-performance HTTP client written in Elixir.

So, let’s add Finch to our list of dependencies:

mix.exs

defmodule CowboyExample.MixProject do
  # ...
  defp deps do
    [
      {:cowboy, "~> 2.8"},
      {:finch, "~> 0.6"}
    ]
  end
end

Running mix deps.get will fetch Finch and all its dependencies.

Now, let’s add a test file to test our server. In the test file, we will be setting up Finch to make HTTP calls to our server. In this section, we will only be testing the happy paths (200 responses) of our root and greet endpoints:

test/cowboy_example/server_test.exs

defmodule CowboyExample.ServerTest do
  use ExUnit.Case
  setup_all do
    Finch.start_link(name: CowboyExample.Finch)
    :ok
  end
  describe "GET /" do
    test "returns Hello World with 200" do
      {:ok, response} =
        :get
        |> Finch.build("http://localhost:4041")
        |> Finch.request(CowboyExample.Finch)
      assert response.body == "Hello World"
      assert response.status == 200
      assert {"content-type", "text/html"} in response.headers
    end
  end
  describe "GET /greeting/:who" do
    test "returns Hello `:who` with 200" do
      {:ok, response} =
        :get
        |> Finch.build("http://localhost:4041/greet/Elixir")
        |> Finch.request(CowboyExample.Finch)
      assert response.body == "Hello Elixir"
      assert response.status == 200
      assert {"content-type", "text/html"} in response.headers
    end
    test "returns `greeting` `:who` with 200" do
      {:ok, response} =
        :get
        |> Finch.build("http://localhost:4041/greet/
                        Elixir?greeting=Hola")
        |> Finch.request(CowboyExample.Finch)
      assert response.body == "Hola Elixir"
      assert response.status == 200
      assert {"content-type", "text/html"} in response.headers
    end
  end
end

As you can see in the preceding module, we have added tests for the two endpoints using Finch. We make calls to our server, running on port 4041 in the test environment, with different request paths and parameters. We then test the response’s body, status, and headers.

This should give you a good idea of how to go about testing a web server. Over the next few chapters, we will be building on top of this foundation and coming up with better ways of testing our web server.

Summary

When I first learned about how a web server works, I was overwhelmed with the number of things that go into building one. Then I decided to look at the code of Puma, a web server written in Ruby, which is also used by Rails. I was surprised by how much more I learned by just looking into Puma than by reading articles about web servers. It is due to that reason that we are kicking off this book by looking at Cowboy. I believe that learning about the basics of Cowboy will better position us to build our own web server in the next few chapters.

In this chapter, we first learned the basics of a web server along with the client-server architecture. We also looked at the high-level architecture of Cowboy and learned about how some of its components such as the router and handlers work. We also added dynamic behavior to our routes by using path variables and query parameters, followed by serving static HTML files. We finished by learning how to test our routes using an HTTP client. In the next chapter, we will use what we learned in this chapter to build our own HTTP server from scratch.

Exercises

Some of you might have noticed that we haven’t tested a few aspects of our web server. Using what you have learned in this chapter, complete these exercises:

  • Write test cases for our web server that would lead to 404 responses
  • Write tests for the static route that respond with HTML files

There are other (better) ways of testing an HTML response, which we haven’t covered in this chapter. We will dig deeper into those testing methods later in this book.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Explore the various web servers available to build robust web applications
  • Leverage Elixir's powerful Plug module to build a request-response pipeline
  • Explore advanced techniques of Elixir to create Domain-Specific languages and build scalable, maintainable web products

Description

Elixir's functional nature and metaprogramming capabilities make it an ideal language for building web frameworks, with Phoenix being the most ubiquitous framework in the Elixir ecosystem and a popular choice for companies seeking scalable web-based products. With an ever-increasing demand for Elixir engineers, developers can accelerate their careers by learning Elixir and the Phoenix web framework. With Build Your Own Web Framework in Elixir, you’ll start by exploring the fundamental concepts of web development using Elixir. You'll learn how to build a robust web server and create a router to direct incoming requests to the correct controller. Then, you'll learn to dispatch requests to controllers to respond with clean, semantic HTML, and explore the power of Domain-Specific Languages (DSL) and metaprogramming in Elixir. You'll develop a deep understanding of Elixir's unique syntax and semantics, allowing you to optimize your code for performance and maintainability. Finally, you'll discover how to effectively test each component of your application for accuracy and performance. By the end of this book, you'll have a thorough understanding of how Elixir components are implemented within Phoenix, and how to leverage its powerful features to build robust web applications.

Who is this book for?

This book is for web developers seeking to deepen their understanding of Elixir's role in the Phoenix framework. Basic familiarity with Elixir is a must.

What you will learn

  • Develop a deep understanding of Phoenix
  • Use metaprogramming with Elixir
  • Get to grips with web development fundamentals
  • Dispatch a request to a controller and respond using HTML embedded with Elixir
  • Use OTP constructs to add concurrency to the web server
  • Streamline your router using Error Handler

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jun 16, 2023
Length: 274 pages
Edition : 1st
Language : English
ISBN-13 : 9781801815901
Languages :
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Jun 16, 2023
Length: 274 pages
Edition : 1st
Language : English
ISBN-13 : 9781801815901
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 130.97
Machine Learning Engineering with MLflow
$43.99
Build Your Own Web Framework in Elixir
$36.99
Python Deep Learning
$49.99
Total $ 130.97 Stars icon
Banner background image

Table of Contents

14 Chapters
Part 1: Web Server Fundamentals Chevron down icon Chevron up icon
Chapter 1: Introducing the Cowboy Web Server Chevron down icon Chevron up icon
Chapter 2: Building an HTTP Server in Elixir Chevron down icon Chevron up icon
Part 2: Router, Controller, and View Chevron down icon Chevron up icon
Chapter 3: Defining Web Application Specifications Using Plug Chevron down icon Chevron up icon
Chapter 4: Working with Controllers Chevron down icon Chevron up icon
Chapter 5: Adding Controller Plugs and Action Fallback Chevron down icon Chevron up icon
Chapter 6: Working with HTML and Embedded Elixir Chevron down icon Chevron up icon
Chapter 7: Working with Views Chevron down icon Chevron up icon
Part 3: DSL Design Chevron down icon Chevron up icon
Chapter 8: Metaprogramming – Code That Writes Code Chevron down icon Chevron up icon
Chapter 9: Controller and View DSL Chevron down icon Chevron up icon
Chapter 10: Building the Router DSL Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.9
(8 Ratings)
5 star 87.5%
4 star 12.5%
3 star 0%
2 star 0%
1 star 0%
Filter icon Filter
Top Reviews

Filter reviews by




Reader Jun 18, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
The author starts with a review of how a web server is built, followed by controllers, routers and serving HTML. From there the book gets into metaprogramming and wraps all the functionality into a nice readable interface. All of this is done while learning how to test everything. Highly recommended for anyone looking to gain deeper insight on how Phoenix works!
Amazon Verified review Amazon
Joel Byler Jun 16, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I’m enjoying this book. It’s a great alternative to just diving straight into phoenix and helps to give more background into what’s happening under the hood in an elixir web application.
Amazon Verified review Amazon
cs44 Jun 20, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I often read tech books to learn a new skill or rehash an older subject, and generally I do enjoy them. However, I was grinning ear to ear with this one.As a seasoned Elixir engineer, primary focused on web technologies, I have a comprehensive understanding of the underlying machinery of, say, Phoenix. Though I often want to peek under the hood, and that’s what this book does.Aditya expertly guides you through the foundation and frameworks powering our web applications from scratch. It’s an excellent vantage point.Aside from that, his elegant style is something I’ll no doubt borrow in my own code.If you’re an Elixir dev searching for a new book, after having read them all, look no further.
Amazon Verified review Amazon
Amazon Customer Jul 19, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
The book is well written. Explains the concepts in a concise and easy-to-understand way. It is easy to read and follow. All of the code fragments I tested compiled successfully and worked correctly. It is a great in-depth introduction to the Elixir Phoenix framework and what's under the hood.
Amazon Verified review Amazon
Patrick Jun 16, 2023
Full star icon Full star icon Full star icon Full star icon Full star icon 5
The book starts off I received early access to this programming book. The book starts with building with Cowboy then leads you into the details of writing a web server from scratch. This step-by-step approach ensures even beginners can follow along without feeling overwhelmed.One standout feature of this book is its emphasis on testing. It not only teaches you how to create a web server but also demonstrates how to add tests for it. This focus on testing practices instills good coding habits and prepares you for real-world scenarios. Furthermore, the book acts as a great stepping stone toward understanding big libraries like Phoenix. By exploring the internals of a web server, you gain valuable insights into the underlying principles that power larger frameworks.The writing style is engaging and approachable, making it enjoyable for readers new to programming. Code examples fill the book. In conclusion, this programming book is a gem for anyone interested in web development. I recommend it to aspiring developers looking to solidify their understanding and take their skills to the next level.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.