Web development has had an interesting history, to say the least. Several (failed) attempts have been made to expand the platform to support different languages. Clunky solutions such as plugins failed to stand the test of time, and limiting a user to a single browser is a recipe for disaster.

WebAssembly was developed as an elegant solution to a problem that has existed since browsers were first able to execute code: If you want to develop for the web, you have to use JavaScript. Fortunately, using JavaScript doesn't have the same negative connotations it had back in the early 2000s, but it continues to have certain limitations as a programming language. In this section, we're going to discuss the technologies that led to WebAssembly to get a better grasp of why this new technology is needed.

JavaScript was created by Brendan Eich in just 10 days back in 1995. Originally seen as a toy language by programmers, it was used primarily to make buttons flash or banners appear on a web page. The last decade has seen JavaScript evolve from a toy to a platform with profound capabilities and a massive following.

In 2008 heavy competition in the browser market resulted in the addition of just-in-time (JIT) compilers, which increased the execution speed of JavaScript by a factor of 10. Node.js debuted in 2009 and represented a paradigm shift in web development. Ryan Dahl combined Google's V8 JavaScript engine, an event loop, and a low-level I/O API to build a platform that allowed for the use of JavaScript across the server and client side. Node.js led to npm, a package manager that allowed for the development of libraries to be used within the Node.js ecosystem. As of the time of writing, there are over 600,000 packages available with hundreds being added every day:

It's not just the Node.js ecosystem that is growing; JavaScript itself is being actively developed. The ECMA Technical Committee 39 (TC39), which dictates the standards for JavaScript and oversees the addition of new language features, releases yearly updates to JavaScript with a community-driven proposal process. Between its wealth of libraries and tooling, constant improvements to the language, and possessing one of the largest communities of programmers, JavaScript has become a force to be reckoned with.

But the language does have some shortcomings:

• Up until recently, JavaScript only included 64-bit floating point numbers. This can cause issues with very large or very small numbers. BigInt, a new numeric primitive that can alleviate some of these issues, is in the the process of being added to the ECMAScript specification, but it may take some time until it's fully supported in browsers.
• JavaScript is weakly typed, which adds to its flexibility, but can cause confusion and bugs. It essentially gives you enough rope to hang yourself.
• JavaScript isn't as performant as compiled languages despite the best efforts of the browser vendors.
• If a developer wants to create a web application, they need to learn JavaScript—whether they like it or not.

To avoid having to write more than a few lines of JavaScript, some developers built transpilers to convert other languages to JavaScript. Transpilers (or source-to-source compilers) are types of compilers that convert source code in one programming language to equivalent source code in another programming language. TypeScript, which is a popular tool for frontend JavaScript development, transpiles TypeScript to valid JavaScript targeted for browsers or Node.js. Pick any programming language and there's a good chance that someone created a JavaScript transpiler for it. For example, if you prefer to write Python, you have about 15 different tools that you can use to generate JavaScript. In the end, though, it's still JavaScript, so you're still subject to the idiosyncrasies of the language.

As the web evolved into a valid platform for building and distributing applications, more and more complex and resource-intensive applications were created. In order to meet the demands of these applications, browser vendors began working on new technologies to integrate into their software without disrupting the normal course of web development. Google and Mozilla, creators of Chrome and Firefox, respectively, took two different paths to achieve this goal, culminating in the creation of WebAssembly.

Google developed Native Client (NaCl) with the intent to safely run native code within a web browser. The executable code would run in a sandbox and offered the performance advantages of native code execution.

NaCl was tied to a specific architecture, while Portable Native Client (PNaCl) was an architecture-independent version of NaCl developed to run on any platform. The technology consisted of two elements:

• Toolchains which could transform C/C++ code to NaCl modules
• Runtime components which were components embedded in the browser that allowed execution of NaCl modules:

NaCl's architecture-specific executable (nexe) was limited to applications and extensions that were installed from Google's Chrome Web Store, but PNaCl executables (pexe) can be freely distributed on the web and embedded in web applications. Portability was made possible with Pepper, an open source API for creating NaCl modules, and its corresponding plugin API (PPAPI). Pepper enabled communication between NaCl modules and the hosting browser, and allowed for access to system-level functions in a safe and portable way. Applications could be easily distributed by including a manifest file and a compiled module (pexe) with the corresponding HTML, CSS, and JavaScript:

NaCl offered promising opportunities to overcome the performance limitations of the web, but it had some drawbacks. Although Chrome had built-in support for PNaCl executables and Pepper, other major browser did not. Detractors of the technology took issue with the black-box nature of the applications as well as the potential security risks and complexity.

Mozilla focused its efforts on improving the performance of JavaScript with asm.js. They wouldn't add support for Pepper to Firefox due to the incompleteness of its API specification and limited documentation. In the end, NaCl was deprecated in May, 2017, in favor of WebAssembly.

Mozilla debuted asm.js in 2013 and provided a way for developers to translate their C and C++ source code to JavaScript. The official specification for asm.js defines it as a strict subset of JavaScript that can be used as a low-level, efficient target language for compilers. It's still valid JavaScript, but the language features are limited to those that are amenable to ahead-of-time (AOT) optimization. AOT is a technique that the browser's JavaScript engine uses to execute code more efficiently by compiling it down to native machine code. asm.js achieves these performance gains by having 100% type consistency and manual memory management.

Using a tool such as Emscripten, C/C++ code can be transpiled down to asm.js and easily distributed using the same means as normal JavaScript. Accessing the functions in an asm.js module requires linking, which involves calling its function to obtain an object with the module's exports.

asm.js is incredibly flexible, however, certain interactions with the module can cause a loss of performance. For example, if an asm.js module is given access to a custom JavaScript function that fails dynamic or static validation, the code can't take advantage of AOT and falls back to the interpreter:

The World Wide Web Consortium (W3C), an international community built to develop web standards, formed the WebAssembly Working Group in April, 2015, to standardize WebAssembly and oversee the specification and proposal process. Since then, the Core Specification and corresponding JavaScript API and Web API have been released. The initial implementation of WebAssembly support in browsers was based on the feature set of asm.js. WebAssembly's binary format and corresponding .wasm file combined facets of asm.js output with PNaCl's concept of a distributed executable.

So how will WebAssembly succeed where NaCl failed? According to Dr. Axel Rauschmayer, there are three reasons detailed at http://2ality.com/2015/06/web-assembly.html#what-is-different-this-time:

WebAssembly has a succinct and descriptive definition on the official site, but it's only a piece of the puzzle. There are several other components that fall under the umbrella of WebAssembly. Understanding the role each component plays will give you a better understanding of the technology as a whole. In this section, we will provide a detailed breakdown of WebAssembly's definition and describe potential use cases.

The official WebAssembly website (https://webassembly.org) offers this definition:

Let's break that definition down into parts to add some clarification.

WebAssembly actually encompasses several elements—a binary format and text format, which are documented in the Core Specification, the corresponding APIs (JavaScript and web), and a compilation target. The binary and text format both map to a common structure in the form of an abstract syntax. To better understand abstract syntax, it can be explained in the context of an abstract syntax tree (AST). An AST is a tree representation of the structure of source code for a programming language. Tools such as ESLint use JavaScript's AST to find linting errors. The following example contains a function and the corresponding AST for JavaScript (taken from https://astexplorer.net).

A simple JavaScript function follows:

function doStuff(thingToDo) {
  console.log(thingToDo);
}

The corresponding AST is as follows:

{
  "type": "Program",
  "start": 0,
  "end": 57,
  "body": [
    {
      "type": "FunctionDeclaration",
      "start": 9,
      "end": 16,
      "id": {
        "type": "Identifier",
        "start": 17,
        "end": 26,
        "name": "doStuff"
      },
      "generator": false,
      "expression": false,
      "params": [
        {
          "type": "Identifier",
          "start": 28,
          "end": 57,
          "name": "thingToDo"
        }
      ],
      "body": {
        "type": "BlockStatement",
        "start": 32,
        "end": 55,
        "body": [
          {
            "type": "ExpressionStatement",
            "start": 32,
            "end": 55,
            "expression": {
              "type": "CallExpression",
              "start": 32,
              "end": 54,
              "callee": {
                "type": "MemberExpression",
                "start": 32,
                "end": 43,
                "object": {
                  "type": "Identifier",
                  "start": 32,
                  "end": 39,
                  "name": "console"
                },
                "property": {
                  "type": "Identifier",
                  "start": 40,
                  "end": 43,
                  "name": "log"
                },
                "computed": false
              },
              "arguments": [
                {
                  "type": "Identifier",
                  "start": 44,
                  "end": 53,
                  "name": "thingToDo"
                }
              ]
            }
          }
        ]
      }
    }
  ],
  "sourceType": "module"
}

An AST may be verbose, but it does an excellent job at describing the components of a program. Representing source code in an AST makes verification and compilation simple and efficient. WebAssembly code in text format is serialized into an AST and compiled to the binary format (as a .wasm file), which is fetched, loaded, and utilized by a web page. When the module is loaded, the browser's JavaScript engine utilizes a decoding stack to decode the .wasm file into an AST, perform type checking, and interpret it to execute functions. WebAssembly started as a binary instruction format for an AST. Due to the performance implications of verifying Wasm expressions that return void, the binary instruction format was updated to target a stack machine.

A stack machine consists of two elements: a stack and instructions. A stack is a data structure with two operations: push and pop. Items are pushed onto the stack and subsequently popped from the stack in last in, first out (LIFO) order. A stack also includes a pointer, which points to the item at the top of the stack. Instructions represent actions to perform on the items in the stack. For example, an ADD instruction might pop the top two items from the stack (the values 100 and 10), and push a single item with the sum back onto the stack (the value 110):

WebAssembly's stack machine operates in the same way. A program counter (pointer) maintains the execution position within the code and a virtual control stack keeps track of blocks and if constructs as they are entered (pushed) and exited (popped). The instructions are executed with no reference to an AST. Thus, the binary instruction format portion of the definition refers to a binary representation of instructions that are in a format readable by the decoding stack in the browser.

WebAssembly was designed from the beginning with portability in mind. Portability in this context means that WebAssembly's binary format can be executed efficiently on a variety of operating systems and instruction set architectures, on and off the web. The specification for WebAssembly defines portability in the context of an execution environment. WebAssembly was designed to run efficiently in environments that meet certain characteristics, most of which are related to memory. WebAssembly's portability can also be attributed to the absence of a specific API around the core technologies. Instead, it defines an import mechanism where the set of available imports is defined by the host environment.

In a nutshell, this means that WebAssembly isn't tied to a specific environment, such as the web or desktop. The WebAssembly Working Group has defined a Web API, but that's separate from the Core Specification. The Web API caters to WebAssembly, not the other way around.

The compilation aspect of the definition indicates that WebAssembly will be simple to compile down to its binary format from source code written in high-level languages. The MVP focuses on two languages, C and C++, but Rust can also be used given its similarities to C++. Compilation will be achieved through the use of a Clang/LLVM backend, although we'll be using Emscripten in this book to generate our Wasm modules. The plan is to eventually add support for other languages and compilers (such as GCC), but the MVP is focused on LLVM.

The official definition gives some high-level insight into the overall technology, but for the sake of completeness, it's worth digging a little deeper. WebAssembly's Core Specification is the official document to reference if you want to understand WebAssembly at a very granular level. If you're interested in learning about the characteristics of the runtime structure with regard to the execution environment, check out section 4: Execution. We won't cover that here, but understanding where the Core Specification fits in will help in establishing a complete definition of WebAssembly.

The Core Specification states WebAssembly encodes a low-level, assembly-like programming language. The specification defines the structure, execution, and validation of this language as well as the details of the binary and text formats. The language itself is structured around the following concepts:

• Values, or rather value types that WebAssembly provides
• Instructions that are executed within the stack machine
• Traps produced under error conditions and abort execution
• Functions into which code is organized, each of which takes a sequence of values as parameters and returns a sequence of values as a result
• Tables, which are arrays of values of a particular element type (such as function references) that are selectable by the executing program
• Linear Memory, which is an array of raw bytes that can be used to store and load values
• Modules, WebAssembly binary (.wasm file) that contains function, tables, and linear memories
• Embedder, the mechanism by which WebAssembly can be executed in a host environment, such as a web browser

Functions, tables, memory, and modules have direct correlations with the JavaScript API and are important to be aware of. These concepts describe the underlying structure of the language itself and how to write or encode WebAssembly. With regard to usage, understanding the corresponding semantic phases of WebAssembly provides a complete definition of the technology:

The Core Specification describes the different phases an encoded module (.wasm file) undergoes when it is being utilized in a host environment (such as a web browser). This aspect of the specification represents how the output is handled and executed:

• Decoding: The binary format is converted into a module
• Validation: The decoded module undergoes validation checks (such as type checking) to ensure the module is well formed and safe
• Execution, Part 1: Instantiation: A module instance, which is the dynamic representation of the module, is instantiated by initializing the Globals, Memories, and Tables, and invokes the module's start() function
• Execution, Part 2: Invocation: Exported functions are called from the module instance:

 The following diagram provides a visual representation of the semantic phases:

The WebAssembly Working Group also released API specifications for interacting with JavaScript and the web, which qualifies them for inclusion in the WebAssembly technology space. The JavaScript API is scoped to the JavaScript language itself, without being specifically tied to an environment (for example, web browsers or Node.js). It defines classes, methods, and objects for interacting with WebAssembly and managing the compilation and instantiation processes. The Web API is an extension of the JavaScript API that defines functionality specific to web browsers. The Web API specification currently only defines two methods, compileStreaming and instantiateStreaming, which are convenience methods that simplify the use of Wasm modules in the browser. These will be covered in greater detail in Chapter 2, Elements of WebAssembly - Wat, Wasm, and the JavaScript API.

WebAssembly's ultimate goal is not to replace JavaScript, but rather to complement it. JavaScript's rich ecosystem and flexibility still makes it the ideal language for the web. WebAssembly's JavaScript API makes interoperability between the two technologies relatively simple. So will you be able to build a web application using just WebAssembly? One of the explicit goals of WebAssembly is portability, and replicating all of JavaScript's functionality could inhibit that goal. However, the official site includes a goal to execute and integrate well with the existing web platform, so only time will tell. It may not be practical to write the entire code base in a language that compiles down to WebAssembly, but moving some of the application logic to Wasm modules could be beneficial in terms of performance and load times.

WebAssembly's official site has an extensive list of potential use cases. I'm not going to cover them all here, but there are several that represent significant enhancements to the capabilities of the web platform:

• Image/video editing
• Games
• Music applications (streaming, caching)
• Image recognition
• Live video augmentation
• VR and augmented reality

Although some of these use cases are technically feasible with JavaScript, HTML, and CSS, using WebAssembly can offer significant performance gains. Serving up a binary file (instead of a single JavaScript file) can greatly reduce the bundle size, and instantiating the Wasm module on page load speeds up code execution.

WebAssembly isn't just limited to the browser. Outside the browser, you could use it to build hybrid native apps on mobile devices or perform server-side computations of untrusted code. Using Wasm modules for phone apps could be incredibly beneficial in terms of power usage and performance.

WebAssembly also offers flexibility with regard to how it can be used. You can write your entire code base in WebAssembly, although this may not be practical in its current form or in the context of a web application. Given WebAssembly's robust JavaScript API, you could write the UI in JavaScript/HTML and use Wasm modules for functionality that doesn't directly access the DOM. Once additional languages are supported, objects can be easily passed between the Wasm module and JavaScript code, which will greatly simplify integration and increase developer adoption.

WebAssembly's high-level goals for their MVP was to provide roughly the same functionality as asm.js. The two technologies are very closely related. C, C++, and Rust are very popular languages that support manual memory allocation, which made them ideal candidates for the initial implementation. In this section, we're going to provide a brief overview of each programming language.

C and C++ are low-level programming languages that have been around for over 30 years. C is procedural and doesn't inherently support object-oriented programming concepts such as classes and inheritance, but it's fast, portable, and widely used. 

C++ was built to fill the gaps in C by adding features such as operator overloading and improved type checking. Both languages consistently rank in the top 10 most popular programming languages, which make them ideally suited for the MVP:

C and C++ support is also baked into Emscripten, so in addition to simplifying the compilation process, it allows you to take advantage of WebAssembly's full capabilities. It is also possible to compile C/C++ code down to a .wasm file using LLVM. LLVM is a collection of modular and reusable compiler and toolchain technologies. In a nutshell, it's a framework that simplifies the configuration of a compilation process from source code to machine code. If you made your own programming language and would like to build a compiler, LLVM has tools to simplify the process. I'll cover how to compile C/C++ into .wasm files using LLVM in Chapter 10, Advanced Tools and Upcoming Features.

The following snippet demonstrates how to print Hello World! to the console using C++:

#include <iostream>

int main() {
    std::cout << "Hello, World!\n";
    return 0;
}

C and C++ were intended to be the primary languages used for WebAssembly, but Rust is a perfectly suitable substitute. Rust is a systems programming language that is syntactically similar to C++. It was designed with memory safety in mind, but still retains the performance advantages of C and C++. The current nightly build of Rust's compiler can generate .wasm files from Rust source code, so if you prefer Rust and are familiar with C++, you should be able to use Rust for most of the examples in this book.

The following snippet demonstrates how to print Hello World! to the console using Rust:

fn main() {
    println!("Hello World!");
}

Various tooling exists to enable the use of WebAssembly with some of the other popular programming languages, although they are mostly experimental:

• C# via Blazor
• Haxe via WebIDL
• Java via TeaVM or Bytecoder
• Kotlin via TeaVM
• TypeScript via AssemblyScript

It is also technically possible to transpile a language to C and consequently compile that to a Wasm module, but the success of compilation is contingent on the output of the transpiler. More than likely, you'd have to make significant changes to the code to get it to work.

Admittedly, WebAssembly is not without its limitations. New features are being actively developed and the technology is constantly evolving, but the MVP functionality represents only a portion of WebAssembly's capabilities. In this section, we'll cover some of these limitations and how they impact the development process.

WebAssembly supports a flat linear memory, which isn't a limitation per se, but requires some understanding of how to explicitly allocate memory to execute code. C and C++ were logical choices for the MVP because memory management is built into the language. The reason why some of the more popular high-level languages such as Java weren't included initially is due to something called garbage collection (GC).

GC is a form of automated memory management wherein memory occupied by objects that are no longer in use by the program is reclaimed automatically. GC is analogous to an automatic transmission on a car. It has been heavily optimized by skilled engineers to operate as efficiently as possible, but limits the amount of control the driver has. Manually allocating memory is like driving a car with a manual transmission. It affords greater control over speed and torque, but misuse or lack of experience can leave you stranded with a severely damaged car. Part of C and C++'s excellent performance and speed can be attributed to the manual allocation of memory.

GC languages allow you to program without having to worry about memory availability or allocation. JavaScript is an example of a GC language. The browser engine employs something called a mark-and-sweep algorithm to collect unreachable objects and free up the corresponding memory. Support for GC languages is currently being worked on in WebAssembly, but it's hard to say exactly when it will be completed.

WebAssembly is unable to access the DOM, so any DOM manipulation needs to be done indirectly through JavaScript or using a tool such as Emscripten. There are plans to add the ability to reference DOM and other Web API objects directly, but that's still in the proposal phase. DOM manipulation will likely go hand in hand with GC languages, since it will allow the seamless passing of objects between WebAssembly and JavaScript code.

Older browsers don't have the global WebAssembly object available to instantiate and load Wasm modules. There are experimental polyfills that utilize asm.js if the object isn't found, but the WebAssembly Working Group currently has no plans to create one. Since asm.js and WebAssembly are closely related, simply serving up an asm.js file if the WebAssembly object is unavailable will still offer performance gains while accommodating for backward compatibility. You can see which browsers currently support WebAssembly at https://caniuse.com/#feat=wasm.

Emscripten is the source-to-source compiler that can generate asm.js from C and C++ source code. We'll use it as a build tool to generate the Wasm modules. In this section, we'll quickly review how Emscripten relates to WebAssembly.

Emscripten is an LLVM-to-JavaScript compiler, which means it takes LLVM bitcode output of a compiler such as Clang (for C and C++), and converts that to JavaScript. It isn't one specific technology, but rather a combination of technologies that work together to build, compile, and run asm.js. To generate Wasm modules, we'll use the Emscripten SDK (EMSDK)  Manager:

In Chapter 4, Installing the Required Dependencies, we'll install the EMSDK and use it to manage the dependencies required to compile C and C++ to Wasm modules. Emscripten uses Binaryen's asm2wasm tool to compile the asm.js output by Emscripten to a .wasm file. Binaryen is a compiler and toolchain infrastructure library that includes tools to compile various formats to WebAssembly modules and vice versa. Understanding the inner workings of Binaryen isn't required to use WebAssembly, but it is important to be aware of the underlying technologies and how they work together. By passing certain flags into the compile command for Emscripten (emcc), we can pipe the resultant asm.js code to Binaryen to output our .wasm file.

In this chapter, we discussed the history of WebAssembly with regard to the technologies that led to its creation. A detailed overview of the definition of WebAssembly was provided to allow for a greater understanding of the underlying technologies involved.

The Core Specification, JavaScript API, and Web API were presented as important elements of WebAssembly and demonstrate how the technology will evolve. We also reviewed potentials use cases, currently supported languages, and tools that enable the use of non-supported languages.

The limitations of WebAssembly are the absence of GC, the inability to communicate directly with the DOM, and the lack of support for older browsers. These were discussed to convey the newness of the technology and shed light on some of its shortcomings. Finally, we discussed Emscripten's role in the development process and where it fits into the WebAssembly development workflow.

In Chapter 2, Elements of WebAssembly - Wat, Wasm, and the JavaScript API, we'll be diving deeper into the elements that make up WebAssembly: the WebAssembly text format (Wat), binary format (Wasm), JavaScript, and Web APIs.

1. Which two technologies influenced the creation of WebAssembly?
2. What is a stack machine and how does it relate to WebAssembly?
3. In what ways does WebAssembly complement JavaScript?
4. Which three programming languages can be compiled to Wasm modules?
5. What role does LLVM play with regard to WebAssembly?
6. What are three potential use cases for WebAssembly?
7. How are DOM access and GC related?
8. What tool does Emscripten use to generate Wasm modules?

• Official WebAssembly site: https://webassembly.org
• Native Client technical overview: https://developer.chrome.com/native-client/overview
• The LLVM Compiler Infrastructure Project: https://llvm.org
• About Emscripten: http://kripken.github.io/emscripten-site/docs/introducing_emscripten/about_emscripten.html
• asm.js specification: http://asmjs.org/spec/latest