This book has an accompanying GitHub repository at https://github.com/PacktPublishing/Building-Your-Own-JavaScript-Framework. In each chapter, we will point to the relevant directories in this repository. Feel free to clone or download the repository as a ZIP file.

You need a desktop or a laptop computer with internet access and a terminal application to install and run the code from this repository. We shall also be utilizing Node.js to run some of the parts of the repository. Node.js is an open source, cross-platform, backend JavaScript runtime environment that runs JavaScript code outside a web browser. The installation for Node.js can be found at nodejs.org. For the code from the repository, you can use any environment that supports a terminal and runs Node.js, such as Windows, macOS, and most varieties of Linux.

As JavaScript progressed and evolved, the innovators who were heavily involved in the language, both companies and individuals, started writing software libraries to help solve the everyday burdens of web application architectures. The initial focus of the most basic JavaScript libraries was to provide singular features, interactivity, and add-ons, which progressively enhanced the web page. At that time, JavaScript gave life to a static page with its interactive components – simple examples that always come to mind are tiny scripts, which enabled creative button effects and mouse cursor effects. In many cases, these scripts were separate from the site’s core functionality and were not essential to allow users to interact with the content. Since the inception of small libraries, these have paved the way for the complex framework systems that we have today. The frontend technology quickly evolved, and now, developers are much more accustomed to megabytes of scripts powering the frontend code.

JavaScript libraries were the next step in the web development evolution, assisting with cross-browser quirks, complex visual effects, network requests, and web page layout. With the use of these libraries, developers were able to have the cross-browser development challenges under control. CSS started to catch up with its focus on layout features and cross-browser standards, improving the web’s styling features. Developers finally began introducing structure and well-crafted systems into web development.

The time has come to finally focus on building scalable and opinionated software on the web, and this is where we began to see glimpses of complex software paradigms introduced into large websites and web applications. Companies and larger enterprises started treating the web as a serious application platform, which led to a few prominent projects written in JavaScript and compiled to JavaScript from languages such as Java. Tracing back to late 2009, we see the first iterations of Model-View-Controller (MVC) frameworks built entirely with HTML, CSS, and JavaScript. This MVC model allows more extensive projects to stay organized, enriches the development workflow, and opens up the world of frontend development to developers who expect a much more structured approach when writing software. The MVC model fit web applications well enough to spawn a renaissance in framework development.

Many development hours were invested into connecting the mechanisms between the JavaScript engines and the browser web APIs. In Figure 1.1, we see a simplified view of how this interaction happens:

Figure 1.1: Interaction between the JavaScript engine and web APIs

The framework code and its internal technologies, such as the virtual DOM, use the DOM and its components to make the web application experience possible. The web has its own approach to the MVC architecture, with DOM and web API events interacting with Controllers defined in JavaScript. The Controllers interface with Views written in HTML or templated HTML. Furthermore, in this paradigm, the components of the applications utilize a Model to model the data within them. Using this approach, we can later communicate with backend services to retrieve the data in a particular manner.

Each new JavaScript MVC framework tried to perfect its implementation or approach in various ways. About five years after the first MVC frameworks appeared and gained popularity, several new paradigms that focused on the observer software design pattern started gaining traction in the JavaScript community. This observer approach is a software design pattern in which an object maintains a list of its dependants, called observers. The object notifies the observers automatically of the state changes within it. At that time, Flux came about, which is an application architecture that concentrates on simplifying the hurdles faced within MVC. The burdens included dealing with views constantly needing to interact with the models, hard-to-debug, deeply nested logic, and the need for adequate testing solutions of complex applications.

In the observer pattern, we define subjects that include a collection of observers informing about state changes. The Flux architecture expanded this existing pattern to fit better with applications built on the web. In the case of the Flux pattern, it consists of stores interacting with the state of a component. These stores get notified by a dispatcher based on data coming from actions that a user in the view took. Many JavaScript frameworks started adapting this pattern, ultimately simplifying how engineers structured their applications while still enforcing a set of applicable rules to keep the separation of concerns intact. The software patterns in all of these frameworks provide a clear separation of concerns between the interface, data model, and the application logic that integrates them. The Flux-based JavaScript frameworks introduced new concepts that emerged from the known MVC patterns. Still, both the MVC and Flux approaches focused on the principle of the separation of concerns in application development.

Furthermore, while simplifying ideas brought to light by Flux, a library called Redux inspired the next generation of frameworks to switch their approach to application state management. Instead of the Flux dispatchers, Redux frameworks rely on a single store with pure reducer functions, taking the current state and returning an updated state. Even today, frontend patterns are still maturing, and building for the web platform is becoming increasingly easier.

While there’s a lot to mention with regard to frontend technologies, JavaScript has also made a great impact in places outside of web browsers. We cover those areas in the next section.

Another monumental event during the appearance of the first frontend frameworks was the emergence of a new open source runtime called Node.js. Node.js allowed developers to use JavaScript to produce server-side scripts, deploy backend systems, build developer tools, and, more importantly, write frameworks using the same language as the one from the web browser. The unique combination of having JavaScript on both sides of the software stack created immense possibilities for software developers. This runtime has since spread into many directions beyond software applications, with frameworks for desktop application development, hardware I/O solutions, and much more.

JavaScript-built frameworks enabled the web platform to become one of the most important technologies within reach of billions of people. It’s almost impossible to imagine starting a new project without relying on the consistency and amiability of using a framework, and even the smallest tasks benefit significantly from using a cohesive and opinionated structure. However, even with the fast-paced evolution of the language and how we build web projects, it took quite a bit of time for JavaScript frameworks to emerge as fully encapsulated platforms that can aid developers in producing efficient applications.

JavaScript prevailed through the rise of mobile platforms, with multiple frameworks being created for mobile and existing systems integrating mobile benchmarks into their release process. The optimizations got to the hardware level, having the ARM (arm.com) processor architecture introduce optimizations to improve JavaScript performance in data type conversion, resulting in performance boosts for many JavaScript applications. That is quite a journey for a scripting language that started with small scripts on plain web pages.

Today, we can create fully fledged applications and services using the web platform by combining the power of web APIs, the JavaScript language, and technologies such as progressive web apps, using the frameworks that bring it all together. It is a fantastic time to start traversing the world of these JavaScript systems and using them to our advantage.

Now that we have an overview of how web development evolved, let’s take a look at how the code bases have changed over time.

While learning about frameworks, it is fascinating to reflect on how building for the web has changed over time. This exploration helps us understand why we build web applications the way we do today and helps us learn from historical shifts. It also allows us to be more mindful concerning framework usability and development decisions when we take on large projects. As technology pushes forward, the requirements and expectations around how websites and web applications are built drastically change. Depending on how long someone has been involved in web development, they either experienced many rapidly evolving shifts to how the code bases are structured or were lucky enough to avoid the times when the tooling and the process were highly tedious.

Initially, the code bases comprised isolated frontend components stitched together, consisting of code repetition and mixes of coding patterns. Code organization, usage of software development patterns, and performance optimizations were not a primary focus for developers. The web application deployment process used to be rudimentary as well. In many cases, the websites were manually updated and did not use source control or version tracking. Testing was also highly manual and would only exist in a few projects that were large enough to enable it. This was before deployment pipelines with continuous integration, deployment automation, and advanced testing infrastructure, rigorously verified every change. There used to be a time when developers had to optimize their CSS selectors for performance reasons.

Luckily, productivity and workflows rapidly started to improve once the industry started focusing more on building complex applications. Today we have source control, we have a myriad of testing and deployment tools to choose from, and we have established software paradigms that considerably improve our lives as developers and vastly improve the quality of the projects we build. Improvements to JavaScript engines unlocked new pathways for frameworks, and enhancements to web browsers fixed slow DOM interactivity with techniques such as the virtual DOM, Shadow DOM, and Web Components. These days, frontend frameworks have a better client platform to target as well, and the more established and improved web standards make it possible to perform much more complex operations. For example, with the help of WebAssembly (webassembly.org) standards, we can now run low-level code with improved performance, all within the browser.

As part of all these developments and growth in popularity, the web application development workflow got a lot more complex in many ways. Almost at every point of interaction with a web application project, there is a tooling set designed to improve our workflow. Some examples of this would be Git source control, various pre- and post-processors of our files, code editors with plugins, browser extensions, and many more. Here we have an example that illustrates the key components of a modern web application code base structure, in this case, generated by SvelteKit:

Figure 1.2: SvelteKit code base structure

We will go over SvelteKit later in the Frameworks that use React section of this chapter, and even if you have never used Svelte, this project file tree will look very familiar if you work with other frameworks. This dynamic structure of tools enables flexibility when it comes to switching out certain functionality. For example, Prettier can be substituted for another code formatting tool if need be, while the rest of the project structure remains the same and functions as it was.

With the establishment of the first frameworks in JavaScript, we experienced the introduction of a build step into our projects, which meant that either external or bundled tooling would help run or build the application. Today, this build step, popularized by Webpack or esbuild, is almost impossible to avoid. As part of this build step, we fetch application dependencies using package managers, process CSS, create code bundles, and run various optimization steps to make our app run fast and consume the least bandwidth. The ecosystem also introduced JavaScript transpilers, which are a type of source-to-source code compiler. They are used to take one particular syntax, which could consist of more modern features or include additional features, and convert them to be compatible with broadly accepted JavaScript syntax. Transpilers, such as Babel, began to see everyday use, integrated with the build step in many projects; this pattern generally motivated the use of the latest language features while also supporting old browser engines. These days, transpilation and build steps apply to files beyond JavaScript, as well as files such as CSS and specific templating formats.

Integrating with the build step are the package managers, such as npm or yarn, which play an essential role in resolving project dependencies. If you want to bootstrap a workflow with a framework, you will likely rely on the package manager to initialize the framework structure and its dependencies. For new projects, it is almost impossible to have a sensible framework workflow without using a package manager or some form of dependency resolution. As the project grows, the package manager facilitates the organization of newer dependencies while keeping track of updates to modules that are already in use. These days text editors, such as Visual Studio Code and IntelliJ WebStorm, adapt to our code bases and provide excellent tooling to enable source control of our code. The editors rely on built-in features and external plugins that encourage better formatting, easier debugging, and framework-specific improvements.

The code bases will keep changing as technology develops further, and the tools will keep improving to enable us to develop applications more quickly. Regarding the framework organization, we can expect higher levels of abstractions that simplify the way we do web development. Many programming languages, such as Java and Swift, have pre-defined development workflows encapsulating all aspects of development. JavaScript code bases so far have been an exception to these rules and allowed for high levels of flexibility. This trend is going to continue for many more years as the rapid pace of tooling and innovation in web development is not slowing down at all.

Now that we understand how the JavaScript ecosystem has evolved and how codebases have changed over time, let us explore what JavaScript frameworks offer in terms of frontend, backend, testing, and beyond.

Though it is challenging to compare all the subtle differences of every framework out there in the ecosystem, we can cover several frameworks that significantly impact the developer community or offer a unique approach to solving a particular problem. The expanded knowledge of the tools helps us notice specific patterns in these frameworks regarding the different strategies for developer experience and feature sets.

There are still ways to build apps and websites without frameworks, but many developers prefer to use established and opinionated frameworks even with the given overhead and learning curves. If you follow the JavaScript community, you will find that it is always passionately abuzz with discussions around frameworks, so let us dive deeper into the needs and benefits of framework use.

The frameworks provide good levels of abstraction to write high-level code without rewriting low-level functionality. Developers can be much more involved in business and product logic and iterate faster on new features. To give an example, up until recently, writing the code to make an asynchronous web request with proper error handling was a very time-consuming task without the aid of a good abstraction. Now that we have the Fetch API (fetch.spec.whatwg.org), this is a much easier endeavor, but Fetch is only part of the story, so the rest of the web APIs, especially the ones from earlier times, still benefit from good abstractions. In cases where we choose to write low-level code, it is a much better approach to find ways to write that code within the framework boundaries. This way it is tested and maintained within the primitives of the framework. This yields the benefits of avoiding extra maintenance and ensuring all the usages of that code are still behind sensible abstractions. Some backend frameworks approach this by providing extensible interfaces to hook into framework internals through plugins or extending the default behavior.

Developing software with groups of people is a challenging endeavor, so small and large teams can benefit from integrating a framework into the engineering workflow. The provided structure of abstractions generally results in much more well-architected systems, given the limits of how developers can write high-level components. The key benefit is enabling everyone involved in the task to understand the code base better and conveniently spend less time deliberating refactors and adding new code.

Now that we have our abstracted high-level code, we can cherish another benefit of frameworks – the performance optimizations they enable. Writing performant code that works in all provided use cases takes skill and takes away significant time from the project at hand. Even the most knowledgeable developers would only be able to come up with good enough solutions in a short amount of time. With frameworks, especially open source ones, you benefit from many minds put together to solve performance bottlenecks, overcome typical encumbrances, and continue to benefit from improvements as the framework develops. The performance benefits come from optimized low-level and well-structured high-level components; notably, some frameworks will guard against code that will slow down the application.

Frameworks make integrating with external systems, such as databases, external APIs, or specific components, easier. For instance, some web frameworks can integrate directly with the GraphQL data query language, simplifying backend systems’ interaction. It’s not just the ease of use, but also these integrations enable safe interaction with components such as databases, which helps avoid problematic queries that can be slow or harmful to execute. For frontend projects, it is important to always keep up with the latest web standards, and this is where frameworks provide another integration benefit.

Finally, as with all software, support plays an important role. Another reason a project may use an established framework is the available support channels through paid, volunteer, and open source help. The shared knowledge of these systems enables developers to help each other build systems and makes it easier to hire new developers who are familiar with these existing systems.

As we see, frameworks benefit us in countless ways – let us recap with these exact reasons. Here’s what frameworks allow us to do:

• Focus on business logic and writing high-level code
• Write less code and follow code conventions defined by the framework
• Benefit from performance gains and rely on future optimizations
• Develop the project with good architecture, abstractions, and organization
• Easily integrate with external systems such as databases and external APIs
• Ability to rely on security fixes, audits, and patches
• Improve developer workflow using framework-specific toolings, such as text-editor integrations and command-line utilities
• Ability to debug issues easily by relying on detailed error messages and consistent logging
• Rely on external support from framework authors and the community
• Hire more developers who are already accustomed to working with the framework of our choice or with similar experience
• Develop better user experiences by leveraging the framework feature set

While a lot of JavaScript frameworks focus on the developer experience, the user experience can sometimes suffer from the overhead of these systems. This is usually relevant in frontend projects – an example of this would be loading a complex web application on budget mobile devices. In backend systems, this can be seen when the APIs are not able to keep up with request load and reliably scale with traffic spikes.

Even if the systems are skillfully built in both of these cases, the framework of choice might not be optimized to cover all use cases. I believe the next iteration of the framework ecosystem will largely focus on the user experience aspects, which means making load times faster, shipping less JavaScript over the network, and ensuring the web applications we create work seamlessly on all platforms. In the following sections, we will examine some of the most popular frameworks that enable these benefits for web application developers.

Frontend frameworks

Since JavaScript frameworks originated in the browser, let us look at modern frontend frameworks as our first exploration.

Ember.js

Suppose we trace the roots of the first JavaScript frameworks through the origins of libraries such as Prototype.js, jQuery, and script.aculo.us. In that case, we will eventually arrive at SproutCore, a framework used by Apple and a handful of other companies to build some of the most complex web experiences many years ago.

Today this early SproutCore project has influenced the Ember.js framework. Ember continues to be a highly opinionated piece of software that allows us to build applications with defined components, services, models, and a powerful router. Like many frameworks we will discuss in this chapter, Ember comes with its own command-line tooling, which helps developers quickly get started on the basics of the application and later generate more code quickly as the project scope grows. The usefulness of the provided framework tooling is immense. The CLI encapsulates the code generation steps and enables a way to run common framework commands, such as running tests or serving application files. With Ember, developers get a complete set of tools such as auto-reload, browser developer tooling, and a package called Ember Data, which helps manage the API-to-model relationship through adapters and serializers. Ultimately, Ember has a steeper learning curve than other frameworks, but its highly opinionated concepts guide developers toward highly functional web applications.

Angular

Angular is another framework with a large following. With TypeScript at its core, it is often used as a subset system for other full stack web frameworks. Angular provides its opinionated approach to component-based architecture. Angular has a complex history of rewrites but is now a more streamlined project with a stable feature set. Angular’s template syntax extends HTML by adding expressions and new attributes. At its core, it uses the pattern of dependency injection. The latest versions of this framework offer a variety of binding techniques, including event, property, and two-way binding.

Vue.js

Vue.js, also written in TypeScript, was created by borrowing the good elements of Angular. Developers love Vue’s simplicity within its component system, syntax, and general ease of use. It utilizes the Model–View–Viewmodel (MVVM) pattern, where a View communicates with a ViewModel using some data binding technique. In the case of Vue.js, for its data, it uses different techniques through HTML classes, HTML elements, and custom binding element attributes to achieve this. The purpose of the given ViewModel is to handle as much of the View’s interaction logic and be the middle structure between the presentation logic and the application’s business logic. Besides using HTML authoring, Vue has the Single-File Component (SFC) format (vuejs.org/api/sfc-spec.html) to encapsulate all aspects of the components – scripts, styling, and templating into one file. The SFC happens as part of the build step and helps the components avoid runtime compilation, scopes the CSS styles to the component, enables Hot Module Replacement, and much more.

About TypeScript

TypeScript is a superset of JavaScript, enabling features such as static typing and language extensions in many JavaScript environments. In recent years, TypeScript has become popular among framework authors and developers. It is also widely supported by many code editors and IDEs. Initially released in 2012 and inspired by ActionScript and Java, TypeScript is a superset of JavaScript, enabling features such as static typing and language extensions in many JavaScript environments. It helps catch errors at compile time instead of runtime error handling. Files with the file extensions .ts and .tsx are TypeScript files that must be compiled to JavaScript to be used in most environments.

Frameworks that use React

These days, we hear about React a lot; even though it is a user interface component library by itself, it has become the cornerstone for many frontend frameworks, such as Gatsby, Remix, Next.js, and others. As part of its introduction, React also debuted JSX, its own set of extensions to JavaScript, making it possible to define the components in a similar-looking syntax to HTML. For instance, the static site framework Gatsby relies on React’s state management and the nested component architecture to compose its web pages. With Gatsby, developers can multiplex data, utilizing GraphQL, from content management systems, e-commerce sources, and other places.

Following along our React route, we get to Remix, which bundles a full stack solution with features for both the server and the client, plus a compiler and a request handler. Remix provides solutions for the View and Controller aspects of the application and relies on the Node.js module ecosystem for the rest, giving flexibility to the developers who need custom solutions from project to project. Based on the experience of creating and maintaining the react-router project for many years, the creators of Remix were able to come up with powerful abstractions while taking advantage of the browser’s web APIs instead of investing in new concepts. To give an example, if you choose Remix for your project, you will find yourself using web standard APIs more than some of the other frameworks.

Next.js is our next React-based framework, which extends the use of the React component architecture as well by bringing it to the server with its built-in server rendering. The server-rendered components allow for a pre-rendered page to be sent to the client, resulting in the client only spending resources on initializing the interactive components. The framework provides the concept of pages, which allows for simpler routing implementations with lazy loading and enables automatic code-splitting. Combining all these features results in a fantastic user experience with fast loading times. In addition, the deployed applications rank highly when indexed by search engines, a feature that makes this framework stand out.

While talking about React frameworks, it is worth mentioning Solid.js. It’s a newer library that creates frontend interfaces. Solid’s benchmarks outperform React and others. It uses features such as JSX, but with a few key differences. With Solid, there is no virtual DOM and no concept of hooks. Instead, it relies on the pattern of signals to update the real DOM nodes, while utilizing reactive primitives. As part of Solid’s approach, it offers the SolidStart app framework, which is very comparable to Next.js. It consists of core support components – router, session, document, actions, data, entrypoints, and server – these are integrated together as part of SolidStart.

SvelteKit

Like SolidStart, there is also SvelteKit, a new framework powered by the Svelte user interface library. SvelteKit framework generator script assembles a project skeleton, which helps you quickly get started and gives you full flexibility over the configuration of the compiler and the TypeScript settings. When setting up SvelteKit, we can use JavaScript with JSDoc formatting or TypeScript directly to write frontend applications. As part of the Svelte integration, it equips developers with a compiler that pre-builds the app before the client processes it. Like Vue’s SFC format, Svelte uses .svelte files, which encapsulate the components with the <script>, <style>, and HTML tags that are coded together. These are compiled into JavaScript output generated by the compiler.

About Vite

Vite (vitejs.dev) is framework-agnostic tooling, meaning it can be used in conjunction with different frameworks. It uses a vite.config.js configuration file. Mainly, it is used as a build tool for frontend projects. It is optimized for speed and it achieves that speed by providing a development server with Hot Module Replacement and a bundler that optimizes JavaScript output using esbuild (esbuild.github.io).

Framework features and patterns

To understand what most modern frameworks enable, we need to understand the following acronyms and features:

• Single-Page Application (SPA): An early term that describes an application that purely uses JavaScript and other frontend frameworks for all interactions with reduced browser routing.
• Server-Side Rendering (SSR): Pre-rendered components on the server side, which are transferred for JavaScript hydration on the client side.
• Client-Side Rendering (CSR): Rendering of components using JavaScript, purely on the browser’s side.
• Static Site Generator (SSG): The concept of pre-generating all pages from source for faster rendering and better search engine optimization.
• Deferred Static Generator (DSG): Renders content on the server when initiated by a request to the server.
• Incremental Static Regeneration (ISR): Another pattern of static content generation. In this case, the static generation is triggered by updates by some external trigger.
• Content Security Policy (CSP): Configuration for serving scripts that helps protect against cross-site scripting attacks.
• Hot Module Replacement (HMR): Technique to replace JavaScript modules as the application is running in the browser, mainly used to improve development speed and avoid page reloads.
• Single-File Component (SFC): A file structure that encapsulates all aspects of a usable framework component, such as styling, templating, logic, and more.
• Model-View-Controller (MVC): A design pattern focusing on the separation of concerns in various types of applications. It approaches this separation by using the following: a Model that represents the data, a View that provides the user with an interface, and a Controller that is the intermediary between the views and the models.
• Model-View-ViewModel (MVVM): Another design pattern that also focuses on the separation of concerns in applications, but the approach to these separations is different. In this case, there are still Views and Models, similar to MVC. However, the ViewModel acts as a connection between those types. This approach uses two-way data binding between the View and the Model.

Besides the features and their acronyms, here is a helpful visual describing both the MVC and MVVM patterns:

Figure 1.3: MVC versus MVVM patterns

During the renaissance of frontend frameworks, an open source project called TodoMVC was established to help developers compare frameworks based on the same To Do app, where anyone can send pull requests with their framework implementations. Besides comparing different frameworks, the project also popularized the approach to complex code organization in JavaScript. Now with the emergence of these new frameworks, we need another iteration of TodoMVC to continue aiding developers with comparisons of these systems.

Backend frameworks

Switching gears from the frontend, let us look at some of the backend frameworks. Node.js plays a vital role in the JavaScript ecosystem, powering a variety of frameworks that allow us to develop backend services. Similar to the frontend, it is impossible to cover all of them, but in this section, we will examine hapi.js, express, Sails.js, nest.js, and AdonisJS.

Hapi.js

As part of framework explorations over the years, I had the opportunity to work with these frameworks in a professional capacity and on small hobby projects. I started with hapi.js, which is a good example of a well-crafted Node.js framework, built with essential defaults that allow it to craft a server backend quickly. It has a unique approach of avoiding middlewares and relying on external modules. As part of its core, it already has validation rules, parsing, logging, and more built right into it. hapi.js doesn’t lock down extensibility; developers can create plugins and register them to execute as part of the different parts of the request lifecycle. Hapi.js’ mission puts an emphasis on avoiding unexpected consequences when combining a lot of application logic. This is evident in how hapi.js approaches dependency management and module namespacing.

Express

In stark contrast to hapi.js, the Node.js ecosystem also has a framework called Express, which is largely an unopinionated approach to building backend services. Thousands of projects and tools usually use Express for its routing, content parsers, and high-performance reasons. Being flexible in almost every way and with support for over a dozen templating engines, Express is the introductory framework for developers starting with Node.js development. For example, a popular MVC framework, Sails.js, builds upon Express’ functionality to offer API generation, database ORM solutions, and support for building real-time features. Generally, it is a good solution for those appreciating the middleware patterns of Express, while having a more structured approach to building backend systems.

NestJS

NestJS, not to be confused with Next.js, is another server-side framework that is worth mentioning. It is similar to Vue, and Angular inspired its approach to application structure, but in this case, for a backend system. By default, it utilizes Express as its default HTTP server and creates an abstraction layer that allows for the ability to change the third-party modules, enabling the developers to swap out Express for some other HTTP framework such as Fastify. In NestJS, we see a similar pattern of dependency injection, which enables developers to architect contained modules. These modules can be reused, overridden, and mocked in tests.

AdonisJS

Our final Node.js framework for this section is AdonisJS. Built entirely with TypeScript, it is packed with features that you would expect from a mature framework, such as the ORM based on the Active Record pattern, schema validators, extensive authentication support, and much more. The built-in and first-party plugin features provide solutions for many mundane problems of backend building. AdonisJS also packs a custom templating engine to render HTML layouts. As an added bonus, AdonisJS has straight-to-the-point and clear documentation, which is a joy to read and explore.

Fresh

Given the focus on essential frameworks in the Node.js ecosystem, we should also mention a backend framework called Fresh, which is powered by the Deno runtime. This runtime is built using a combination of technologies – JavaScript, TypeScript, WebAssembly, and the Rust programming language. Fresh takes a simplistic approach with its emphasis on having no build steps, minimal configuration, and just-in-time rendering of components on the server. Routing is taken care of by creating files in the directories of your project, called File-system routing, a similar pattern in other frameworks.

Looking back at all the Node.js frameworks we covered in this section, there is a healthy framework diversity that delivers solutions for projects of any type.

Native frameworks

The knowledge of JavaScript also allows us to build for native operating system environments and interact with hardware platforms. The availability of the runtime in other environments makes it possible for us to create unique solutions that can help web developers apply their skills in areas beyond the browser. In this section, we cover some of the frameworks created for native JavaScript development.

Electron

The idea of packaging a web app as a native app is not new, but it has been perfected with Electron. Electron allows developers to use familiar frontend technologies to build fully capable cross-platform applications that run on popular desktop platforms. It has feature supports features such as auto-updates and inter-process communication, as well as having a collection of plugins that tap into operating system functionality. Besides the advanced framework features, it is beneficial to have a single code base targeting all the platforms, which helps with efficiently building new features and bug fixing. These days millions of people use applications built with Electron, in many cases without knowing it. Applications such as Microsoft Teams, Slack, 1Password, Discord, Figma, Notion, and many more utilize Electron. Even more examples can be found at electronjs.org/apps.

React Native

Another framework that helps us create for native platforms is React Native, which unlocks the world of mobile development to those experienced with JavaScript. Targeting iOS and Android mobile platforms, just like Electron on desktop, it brings all the benefits of React user-interface building blocks, a unified codebase, and a strong, established community.

Johnny-Five

The Node.js ecosystem also offers hardware frameworks such as Johnny-Five, which allows for creative learning use cases of robotics programming using JavaScript and the Firmata protocol. Johnny-Five is an IoT platform supporting over 30 hardware boards. Mainly, it offers interfaces to interact with LEDs, services, motors, switches, and more.

All the frameworks so far deal with building out application logic, but there are also other types of frameworks in JavaScript that play an important role in the development process – these would be the testing frameworks.

Testing frameworks

Testing frameworks in software development are essential for ensuring our projects function as expected. With JavaScript and its supported runtime environments, we have a much more challenging task at hand – we have to test in different browser engines and mock the native web APIs. In some cases, mocking built-in and external libraries can also be challenging. The asynchronous behavior of the language brings its own obstacles as well. Luckily, the JavaScript ecosystem came up with various testing frameworks addressing many software testing challenges – unit, integration, functional, end-to-end testing, and beyond. To name a few, Jest, Playwright, and Vitest all offer great solutions to testing challenges. We will discuss them next.

Jest

As we develop our web applications, we want to ensure that the components we build are functioning as intended; this is where a framework such as Jest comes in. Jest is a unit testing framework that integrates well into other projects. If we are given a project with one of the frameworks that we already saw in this chapter, Jest would equip us with reliable testing solutions. It is ergonomic, with minimal or zero configuration, and provides us with interfaces for easy mocking, object snapshotting, code coverage, and most importantly, an easy-to-understand API for organizing our tests.

Vitest

Vitest is a similar unit testing framework, offering the same interfaces to mock modules in web projects. It focuses on speed and support for components of many frameworks, including Vue, React, Svelte, and even Web Components. It is designed for developer productivity and has a mode for smart test watching, a multi-threaded test runner, and a familiar snapshotting mechanism.

Playwright

Besides unit testing, our software projects benefit highly from end-to-end testing; this is where a testing framework such as Playwright is a good contender. It delivers cross-browser and cross-platform testing for web applications. Playwright comes with a set of test interfaces to automate various browsers, including navigating to URLs and clicking buttons. Historically, this has been a challenging problem due to the asynchronous nature of web pages, but this framework is equipped with ways to avoid flaky tests through retries and await behavior.

Depending on the requirements of the JavaScript project you are involved in, you might have to create new testing workflows or customize the existing testing infrastructure to fit your use case – this is where experience in building testing frameworks would be advantageous.

Framework showcase

Here’s a breakdown of the frameworks we covered in this chapter.

These are some of the noteworthy web application frameworks that we will be focusing on in this book:

Figure 1.4: Examples of frontend and full stack frameworks

These are some of the backend frameworks that will serve as good examples and help us learn certain framework development patterns:

Figure 1.5: Examples of backend frameworks

Other frameworks that use frontend technologies to target native or hardware development are as follows:

Figure 1.6: Examples of native and hardware frameworks

Here are testing framework examples, which are useful to integrate and use in web application projects:

Figure 1.7: Examples of testing frameworks

One of the best resources to keep up with the current direction of framework development is stateofjs.com. It is a yearly survey with results from thousands of developers, and it provides an outlook of where the technologies are shifting. For example, if we look at the frontend framework rankings of 2022 (2022.stateofjs.com/en-US/libraries/front-end-frameworks), we can already start to see the retention and interest in React slowly dropping, which could potentially indicate that the industry is slowly shifting to other solutions. Due to this constant change of use, awareness, and popularity of all these frameworks, instead of focusing on many of the frameworks of today, we are going to cover the core patterns that could be applicable to new frameworks in the future. These patterns will be helpful to you while you explore creating your own framework.

Now it’s time to try out some of the frameworks mentioned in this chapter using the GitHub repository for this book mentioned in the Technical requirements section. You can follow these steps:

1. Install Node.js version 20 from nodejs.org.
2. Clone the repository from https://github.com/PacktPublishing/Building-Your-Own-JavaScript-Framework.
3. Using your terminal, change directories into the chapter1 directory of the repository.
4. Run npm install and then npm start.
5. Follow the interactive prompt to run the examples.

The showcase focuses on reproducing the same example based on the framework type. All frontend frameworks demonstrate the same component written in different structures. Note that some examples might take a while to install and run.

In the final part of this chapter, we shall take a look at my notable personal experiences with frameworks in web development.

My professional web development career initially started with building basic websites before established frameworks or libraries were around. Looking ahead, I want to share my framework experiences of professionally utilizing them and contributing to some of the open source ones. These days, the accumulated knowledge gathered from these experiences has helped me better assess framework usefulness and introduce new software paradigms into my work. Many find it challenging to keep up with the latest innovations in the JavaScript field, but building even the smallest projects helps one grow as a developer in many ways.

Here are some examples from the full stack development areas that helped me become much more effective as a web developer.

Frontend development

The first few professional websites I built were developed for Internet Explorer 6 and early versions of Firefox web browsers. As we learned from this chapter, there were no frameworks for building web applications at the time, and I had to utilize the few libraries at my disposal. These would help add interactivity for components such as image galleries and dynamic page layouts. Luckily, when my focus switched to larger projects, the jQuery frontend library came along and started growing in popularity. Even to this day, jQuery is still a popular tool of choice for a large chunk of websites. I now had the opportunity to hand-craft a basic framework, which could be reused from project to project. This series of scripts was incredibly convenient and foreshadowed the bright future of frameworks that we have today. It was clear that the trend of single-page JavaScript applications was heading towards structured and opinionated solutions.

During one of my large early-on projects – specifically the Firefox Accounts frontend (accounts.firefox.com), I had the opportunity to use Backbone.js, with the help of jQuery and multiple extension libraries to make it more suitable for large projects. The Firefox Accounts frontend, which is serving millions of users, is still using Backbone.js to this day. The way the Backbone.js framework is structured allows for a soft dependency on jQuery, so it did feel like a natural continuation of my earlier approach to web application development. My key takeaways from this experience are that Backbone.js wasn’t the perfect answer to the challenges of frontend web applications, but it was beneficial in many ways. For example, it allowed the project to stay flexible with the ever-evolving JavaScript ecosystem and helped diverse developers work on the application while following a solid set of application guidelines. The unique opportunity to work on the client and the integrated services of the Firefox web browser taught me how to produce JavaScript components for a desktop client that runs on millions of computers worldwide.

Throughout many professional projects, I had the chance to work with Ember.js, Angular, and various React frameworks. I was impressed by how empowering these frameworks can be on these occasions. A notable mention from my experience is the Meteor web framework, released in early 2012. One of the big selling features of Meteor was the isomorphic or so-called universal JavaScript approach, where the code runs on both the client and the server. In many ways, we see similar approaches in popular frameworks today, where a full stack framework lets developers write JavaScript to develop on both sides of the stack. I have built a few applications and some plugins for this framework, and while it felt so easy to get started with Meteor, I have experienced hurdles while trying to build something that didn’t fit exactly into the scope of what Meteor supported, especially in the early releases of the framework. A particular example of fighting with the framework’s constraints was developing a feature with a synchronized document state across multiple clients. This ended up being challenging to implement with Meteor’s feature set at the time and had to be rebuilt with alternative tooling. Luckily this was not a critical project, but for times when it is important, it is a good idea to evaluate whether the framework of your choice is the right tool for what you are trying to build.

Backend development

During the early years of Node.js, I had the chance to work on several projects utilizing the microservices architecture, and these involved using the Express and hapi frameworks. I felt the contrast between the open-ended approach of the express framework versus the rigorous set of rules and options that were defined in hapi.js. To give some examples, overriding and customizing certain behaviors in hapi.js was quite difficult, and keeping the framework up to date required difficult migrations to the code base.

I still remember combing through the changelog of every new version of hapi.js, making sure not to miss any breaking changes that would make my projects dysfunctional. Even with the hardships of hapi.js, it did feel like the framework was providing a good set of abstractions. In many ways, following the existing examples from something like Flask in Python, hapi had the necessary components to build highly usable services. Meanwhile, my experiences with Express seemed more reminiscent of working with the jQuery and Backbone.js days. In the Express projects, I could have a highly flexible development environment, combining different Node.js modules to achieve what I wanted from the framework. This made me realize that the perfect framework for me would be something between Express and hapi, in the sense that it would allow me to stay creative, highly productive, and able to utilize the runtime ecosystem to the fullest, while at the same time having a strong opinionated framework core, which would keep my application efficient and reliable.

Developer tooling and more

As part of my profession, I have always been passionate about open source, so I focused my efforts on contributing to developer tooling and testing frameworks. I have been a maintainer of Grunt.js (gruntjs.com), a JavaScript task runner for many years. Grunt.js has been a core component of frameworks, such as Yeoman, and has been used as a tool of choice in the early version of AngularJS. The task runner conventions in Node.js have changed a lot since then, but there is still a solid number of projects that use Grunt.js. Maintaining this project for many years feels similar to maintaining a large framework project – releasing new versions, keeping a stable API, supporting it through security bounties, and much more. There is also a huge list of issues, feature demands, pull requests, and plugins to support.

In terms of my testing framework contributions, I was involved in developing the Intern.js testing framework (github.com/theintern), which enabled unit and functional testing for web applications. I was both the contributor and the consumer of this framework in my daily projects, which gave me a unique angle on the project. I was inspired to provide a good integration experience because it would aid my own projects. As part of this effort, besides learning how a testing framework is built, I focused on developing integration examples and documentation for other application frameworks. Covering many integration scenarios in the provided examples made it much easier for developers to integrate this testing system into their applications.

A final notable framework from my personal experience would be with voxel.js – an open source voxel game-building toolkit. While not that popular, it is a great example of creative use of JavaScript, combining both frontend and backend technologies. It is a framework built by a small team that fills in a niche for an audience of developers, who are looking into working on games and visualizations. voxel.js did not set out to be a world-changing framework; instead, it was a creative outlet for many to create. While exploring voxel.js in my personal project, I learned a lot about unique framework and module structures, and it was fun to experiment with systems that enable more imaginative thinking.

Contributing to new projects

These experiences with JavaScript frameworks in frontend, backend, and developer systems were incredibly valuable to me as part of my career. I have learned the importance of following best practices, adhering to software patterns, and developing for various runtime environments, which ultimately helped me write better code and deliver successful projects. As part of this book, I am sharing my learnings and as much knowledge as possible for the next generation of devoted JavaScript developers to build and contribute to their own framework projects.

The projects that I have been involved in always had different origins. In my case, I had to work with both private and open source frameworks. In work projects, I have focused on combining open source tooling with the context of the larger business organization. This approach helped align the existing tooling with the requirements of particular projects. In the open source context, I have been lucky to contribute to projects that have improved the overall developer experience. In many of the scenarios, I got to work on projects that were innovative and were firsts of their kind in the JavaScript ecosystem. For example, when Grunt.js was coming along, there were task runner tools from other languages, but the JavaScript tools were in their inception. Contributing to voxel.js was a similar experience; as more HTML5 APIs and WebGL enable more advanced graphics on the web, it enabled voxel.js as a project and created the contributor community.

During my contributions to the Intern.js testing framework, the overall feeling was that there were not fully fledged testing frameworks that solved all of the needs of web application testing. The goal of this project was to create an all-in solution for testing using the same types of testing APIs.

The framework that we create in this book focuses on the use of modern technologies, such as Web Components, intermixed with popular JavaScript libraries. The Web Components field does not feel as explored in the ecosystem just yet; therefore, with this book, we are taking aim to further widen the knowledge of these technologies among web developers. Besides expanding those skills, one of the greater goals is to make the framework development process more approachable and demystify the established JavaScript systems.

This first chapter began our exploration into how the web application development process has changed from pure basics into a full-fledged software platform. We have looked at how the innovations and challenges of the web shape the frameworks discussed in this chapter and play a huge role in offering a variety of useful features to web developers. As part of my career journey, working on various projects made me appreciate how much can be achieved by combining elegant patterns and the creative use of the JavaScript programming language.

As part of the framework showcase, it is evident that the ecosystem has a lot of options for the browser and other places where JavaScript runtime functions. However, there is always room for improvement in speed, features, and unique ideas, which can help us enhance the development processes. The significant part that stimulates this ecosystem is the ever-evolving web platform, development of the ECMAScript specification, and of course, the hard work of maintainers of runtimes such as Node.js and Deno.

In the upcoming chapters, we will dive deeper into software paradigms, focusing on framework organization and their architectural patterns. In Chapter 2, we are going to look at how frameworks are structured and organized.