Chapter 10. Web Development Done Right

	"Don't believe everything you read on the Web."
	--Confucius

In this chapter, we're going to work on a website together. By working on a small project, my aim is to open a window for you to take a peek on what web development is, along with the main concepts and tools you should know if you want to be successful with it.

What is the Web?

The World Wide Web, or simply Web, is a way of accessing information through the use of a medium called the Internet. The Internet is a huge network of networks, a networking infrastructure. Its purpose is to connect billions of devices together, all around the globe, so that they can communicate with one another. Information travels through the Internet in a rich variety of languages called protocols, which allow different devices to speak the same tongue in order to share content.

The Web is an information-sharing model, built on top of the Internet, which employs the Hypertext Transfer Protocol (HTTP) as a basis for data communication. The Web, therefore, is just one of the several different ways information can be exchanged over the Internet: e-mail, instant messaging, news groups, and so on, they all rely on different protocols.

How does the Web work?

In a nutshell, HTTP is an asymmetric request-response client-server protocol. An HTTP client sends a request message to an HTTP server. The server, in turn, returns a response message. In other words, HTTP is a pull protocol in which the client pulls information from the server (as opposed to a push protocol in which the server pushes information down to the client). Take a look at the following image:

HTTP is based on TCP/IP (Transmission Control Protocol/Internet Protocol), which provides the tools for a reliable communication exchange.

An important feature of the HTTP protocol is that it's stateless. This means that the current request has no knowledge about what happened in previous requests. This is a limitation, but you can browse a website with the illusion of being logged in. Under the covers though, what happens is that, on login, a token of user information is saved (most often on the client side, in special files called cookies) so that each request the user makes carries the means for the server to recognize the user and provide a custom interface by showing their name, keeping their basket populated, and so on.

Even though it's very interesting, we're not going to delve into the rich details of HTTP and how it works. However, we're going to write a small website, which means we'll have to write the code to handle HTTP requests and return HTTP responses. I won't keep prepending HTTP to the terms request and response from now on, as I trust there won't be any confusion.

The Django web framework

For our project, we're going to use one of the most popular web frameworks you can find in the Python ecosystem: Django.

A web framework is a set of tools (libraries, functions, classes, and so on) that we can use to code a website. We need to decide what kind of requests we want to allow to be issued against our web server and how we respond to them. A web framework is the perfect tool to do that because it takes care of many things for us so that we can concentrate only on the important bits without having to reinvent the wheel.

Note

There are different types of frameworks. Not all of them are designed for writing code for the web. In general, a framework is a tool that provides functionalities to facilitate the development of software applications, products and solutions.

Django design philosophy

Django is designed according to the following principles:

• DRY: As in, Don't Repeat Yourself. Don't repeat code, and code in a way that makes the framework deduce as much as possible from as little as possible.
• Loose coupling: The various layers of the framework shouldn't know about each other (unless absolutely necessary for whatever reason). Loose coupling works best when paralleled with high cohesion. To quote Robert Martin: putting together things which change for the same reason, and spreading apart those which change for different reasons.
• Less code: Applications should use the least possible amount of code, and be written in a way that favors reuse as much as possible.
• Consistency: When using the Django framework, regardless of which layer you're coding against, your experience will be very consistent with the design patterns and paradigms that were chosen to lay out the project.

The framework itself is designed around the model-template-view (MTV) pattern, which is a variant of model-view-controller (MVC), which is widely employed by other frameworks. The purpose of such patterns is to separate concerns and promote code reuse and quality.

The model layer

Of the three layers, this is the one that defines the structure of the data that is handled by the application, and deals with data sources. A model is a class that represents a data structure. Through some Django magic, models are mapped to database tables so that you can store your data in a relational database.

Note

A relational database stores data in tables in which each column is a property of the data and each row represents a single item or entry in the collection represented by that table. Through the primary key of each table, which is that part of the data that allows to uniquely identify each item, it is possible to establish relationships between items belonging to different tables, that is, to put them into relation.

The beauty of this system is that you don't have to write database-specific code in order to handle your data. You will just have to configure your models correctly and simply use them. The work on the database is done for you by the Django object-relational mapping (ORM), which takes care of translating operations done on Python objects into a language that a relational database can understand: SQL (Structured Query Language).

One benefit of this approach is that you will be able to change databases without rewriting your code since all the database specific code is produced by Django on the fly, according to which database it's connected to. Relational databases speak SQL, but each of them has its own unique flavor of it; therefore, not having to hardcode any SQL in our application is a tremendous advantage.

Django allows you to modify your models at any time. When you do, you can run a command that creates a migration, which is the set of instructions needed to port the database in a state that represents the current definition of your models.

To summarize, this layer deals with defining the data structures you need to handle in your website and gives you the means to save and load them from and to the database by simply accessing the models, which are Python objects.

The view layer

The function of a view is handling a request, performing whatever action needs to be carried out, and eventually returning a response. For example, if you open your browser and request a page corresponding to a category of products in an e-commerce shop, the view will likely talk to the database, asking for all the categories that are children of the selected category (for example, to display them in a navigation sidebar) and for all the products that belong to the selected category, in order to display them on the page.

Therefore, the view is the mechanism through which we can fulfill a request. Its result, the response object, can assume several different forms: a JSON payload, text, an HTML page, and so on. When you code a website, your responses usually consist of HTML or JSON.

Note

The Hypertext Markup Language, or HTML, is the standard markup language used to create web pages. Web browsers run engines that are capable of interpreting HTML code and render it into what we see when we open a page of a website.

The template layer

This is the layer that provides the bridge between backend and frontend development. When a view has to return HTML, it usually does it by preparing a context object (a dict) with some data, and then it feeds this context to a template, which is rendered (that is to say, transformed into HTML) and returned to the caller in the form of a response (more precisely, the body of the response). This mechanism allows for maximum code reuse. If you go back to the category example, it's easy to see that, if you browse a website that sells products, it doesn't really matter which category you click on or what type of search you perform, the layout of the products page doesn't change. What does change is the data with which that page is populated.

Therefore, the layout of the page is defined by a template, which is written in a mixture of HTML and Django template language. The view that serves that page collects all the products to be displayed in the context dict, and feeds it to the template which will be rendered into an HTML page by the Django template engine.

The Django URL dispatcher

The way Django associates a Uniform Resource Locator (URL) with a view is through matching the requested URL with the patterns that are registered in a special file. A URL represents a page in a website so, for example, http://mysite.com/categories?id=123 would probably point to the page for the category with ID 123 on my website, while https://mysite.com/login would probably be the user login page.

Tip

The difference between HTTP and HTTPS is that the latter adds encryption to the protocol so that the data that you exchange with the website is secured. When you put your credit card details on a website, or log in anywhere, or do anything around sensitive data, you want to make sure that you're using HTTPS.

Regular expressions

The way Django matches URLs to patterns is through a regular expression. A regular expression is a sequence of characters that defines a search pattern with which we can carry out operations such as pattern and string matching, find/replace, and so on.

Regular expressions have a special syntax to indicate things like digits, letters, spaces, and so on, as well as how many times we expect a character to appear, and much more. A complete explanation of this topic is beyond of the scope of the book. However, it is a very important topic, so the project we're going to work on together will evolve around it, in the hope that you will be stimulated to find the time to explore it a bit more on your own.

To give you a quick example, imagine that you wanted to specify a pattern to match a date such as "26-12-1947". This string consists of two digits, one dash, two digits, one dash, and finally four digits. Therefore, we could write it like this: r'[0-9]{2}-[0-9]{2}-[0-9]{4}'. We created a class by using square brackets, and we defined a range of digits inside, from 0 to 9, hence all the possible digits. Then, between curly braces, we say that we expect two of them. Then a dash, then we repeat this pattern once as it is, and once more, by changing how many digits we expect, and without the final dash. Having a class like [0-9] is such a common pattern that a special notation has been created as a shortcut: '\d'. Therefore, we can rewrite the pattern like this: r'\d{2}-\d{2}-\d{4}' and it will work exactly the same. That r in front of the string stands for raw, and its purpose is to alter the way every backslash '\' is interpreted by the regular expression engine.

Django design philosophy

Django is designed according to the following principles:

• DRY: As in, Don't Repeat Yourself. Don't repeat code, and code in a way that makes the framework deduce as much as possible from as little as possible.
• Loose coupling: The various layers of the framework shouldn't know about each other (unless absolutely necessary for whatever reason). Loose coupling works best when paralleled with high cohesion. To quote Robert Martin: putting together things which change for the same reason, and spreading apart those which change for different reasons.
• Less code: Applications should use the least possible amount of code, and be written in a way that favors reuse as much as possible.
• Consistency: When using the Django framework, regardless of which layer you're coding against, your experience will be very consistent with the design patterns and paradigms that were chosen to lay out the project.

The framework itself is designed around the model-template-view (MTV) pattern, which is a variant of model-view-controller (MVC), which is widely employed by other frameworks. The purpose of such patterns is to separate concerns and promote code reuse and quality.

The model layer

Of the three layers, this is the one that defines the structure of the data that is handled by the application, and deals with data sources. A model is a class that represents a data structure. Through some Django magic, models are mapped to database tables so that you can store your data in a relational database.

Note

A relational database stores data in tables in which each column is a property of the data and each row represents a single item or entry in the collection represented by that table. Through the primary key of each table, which is that part of the data that allows to uniquely identify each item, it is possible to establish relationships between items belonging to different tables, that is, to put them into relation.

The beauty of this system is that you don't have to write database-specific code in order to handle your data. You will just have to configure your models correctly and simply use them. The work on the database is done for you by the Django object-relational mapping (ORM), which takes care of translating operations done on Python objects into a language that a relational database can understand: SQL (Structured Query Language).

One benefit of this approach is that you will be able to change databases without rewriting your code since all the database specific code is produced by Django on the fly, according to which database it's connected to. Relational databases speak SQL, but each of them has its own unique flavor of it; therefore, not having to hardcode any SQL in our application is a tremendous advantage.

Django allows you to modify your models at any time. When you do, you can run a command that creates a migration, which is the set of instructions needed to port the database in a state that represents the current definition of your models.

To summarize, this layer deals with defining the data structures you need to handle in your website and gives you the means to save and load them from and to the database by simply accessing the models, which are Python objects.

The view layer

The function of a view is handling a request, performing whatever action needs to be carried out, and eventually returning a response. For example, if you open your browser and request a page corresponding to a category of products in an e-commerce shop, the view will likely talk to the database, asking for all the categories that are children of the selected category (for example, to display them in a navigation sidebar) and for all the products that belong to the selected category, in order to display them on the page.

Therefore, the view is the mechanism through which we can fulfill a request. Its result, the response object, can assume several different forms: a JSON payload, text, an HTML page, and so on. When you code a website, your responses usually consist of HTML or JSON.

Note

The Hypertext Markup Language, or HTML, is the standard markup language used to create web pages. Web browsers run engines that are capable of interpreting HTML code and render it into what we see when we open a page of a website.

The template layer

This is the layer that provides the bridge between backend and frontend development. When a view has to return HTML, it usually does it by preparing a context object (a dict) with some data, and then it feeds this context to a template, which is rendered (that is to say, transformed into HTML) and returned to the caller in the form of a response (more precisely, the body of the response). This mechanism allows for maximum code reuse. If you go back to the category example, it's easy to see that, if you browse a website that sells products, it doesn't really matter which category you click on or what type of search you perform, the layout of the products page doesn't change. What does change is the data with which that page is populated.

Therefore, the layout of the page is defined by a template, which is written in a mixture of HTML and Django template language. The view that serves that page collects all the products to be displayed in the context dict, and feeds it to the template which will be rendered into an HTML page by the Django template engine.

The Django URL dispatcher

The way Django associates a Uniform Resource Locator (URL) with a view is through matching the requested URL with the patterns that are registered in a special file. A URL represents a page in a website so, for example, http://mysite.com/categories?id=123 would probably point to the page for the category with ID 123 on my website, while https://mysite.com/login would probably be the user login page.

Tip

The difference between HTTP and HTTPS is that the latter adds encryption to the protocol so that the data that you exchange with the website is secured. When you put your credit card details on a website, or log in anywhere, or do anything around sensitive data, you want to make sure that you're using HTTPS.

Regular expressions

The way Django matches URLs to patterns is through a regular expression. A regular expression is a sequence of characters that defines a search pattern with which we can carry out operations such as pattern and string matching, find/replace, and so on.

Regular expressions have a special syntax to indicate things like digits, letters, spaces, and so on, as well as how many times we expect a character to appear, and much more. A complete explanation of this topic is beyond of the scope of the book. However, it is a very important topic, so the project we're going to work on together will evolve around it, in the hope that you will be stimulated to find the time to explore it a bit more on your own.

To give you a quick example, imagine that you wanted to specify a pattern to match a date such as "26-12-1947". This string consists of two digits, one dash, two digits, one dash, and finally four digits. Therefore, we could write it like this: r'[0-9]{2}-[0-9]{2}-[0-9]{4}'. We created a class by using square brackets, and we defined a range of digits inside, from 0 to 9, hence all the possible digits. Then, between curly braces, we say that we expect two of them. Then a dash, then we repeat this pattern once as it is, and once more, by changing how many digits we expect, and without the final dash. Having a class like [0-9] is such a common pattern that a special notation has been created as a shortcut: '\d'. Therefore, we can rewrite the pattern like this: r'\d{2}-\d{2}-\d{4}' and it will work exactly the same. That r in front of the string stands for raw, and its purpose is to alter the way every backslash '\' is interpreted by the regular expression engine.

The model layer

Of the three layers, this is the one that defines the structure of the data that is handled by the application, and deals with data sources. A model is a class that represents a data structure. Through some Django magic, models are mapped to database tables so that you can store your data in a relational database.

Note

A relational database stores data in tables in which each column is a property of the data and each row represents a single item or entry in the collection represented by that table. Through the primary key of each table, which is that part of the data that allows to uniquely identify each item, it is possible to establish relationships between items belonging to different tables, that is, to put them into relation.

The beauty of this system is that you don't have to write database-specific code in order to handle your data. You will just have to configure your models correctly and simply use them. The work on the database is done for you by the Django object-relational mapping (ORM), which takes care of translating operations done on Python objects into a language that a relational database can understand: SQL (Structured Query Language).

One benefit of this approach is that you will be able to change databases without rewriting your code since all the database specific code is produced by Django on the fly, according to which database it's connected to. Relational databases speak SQL, but each of them has its own unique flavor of it; therefore, not having to hardcode any SQL in our application is a tremendous advantage.

Django allows you to modify your models at any time. When you do, you can run a command that creates a migration, which is the set of instructions needed to port the database in a state that represents the current definition of your models.

To summarize, this layer deals with defining the data structures you need to handle in your website and gives you the means to save and load them from and to the database by simply accessing the models, which are Python objects.

The view layer

The function of a view is handling a request, performing whatever action needs to be carried out, and eventually returning a response. For example, if you open your browser and request a page corresponding to a category of products in an e-commerce shop, the view will likely talk to the database, asking for all the categories that are children of the selected category (for example, to display them in a navigation sidebar) and for all the products that belong to the selected category, in order to display them on the page.

Therefore, the view is the mechanism through which we can fulfill a request. Its result, the response object, can assume several different forms: a JSON payload, text, an HTML page, and so on. When you code a website, your responses usually consist of HTML or JSON.

Note

The Hypertext Markup Language, or HTML, is the standard markup language used to create web pages. Web browsers run engines that are capable of interpreting HTML code and render it into what we see when we open a page of a website.

The template layer

This is the layer that provides the bridge between backend and frontend development. When a view has to return HTML, it usually does it by preparing a context object (a dict) with some data, and then it feeds this context to a template, which is rendered (that is to say, transformed into HTML) and returned to the caller in the form of a response (more precisely, the body of the response). This mechanism allows for maximum code reuse. If you go back to the category example, it's easy to see that, if you browse a website that sells products, it doesn't really matter which category you click on or what type of search you perform, the layout of the products page doesn't change. What does change is the data with which that page is populated.

Therefore, the layout of the page is defined by a template, which is written in a mixture of HTML and Django template language. The view that serves that page collects all the products to be displayed in the context dict, and feeds it to the template which will be rendered into an HTML page by the Django template engine.

The Django URL dispatcher

The way Django associates a Uniform Resource Locator (URL) with a view is through matching the requested URL with the patterns that are registered in a special file. A URL represents a page in a website so, for example, http://mysite.com/categories?id=123 would probably point to the page for the category with ID 123 on my website, while https://mysite.com/login would probably be the user login page.

Tip

The difference between HTTP and HTTPS is that the latter adds encryption to the protocol so that the data that you exchange with the website is secured. When you put your credit card details on a website, or log in anywhere, or do anything around sensitive data, you want to make sure that you're using HTTPS.

Regular expressions

The way Django matches URLs to patterns is through a regular expression. A regular expression is a sequence of characters that defines a search pattern with which we can carry out operations such as pattern and string matching, find/replace, and so on.

Regular expressions have a special syntax to indicate things like digits, letters, spaces, and so on, as well as how many times we expect a character to appear, and much more. A complete explanation of this topic is beyond of the scope of the book. However, it is a very important topic, so the project we're going to work on together will evolve around it, in the hope that you will be stimulated to find the time to explore it a bit more on your own.

To give you a quick example, imagine that you wanted to specify a pattern to match a date such as "26-12-1947". This string consists of two digits, one dash, two digits, one dash, and finally four digits. Therefore, we could write it like this: r'[0-9]{2}-[0-9]{2}-[0-9]{4}'. We created a class by using square brackets, and we defined a range of digits inside, from 0 to 9, hence all the possible digits. Then, between curly braces, we say that we expect two of them. Then a dash, then we repeat this pattern once as it is, and once more, by changing how many digits we expect, and without the final dash. Having a class like [0-9] is such a common pattern that a special notation has been created as a shortcut: '\d'. Therefore, we can rewrite the pattern like this: r'\d{2}-\d{2}-\d{4}' and it will work exactly the same. That r in front of the string stands for raw, and its purpose is to alter the way every backslash '\' is interpreted by the regular expression engine.

The view layer

The function of a view is handling a request, performing whatever action needs to be carried out, and eventually returning a response. For example, if you open your browser and request a page corresponding to a category of products in an e-commerce shop, the view will likely talk to the database, asking for all the categories that are children of the selected category (for example, to display them in a navigation sidebar) and for all the products that belong to the selected category, in order to display them on the page.

Therefore, the view is the mechanism through which we can fulfill a request. Its result, the response object, can assume several different forms: a JSON payload, text, an HTML page, and so on. When you code a website, your responses usually consist of HTML or JSON.

Note

The Hypertext Markup Language, or HTML, is the standard markup language used to create web pages. Web browsers run engines that are capable of interpreting HTML code and render it into what we see when we open a page of a website.

The template layer

This is the layer that provides the bridge between backend and frontend development. When a view has to return HTML, it usually does it by preparing a context object (a dict) with some data, and then it feeds this context to a template, which is rendered (that is to say, transformed into HTML) and returned to the caller in the form of a response (more precisely, the body of the response). This mechanism allows for maximum code reuse. If you go back to the category example, it's easy to see that, if you browse a website that sells products, it doesn't really matter which category you click on or what type of search you perform, the layout of the products page doesn't change. What does change is the data with which that page is populated.

Therefore, the layout of the page is defined by a template, which is written in a mixture of HTML and Django template language. The view that serves that page collects all the products to be displayed in the context dict, and feeds it to the template which will be rendered into an HTML page by the Django template engine.

The Django URL dispatcher

The way Django associates a Uniform Resource Locator (URL) with a view is through matching the requested URL with the patterns that are registered in a special file. A URL represents a page in a website so, for example, http://mysite.com/categories?id=123 would probably point to the page for the category with ID 123 on my website, while https://mysite.com/login would probably be the user login page.

Tip

The difference between HTTP and HTTPS is that the latter adds encryption to the protocol so that the data that you exchange with the website is secured. When you put your credit card details on a website, or log in anywhere, or do anything around sensitive data, you want to make sure that you're using HTTPS.

Regular expressions

The way Django matches URLs to patterns is through a regular expression. A regular expression is a sequence of characters that defines a search pattern with which we can carry out operations such as pattern and string matching, find/replace, and so on.

Regular expressions have a special syntax to indicate things like digits, letters, spaces, and so on, as well as how many times we expect a character to appear, and much more. A complete explanation of this topic is beyond of the scope of the book. However, it is a very important topic, so the project we're going to work on together will evolve around it, in the hope that you will be stimulated to find the time to explore it a bit more on your own.

To give you a quick example, imagine that you wanted to specify a pattern to match a date such as "26-12-1947". This string consists of two digits, one dash, two digits, one dash, and finally four digits. Therefore, we could write it like this: r'[0-9]{2}-[0-9]{2}-[0-9]{4}'. We created a class by using square brackets, and we defined a range of digits inside, from 0 to 9, hence all the possible digits. Then, between curly braces, we say that we expect two of them. Then a dash, then we repeat this pattern once as it is, and once more, by changing how many digits we expect, and without the final dash. Having a class like [0-9] is such a common pattern that a special notation has been created as a shortcut: '\d'. Therefore, we can rewrite the pattern like this: r'\d{2}-\d{2}-\d{4}' and it will work exactly the same. That r in front of the string stands for raw, and its purpose is to alter the way every backslash '\' is interpreted by the regular expression engine.

The template layer

This is the layer that provides the bridge between backend and frontend development. When a view has to return HTML, it usually does it by preparing a context object (a dict) with some data, and then it feeds this context to a template, which is rendered (that is to say, transformed into HTML) and returned to the caller in the form of a response (more precisely, the body of the response). This mechanism allows for maximum code reuse. If you go back to the category example, it's easy to see that, if you browse a website that sells products, it doesn't really matter which category you click on or what type of search you perform, the layout of the products page doesn't change. What does change is the data with which that page is populated.

Therefore, the layout of the page is defined by a template, which is written in a mixture of HTML and Django template language. The view that serves that page collects all the products to be displayed in the context dict, and feeds it to the template which will be rendered into an HTML page by the Django template engine.

The Django URL dispatcher

The way Django associates a Uniform Resource Locator (URL) with a view is through matching the requested URL with the patterns that are registered in a special file. A URL represents a page in a website so, for example, http://mysite.com/categories?id=123 would probably point to the page for the category with ID 123 on my website, while https://mysite.com/login would probably be the user login page.

Tip

The difference between HTTP and HTTPS is that the latter adds encryption to the protocol so that the data that you exchange with the website is secured. When you put your credit card details on a website, or log in anywhere, or do anything around sensitive data, you want to make sure that you're using HTTPS.

Regular expressions

The way Django matches URLs to patterns is through a regular expression. A regular expression is a sequence of characters that defines a search pattern with which we can carry out operations such as pattern and string matching, find/replace, and so on.

Regular expressions have a special syntax to indicate things like digits, letters, spaces, and so on, as well as how many times we expect a character to appear, and much more. A complete explanation of this topic is beyond of the scope of the book. However, it is a very important topic, so the project we're going to work on together will evolve around it, in the hope that you will be stimulated to find the time to explore it a bit more on your own.

To give you a quick example, imagine that you wanted to specify a pattern to match a date such as "26-12-1947". This string consists of two digits, one dash, two digits, one dash, and finally four digits. Therefore, we could write it like this: r'[0-9]{2}-[0-9]{2}-[0-9]{4}'. We created a class by using square brackets, and we defined a range of digits inside, from 0 to 9, hence all the possible digits. Then, between curly braces, we say that we expect two of them. Then a dash, then we repeat this pattern once as it is, and once more, by changing how many digits we expect, and without the final dash. Having a class like [0-9] is such a common pattern that a special notation has been created as a shortcut: '\d'. Therefore, we can rewrite the pattern like this: r'\d{2}-\d{2}-\d{4}' and it will work exactly the same. That r in front of the string stands for raw, and its purpose is to alter the way every backslash '\' is interpreted by the regular expression engine.

The Django URL dispatcher

The way Django associates a Uniform Resource Locator (URL) with a view is through matching the requested URL with the patterns that are registered in a special file. A URL represents a page in a website so, for example, http://mysite.com/categories?id=123 would probably point to the page for the category with ID 123 on my website, while https://mysite.com/login would probably be the user login page.

Tip

The difference between HTTP and HTTPS is that the latter adds encryption to the protocol so that the data that you exchange with the website is secured. When you put your credit card details on a website, or log in anywhere, or do anything around sensitive data, you want to make sure that you're using HTTPS.

Regular expressions

The way Django matches URLs to patterns is through a regular expression. A regular expression is a sequence of characters that defines a search pattern with which we can carry out operations such as pattern and string matching, find/replace, and so on.

Regular expressions have a special syntax to indicate things like digits, letters, spaces, and so on, as well as how many times we expect a character to appear, and much more. A complete explanation of this topic is beyond of the scope of the book. However, it is a very important topic, so the project we're going to work on together will evolve around it, in the hope that you will be stimulated to find the time to explore it a bit more on your own.

To give you a quick example, imagine that you wanted to specify a pattern to match a date such as "26-12-1947". This string consists of two digits, one dash, two digits, one dash, and finally four digits. Therefore, we could write it like this: r'[0-9]{2}-[0-9]{2}-[0-9]{4}'. We created a class by using square brackets, and we defined a range of digits inside, from 0 to 9, hence all the possible digits. Then, between curly braces, we say that we expect two of them. Then a dash, then we repeat this pattern once as it is, and once more, by changing how many digits we expect, and without the final dash. Having a class like [0-9] is such a common pattern that a special notation has been created as a shortcut: '\d'. Therefore, we can rewrite the pattern like this: r'\d{2}-\d{2}-\d{4}' and it will work exactly the same. That r in front of the string stands for raw, and its purpose is to alter the way every backslash '\' is interpreted by the regular expression engine.

Regular expressions

The way Django matches URLs to patterns is through a regular expression. A regular expression is a sequence of characters that defines a search pattern with which we can carry out operations such as pattern and string matching, find/replace, and so on.

Regular expressions have a special syntax to indicate things like digits, letters, spaces, and so on, as well as how many times we expect a character to appear, and much more. A complete explanation of this topic is beyond of the scope of the book. However, it is a very important topic, so the project we're going to work on together will evolve around it, in the hope that you will be stimulated to find the time to explore it a bit more on your own.

To give you a quick example, imagine that you wanted to specify a pattern to match a date such as "26-12-1947". This string consists of two digits, one dash, two digits, one dash, and finally four digits. Therefore, we could write it like this: r'[0-9]{2}-[0-9]{2}-[0-9]{4}'. We created a class by using square brackets, and we defined a range of digits inside, from 0 to 9, hence all the possible digits. Then, between curly braces, we say that we expect two of them. Then a dash, then we repeat this pattern once as it is, and once more, by changing how many digits we expect, and without the final dash. Having a class like [0-9] is such a common pattern that a special notation has been created as a shortcut: '\d'. Therefore, we can rewrite the pattern like this: r'\d{2}-\d{2}-\d{4}' and it will work exactly the same. That r in front of the string stands for raw, and its purpose is to alter the way every backslash '\' is interpreted by the regular expression engine.

A regex website

So, here we are. We'll code a website that stores regular expressions so that we'll be able to play with them a little bit.

Note

Before we proceed creating the project, I'd like to spend a word about CSS. CSS (Cascading Style Sheets) are files in which we specify how the various elements on an HTML page look. You can set all sorts of properties such as shape, size, color, margins, borders, fonts, and so on. In this project, I have tried my best to achieve a decent result on the pages, but I'm neither a frontend developer nor a designer, so please don't pay too much attention to how things look. Try and focus on how they work.

Setting up Django

On the Django website (https://www.djangoproject.com/), you can follow the tutorial, which gives you a pretty good idea of Django's capabilities. If you want, you can follow that tutorial first and then come back to this example. So, first things first; let's install Django in your virtual environment:

$ pip install django

When this command is done, you can test it within a console (try doing it with bpython, it gives you a shell similar to IPython but with nice introspection capabilities):

>>> import django
>>> django.VERSION
(1, 8, 4, 'final', 0)

Now that Django is installed, we're good to go. We'll have to do some scaffolding, so I'll quickly guide you through that.

Starting the project

Choose a folder in the book's environment and change into that. I'll use ch10. From there, we start a Django project with the following command:

$ django-admin startproject regex

This will prepare the skeleton for a Django project called regex. Change into the regex folder and run the following:

$ python manage.py runserver

You should be able to go to http://127.0.0.1:8000/ with your browser and see the It worked! default Django page. This means that the project is correctly set up. When you've seen the page, kill the server with Ctrl + C (or whatever it says in the console). I'll paste the final structure for the project now so that you can use it as a reference:

$ tree -A regex  # from the ch10 folder
regex
├── db.sqlite3
├── entries
│   ├── admin.py
│   ├── forms.py
│   ├── __init__.py
│   ├── migrations
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   ├── models.py
│   ├── static
│   │   └── entries
│   │       └── css
│   │           └── main.css
│   ├── templates
│   │   └── entries
│   │       ├── base.html
│   │       ├── footer.html
│   │       ├── home.html
│   │       ├── insert.html
│   │       └── list.html
│   └── views.py
├── manage.py
└── regex
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

Don't worry if you're missing files, we'll get there. A Django project is typically a collection of several different applications. Each application is meant to provide a functionality in a self-contained, reusable fashion. We'll create just one, called entries:

$ python manage.py startapp entries

Within the entries folder that has been created, you can get rid of the tests.py module.

Now, let's fix the regex/settings.py file in the regex folder. We need to add our application to the INSTALLED_APPS tuple so that we can use it (add it at the bottom of the tuple):

INSTALLED_APPS = (
    ... django apps ...
    'entries',
)

Then, you may want to fix the language and time zone according to your personal preference. I live in London, so I set them like this:

LANGUAGE_CODE = 'en-gb'
TIME_ZONE = 'Europe/London'

There is nothing else to do in this file, so you can save and close it.

Now it's time to apply the migrations to the database. Django needs database support to handle users, sessions, and things like that, so we need to create a database and populate it with the necessary data. Luckily, this is very easily done with the following command:

$ python manage.py migrate

Note

For this project, we use a SQLite database, which is basically just a file. On a real project, you would probably use a different database engine like MySQL or PostgreSQL.

Creating users

Now that we have a database, we can create a superuser using the console.

$ python manage.py createsuperuser

After entering username and other details, we have a user with admin privileges. This is enough to access the Django admin section, so try and start the server:

$ python manage.py runserver

This will start the Django development server, which is a very useful built-in web server that you can use while working with Django. Now that the server is running, we can access the admin page at http://localhost:8000/admin/. I will show you a screenshot of this section later. If you log in with the credentials of the user you just created and head to the Authentication and Authorization section, you'll find Users. Open that and you will be able to see the list of users. You can edit the details of any user you want as an admin. In our case, make sure you create a different one so that there are at least two users in the system (we'll need them later). I'll call the first user Fabrizio (username: fab) and the second one Adriano (username: adri) in honor of my father.

By the way, you should see that the Django admin panel comes for free automatically. You define your models, hook them up, and that's it. This is an incredible tool that shows how advanced Django's introspection capabilities are. Moreover, it is completely customizable and extendable. It's truly an excellent piece of work.

Adding the Entry model

Now that the boilerplate is out of the way, and we have a couple of users, we're ready to code. We start by adding the Entry model to our application so that we can store objects in the database. Here's the code you'll need to add (remember to use the project tree for reference):

entries/models.py

from django.db import models
from django.contrib.auth.models import User
from django.utils import timezone

class Entry(models.Model):
    user = models.ForeignKey(User)
    pattern = models.CharField(max_length=255)
    test_string = models.CharField(max_length=255)
    date_added = models.DateTimeField(default=timezone.now)

    class Meta:
        verbose_name_plural = 'entries'

This is the model we'll use to store regular expressions in our system. We'll store a pattern, a test string, a reference to the user who created the entry, and the moment of creation. You can see that creating a model is actually quite easy, but nonetheless, let's go through it line by line.

First we need to import the models module from django.db. This will give us the base class for our Entry model. Django models are special classes and much is done for us behind the scenes when we inherit from models.Model.

We want a reference to the user who created the entry, so we need to import the User model from Django's authorization application and we also need to import the timezone model to get access to the timezone.now() function, which provides us with a timezone-aware version of datetime.now(). The beauty of this is that it's hooked up with the TIME_ZONE settings I showed you before.

As for the primary key for this class, if we don't set one explicitly, Django will add one for us. A primary key is a key that allows us to uniquely identify an Entry object in the database (in this case, Django will add an auto-incrementing integer ID).

So, we define our class, and we set up four class attributes. We have a ForeignKey attribute that is our reference to the User model. We also have two CharField attributes that hold the pattern and test strings for our regular expressions. We also have a DateTimeField, whose default value is set to timezone.now. Note that we don't call timezone.now right there, it's now, not now(). So, we're not passing a DateTime instance (set at the moment in time when that line is parsed) rather, we're passing a callable, a function that is called when we save an entry in the database. This is similar to the callback mechanism we used in Chapter 8, The Edges – GUIs and Scripts, when we were assigning commands to button clicks.

The last two lines are very interesting. We define a class Meta within the Entry class itself. The Meta class is used by Django to provide all sorts of extra information for a model. Django has a great deal of logic under the hood to adapt its behavior according to the information we put in the Meta class. In this case, in the admin panel, the pluralized version of Entry would be Entrys, which is wrong, therefore we need to manually set it. We specify the plural all lowercase, as Django takes care of capitalizing it for us when needed.

Now that we have a new model, we need to update the database to reflect the new state of the code. In order to do this, we need to instruct Django that it needs to create the code to update the database. This code is called migration. Let's create it and execute it:

$ python manage.py makemigrations entries
$ python manage.py migrate

After these two instructions, the database will be ready to store Entry objects.

Note

There are two different kinds of migrations: data and schema migration. Data migrations port data from one state to another without altering its structure. For example, a data migration could set all products for a category as out of stock by switching a flag to False or 0. A schema migration is a set of instructions that alter the structure of the database schema. For example, that could be adding an age column to a Person table, or increasing the maximum length of a field to account for very long addresses. When developing with Django, it's quite common to have to perform both kinds of migrations over the course of development. Data evolves continuously, especially if you code in an agile environment.

Customizing the admin panel

The next step is to hook the Entry model up with the admin panel. You can do it with one line of code, but in this case, I want to add some options to customize a bit the way the admin panel shows the entries, both in the list view of all entry items in the database and in the form view that allows us to create and modify them.

All we need to do is to add the following code:

entries/admin.py

from django.contrib import admin
from .models import Entry

@admin.register(Entry)
class EntryAdmin(admin.ModelAdmin):
    fieldsets = [
        ('Regular Expression',
         {'fields': ['pattern', 'test_string']}),
        ('Other Information',
         {'fields': ['user', 'date_added']}),
    ]
    list_display = ('pattern', 'test_string', 'user')
    list_filter = ['user']
    search_fields = ['test_string']

This is simply beautiful. My guess is that you probably already understand most of it, even if you're new to Django.

So, we start by importing the admin module and the Entry model. Because we want to foster code reuse, we import the Entry model using a relative import (there's a dot before models). This will allow us to move or rename the app without too much trouble. Then, we define the EntryAdmin class, which inherits from admin.ModelAdmin. The decoration on the class tells Django to display the Entry model in the admin panel, and what we put in the EntryAdmin class tells Django how to customize the way it handles this model.

Firstly, we specify the fieldsets for the create/edit page. This will divide the page into two sections so that we get a better visualization of the content (pattern and test string) and the other details (user and timestamp) separately.

Then, we customize the way the list page displays the results. We want to see all the fields, but not the date. We also want to be able to filter on the user so that we can have a list of all the entries by just one user, and we want to be able to search on test_string.

I will go ahead and add three entries, one for myself and two on behalf of my father. The result is shown in the next two images. After inserting them, the list page looks like this:

I have highlighted the three parts of this view that we customized in the EntryAdmin class. We can filter by user, we can search and we have all the fields displayed. If you click on a pattern, the edit view opens up.

After our customization, it looks like this:

Notice how we have two sections: Regular Expression and Other Information, thanks to our custom EntryAdmin class. Have a go with it, add some entries to a couple of different users, get familiar with the interface. Isn't it nice to have all this for free?

Creating the form

Every time you fill in your details on a web page, you're inserting data in form fields. A form is a part of the HTML Document Object Model (DOM) tree. In HTML, you create a form by using the form tag. When you click on the submit button, your browser normally packs the form data together and puts it in the body of a POST request. As opposed to GET requests, which are used to ask the web server for a resource, a POST request normally sends data to the web server with the aim of creating or updating a resource. For this reason, handling POST requests usually requires more care than GET requests.

When the server receives data from a POST request, that data needs to be validated. Moreover, the server needs to employ security mechanisms to protect against various types of attacks. One attack that is very dangerous is the cross-site request forgery (CSRF) attack, which happens when data is sent from a domain that is not the one the user is authenticated on. Django allows you to handle this issue in a very elegant way.

So, instead of being lazy and using the Django admin to create the entries, I'm going to show you how to do it using a Django form. By using the tools the framework gives you, you get a very good degree of validation work already done (in fact, we won't need to add any custom validation ourselves).

There are two kinds of form classes in Django: Form and ModelForm. You use the former to create a form whose shape and behavior depends on how you code the class, what fields you add, and so on. On the other hand, the latter is a type of form that, albeit still customizable, infers fields and behavior from a model. Since we need a form for the Entry model, we'll use that one.

entries/forms.py

from django.forms import ModelForm
from .models import Entry

class EntryForm(ModelForm):
    class Meta:
        model = Entry
        fields = ['pattern', 'test_string']

Amazingly enough, this is all we have to do to have a form that we can put on a page. The only notable thing here is that we restrict the fields to only pattern and test_string. Only logged-in users will be allowed access to the insert page, and therefore we don't need to ask who the user is: we know that. As for the date, when we save an Entry, the date_added field will be set according to its default, therefore we don't need to specify that as well. We'll see in the view how to feed the user information to the form before saving. So, all the background work is done, all we need is the views and the templates. Let's start with the views.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Setting up Django

On the Django website (https://www.djangoproject.com/), you can follow the tutorial, which gives you a pretty good idea of Django's capabilities. If you want, you can follow that tutorial first and then come back to this example. So, first things first; let's install Django in your virtual environment:

$ pip install django

When this command is done, you can test it within a console (try doing it with bpython, it gives you a shell similar to IPython but with nice introspection capabilities):

>>> import django
>>> django.VERSION
(1, 8, 4, 'final', 0)

Now that Django is installed, we're good to go. We'll have to do some scaffolding, so I'll quickly guide you through that.

Starting the project

Choose a folder in the book's environment and change into that. I'll use ch10. From there, we start a Django project with the following command:

$ django-admin startproject regex

This will prepare the skeleton for a Django project called regex. Change into the regex folder and run the following:

$ python manage.py runserver

You should be able to go to http://127.0.0.1:8000/ with your browser and see the It worked! default Django page. This means that the project is correctly set up. When you've seen the page, kill the server with Ctrl + C (or whatever it says in the console). I'll paste the final structure for the project now so that you can use it as a reference:

$ tree -A regex  # from the ch10 folder
regex
├── db.sqlite3
├── entries
│   ├── admin.py
│   ├── forms.py
│   ├── __init__.py
│   ├── migrations
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   ├── models.py
│   ├── static
│   │   └── entries
│   │       └── css
│   │           └── main.css
│   ├── templates
│   │   └── entries
│   │       ├── base.html
│   │       ├── footer.html
│   │       ├── home.html
│   │       ├── insert.html
│   │       └── list.html
│   └── views.py
├── manage.py
└── regex
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

Don't worry if you're missing files, we'll get there. A Django project is typically a collection of several different applications. Each application is meant to provide a functionality in a self-contained, reusable fashion. We'll create just one, called entries:

$ python manage.py startapp entries

Within the entries folder that has been created, you can get rid of the tests.py module.

Now, let's fix the regex/settings.py file in the regex folder. We need to add our application to the INSTALLED_APPS tuple so that we can use it (add it at the bottom of the tuple):

INSTALLED_APPS = (
    ... django apps ...
    'entries',
)

Then, you may want to fix the language and time zone according to your personal preference. I live in London, so I set them like this:

LANGUAGE_CODE = 'en-gb'
TIME_ZONE = 'Europe/London'

There is nothing else to do in this file, so you can save and close it.

Now it's time to apply the migrations to the database. Django needs database support to handle users, sessions, and things like that, so we need to create a database and populate it with the necessary data. Luckily, this is very easily done with the following command:

$ python manage.py migrate

Note

For this project, we use a SQLite database, which is basically just a file. On a real project, you would probably use a different database engine like MySQL or PostgreSQL.

Creating users

Now that we have a database, we can create a superuser using the console.

$ python manage.py createsuperuser

After entering username and other details, we have a user with admin privileges. This is enough to access the Django admin section, so try and start the server:

$ python manage.py runserver

This will start the Django development server, which is a very useful built-in web server that you can use while working with Django. Now that the server is running, we can access the admin page at http://localhost:8000/admin/. I will show you a screenshot of this section later. If you log in with the credentials of the user you just created and head to the Authentication and Authorization section, you'll find Users. Open that and you will be able to see the list of users. You can edit the details of any user you want as an admin. In our case, make sure you create a different one so that there are at least two users in the system (we'll need them later). I'll call the first user Fabrizio (username: fab) and the second one Adriano (username: adri) in honor of my father.

By the way, you should see that the Django admin panel comes for free automatically. You define your models, hook them up, and that's it. This is an incredible tool that shows how advanced Django's introspection capabilities are. Moreover, it is completely customizable and extendable. It's truly an excellent piece of work.

Adding the Entry model

Now that the boilerplate is out of the way, and we have a couple of users, we're ready to code. We start by adding the Entry model to our application so that we can store objects in the database. Here's the code you'll need to add (remember to use the project tree for reference):

entries/models.py

from django.db import models
from django.contrib.auth.models import User
from django.utils import timezone

class Entry(models.Model):
    user = models.ForeignKey(User)
    pattern = models.CharField(max_length=255)
    test_string = models.CharField(max_length=255)
    date_added = models.DateTimeField(default=timezone.now)

    class Meta:
        verbose_name_plural = 'entries'

This is the model we'll use to store regular expressions in our system. We'll store a pattern, a test string, a reference to the user who created the entry, and the moment of creation. You can see that creating a model is actually quite easy, but nonetheless, let's go through it line by line.

First we need to import the models module from django.db. This will give us the base class for our Entry model. Django models are special classes and much is done for us behind the scenes when we inherit from models.Model.

We want a reference to the user who created the entry, so we need to import the User model from Django's authorization application and we also need to import the timezone model to get access to the timezone.now() function, which provides us with a timezone-aware version of datetime.now(). The beauty of this is that it's hooked up with the TIME_ZONE settings I showed you before.

As for the primary key for this class, if we don't set one explicitly, Django will add one for us. A primary key is a key that allows us to uniquely identify an Entry object in the database (in this case, Django will add an auto-incrementing integer ID).

So, we define our class, and we set up four class attributes. We have a ForeignKey attribute that is our reference to the User model. We also have two CharField attributes that hold the pattern and test strings for our regular expressions. We also have a DateTimeField, whose default value is set to timezone.now. Note that we don't call timezone.now right there, it's now, not now(). So, we're not passing a DateTime instance (set at the moment in time when that line is parsed) rather, we're passing a callable, a function that is called when we save an entry in the database. This is similar to the callback mechanism we used in Chapter 8, The Edges – GUIs and Scripts, when we were assigning commands to button clicks.

The last two lines are very interesting. We define a class Meta within the Entry class itself. The Meta class is used by Django to provide all sorts of extra information for a model. Django has a great deal of logic under the hood to adapt its behavior according to the information we put in the Meta class. In this case, in the admin panel, the pluralized version of Entry would be Entrys, which is wrong, therefore we need to manually set it. We specify the plural all lowercase, as Django takes care of capitalizing it for us when needed.

Now that we have a new model, we need to update the database to reflect the new state of the code. In order to do this, we need to instruct Django that it needs to create the code to update the database. This code is called migration. Let's create it and execute it:

$ python manage.py makemigrations entries
$ python manage.py migrate

After these two instructions, the database will be ready to store Entry objects.

Note

There are two different kinds of migrations: data and schema migration. Data migrations port data from one state to another without altering its structure. For example, a data migration could set all products for a category as out of stock by switching a flag to False or 0. A schema migration is a set of instructions that alter the structure of the database schema. For example, that could be adding an age column to a Person table, or increasing the maximum length of a field to account for very long addresses. When developing with Django, it's quite common to have to perform both kinds of migrations over the course of development. Data evolves continuously, especially if you code in an agile environment.

Customizing the admin panel

The next step is to hook the Entry model up with the admin panel. You can do it with one line of code, but in this case, I want to add some options to customize a bit the way the admin panel shows the entries, both in the list view of all entry items in the database and in the form view that allows us to create and modify them.

All we need to do is to add the following code:

entries/admin.py

from django.contrib import admin
from .models import Entry

@admin.register(Entry)
class EntryAdmin(admin.ModelAdmin):
    fieldsets = [
        ('Regular Expression',
         {'fields': ['pattern', 'test_string']}),
        ('Other Information',
         {'fields': ['user', 'date_added']}),
    ]
    list_display = ('pattern', 'test_string', 'user')
    list_filter = ['user']
    search_fields = ['test_string']

This is simply beautiful. My guess is that you probably already understand most of it, even if you're new to Django.

So, we start by importing the admin module and the Entry model. Because we want to foster code reuse, we import the Entry model using a relative import (there's a dot before models). This will allow us to move or rename the app without too much trouble. Then, we define the EntryAdmin class, which inherits from admin.ModelAdmin. The decoration on the class tells Django to display the Entry model in the admin panel, and what we put in the EntryAdmin class tells Django how to customize the way it handles this model.

Firstly, we specify the fieldsets for the create/edit page. This will divide the page into two sections so that we get a better visualization of the content (pattern and test string) and the other details (user and timestamp) separately.

Then, we customize the way the list page displays the results. We want to see all the fields, but not the date. We also want to be able to filter on the user so that we can have a list of all the entries by just one user, and we want to be able to search on test_string.

I will go ahead and add three entries, one for myself and two on behalf of my father. The result is shown in the next two images. After inserting them, the list page looks like this:

I have highlighted the three parts of this view that we customized in the EntryAdmin class. We can filter by user, we can search and we have all the fields displayed. If you click on a pattern, the edit view opens up.

After our customization, it looks like this:

Notice how we have two sections: Regular Expression and Other Information, thanks to our custom EntryAdmin class. Have a go with it, add some entries to a couple of different users, get familiar with the interface. Isn't it nice to have all this for free?

Creating the form

Every time you fill in your details on a web page, you're inserting data in form fields. A form is a part of the HTML Document Object Model (DOM) tree. In HTML, you create a form by using the form tag. When you click on the submit button, your browser normally packs the form data together and puts it in the body of a POST request. As opposed to GET requests, which are used to ask the web server for a resource, a POST request normally sends data to the web server with the aim of creating or updating a resource. For this reason, handling POST requests usually requires more care than GET requests.

When the server receives data from a POST request, that data needs to be validated. Moreover, the server needs to employ security mechanisms to protect against various types of attacks. One attack that is very dangerous is the cross-site request forgery (CSRF) attack, which happens when data is sent from a domain that is not the one the user is authenticated on. Django allows you to handle this issue in a very elegant way.

So, instead of being lazy and using the Django admin to create the entries, I'm going to show you how to do it using a Django form. By using the tools the framework gives you, you get a very good degree of validation work already done (in fact, we won't need to add any custom validation ourselves).

There are two kinds of form classes in Django: Form and ModelForm. You use the former to create a form whose shape and behavior depends on how you code the class, what fields you add, and so on. On the other hand, the latter is a type of form that, albeit still customizable, infers fields and behavior from a model. Since we need a form for the Entry model, we'll use that one.

entries/forms.py

from django.forms import ModelForm
from .models import Entry

class EntryForm(ModelForm):
    class Meta:
        model = Entry
        fields = ['pattern', 'test_string']

Amazingly enough, this is all we have to do to have a form that we can put on a page. The only notable thing here is that we restrict the fields to only pattern and test_string. Only logged-in users will be allowed access to the insert page, and therefore we don't need to ask who the user is: we know that. As for the date, when we save an Entry, the date_added field will be set according to its default, therefore we don't need to specify that as well. We'll see in the view how to feed the user information to the form before saving. So, all the background work is done, all we need is the views and the templates. Let's start with the views.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Starting the project

Choose a folder in the book's environment and change into that. I'll use ch10. From there, we start a Django project with the following command:

$ django-admin startproject regex

This will prepare the skeleton for a Django project called regex. Change into the regex folder and run the following:

$ python manage.py runserver

You should be able to go to http://127.0.0.1:8000/ with your browser and see the It worked! default Django page. This means that the project is correctly set up. When you've seen the page, kill the server with Ctrl + C (or whatever it says in the console). I'll paste the final structure for the project now so that you can use it as a reference:

$ tree -A regex  # from the ch10 folder
regex
├── db.sqlite3
├── entries
│   ├── admin.py
│   ├── forms.py
│   ├── __init__.py
│   ├── migrations
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   ├── models.py
│   ├── static
│   │   └── entries
│   │       └── css
│   │           └── main.css
│   ├── templates
│   │   └── entries
│   │       ├── base.html
│   │       ├── footer.html
│   │       ├── home.html
│   │       ├── insert.html
│   │       └── list.html
│   └── views.py
├── manage.py
└── regex
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

Don't worry if you're missing files, we'll get there. A Django project is typically a collection of several different applications. Each application is meant to provide a functionality in a self-contained, reusable fashion. We'll create just one, called entries:

$ python manage.py startapp entries

Within the entries folder that has been created, you can get rid of the tests.py module.

Now, let's fix the regex/settings.py file in the regex folder. We need to add our application to the INSTALLED_APPS tuple so that we can use it (add it at the bottom of the tuple):

INSTALLED_APPS = (
    ... django apps ...
    'entries',
)

Then, you may want to fix the language and time zone according to your personal preference. I live in London, so I set them like this:

LANGUAGE_CODE = 'en-gb'
TIME_ZONE = 'Europe/London'

There is nothing else to do in this file, so you can save and close it.

Now it's time to apply the migrations to the database. Django needs database support to handle users, sessions, and things like that, so we need to create a database and populate it with the necessary data. Luckily, this is very easily done with the following command:

$ python manage.py migrate

Note

For this project, we use a SQLite database, which is basically just a file. On a real project, you would probably use a different database engine like MySQL or PostgreSQL.

Creating users

Now that we have a database, we can create a superuser using the console.

$ python manage.py createsuperuser

After entering username and other details, we have a user with admin privileges. This is enough to access the Django admin section, so try and start the server:

$ python manage.py runserver

This will start the Django development server, which is a very useful built-in web server that you can use while working with Django. Now that the server is running, we can access the admin page at http://localhost:8000/admin/. I will show you a screenshot of this section later. If you log in with the credentials of the user you just created and head to the Authentication and Authorization section, you'll find Users. Open that and you will be able to see the list of users. You can edit the details of any user you want as an admin. In our case, make sure you create a different one so that there are at least two users in the system (we'll need them later). I'll call the first user Fabrizio (username: fab) and the second one Adriano (username: adri) in honor of my father.

By the way, you should see that the Django admin panel comes for free automatically. You define your models, hook them up, and that's it. This is an incredible tool that shows how advanced Django's introspection capabilities are. Moreover, it is completely customizable and extendable. It's truly an excellent piece of work.

Adding the Entry model

Now that the boilerplate is out of the way, and we have a couple of users, we're ready to code. We start by adding the Entry model to our application so that we can store objects in the database. Here's the code you'll need to add (remember to use the project tree for reference):

entries/models.py

from django.db import models
from django.contrib.auth.models import User
from django.utils import timezone

class Entry(models.Model):
    user = models.ForeignKey(User)
    pattern = models.CharField(max_length=255)
    test_string = models.CharField(max_length=255)
    date_added = models.DateTimeField(default=timezone.now)

    class Meta:
        verbose_name_plural = 'entries'

This is the model we'll use to store regular expressions in our system. We'll store a pattern, a test string, a reference to the user who created the entry, and the moment of creation. You can see that creating a model is actually quite easy, but nonetheless, let's go through it line by line.

First we need to import the models module from django.db. This will give us the base class for our Entry model. Django models are special classes and much is done for us behind the scenes when we inherit from models.Model.

We want a reference to the user who created the entry, so we need to import the User model from Django's authorization application and we also need to import the timezone model to get access to the timezone.now() function, which provides us with a timezone-aware version of datetime.now(). The beauty of this is that it's hooked up with the TIME_ZONE settings I showed you before.

As for the primary key for this class, if we don't set one explicitly, Django will add one for us. A primary key is a key that allows us to uniquely identify an Entry object in the database (in this case, Django will add an auto-incrementing integer ID).

So, we define our class, and we set up four class attributes. We have a ForeignKey attribute that is our reference to the User model. We also have two CharField attributes that hold the pattern and test strings for our regular expressions. We also have a DateTimeField, whose default value is set to timezone.now. Note that we don't call timezone.now right there, it's now, not now(). So, we're not passing a DateTime instance (set at the moment in time when that line is parsed) rather, we're passing a callable, a function that is called when we save an entry in the database. This is similar to the callback mechanism we used in Chapter 8, The Edges – GUIs and Scripts, when we were assigning commands to button clicks.

The last two lines are very interesting. We define a class Meta within the Entry class itself. The Meta class is used by Django to provide all sorts of extra information for a model. Django has a great deal of logic under the hood to adapt its behavior according to the information we put in the Meta class. In this case, in the admin panel, the pluralized version of Entry would be Entrys, which is wrong, therefore we need to manually set it. We specify the plural all lowercase, as Django takes care of capitalizing it for us when needed.

Now that we have a new model, we need to update the database to reflect the new state of the code. In order to do this, we need to instruct Django that it needs to create the code to update the database. This code is called migration. Let's create it and execute it:

$ python manage.py makemigrations entries
$ python manage.py migrate

After these two instructions, the database will be ready to store Entry objects.

Note

There are two different kinds of migrations: data and schema migration. Data migrations port data from one state to another without altering its structure. For example, a data migration could set all products for a category as out of stock by switching a flag to False or 0. A schema migration is a set of instructions that alter the structure of the database schema. For example, that could be adding an age column to a Person table, or increasing the maximum length of a field to account for very long addresses. When developing with Django, it's quite common to have to perform both kinds of migrations over the course of development. Data evolves continuously, especially if you code in an agile environment.

Customizing the admin panel

The next step is to hook the Entry model up with the admin panel. You can do it with one line of code, but in this case, I want to add some options to customize a bit the way the admin panel shows the entries, both in the list view of all entry items in the database and in the form view that allows us to create and modify them.

All we need to do is to add the following code:

entries/admin.py

from django.contrib import admin
from .models import Entry

@admin.register(Entry)
class EntryAdmin(admin.ModelAdmin):
    fieldsets = [
        ('Regular Expression',
         {'fields': ['pattern', 'test_string']}),
        ('Other Information',
         {'fields': ['user', 'date_added']}),
    ]
    list_display = ('pattern', 'test_string', 'user')
    list_filter = ['user']
    search_fields = ['test_string']

This is simply beautiful. My guess is that you probably already understand most of it, even if you're new to Django.

So, we start by importing the admin module and the Entry model. Because we want to foster code reuse, we import the Entry model using a relative import (there's a dot before models). This will allow us to move or rename the app without too much trouble. Then, we define the EntryAdmin class, which inherits from admin.ModelAdmin. The decoration on the class tells Django to display the Entry model in the admin panel, and what we put in the EntryAdmin class tells Django how to customize the way it handles this model.

Firstly, we specify the fieldsets for the create/edit page. This will divide the page into two sections so that we get a better visualization of the content (pattern and test string) and the other details (user and timestamp) separately.

Then, we customize the way the list page displays the results. We want to see all the fields, but not the date. We also want to be able to filter on the user so that we can have a list of all the entries by just one user, and we want to be able to search on test_string.

I will go ahead and add three entries, one for myself and two on behalf of my father. The result is shown in the next two images. After inserting them, the list page looks like this:

I have highlighted the three parts of this view that we customized in the EntryAdmin class. We can filter by user, we can search and we have all the fields displayed. If you click on a pattern, the edit view opens up.

After our customization, it looks like this:

Notice how we have two sections: Regular Expression and Other Information, thanks to our custom EntryAdmin class. Have a go with it, add some entries to a couple of different users, get familiar with the interface. Isn't it nice to have all this for free?

Creating the form

Every time you fill in your details on a web page, you're inserting data in form fields. A form is a part of the HTML Document Object Model (DOM) tree. In HTML, you create a form by using the form tag. When you click on the submit button, your browser normally packs the form data together and puts it in the body of a POST request. As opposed to GET requests, which are used to ask the web server for a resource, a POST request normally sends data to the web server with the aim of creating or updating a resource. For this reason, handling POST requests usually requires more care than GET requests.

When the server receives data from a POST request, that data needs to be validated. Moreover, the server needs to employ security mechanisms to protect against various types of attacks. One attack that is very dangerous is the cross-site request forgery (CSRF) attack, which happens when data is sent from a domain that is not the one the user is authenticated on. Django allows you to handle this issue in a very elegant way.

So, instead of being lazy and using the Django admin to create the entries, I'm going to show you how to do it using a Django form. By using the tools the framework gives you, you get a very good degree of validation work already done (in fact, we won't need to add any custom validation ourselves).

There are two kinds of form classes in Django: Form and ModelForm. You use the former to create a form whose shape and behavior depends on how you code the class, what fields you add, and so on. On the other hand, the latter is a type of form that, albeit still customizable, infers fields and behavior from a model. Since we need a form for the Entry model, we'll use that one.

entries/forms.py

from django.forms import ModelForm
from .models import Entry

class EntryForm(ModelForm):
    class Meta:
        model = Entry
        fields = ['pattern', 'test_string']

Amazingly enough, this is all we have to do to have a form that we can put on a page. The only notable thing here is that we restrict the fields to only pattern and test_string. Only logged-in users will be allowed access to the insert page, and therefore we don't need to ask who the user is: we know that. As for the date, when we save an Entry, the date_added field will be set according to its default, therefore we don't need to specify that as well. We'll see in the view how to feed the user information to the form before saving. So, all the background work is done, all we need is the views and the templates. Let's start with the views.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Creating users

Now that we have a database, we can create a superuser using the console.

$ python manage.py createsuperuser

After entering username and other details, we have a user with admin privileges. This is enough to access the Django admin section, so try and start the server:

$ python manage.py runserver

This will start the Django development server, which is a very useful built-in web server that you can use while working with Django. Now that the server is running, we can access the admin page at http://localhost:8000/admin/. I will show you a screenshot of this section later. If you log in with the credentials of the user you just created and head to the Authentication and Authorization section, you'll find Users. Open that and you will be able to see the list of users. You can edit the details of any user you want as an admin. In our case, make sure you create a different one so that there are at least two users in the system (we'll need them later). I'll call the first user Fabrizio (username: fab) and the second one Adriano (username: adri) in honor of my father.

By the way, you should see that the Django admin panel comes for free automatically. You define your models, hook them up, and that's it. This is an incredible tool that shows how advanced Django's introspection capabilities are. Moreover, it is completely customizable and extendable. It's truly an excellent piece of work.

Adding the Entry model

Now that the boilerplate is out of the way, and we have a couple of users, we're ready to code. We start by adding the Entry model to our application so that we can store objects in the database. Here's the code you'll need to add (remember to use the project tree for reference):

entries/models.py

from django.db import models
from django.contrib.auth.models import User
from django.utils import timezone

class Entry(models.Model):
    user = models.ForeignKey(User)
    pattern = models.CharField(max_length=255)
    test_string = models.CharField(max_length=255)
    date_added = models.DateTimeField(default=timezone.now)

    class Meta:
        verbose_name_plural = 'entries'

This is the model we'll use to store regular expressions in our system. We'll store a pattern, a test string, a reference to the user who created the entry, and the moment of creation. You can see that creating a model is actually quite easy, but nonetheless, let's go through it line by line.

First we need to import the models module from django.db. This will give us the base class for our Entry model. Django models are special classes and much is done for us behind the scenes when we inherit from models.Model.

We want a reference to the user who created the entry, so we need to import the User model from Django's authorization application and we also need to import the timezone model to get access to the timezone.now() function, which provides us with a timezone-aware version of datetime.now(). The beauty of this is that it's hooked up with the TIME_ZONE settings I showed you before.

As for the primary key for this class, if we don't set one explicitly, Django will add one for us. A primary key is a key that allows us to uniquely identify an Entry object in the database (in this case, Django will add an auto-incrementing integer ID).

So, we define our class, and we set up four class attributes. We have a ForeignKey attribute that is our reference to the User model. We also have two CharField attributes that hold the pattern and test strings for our regular expressions. We also have a DateTimeField, whose default value is set to timezone.now. Note that we don't call timezone.now right there, it's now, not now(). So, we're not passing a DateTime instance (set at the moment in time when that line is parsed) rather, we're passing a callable, a function that is called when we save an entry in the database. This is similar to the callback mechanism we used in Chapter 8, The Edges – GUIs and Scripts, when we were assigning commands to button clicks.

The last two lines are very interesting. We define a class Meta within the Entry class itself. The Meta class is used by Django to provide all sorts of extra information for a model. Django has a great deal of logic under the hood to adapt its behavior according to the information we put in the Meta class. In this case, in the admin panel, the pluralized version of Entry would be Entrys, which is wrong, therefore we need to manually set it. We specify the plural all lowercase, as Django takes care of capitalizing it for us when needed.

Now that we have a new model, we need to update the database to reflect the new state of the code. In order to do this, we need to instruct Django that it needs to create the code to update the database. This code is called migration. Let's create it and execute it:

$ python manage.py makemigrations entries
$ python manage.py migrate

After these two instructions, the database will be ready to store Entry objects.

Note

There are two different kinds of migrations: data and schema migration. Data migrations port data from one state to another without altering its structure. For example, a data migration could set all products for a category as out of stock by switching a flag to False or 0. A schema migration is a set of instructions that alter the structure of the database schema. For example, that could be adding an age column to a Person table, or increasing the maximum length of a field to account for very long addresses. When developing with Django, it's quite common to have to perform both kinds of migrations over the course of development. Data evolves continuously, especially if you code in an agile environment.

Customizing the admin panel

The next step is to hook the Entry model up with the admin panel. You can do it with one line of code, but in this case, I want to add some options to customize a bit the way the admin panel shows the entries, both in the list view of all entry items in the database and in the form view that allows us to create and modify them.

All we need to do is to add the following code:

entries/admin.py

from django.contrib import admin
from .models import Entry

@admin.register(Entry)
class EntryAdmin(admin.ModelAdmin):
    fieldsets = [
        ('Regular Expression',
         {'fields': ['pattern', 'test_string']}),
        ('Other Information',
         {'fields': ['user', 'date_added']}),
    ]
    list_display = ('pattern', 'test_string', 'user')
    list_filter = ['user']
    search_fields = ['test_string']

This is simply beautiful. My guess is that you probably already understand most of it, even if you're new to Django.

So, we start by importing the admin module and the Entry model. Because we want to foster code reuse, we import the Entry model using a relative import (there's a dot before models). This will allow us to move or rename the app without too much trouble. Then, we define the EntryAdmin class, which inherits from admin.ModelAdmin. The decoration on the class tells Django to display the Entry model in the admin panel, and what we put in the EntryAdmin class tells Django how to customize the way it handles this model.

Firstly, we specify the fieldsets for the create/edit page. This will divide the page into two sections so that we get a better visualization of the content (pattern and test string) and the other details (user and timestamp) separately.

Then, we customize the way the list page displays the results. We want to see all the fields, but not the date. We also want to be able to filter on the user so that we can have a list of all the entries by just one user, and we want to be able to search on test_string.

I will go ahead and add three entries, one for myself and two on behalf of my father. The result is shown in the next two images. After inserting them, the list page looks like this:

I have highlighted the three parts of this view that we customized in the EntryAdmin class. We can filter by user, we can search and we have all the fields displayed. If you click on a pattern, the edit view opens up.

After our customization, it looks like this:

Notice how we have two sections: Regular Expression and Other Information, thanks to our custom EntryAdmin class. Have a go with it, add some entries to a couple of different users, get familiar with the interface. Isn't it nice to have all this for free?

Creating the form

Every time you fill in your details on a web page, you're inserting data in form fields. A form is a part of the HTML Document Object Model (DOM) tree. In HTML, you create a form by using the form tag. When you click on the submit button, your browser normally packs the form data together and puts it in the body of a POST request. As opposed to GET requests, which are used to ask the web server for a resource, a POST request normally sends data to the web server with the aim of creating or updating a resource. For this reason, handling POST requests usually requires more care than GET requests.

When the server receives data from a POST request, that data needs to be validated. Moreover, the server needs to employ security mechanisms to protect against various types of attacks. One attack that is very dangerous is the cross-site request forgery (CSRF) attack, which happens when data is sent from a domain that is not the one the user is authenticated on. Django allows you to handle this issue in a very elegant way.

So, instead of being lazy and using the Django admin to create the entries, I'm going to show you how to do it using a Django form. By using the tools the framework gives you, you get a very good degree of validation work already done (in fact, we won't need to add any custom validation ourselves).

There are two kinds of form classes in Django: Form and ModelForm. You use the former to create a form whose shape and behavior depends on how you code the class, what fields you add, and so on. On the other hand, the latter is a type of form that, albeit still customizable, infers fields and behavior from a model. Since we need a form for the Entry model, we'll use that one.

entries/forms.py

from django.forms import ModelForm
from .models import Entry

class EntryForm(ModelForm):
    class Meta:
        model = Entry
        fields = ['pattern', 'test_string']

Amazingly enough, this is all we have to do to have a form that we can put on a page. The only notable thing here is that we restrict the fields to only pattern and test_string. Only logged-in users will be allowed access to the insert page, and therefore we don't need to ask who the user is: we know that. As for the date, when we save an Entry, the date_added field will be set according to its default, therefore we don't need to specify that as well. We'll see in the view how to feed the user information to the form before saving. So, all the background work is done, all we need is the views and the templates. Let's start with the views.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Adding the Entry model

Now that the boilerplate is out of the way, and we have a couple of users, we're ready to code. We start by adding the Entry model to our application so that we can store objects in the database. Here's the code you'll need to add (remember to use the project tree for reference):

entries/models.py

from django.db import models
from django.contrib.auth.models import User
from django.utils import timezone

class Entry(models.Model):
    user = models.ForeignKey(User)
    pattern = models.CharField(max_length=255)
    test_string = models.CharField(max_length=255)
    date_added = models.DateTimeField(default=timezone.now)

    class Meta:
        verbose_name_plural = 'entries'

This is the model we'll use to store regular expressions in our system. We'll store a pattern, a test string, a reference to the user who created the entry, and the moment of creation. You can see that creating a model is actually quite easy, but nonetheless, let's go through it line by line.

First we need to import the models module from django.db. This will give us the base class for our Entry model. Django models are special classes and much is done for us behind the scenes when we inherit from models.Model.

We want a reference to the user who created the entry, so we need to import the User model from Django's authorization application and we also need to import the timezone model to get access to the timezone.now() function, which provides us with a timezone-aware version of datetime.now(). The beauty of this is that it's hooked up with the TIME_ZONE settings I showed you before.

As for the primary key for this class, if we don't set one explicitly, Django will add one for us. A primary key is a key that allows us to uniquely identify an Entry object in the database (in this case, Django will add an auto-incrementing integer ID).

So, we define our class, and we set up four class attributes. We have a ForeignKey attribute that is our reference to the User model. We also have two CharField attributes that hold the pattern and test strings for our regular expressions. We also have a DateTimeField, whose default value is set to timezone.now. Note that we don't call timezone.now right there, it's now, not now(). So, we're not passing a DateTime instance (set at the moment in time when that line is parsed) rather, we're passing a callable, a function that is called when we save an entry in the database. This is similar to the callback mechanism we used in Chapter 8, The Edges – GUIs and Scripts, when we were assigning commands to button clicks.

The last two lines are very interesting. We define a class Meta within the Entry class itself. The Meta class is used by Django to provide all sorts of extra information for a model. Django has a great deal of logic under the hood to adapt its behavior according to the information we put in the Meta class. In this case, in the admin panel, the pluralized version of Entry would be Entrys, which is wrong, therefore we need to manually set it. We specify the plural all lowercase, as Django takes care of capitalizing it for us when needed.

Now that we have a new model, we need to update the database to reflect the new state of the code. In order to do this, we need to instruct Django that it needs to create the code to update the database. This code is called migration. Let's create it and execute it:

$ python manage.py makemigrations entries
$ python manage.py migrate

After these two instructions, the database will be ready to store Entry objects.

Note

There are two different kinds of migrations: data and schema migration. Data migrations port data from one state to another without altering its structure. For example, a data migration could set all products for a category as out of stock by switching a flag to False or 0. A schema migration is a set of instructions that alter the structure of the database schema. For example, that could be adding an age column to a Person table, or increasing the maximum length of a field to account for very long addresses. When developing with Django, it's quite common to have to perform both kinds of migrations over the course of development. Data evolves continuously, especially if you code in an agile environment.

Customizing the admin panel

The next step is to hook the Entry model up with the admin panel. You can do it with one line of code, but in this case, I want to add some options to customize a bit the way the admin panel shows the entries, both in the list view of all entry items in the database and in the form view that allows us to create and modify them.

All we need to do is to add the following code:

entries/admin.py

from django.contrib import admin
from .models import Entry

@admin.register(Entry)
class EntryAdmin(admin.ModelAdmin):
    fieldsets = [
        ('Regular Expression',
         {'fields': ['pattern', 'test_string']}),
        ('Other Information',
         {'fields': ['user', 'date_added']}),
    ]
    list_display = ('pattern', 'test_string', 'user')
    list_filter = ['user']
    search_fields = ['test_string']

This is simply beautiful. My guess is that you probably already understand most of it, even if you're new to Django.

So, we start by importing the admin module and the Entry model. Because we want to foster code reuse, we import the Entry model using a relative import (there's a dot before models). This will allow us to move or rename the app without too much trouble. Then, we define the EntryAdmin class, which inherits from admin.ModelAdmin. The decoration on the class tells Django to display the Entry model in the admin panel, and what we put in the EntryAdmin class tells Django how to customize the way it handles this model.

Firstly, we specify the fieldsets for the create/edit page. This will divide the page into two sections so that we get a better visualization of the content (pattern and test string) and the other details (user and timestamp) separately.

Then, we customize the way the list page displays the results. We want to see all the fields, but not the date. We also want to be able to filter on the user so that we can have a list of all the entries by just one user, and we want to be able to search on test_string.

I will go ahead and add three entries, one for myself and two on behalf of my father. The result is shown in the next two images. After inserting them, the list page looks like this:

I have highlighted the three parts of this view that we customized in the EntryAdmin class. We can filter by user, we can search and we have all the fields displayed. If you click on a pattern, the edit view opens up.

After our customization, it looks like this:

Notice how we have two sections: Regular Expression and Other Information, thanks to our custom EntryAdmin class. Have a go with it, add some entries to a couple of different users, get familiar with the interface. Isn't it nice to have all this for free?

Creating the form

Every time you fill in your details on a web page, you're inserting data in form fields. A form is a part of the HTML Document Object Model (DOM) tree. In HTML, you create a form by using the form tag. When you click on the submit button, your browser normally packs the form data together and puts it in the body of a POST request. As opposed to GET requests, which are used to ask the web server for a resource, a POST request normally sends data to the web server with the aim of creating or updating a resource. For this reason, handling POST requests usually requires more care than GET requests.

When the server receives data from a POST request, that data needs to be validated. Moreover, the server needs to employ security mechanisms to protect against various types of attacks. One attack that is very dangerous is the cross-site request forgery (CSRF) attack, which happens when data is sent from a domain that is not the one the user is authenticated on. Django allows you to handle this issue in a very elegant way.

So, instead of being lazy and using the Django admin to create the entries, I'm going to show you how to do it using a Django form. By using the tools the framework gives you, you get a very good degree of validation work already done (in fact, we won't need to add any custom validation ourselves).

There are two kinds of form classes in Django: Form and ModelForm. You use the former to create a form whose shape and behavior depends on how you code the class, what fields you add, and so on. On the other hand, the latter is a type of form that, albeit still customizable, infers fields and behavior from a model. Since we need a form for the Entry model, we'll use that one.

entries/forms.py

from django.forms import ModelForm
from .models import Entry

class EntryForm(ModelForm):
    class Meta:
        model = Entry
        fields = ['pattern', 'test_string']

Amazingly enough, this is all we have to do to have a form that we can put on a page. The only notable thing here is that we restrict the fields to only pattern and test_string. Only logged-in users will be allowed access to the insert page, and therefore we don't need to ask who the user is: we know that. As for the date, when we save an Entry, the date_added field will be set according to its default, therefore we don't need to specify that as well. We'll see in the view how to feed the user information to the form before saving. So, all the background work is done, all we need is the views and the templates. Let's start with the views.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Customizing the admin panel

The next step is to hook the Entry model up with the admin panel. You can do it with one line of code, but in this case, I want to add some options to customize a bit the way the admin panel shows the entries, both in the list view of all entry items in the database and in the form view that allows us to create and modify them.

All we need to do is to add the following code:

entries/admin.py

from django.contrib import admin
from .models import Entry

@admin.register(Entry)
class EntryAdmin(admin.ModelAdmin):
    fieldsets = [
        ('Regular Expression',
         {'fields': ['pattern', 'test_string']}),
        ('Other Information',
         {'fields': ['user', 'date_added']}),
    ]
    list_display = ('pattern', 'test_string', 'user')
    list_filter = ['user']
    search_fields = ['test_string']

This is simply beautiful. My guess is that you probably already understand most of it, even if you're new to Django.

So, we start by importing the admin module and the Entry model. Because we want to foster code reuse, we import the Entry model using a relative import (there's a dot before models). This will allow us to move or rename the app without too much trouble. Then, we define the EntryAdmin class, which inherits from admin.ModelAdmin. The decoration on the class tells Django to display the Entry model in the admin panel, and what we put in the EntryAdmin class tells Django how to customize the way it handles this model.

Firstly, we specify the fieldsets for the create/edit page. This will divide the page into two sections so that we get a better visualization of the content (pattern and test string) and the other details (user and timestamp) separately.

Then, we customize the way the list page displays the results. We want to see all the fields, but not the date. We also want to be able to filter on the user so that we can have a list of all the entries by just one user, and we want to be able to search on test_string.

I will go ahead and add three entries, one for myself and two on behalf of my father. The result is shown in the next two images. After inserting them, the list page looks like this:

I have highlighted the three parts of this view that we customized in the EntryAdmin class. We can filter by user, we can search and we have all the fields displayed. If you click on a pattern, the edit view opens up.

After our customization, it looks like this:

Notice how we have two sections: Regular Expression and Other Information, thanks to our custom EntryAdmin class. Have a go with it, add some entries to a couple of different users, get familiar with the interface. Isn't it nice to have all this for free?

Creating the form

Every time you fill in your details on a web page, you're inserting data in form fields. A form is a part of the HTML Document Object Model (DOM) tree. In HTML, you create a form by using the form tag. When you click on the submit button, your browser normally packs the form data together and puts it in the body of a POST request. As opposed to GET requests, which are used to ask the web server for a resource, a POST request normally sends data to the web server with the aim of creating or updating a resource. For this reason, handling POST requests usually requires more care than GET requests.

When the server receives data from a POST request, that data needs to be validated. Moreover, the server needs to employ security mechanisms to protect against various types of attacks. One attack that is very dangerous is the cross-site request forgery (CSRF) attack, which happens when data is sent from a domain that is not the one the user is authenticated on. Django allows you to handle this issue in a very elegant way.

So, instead of being lazy and using the Django admin to create the entries, I'm going to show you how to do it using a Django form. By using the tools the framework gives you, you get a very good degree of validation work already done (in fact, we won't need to add any custom validation ourselves).

There are two kinds of form classes in Django: Form and ModelForm. You use the former to create a form whose shape and behavior depends on how you code the class, what fields you add, and so on. On the other hand, the latter is a type of form that, albeit still customizable, infers fields and behavior from a model. Since we need a form for the Entry model, we'll use that one.

entries/forms.py

from django.forms import ModelForm
from .models import Entry

class EntryForm(ModelForm):
    class Meta:
        model = Entry
        fields = ['pattern', 'test_string']

Amazingly enough, this is all we have to do to have a form that we can put on a page. The only notable thing here is that we restrict the fields to only pattern and test_string. Only logged-in users will be allowed access to the insert page, and therefore we don't need to ask who the user is: we know that. As for the date, when we save an Entry, the date_added field will be set according to its default, therefore we don't need to specify that as well. We'll see in the view how to feed the user information to the form before saving. So, all the background work is done, all we need is the views and the templates. Let's start with the views.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Creating the form

Every time you fill in your details on a web page, you're inserting data in form fields. A form is a part of the HTML Document Object Model (DOM) tree. In HTML, you create a form by using the form tag. When you click on the submit button, your browser normally packs the form data together and puts it in the body of a POST request. As opposed to GET requests, which are used to ask the web server for a resource, a POST request normally sends data to the web server with the aim of creating or updating a resource. For this reason, handling POST requests usually requires more care than GET requests.

When the server receives data from a POST request, that data needs to be validated. Moreover, the server needs to employ security mechanisms to protect against various types of attacks. One attack that is very dangerous is the cross-site request forgery (CSRF) attack, which happens when data is sent from a domain that is not the one the user is authenticated on. Django allows you to handle this issue in a very elegant way.

So, instead of being lazy and using the Django admin to create the entries, I'm going to show you how to do it using a Django form. By using the tools the framework gives you, you get a very good degree of validation work already done (in fact, we won't need to add any custom validation ourselves).

There are two kinds of form classes in Django: Form and ModelForm. You use the former to create a form whose shape and behavior depends on how you code the class, what fields you add, and so on. On the other hand, the latter is a type of form that, albeit still customizable, infers fields and behavior from a model. Since we need a form for the Entry model, we'll use that one.

entries/forms.py

from django.forms import ModelForm
from .models import Entry

class EntryForm(ModelForm):
    class Meta:
        model = Entry
        fields = ['pattern', 'test_string']

Amazingly enough, this is all we have to do to have a form that we can put on a page. The only notable thing here is that we restrict the fields to only pattern and test_string. Only logged-in users will be allowed access to the insert page, and therefore we don't need to ask who the user is: we know that. As for the date, when we save an Entry, the date_added field will be set according to its default, therefore we don't need to specify that as well. We'll see in the view how to feed the user information to the form before saving. So, all the background work is done, all we need is the views and the templates. Let's start with the views.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Writing the views

We need to write three views. We need one for the home page, one to display the list of all entries for a user, and one to create a new entry. We also need views to log in and log out. But thanks to Django, we don't need to write them. I'll paste all the code, and then we'll go through it together, step by step.

entries/views.py

import re
from django.contrib.auth.decorators import login_required
from django.contrib.messages.views import SuccessMessageMixin
from django.core.urlresolvers import reverse_lazy
from django.utils.decorators import method_decorator
from django.views.generic import FormView, TemplateView
from .forms import EntryForm
from .models import Entry

class HomeView(TemplateView):
    template_name = 'entries/home.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        return self.render_to_response(context)

class EntryListView(TemplateView):
    template_name = 'entries/list.html'

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        context = self.get_context_data(**kwargs)
        entries = Entry.objects.filter(
            user=request.user).order_by('-date_added')
        matches = (self._parse_entry(entry) for entry in entries)
        context['entries'] = list(zip(entries, matches))
        return self.render_to_response(context)

    def _parse_entry(self, entry):
        match = re.search(entry.pattern, entry.test_string)
        if match is not None:
            return (
                match.group(),
                match.groups() or None,
                match.groupdict() or None
            )
        return None

class EntryFormView(SuccessMessageMixin, FormView):
    template_name = 'entries/insert.html'
    form_class = EntryForm
    success_url = reverse_lazy('insert')
    success_message = "Entry was created successfully"

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def get(self, request, *args, **kwargs):
        return super(EntryFormView, self).get(
            request, *args, **kwargs)

    @method_decorator(
        login_required(login_url=reverse_lazy('login')))
    def post(self, request, *args, **kwargs):
        return super(EntryFormView, self).post(
            request, *args, **kwargs)

    def form_valid(self, form):
        self._save_with_user(form)
        return super(EntryFormView, self).form_valid(form)

    def _save_with_user(self, form):
        self.object = form.save(commit=False)
        self.object.user = self.request.user
        self.object.save()

Let's start with the imports. We need the re module to handle regular expressions, then we need a few classes and functions from Django, and finally, we need the Entry model and the EntryForm form.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

The home view

The first view is HomeView. It inherits from TemplateView, which means that the response will be created by rendering a template with the context we'll create in the view. All we have to do is specify the template_name class attribute to point to the correct template. Django promotes code reuse to a point that if we didn't need to make this view accessible only to logged-in users, the first two lines would have been all we needed.

However, we want this view to be accessible only to logged-in users; therefore, we need to decorate it with login_required. Now, historically views in Django used to be functions; therefore, this decorator was designed to accept a function not a method like we have in this class. We're using Django class-based views in this project so, in order to make things work, we need to transform login_required so that it accepts a method (the difference being in the first argument: self). We do this by passing login_required to method_decorator.

We also need to feed the login_required decorator with login_url information, and here comes another wonderful feature of Django. As you'll see after we're done with the views, in Django, you tie a view to a URL through a pattern, consisting of a regular expression and other information. You can give a name to each entry in the urls.py file so that when you want to refer to a URL, you don't have to hardcode its value into your code. All you have to do is get Django to reverse-engineer that URL from the name we gave to the entry in urls.py defining the URL and the view that is tied to it. This mechanism will become clearer later. For now, just think of reverse('...') as a way of getting a URL from an identifier. In this way, you only write the actual URL once, in the urls.py file, which is brilliant. In the views.py code, we need to use reverse_lazy, which works exactly like reverse with one major difference: it only finds the URL when we actually need it (in a lazy fashion). This is needed when the urls.py file hasn't been loaded yet when the reverse function is used.

The get method, which we just decorated, simply calls the get method of the parent class. Of course, the get method is the method that Django calls when a GET request is performed against the URL tied to this view.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

The entry list view

This view is much more interesting than the previous one. First of all, we decorate the get method as we did before. Inside of it, we need to prepare a list of Entry objects and feed it to the template, which shows it to the user. In order to do so, we start by getting the context dict like we're supposed to do, by calling the get_context_data method of the TemplateView class. Then, we use the ORM to get a list of the entries. We do this by accessing the objects manager, and calling a filter on it. We filter the entries according to which user is logged in, and we ask for them to be sorted in a descending order (that '-' in front of the name specifies the descending order). The objects manager is the default manager every Django model is augmented with on creation, it allows us to interact with the database through its methods.

We parse each entry to get a list of matches (actually, I coded it so that matches is a generator expression). Finally, we add to the context an 'entries' key whose value is the coupling of entries and matches, so that each Entry instance is paired with the resulting match of its pattern and test string.

On the last line, we simply ask Django to render the template using the context we created.

Take a look at the _parse_entry method. All it does is perform a search on the entry.test_string with the entry.pattern. If the resulting match object is not None, it means that we found something. If so, we return a tuple with three elements: the overall group, the subgroups, and the group dictionary. If you're not familiar with these terms, don't worry, you'll see a screenshot soon with an example. We return None if there is no match.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

The form view

Finally, let's examine EntryFormView. This is particularly interesting for a few reasons. Firstly, it shows us a nice example of Python's multiple inheritance. We want to display a message on the page, after having inserted an Entry, so we inherit from SuccessMessageMixin. But we want to handle a form as well, so we also inherit from FormView.

Note

Note that, when you deal with mixins and inheritance, you may have to consider the order in which you specify the base classes in the class declaration.

In order to set up this view correctly, we need to specify a few attributes at the beginning: the template to be rendered, the form class to be used to handle the data from the POST request, the URL we need to redirect the user to in the case of success, and the success message.

Another interesting feature is that this view needs to handle both GET and POST requests. When we land on the form page for the first time, the form is empty, and that is the GET request. On the other hand, when we fill in the form and want to submit the Entry, we make a POST request. You can see that the body of get is conceptually identical to HomeView. Django does everything for us.

The post method is just like get. The only reason we need to code these two methods is so that we can decorate them to require login.

Within the Django form handling process (in the FormView class), there are a few methods that we can override in order to customize the overall behavior. We need to do it with the form_valid method. This method will be called when the form validation is successful. Its purpose is to save the form so that an Entry object is created out of it, and then stored in the database.

The only problem is that our form is missing the user. We need to intercept that moment in the chain of calls and put the user information in ourselves. This is done by calling the _save_with_user method, which is very simple.

Firstly, we ask Django to save the form with the commit argument set to False. This creates an Entry instance without attempting to save it to the database. Saving it immediately would fail because the user information is not there.

The next line updates the Entry instance (self.object), adding the user information and, on the last line, we can safely save it. The reason I called it object and set it on the instance like that was to follow what the original FormView class does.

We're fiddling with the Django mechanism here, so if we want the whole thing to work, we need to pay attention to when and how we modify its behavior, and make sure we don't alter it incorrectly. For this reason, it's very important to remember to call the form_valid method of the base class (we use super for that) at the end of our own customized version, to make sure that every other action that method usually performs is carried out correctly.

Note how the request is tied to each view instance (self.request) so that we don't need to pass it through when we refactor our logic into methods. Note also that the user information has been added to the request automatically by Django. Finally, note that the reason why all the process is split into very small methods like these is so that we can only override those that we need to customize. All this removes the need to write a lot of code.

Now that we have the views covered, let's see how we couple them to the URLs.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Tying up URLs and views

In the urls.py module, we tie each view to a URL. There are many ways of doing this. I chose the simplest one, which works perfectly for the extent of this exercise, but you may want to explore this argument more deeply if you intend to work with Django. This is the core around which the whole website logic will revolve; therefore, you should try to get it down correctly. Note that the urls.py module belongs to the project folder.

regex/urls.py

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.auth import views as auth_views
from django.core.urlresolvers import reverse_lazy
from entries.views import HomeView, EntryListView, EntryFormView

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^entries/$', EntryListView.as_view(), name='entries'),
    url(r'^entries/insert$',
        EntryFormView.as_view(),
        name='insert'),

    url(r'^login/$',
        auth_views.login,
        kwargs={'template_name': 'admin/login.html'},
        name='login'),
    url(r'^logout/$',
        auth_views.logout,
        kwargs={'next_page': reverse_lazy('home')},
        name='logout'),

    url(r'^$', HomeView.as_view(), name='home'),
]

As you can see, the magic comes from the url function. Firstly, we pass it a regular expression; then the view; and finally, a name, which is what we will use in the reverse and reverse_lazy functions to recover the URL.

Note that, when using class-based views, we have to transform them into functions, which is what url is expecting. To do that, we call the as_view() method on them.

Note also that the first url entry, for the admin, is special. Instead of specifying a URL and a view, it specifies a URL prefix and another urls.py module (from the admin.site package). In this way, Django will complete all the URLs for the admin section by prepending 'admin/' to all the URLs specified in admin.site.urls. We could have done the same for our entries app (and we should have), but I feel it would have been a bit too much for this simple project.

In the regular expression language, the '^' and '$' symbols represent the start and end of a string. Note that if you use the inclusion technique, as for the admin, the '$' is missing. Of course, this is because 'admin/' is just a prefix, which needs to be completed by all the definitions in the included urls module.

Something else worth noticing is that we can also include the stringified version of a path to a view, which we do for the login and logout views. We also add information about which templates to use with the kwargs argument. These views come straight from the django.contrib.auth package, by the way, so that we don't need to write a single line of code to handle authentication. This is brilliant and saves us a lot of time.

Each url declaration must be done within the urlpatterns list and on this matter, it's important to consider that, when Django is trying to find a view for a URL that has been requested, the patterns are exercised in order, from top to bottom. The first one that matches is the one that will provide the view for it so, in general, you have to put specific patterns before generic ones, otherwise they will never get a chance to be caught. For example, '^shop/categories/$' needs to come before '^shop' (note the absence of the '$' in the latter), otherwise it would never be called. Our example for the entries works fine because I thoroughly specified URLs using the '$' at the end.

So, models, forms, admin, views and URLs are all done. All that is left to do is take care of the templates. I'll have to be very brief on this part because HTML can be very verbose.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

Writing the templates

All templates inherit from a base one, which provides the HTML structure for all others, in a very OOP type of fashion. It also specifies a few blocks, which are areas that can be overridden by children so that they can provide custom content for those areas. Let's start with the base template:

entries/templates/entries/base.html

{% load static from staticfiles %}
<!DOCTYPE html>
<html lang="en">
  <head>
    {% block meta %}
      <meta charset="utf-8">
      <meta name="viewport"
       content="width=device-width, initial-scale=1.0">
    {% endblock meta %}

    {% block styles %}
      <link href="{% static "entries/css/main.css" %}"
       rel="stylesheet">
    {% endblock styles %}

    <title> {% block title %}Title{% endblock title %} </title>
  </head>

  <body>
    <div id="page-content">
      {% block page-content %}
      {% endblock page-content %}
    </div>
    <div id="footer">
      {% block footer %}
      {% endblock footer %}
    </div>
  </body>
</html>

There is a good reason to repeat the entries folder from the templates one. When you deploy a Django website, you collect all the template files under one folder. If you don't specify the paths like I did, you may get a base.html template in the entries app, and a base.html template in another app. The last one to be collected will override any other file with the same name. For this reason, by putting them in a templates/entries folder and using this technique for each Django app you write, you avoid the risk of name collisions (the same goes for any other static file).

There is not much to say about this template, really, apart from the fact that it loads the static tag so that we can get easy access to the static path without hardcoding it in the template by using {% static ... %}. The code in the special {% ... %} sections is code that defines logic. The code in the special {{ ... }} represents variables that will be rendered on the page.

We define three blocks: title, page-content, and footer, whose purpose is to hold the title, the content of the page, and the footer. Blocks can be optionally overridden by child templates in order to provide different content within them.

Here's the footer:

entries/templates/entries/footer.html

<div class="footer">
  Go back <a href="{% url "home" %}">home</a>.
</div>

It gives us a nice link to the home page.

The home page template is the following:

entries/templates/entries/home.html

{% extends "entries/base.html" %}
{% block title%}Welcome to the Entry website.{% endblock title %}

{% block page-content %}
  <h1>Welcome {{ user.first_name }}!</h1>

  <div class="home-option">To see the list of your entries
    please click <a href="{% url "entries" %}">here.</a>
  </div>
  <div class="home-option">To insert a new entry please click
    <a href="{% url "insert" %}">here.</a>
  </div>
  <div class="home-option">To login as another user please click
    <a href="{% url "logout" %}">here.</a>
  </div>
    <div class="home-option">To go to the admin panel
    please click <a href="{% url "admin:index" %}">here.</a>
  </div>
{% endblock page-content %}

It extends the base.html template, and overrides title and page-content. You can see that basically all it does is provide four links to the user. These are the list of entries, the insert page, the logout page, and the admin page. All of this is done without hardcoding a single URL, through the use of the {% url ... %} tag, which is the template equivalent of the reverse function.

The template for inserting an Entry is as follows:

entries/templates/entries/insert.html

{% extends "entries/base.html" %}
{% block title%}Insert a new Entry{% endblock title %}

{% block page-content %}
  {% if messages %}
    {% for message in messages %}
      <p class="{{ message.tags }}">{{ message }}</p>
    {% endfor %}
  {% endif %}

  <h1>Insert a new Entry</h1>
  <form action="{% url "insert" %}" method="post">
    {% csrf_token %}{{ form.as_p }}
    <input type="submit" value="Insert">
  </form><br>
{% endblock page-content %}

{% block footer %}
  <div><a href="{% url "entries" %}">See your entries.</a></div>
  {% include "entries/footer.html" %}
{% endblock footer %}

There is some conditional logic at the beginning to display messages, if any, and then we define the form. Django gives us the ability to render a form by simply calling {{ form.as_p }} (alternatively, form.as_ul or form.as_table). This creates all the necessary fields and labels for us. The difference between the three commands is in the way the form is laid out: as a paragraph, as an unordered list or as a table. We only need to wrap it in form tags and add a submit button. This behavior was designed for our convenience; we need the freedom to shape that <form> tag as we want, so Django isn't intrusive on that. Also, note that {% csrf_token %}. It will be rendered into a token by Django and will become part of the data sent to the server on submission. This way Django will be able to verify that the request was from an allowed source, thus avoiding the aforementioned cross-site request forgery issue. Did you see how we handled the token when we wrote the view for the Entry insertion? Exactly. We didn't write a single line of code for it. Django takes care of it automatically thanks to a middleware class (CsrfViewMiddleware). Please refer to the official Django documentation to explore this subject further.

For this page, we also use the footer block to display a link to the home page. Finally, we have the list template, which is the most interesting one.

entries/templates/entries/list.html

{% extends "entries/base.html" %}
{% block title%} Entries list {% endblock title %}

{% block page-content %}
 {% if entries %}
  <h1>Your entries ({{ entries|length }} found)</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>

  <table class="entries-table">
   <thead>
     <tr><th>Entry</th><th>Matches</th></tr>
   </thead>
   <tbody>
    {% for entry, match in entries %}
     <tr class="entries-list {% cycle 'light-gray' 'white' %}">
      <td>
        Pattern: <code class="code">
         "{{ entry.pattern }}"</code><br>
        Test String: <code class="code">
         "{{ entry.test_string }}"</code><br>
        Added: {{ entry.date_added }}
      </td>
      <td>
        {% if match %}
         Group: {{ match.0 }}<br>
         Subgroups:
          {{ match.1|default_if_none:"none" }}<br>
         Group Dict: {{ match.2|default_if_none:"none" }}
        {% else %}
         No matches found.
        {% endif %}
      </td>
     </tr>
    {% endfor %}
   </tbody>
  </table>
 {% else %}
  <h1>You have no entries</h1>
  <div><a href="{% url "insert" %}">Insert new entry.</a></div>
 {% endif %}
{% endblock page-content %}

{% block footer %}
 {% include "entries/footer.html" %}
{% endblock footer %}

It may take you a while to get used to the template language, but really, all there is to it is the creation of a table using a for loop. We start by checking if there are any entries and, if so, we create a table. There are two columns, one for the Entry, and the other for the match.

In the Entry column, we display the Entry object (apart from the user) and in the Matches column, we display that 3-tuple we created in the EntryListView. Note that to access the attributes of an object, we use the same dot syntax we use in Python, for example {{ entry.pattern }} or {{ entry.test_string }}, and so on.

When dealing with lists and tuples, we cannot access items using the square brackets syntax, so we use the dot one as well ({{ match.0 }} is equivalent to match[0], and so on.). We also use a filter, through the pipe (|) operator to display a custom value if a match is None.

The Django template language (which is not properly Python) is kept simple for a precise reason. If you find yourself limited by the language, it means you're probably trying to do something in the template that should actually be done in the view, where that logic is more relevant.

Allow me to show you a couple of screenshots of the list and insert templates. This is what the list of entries looks like for my father:

Note how the use of the cycle tag alternates the background color of the rows from white to light gray. Those classes are defined in the main.css file.

The Entry insertion page is smart enough to provide a few different scenarios. When you land on it at first, it presents you with just an empty form. If you fill it in correctly, it will display a nice message for you (see the following picture). However, if you fail to fill in both fields, it will display an error message before them, alerting you that those fields are required.

Note also the custom footer, which includes both a link to the entries list and a link to the home page:

And that's it! You can play around with the CSS styles if you wish. Download the code for the book and have fun exploring and extending this project. Add something else to the model, create and apply a migration, play with the templates, there's lots to do!

Django is a very powerful framework, and offers so much more than what I've been able to show you in this chapter, so you definitely want to check it out. The beauty of it is that it's Python, so reading its source code is a very useful exercise.

The future of web development

Computer science is a very young subject, compared to other branches of science that have existed alongside humankind for centuries or more. One of its main characteristics is that it moves extremely fast. It leaps forward with such speed that, in just a few years, you can see changes that are comparable to real world changes that took a century to happen. Therefore, as a coder, you must pay attention to what happens in this world, all the time.

Something that is happening now is that because powerful computers are now quite cheap and almost everyone has access to them, the trend is to try and avoid putting too much workload on the backend, and let the frontend handle part of it. Therefore, in the last few years, JavaScript frameworks and libraries like jQuery and Backbone have become very popular and web development has shifted from a paradigm where the backend takes care of handling data, preparing it, and serving it to the frontend to display it, to a paradigm where the backend is sometimes just used as an API, a sheer data provider. The frontend fetches the data from the backend with an API call, and then it takes care of the rest. This shift facilitates the existence of paradigms like Single-Page Application (SPA), where, ideally, the whole page is loaded once and then evolves, based on the content that usually comes from the backend. E-commerce websites that load the results of a search in a page that doesn't refresh the surrounding structure, are made with similar techniques. Browsers can perform asynchronous calls (AJAX) that can return data which can be read, manipulated and injected back into the page with JavaScript code.

So, if you're planning to work on web development, I strongly suggest you to get acquainted with JavaScript (if you're not already), and also with APIs. In the last few pages of this chapter, I'll give you an example of how to make a simple API using two different Python microframeworks: Flask and Falcon.

Writing a Flask view

Flask (http://flask.pocoo.org/) is a Python microframework. It provides fewer features than Django, but it's supposedly faster and quicker to get up and running. To be honest, getting Django up and running nowadays is also very quickly done, but Flask is so popular that it's good to see an example of it, nonetheless.

In your ch10 folder, create a flask folder with the following structure:

$ tree -A flask  # from the ch10 folder
flask
├── main.py
└── templates
    └── main.html

Basically, we're going to code two simple files: a Flask application and an HTML template. Flask uses Jinja2 as template engine. It's extremely popular and very fast, and just recently even Django has started to offer native support for it, which is something that Python coders have longed for, for a long time.

flask/templates/main.html

<!doctype html>
<title>Hello from Flask</title>
<h1>
  {% if name %}
    Hello {{ name }}!
  {% else %}
    Hello shy person!
  {% endif %}
</h1>

The template is almost offensively simple; all it does is to change the greeting according to the presence of the name variable. A bit more interesting is the Flask application that renders it:

flask/main.py

from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
@app.route('/<name>')
def hello(name=None):
    return render_template('main.html', name=name)

if __name__ == '__main__':
    app.run()

We create an app object, which is a Flask application. We only feed the fully-qualified name of the module, which is stored in __name__.

Then, we write a simple hello view, which takes an optional name argument. In the body of the view, we simply render the main.html template, passing to it the name argument, regardless of its value.

What's interesting is the routing. Differently from Django's way of tying up views and URLs (the urls.py module), in Flask you decorate your view with one or more @app.route decorators. In this case, we accept both the root URL without anything else, or with name information.

Change into the flask folder and type (make sure you have Flask installed with $ pip install flask):

$ python main.py

You can open a browser and go to http://127.0.0.1:5000/. This URL has no name information; therefore, you will see Hello shy person! It is written all nice and big. Try to add something to that URL like http://127.0.0.1:5000/Adriano. Hit Enter and the page will change to Hello Adriano!.

Of course, Flask offers you much more than this but we don't have the room to see a more complex example. It's definitely worth exploring, though. Several projects use it successfully and it's fun and it is nice to create websites or APIs with it. Flask's author, Armin Ronacher, is a successful and very prolific coder. He also created or collaborated on several other interesting projects like Werkzeug, Jinja2, Click, and Sphinx.

Building a JSON quote server in Falcon

Falcon (http://falconframework.org/) is another microframework written in Python, which was designed to be light, fast and flexible. I think this relatively young project will evolve to become something really popular due to its speed, which is impressive, so I'm happy to show you a tiny example using it.

We're going to build a view that returns a randomly chosen quote from the Buddha.

In your ch10 folder, create a new one called falcon. We'll have two files: quotes.py and main.py. To run this example, install Falcon and Gunicorn ($ pip install falcon gunicorn). Falcon is the framework, and Gunicorn (Green Unicorn) is a Python WSGI HTTP Server for Unix (which, in layman terms, means the technology that is used to run the server). When you're all set up, start by creating the quotes.py file.

falcon/quotes.py

quotes = [
    "Thousands of candles can be lighted from a single candle, "
    "and the life of the candle will not be shortened. "
    "Happiness never decreases by being shared.",
    ...
    "Peace comes from within. Do not seek it without.",
]

You will find the complete list of quotes in the source code for this book. If you don't have it, you can also fill in your favorite quotes. Note that not every line has a comma at the end. In Python, it's possible to concatenate strings like that, as long as they are in brackets (or braces). It's called implicit concatenation.

The code for the main app is not long, but it is interesting:

falcon/main.py

import json
import random
import falcon
from quotes import quotes

class QuoteResource:
    def on_get(self, req, resp):
        quote = {
            'quote': random.choice(quotes),
            'author': 'The Buddha'
        }
        resp.body = json.dumps(quote)

api = falcon.API()
api.add_route('/quote', QuoteResource())

Let's start with the class. In Django we had a get method, in Flask we defined a function, and here we write an on_get method, a naming style that reminds me of C# event handlers. It takes a request and a response argument, both automatically fed by the framework. In its body, we define a dict with a randomly chosen quote, and the author information. Then we dump that dict to a JSON string and set the response body to its value. We don't need to return anything, Falcon will take care of it for us.

At the end of the file, we create the Falcon application, and we call add_route on it to tie the handler we have just written to the URL we want.

When you're all set up, change to the falcon folder and type:

$ gunicorn main:api

Then, make a request (or simply open the page with your browser) to http://127.0.0.1:8000/quote. When I did it, I got this JSON in response:

{
    quote: "The mind is everything. What you think you become.",
    author: "The Buddha"
}

Whatever the framework you end up using for your web development, try and keep yourself informed about other choices too. Sometimes you may be in situations where a different framework is the right way to go, and having a working knowledge of different tools will give you an advantage.

Writing a Flask view

Flask (http://flask.pocoo.org/) is a Python microframework. It provides fewer features than Django, but it's supposedly faster and quicker to get up and running. To be honest, getting Django up and running nowadays is also very quickly done, but Flask is so popular that it's good to see an example of it, nonetheless.

In your ch10 folder, create a flask folder with the following structure:

$ tree -A flask  # from the ch10 folder
flask
├── main.py
└── templates
    └── main.html

Basically, we're going to code two simple files: a Flask application and an HTML template. Flask uses Jinja2 as template engine. It's extremely popular and very fast, and just recently even Django has started to offer native support for it, which is something that Python coders have longed for, for a long time.

flask/templates/main.html

<!doctype html>
<title>Hello from Flask</title>
<h1>
  {% if name %}
    Hello {{ name }}!
  {% else %}
    Hello shy person!
  {% endif %}
</h1>

The template is almost offensively simple; all it does is to change the greeting according to the presence of the name variable. A bit more interesting is the Flask application that renders it:

flask/main.py

from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
@app.route('/<name>')
def hello(name=None):
    return render_template('main.html', name=name)

if __name__ == '__main__':
    app.run()

We create an app object, which is a Flask application. We only feed the fully-qualified name of the module, which is stored in __name__.

Then, we write a simple hello view, which takes an optional name argument. In the body of the view, we simply render the main.html template, passing to it the name argument, regardless of its value.

What's interesting is the routing. Differently from Django's way of tying up views and URLs (the urls.py module), in Flask you decorate your view with one or more @app.route decorators. In this case, we accept both the root URL without anything else, or with name information.

Change into the flask folder and type (make sure you have Flask installed with $ pip install flask):

$ python main.py

You can open a browser and go to http://127.0.0.1:5000/. This URL has no name information; therefore, you will see Hello shy person! It is written all nice and big. Try to add something to that URL like http://127.0.0.1:5000/Adriano. Hit Enter and the page will change to Hello Adriano!.

Of course, Flask offers you much more than this but we don't have the room to see a more complex example. It's definitely worth exploring, though. Several projects use it successfully and it's fun and it is nice to create websites or APIs with it. Flask's author, Armin Ronacher, is a successful and very prolific coder. He also created or collaborated on several other interesting projects like Werkzeug, Jinja2, Click, and Sphinx.

Building a JSON quote server in Falcon

Falcon (http://falconframework.org/) is another microframework written in Python, which was designed to be light, fast and flexible. I think this relatively young project will evolve to become something really popular due to its speed, which is impressive, so I'm happy to show you a tiny example using it.

We're going to build a view that returns a randomly chosen quote from the Buddha.

In your ch10 folder, create a new one called falcon. We'll have two files: quotes.py and main.py. To run this example, install Falcon and Gunicorn ($ pip install falcon gunicorn). Falcon is the framework, and Gunicorn (Green Unicorn) is a Python WSGI HTTP Server for Unix (which, in layman terms, means the technology that is used to run the server). When you're all set up, start by creating the quotes.py file.

falcon/quotes.py

quotes = [
    "Thousands of candles can be lighted from a single candle, "
    "and the life of the candle will not be shortened. "
    "Happiness never decreases by being shared.",
    ...
    "Peace comes from within. Do not seek it without.",
]

You will find the complete list of quotes in the source code for this book. If you don't have it, you can also fill in your favorite quotes. Note that not every line has a comma at the end. In Python, it's possible to concatenate strings like that, as long as they are in brackets (or braces). It's called implicit concatenation.

The code for the main app is not long, but it is interesting:

falcon/main.py

import json
import random
import falcon
from quotes import quotes

class QuoteResource:
    def on_get(self, req, resp):
        quote = {
            'quote': random.choice(quotes),
            'author': 'The Buddha'
        }
        resp.body = json.dumps(quote)

api = falcon.API()
api.add_route('/quote', QuoteResource())

Let's start with the class. In Django we had a get method, in Flask we defined a function, and here we write an on_get method, a naming style that reminds me of C# event handlers. It takes a request and a response argument, both automatically fed by the framework. In its body, we define a dict with a randomly chosen quote, and the author information. Then we dump that dict to a JSON string and set the response body to its value. We don't need to return anything, Falcon will take care of it for us.

At the end of the file, we create the Falcon application, and we call add_route on it to tie the handler we have just written to the URL we want.

When you're all set up, change to the falcon folder and type:

$ gunicorn main:api

Then, make a request (or simply open the page with your browser) to http://127.0.0.1:8000/quote. When I did it, I got this JSON in response:

{
    quote: "The mind is everything. What you think you become.",
    author: "The Buddha"
}

Whatever the framework you end up using for your web development, try and keep yourself informed about other choices too. Sometimes you may be in situations where a different framework is the right way to go, and having a working knowledge of different tools will give you an advantage.

Building a JSON quote server in Falcon

Falcon (http://falconframework.org/) is another microframework written in Python, which was designed to be light, fast and flexible. I think this relatively young project will evolve to become something really popular due to its speed, which is impressive, so I'm happy to show you a tiny example using it.

We're going to build a view that returns a randomly chosen quote from the Buddha.

In your ch10 folder, create a new one called falcon. We'll have two files: quotes.py and main.py. To run this example, install Falcon and Gunicorn ($ pip install falcon gunicorn). Falcon is the framework, and Gunicorn (Green Unicorn) is a Python WSGI HTTP Server for Unix (which, in layman terms, means the technology that is used to run the server). When you're all set up, start by creating the quotes.py file.

falcon/quotes.py

quotes = [
    "Thousands of candles can be lighted from a single candle, "
    "and the life of the candle will not be shortened. "
    "Happiness never decreases by being shared.",
    ...
    "Peace comes from within. Do not seek it without.",
]

You will find the complete list of quotes in the source code for this book. If you don't have it, you can also fill in your favorite quotes. Note that not every line has a comma at the end. In Python, it's possible to concatenate strings like that, as long as they are in brackets (or braces). It's called implicit concatenation.

The code for the main app is not long, but it is interesting:

falcon/main.py

import json
import random
import falcon
from quotes import quotes

class QuoteResource:
    def on_get(self, req, resp):
        quote = {
            'quote': random.choice(quotes),
            'author': 'The Buddha'
        }
        resp.body = json.dumps(quote)

api = falcon.API()
api.add_route('/quote', QuoteResource())

Let's start with the class. In Django we had a get method, in Flask we defined a function, and here we write an on_get method, a naming style that reminds me of C# event handlers. It takes a request and a response argument, both automatically fed by the framework. In its body, we define a dict with a randomly chosen quote, and the author information. Then we dump that dict to a JSON string and set the response body to its value. We don't need to return anything, Falcon will take care of it for us.

At the end of the file, we create the Falcon application, and we call add_route on it to tie the handler we have just written to the URL we want.

When you're all set up, change to the falcon folder and type:

$ gunicorn main:api

Then, make a request (or simply open the page with your browser) to http://127.0.0.1:8000/quote. When I did it, I got this JSON in response:

{
    quote: "The mind is everything. What you think you become.",
    author: "The Buddha"
}

Whatever the framework you end up using for your web development, try and keep yourself informed about other choices too. Sometimes you may be in situations where a different framework is the right way to go, and having a working knowledge of different tools will give you an advantage.

Summary

In this chapter, we caught a glimpse of web development. We talked about important concepts like the DRY philosophy and the concept of a framework as a tool that provides us with many things we need in order to write code to serve requests. We also talked about the MTV pattern, and how nicely these three layers play together to realize a request-response path.

Later on, we briefly introduced regular expressions, which is a subject of paramount importance, and it's the layer which provides the tools for URL routing.

There are many different frameworks out there, and Django is definitely one of the best and most widely used, so it's definitely worth exploring, especially its source code, which is very well written.

There are other very interesting and important frameworks too, like Flask. They provide fewer features but, in general, they are faster, both in execution time and to set up. One that is extremely fast is the relatively young Falcon project, whose benchmarks are outstanding.

It's important to get a solid understanding of how the request-response mechanism works, and how the Web in general works, so that eventually it won't matter too much which framework you have to use. You will be able to pick it up quickly because it will only be a matter of getting familiar with a way of doing something you already know a lot about.

Explore at least three frameworks and try to come up with different use cases to decide which one of them could be the ideal choice. When you are able to make that choice, you will know you have a good enough understanding of them.

The next chapter is about debugging and troubleshooting. We'll learn how to deal with errors and issues so that if you get in trouble when coding (don't worry, normally it only happens about all the time), you will be able to quickly find the solution to your problem and move on.