Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds

How-To Tutorials - Web Design

132 Articles
article-image-installing-drupal-7
Packt
24 Nov 2010
9 min read
Save for later

Installing Drupal 7

Packt
24 Nov 2010
9 min read
Drupal 7 First Look Learn the new features of Drupal 7, how they work and how they will impact you Get to grips with all of the new features in Drupal 7 Upgrade your Drupal 6 site, themes, and modules to Drupal 7 Explore the new Drupal 7 administration interface and map your Drupal 6 administration interface to the new Drupal 7 structure Complete coverage of the DBTNG database layer with usage examples and all API changes for both Themes and Modules         Read more about this book       (For more resources on Drupal, see here.) Drupal's installation process has always been very easy to use, and the Drupal 7 installation makes things even easier. Before beginning to install Drupal 7, you will need a web server running the Apache HTTPD web server. You can also use IIS on Microsoft Windows, but the Apache server is preferred and you will be able to obtain support from the community more easily if you use the Apache server. Want to easily install Apache onto a Microsoft Windows machine? Try XAMPP, which is published by Apache Friends. This package includes Apache, MySQL, and PHP with a standard Microsoft Windows installer. You can download XAMPP from http://www.apachefriends. org/en/xampp.html. Other options include WAMP (http://www. wampserver.com/en/) and MoWeS Portable (http://www. chsoftware.net/en/mowes/mowesportable/mowes.htm). Your server will also need PHP installed on it. Drupal requires at least PHP version 5.2.0. As of this writing, there are some hosts that still do not have PHP 5.2.0 or later installed on their shared hosting accounts, and Red Hat does not include PHP 5.2.0 or later in its default distribution. Check with your host or system administrator before installing Drupal to make sure that the correct version is available. In addition to the web server and PHP, you will also need a database. MySQL and PostgreSQL are the databases that are most frequently used with Drupal, and of the two, MySQL is much more widely used. That being said, you can use Drupal with many different databases and the new DBTNG database abstraction layer will make it easier to deploy to any database. If you are using MySQL, you will need version 5.0.15 or later installed. If you are using PostgreSQL, you will need PostgreSQL 8.3.0 or later. SQLite is also officially supported for use with Drupal and you will need version 3.4.2 or later. After you have a server set up with the proper software, you can download Drupal and begin the installation process. Obtaining Drupal If you have used previous versions of Drupal, the process for downloading Drupal is the same as always. If you are new to Drupal, you will use the following process: Go to the Drupal project page on Drupal.org: http://drupal.org/project/ drupal. Find the latest official release of Drupal 7 and click on the Download link. The release will be named 7.0 or similar. Your browser will ask whether you want to download or Open the file. Make sure to download it to your computer. The file you downloaded is a .tar.gz file, which is a compressed archive similar to a .zip file. You will need to extract the files from this archive onto your computer. If your computer doesn't already have a program that can open .tar.gz files, try 7-Zip, an open source application that easily handles these files. You can download 7-Zip from http://www.7-zip.org. After you have extracted the files, you will need to copy them to your web server's document root. You are now ready to start the installation process. Simply navigate to http://yoursite.com/install.php. Let's step through the installation process in detail now. Selecting an installation profile The first step in the installation process is selecting an installation profile. Drupal prompts you with a screen asking for which installation profile you want to use during the installation: By default, Drupal comes with two installation profiles, the Standard profile and the Minimal profile. Custom distributions may come with additional profiles. Minimal profile The Minimal profile installs a basic configuration of Drupal with only the required functionality enabled. This profile is even more minimal than the base Drupal 6 installation. This profile should be used if you are very familiar with setting up Drupal and don't want some of the additional features activated in the Standard profile. Standard profile The Standard Drupal profile installs and activates several commonly-used features to make your Drupal site more useful immediately. These additional features include: Search form installed on the left sidebar. Powered by Drupal block enabled in the footer. A basic page content type is automatically created to store static content on your site. An article content type is automatically created to store time-specific content. The article content type replaces the story content type from Drupal 6. Both content types are set up with RDF capabilities. User profiles have pictures enabled by default. Profile pictures can have a maximum size of 1024x1024 pixels and be up to 800 KB when they are uploaded. They will be displayed using the thumbnail image style. A taxonomy called Tags is created to allow easy categorization of content on your site. The article content type is enhanced by adding an image field, which allows PNG, GIF, and JPG files to be attached to the article. An administrator role is created that has all permissions activated for it. As new modules are activated, the administrator role will automatically be updated with the permissions for the new module. The Seven theme is activated for the administration section of the site. In most cases, you will want to start with the Standard installation profile, especially if you are setting up an entirely new site or if you are new to Drupal. Language selection The next step in the installation is choosing the language with which you want to install Drupal. By default, Drupal only includes an English installer. If you want to want to install Drupal in another language, you will need to download a translation from Drupal.org. A complete list of translations is available at http://drupal.org/ project/translations. After you download the translation you want to use, you will need to unpack the translation and copy it to your document folder. The process to unpack and copy the files is similar to the process we used when we unpacked and copied the core Drupal files to your server. For now, we will continue with the English installation. Requirements check Drupal will now check the requirements of your server to ensure that it meets the minimum requirements to run Drupal and to ensure that everything is ready for the installation to proceed. The requirements check will appear similar to the following: If Drupal does discover any problems, it will give you information about how to correct the problem. In our case, it looks like we forgot to set up our settings file. The settings file tells Drupal which database to connect to as well as the connection information. To create a settings file, navigate to your document root and then navigate to the sites/default folder. Copy the default.settings.php file to settings.php. You do not need to change any of the information within the file. After you have corrected any problems, click on the proceed with the installation link. Drupal will re-evaluate the requirements and let you know if anything else needs to be changed. This screen has been enhanced in Drupal 7 to provide much more information about your current server settings. Database configuration The next step in installing Drupal is configuring the database where Drupal will store the content and configuration information for your site. The functionality of this screen has also been enhanced in Drupal 7. The key difference is that Drupal 7 will automatically check which types of databases are available to you based on your server setup. Then, it will only allow you to select a database which will work. If you want to run Drupal using a different database server than your web server, you can use the ADVANCED OPTIONS link to configure the database server and port. You can also use ADVANCED OPTIONS if you are setting up multiple sites within a single database. For a Standard installation, enter the name of your database as well as the username and password for the database. This functionality remains the same as in Drupal 6. You will need to create a database outside of the Drupal installation. The actual steps for creating a new database vary depending on your website host. Many hosts have installed phpMyAdmin, which allows you to manage your databases with an easy-to-use web-based interface. If you use phpMyAdmin to create your database, you will need to log in to phpMyAdmin and create a database. You can create a new database from the home page, which should appear similar to the following screenshot depending on the version of phpMyAdmin you are using: You can create a new user for the database in the Privileges tab. After you have entered your database settings, click on the Save and continue button. Drupal will now configure the database and set up your site. As the installation proceeds, Drupal will display its progress. The installation may take several minutes to complete. In the unlikely event that you have problems during the installation, try emptying the database, increasing the amount of memory available to Drupal, and increasing the maximum execution time for a PHP script. You can increase the available memory and execution time in your php.ini file. The relevant sections in php.ini to control memory and execution time are shown in the following screenshot:
Read more
  • 0
  • 0
  • 2005

article-image-getting-started-selenium-grid
Packt
23 Nov 2010
6 min read
Save for later

Getting Started with Selenium Grid

Packt
23 Nov 2010
6 min read
Important preliminary points For this section you will need to have Apache Ant on the machine that you are going to have running Grid instances. You can get this from http://ant.apache.org/bindownload.cgi for Windows and Mac. If you have Ubuntu you can simply do sudo apt-get install ant1.8, which will install all the relevant items that are needed onto your Linux machine. Visit the project site to download Selenium Grid. Understanding Selenium Grid Selenium Grid is a version of Selenium that allows teams to set up a number of Selenium instances and then have one central point to send your Selenium commands to. This differs from what we saw in Selenium Remote Control (RC) where we always had to explicitly say where the Selenium RC is as well as know what browsers that Remote Control can handle. With Selenium Grid, we just ask for a specific browser, and then the hub that is part of Selenium Grid will route all the Selenium commands through to the Remote Control you want. Selenium Grid also allows us to, with the help of the configuration file, assign friendly names to the Selenium RC instances so that when the tests want to run against Firefox on Linux, the hub will find a free instance and then route all the Selenium Commands from your test through to the instance that is registered with that environment. We can see an example of this in the next diagram. We will see how to create tests for this later in the chapter, but for now let's have a look at making sure we have all the necessary items ready for the grid. Checking that we have the necessary items for Selenium Grid Now that you have downloaded Selenium Grid and Ant, it is always good to run a sanity check on Selenium Grid to make sure that we are ready to go. To do this we run a simple command in a console or Command Prompt. Let's see this in action. Time for action – doing a sanity check on Selenium Grid Open a Command Prompt or console window. Run the command ant sanity-check. When it is complete you should see something similar to the next screenshot: What just happened? We have just checked whether we have all the necessary items to run Selenium Grid. If there was something that Selenium relied on, the sanity check script would output what was needed so that you could easily correct this. Now that everything is ready, let us start setting up the Grid. Selenium Grid Hub Selenium Grid works by having a central point that tests can connect to, and commands are then pushed to the Selenium Remote Control instances connected to that hub. The hub has a web interface that tells you about the Selenium Remote Control instances that are connected to the Hub, and whether they are currently in use. Time for action – launching the hub Now that we are ready to start working with Selenium Grid we need to set up the Grid. This is a simple command that we run in the console or Command Prompt. Open a Command Prompt or console window. Run the command ant launch-hub. When that happens you should see something similar to the following screenshot: We can see that this is running in the command prompt or console. We can also see the hub running from within a browser. If we put http://nameofmachine:4444/console where nameofmachine is the name of the machine with the hub. If it is on your machine then you can place http://localhost:4444/console. We can see that in the next screenshot: What just happened? We have successfully started Selenium Grid Hub. This is the central point of our tests and Selenium Grid instances. We saw that when we start Selenium Grid it showed us what items were available according to the configuration file that is with the normal install. We then had a look at how we can see what the Grid is doing by having a look at the hub in a browser. We did this by putting the URL http://nameofmachine:4444/console where nameofmachine is the name of the machine that we would like to access with the hub. It shows what configured environments the hub can handle, what grid instances are available and which instances are currently active. Now that we have the hub ready we can have a look at starting up instances. Adding instances to the hub Now that we have successfully started the Selenium Grid Hub, we will need to have a look at how we can start adding Selenium Remote Controls to the hub so that it starts forming the grid of computers that we are expecting. As with everything in Selenium Grid, we need Ant to start the instances that connect. In the next few Time for action sections we will see the different arguments needed to start instances to join the grid. Time for action – adding a remote control with the defaults In this section we are going to launch Selenium Remote Control and get it to register with the hub. We are going to assume that the browser you would like it to register for is Firefox, and the hub is on the same machine as the Remote Control. We will pass in only one required argument, which is the port that we wish it to run on. However, when starting instances, we will always need to pass in the port since Selenium cannot work out if there are any free ports on the host machine. Open a Command Prompt or console window. Enter the command ant –Dport=5555 launch-remote-control and press Return. You should see the following in your Command Prompt or console: And this in the Selenium Grid Hub site: What just happened? We have added the first machine to our own Selenium Grid. It has used all the defaults that are in the Ant build script and it has created a Selenium Remote Control that will take any Firefox requests, located on the same machine as the host of Selenium Remote Control Grid. This is a useful way to set up the grid if you just want a large number of Firefox-controlling Selenium Remote Controls.
Read more
  • 0
  • 0
  • 2078

article-image-first-steps-selenium-rc
Packt
23 Nov 2010
4 min read
Save for later

First Steps with Selenium RC

Packt
23 Nov 2010
4 min read
Selenium 1.0 Testing Tools: Beginner’s Guide Important preliminary points To complete the examples of this article you will need to make sure that you have at least Java JRE installed. You can download it from http://java.sun.com. Selenium Remote Control has been written in Java to allow it to be cross platform, so we can test on Mac, Linux, and Windows. What is Selenium Remote Control Selenium IDE only works with Firefox so we have only been checking a small subsection of the browsers that our users prefer. We, as web developers and testers, know that unfortunately our users do not just use one browser. Some may use Internet Explorer, others may use Mozilla Firefox. This is not to mention the growth of browsers such as Google Chrome and Opera. Selenium Remote Control was initially developed by Patrick Lightbody as a way to test all of these different web browsers without having to install Selenium Core on the web server. It was developed to act as a proxy between the application under test and the test scripts. Selenium Core is bundled with Selenium Remote Control instead of being installed on the server. This change to the way that Selenium tests are run allowed developers to interact with the proxy directly giving developers and testers a chance to use one of the most prominent programming languages to send commands to the browser. Java and C# have been the main languages used by developers to create Selenium Tests. This is due to most web applications being created in one of those languages. We have seen language bindings for dynamic languages being created and supported as more developers move their web applications to those languages. Ruby and Python are the most popular languages that people are moving to. Using programming languages to write your tests instead of using the HTML-style tests with Selenium IDE allows you, as a developer or tester, to make your tests more robust and take advantage of all setups and tear down those that are common in most testing frameworks. Now that we understand how Selenium Remote Control works, let us have a look at setting it up. Setting up Selenium Remote Control Selenium Remote Control is required on all machines that will be used to run tests. It is good practice to limit the number of Selenium Remote Control instances to one per CPU core. This is due to the fact that web applications are becoming more "chatty" since we use more AJAX in them. Limiting the Selenium instances to one per core makes sure that the browsers load cleanly and Selenium will run as quickly as possible. Time for action – setting up Selenium Remote Control Download Selenium Remote Control from http://seleniumhq.org/download. Extract the ZIP file. Start a Command Prompt or a console window and navigate to where the ZIP file was extracted. Run the command java –jar selenium-server-standalone.jar and the output should appear similar to the following screenshot: What just happened? We have successfully set up Selenium Remote Control. This is the proxy that our tests will communicate with. It works by language bindings, sending commands through to Selenium Remote Control which it then passes through to the relevant browser. It does this by keeping track of browsers by having a unique ID attached to the browser, and each command needs to have that ID in the request. Now that we have finished setting up Selenium Remote Control we can have a look at running our first set of tests in a number of different browsers. Pop quiz – setting up Selenium Remote Control Where can you download Selenium Remote Control from? Once you have placed Selenium Remote Control somewhere accessible, how do you start Selenium Remote Control? Running Selenium IDE tests with Selenium Remote Control The Selenium IDE to create all the tests have only been tested on applications in Firefox. This means the testing coverage that you are offering is very limited. Users will use a number of different browsers to interact with your application. Browser and operating system combinations can mean that a developer or tester will have to run your tests more than nine times. This is to make sure that you cover all the popular browser and operating system combinations. Now let's have a look at running the IDE tests with Selenium Remote Control.
Read more
  • 0
  • 0
  • 6757

article-image-administrating-mysql-server-phpmyadmin
Packt
13 Oct 2010
8 min read
Save for later

Administrating the MySQL Server with phpMyAdmin

Packt
13 Oct 2010
8 min read
  Mastering phpMyAdmin 3.3.x for Effective MySQL Management A complete guide to get started with phpMyAdmin 3.3 and master its features The best introduction to phpMyAdmin available Written by the project leader of phpMyAdmin, and improved over several editions A step-by-step tutorial for manipulating data with phpMyAdmin Learn to do things with your MySQL database and phpMyAdmin that you didn't know were possible! Managing users and their privileges The Privileges subpage (visible only if we are logged in as a privileged user) contains dialogs to manage MySQL user accounts. It also contains dialogs to manage privileges on the global, database, and table levels. This subpage is hierarchical. For example, when editing a user's privileges, we can see the global privileges as well as the database-specific privileges. We can then go deeper to see the table-specific privileges for this database-user combination. The user overview The first page displayed when we enter the Privileges subpage is called User verview. This shows all user accounts and a summary of their global privileges, as shown in the next screenshot: From this page, we can: Edit a user's privileges, via the Edit link for this user Use the checkboxes to remove users, via the Remove selected users dialog Access the page when the Add a new User dialog is available The displayed users' list has columns with the following characteristics: Privileges reload At the bottom of User Overview, the following message is displayed: Note: phpMyAdmin gets the users' privileges directly from MySQL's privilege tables. The content of these tables may differ from the privileges the server uses, if they have been changed manually. In this case, you should reload the privileges before you continue. Here, the text reload the privileges is clickable. The effective privileges (the ones against which the server bases its access decisions) are the privileges that are located in the server's memory. Privilege modifications that are made from the User overview page are made both in memory and on disk, in the mysql database. Modifications made directly to the mysql database do not have immediate effect. The reload the privileges operation reads the privileges from the database and makes them effective in memory. Adding a user The Add a new User link opens a dialog for user account creation. First, we see the panel where we'll describe the account itself: The second part of the Add a new User dialog is where we'll specify the user's global privileges, which apply to the server as a whole. Entering the username The User name menu offers two choices. Firstly, we can choose Use text field and enter a username in the box, or we can choose Any user to create an anonymous user (the blank user). Let's choose Use text field and enter bill. Assigning a host value By default, this menu is set to Any host, with % as the host value. The Local choice means "localhost". The Use host table choice (which creates a blank value in the host field) means to look in the mysql.hosts table for database-specific privileges. Choosing Use text field allows us to enter the exact host value we want. Let's choose Local. Setting passwords Even though it's possible to create a user without a password (by selecting the No password option), it's best to have a password. We have to enter it twice (as we cannot see what is entered) to confirm the intended password. A secure password should have more than eight characters, and should contain a mixture of uppercase and lowercase characters, digits, and special characters. Therefore, it's recommended to have phpMyAdmin generate a password—this is possible in JavaScript-enabled browsers. In the Generate Password dialog, clicking on Generate enters a random password (in clear text) on the screen and fills the Password and Re-type input fields with the generated password. At this point, we should note the password so that we can pass it on to the user. Understanding rights for database creation A frequent convention is to assign a user the rights to a database having the same name as this user. To accomplish this, the Database for user section offers the checkbox Create database with same name and grant all privileges. Selecting this checkbox automates the process by creating both the database (if it does not already exist) and the corresponding rights. Please note that, with this method, each user would be limited to one database (user bill, database bill). Another possibility is to allow users to create databases that have the same prefix as their usernames. Therefore, the other choice, Grant all privileges on wildcard name (username_%), performs this function by assigning a wildcard privilege. With this in place, user bill could create the databases bill_test, bill_2, bill_payroll, and so on; phpMyAdmin does not pre-create the databases in this case. Assigning global privileges Global privileges determine the user's access to all databases. Hence, these are sometimes known as "superuser privileges". A normal user should not have any of these privileges unless there is a good reason for this. Of course, if we are really creating a superuser, we will select every global privilege that he or she needs. These privileges are further divided into Data, Structure, and Administration groups. In our example, bill will not have any global privileges. Limiting the resources used We can limit the resources used by this user on this server (for example, the maximum queries per hour). Zero means no limit. We will not impose any resource limits on bill. The following screenshot shows the status of the screen just before hitting Go to create this user's definition (with the remaining fields being set to default): Editing a user profile The page used to edit a user's profile appears after a user's creation, or whenever we click on Edit for a user in the User overview page. There are four sections on this page, each with its own Go button. Hence, each section is operated independently and has a distinct purpose. Editing privileges The section for editing the user's privileges has the same look as the Add a new User dialog, and is used to view and change global privileges. Assigning database-specific privileges In this section, we define the databases to which our user has access, and his or her exact privileges on these databases. As shown in the previous screenshot, we see None because we haven't defined any privileges yet. There are two ways of defining database privileges. First, we can choose one of the existing databases from the drop-down menu: This assigns privileges only for the chosen database. We can also choose Use text field and enter a database name. We could enter a non-existent database name, so that the user can create it later (provided that we give him or her the CREATE privilege in the next panel). We can also use special characters, such as the underscore and the percent sign, for wildcards. For example, entering bill here would enable him to create a bill database, and entering bill% would enable him to create a database with any name that starts with bill. For our example, we will enter bill and then click on Go. The next screen is used to set bill's privileges on the bill database, and create table-specific privileges. To learn more about the meaning of a specific privilege, we can move the mouse over a privilege name (which is always in English), and an explanation about this privilege appears in the current language. We give SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, and DROP privileges to bill on this database. We then click on Go. After the privileges have been assigned, the interface stays at the same place, so that we can refine these privileges further. We cannot assign table-specific privileges for the moment, as the database does not yet exist. To go back to the general privileges page of bill, click on the 'bill'@'localhost' title. This brings us back to the following, familiar page, except for a change in one section: We see the existing privileges (which we can Edit or Revoke) on the bill database for user bill, and we can add privileges for bill on another database. We can also see that bill has no table-specific privileges on the bill database. Changing the password The Change password dialog is part of the Edit user page, and we can use it either to change bill's password or to remove it. Removing the password will enable bill to login without a password. The dialog offers a choice of password hashing options, and it's recommended to keep the default of MySQL 4.1+ hashing. For more details about hashing, please visit http://dev.mysql.com/doc/refman/5.1/en/password-hashing.html
Read more
  • 0
  • 0
  • 7697

article-image-importing-structure-and-data-phpmyadmin-33x-effective-mysql-management
Packt
12 Oct 2010
11 min read
Save for later

Importing Structure and Data in phpMyAdmin 3.3.x for MySQL Management

Packt
12 Oct 2010
11 min read
In general, an exported file can be imported either to the same database it came from or to any other database; the XML format is an exception to this and a workaround is given in the XML section later in this chapter. Also, a file generated from an older phpMyAdmin version should have no problem being imported by the current version, but the difference between the MySQL version at the time of export and the one at the time of import might play a bigger role regarding compatibility. It's difficult to evaluate how future MySQL releases will change the language's syntax, which could result in import challenges. The import feature can be accessed from several panels: The Import menu available from the homepage, the Database view, or the Table view The Import files menu offered inside the Query window An import file may contain the DELIMITER keyword. This enables phpMyAdmin to mimic the mysql command-line interpreter. The DELIMITER separator is used to delineate the part of the file containing a stored procedure, as these procedures can themselves contain semicolons. The default values for the Import interface are defned in $cfg['Import']. Before examining the actual import dialog, let's discuss some limits issues. Limits for the transfer When we import, the source file is usually on our client machine and therefore must travel to the server via HTTP. This transfer takes time and uses resources that may be limited in the web server's PHP configuration. Instead of using HTTP, we can upload our file to the server by using a protocol such as FTP, as described in the Reading files from a web server upload directory section. This method circumvents the web server's PHP upload limits. Time limits First, let's consider the time limit. In config.inc.php, the $cfg['ExecTimeLimit'] configuration directive assigns, by default, a maximum execution time of 300 seconds (five minutes) for any phpMyAdmin script, including the scripts that process data after the file has been uploaded. A value of 0 removes the limit, and in theory, gives us infinite time to complete the import operation. If the PHP server is running in safe mode, modifying $cfg['ExecTimeLimit'] will have no effect. This is because the limits set in php.ini or the user-related web server configuration file, (such as .htaccess or the virtual host configuration files) take precedence over this parameter. Of course, the time it effectively takes depends on two key factors: Web server load MySQL server load The time taken by the file, as it travels between the client and the server does not count as execution time because the PHP script only starts to execute after the file has been received on the server. Therefore, the $cfg['ExecTimeLimit'] parameter has an impact only on the time used to process data (like decompression or sending it to the MySQL server). Other limits The system administrator can use the php.ini file or the web server's virtual host configuration file to control uploads on the server. The upload_max_filesize parameter specifies the upper limit or maximum file size that can be uploaded via HTTP. This one is obvious, but another less obvious parameter is post_max_size. As HTTP uploading is done via the POST method, this parameter may limit our transfers. For more details about the POST method, please refer to http://en.wikipedia.org/wiki/Http#Request_methods. The memory_limit parameter is provided to prevent web server child processes from grabbing too much of the server's memory—phpMyAdmin runs inside a child process. Thus, the handling of normal file uploads, especially compressed dumps, can be compromised by giving this parameter a small value. Here, no preferred value can be recommended; the value depends on the size of uploaded data we want to handle and on the size of the physical memory. The memory limit can also be tuned via the $cfg['MemoryLimit'] parameter in config.inc.php, as seen in Chapter 6, Exporting Structure and Data (Backup). Finally, file uploads must be allowed by setting file_uploads to On; otherwise, phpMyAdmin won't even show the Location of the textfile dialog. It would be useless to display this dialog as the connection would be refused later by the PHP component of the web server. Handling big export files If the file is too big, there are ways in which we can resolve the situation. If the original data is still accessible via phpMyAdmin, we could use phpMyAdmin to generate smaller CSV export files, choosing the Dump n rows starting at record # n dialog. If this were not possible, we could use a spreadsheet program or a text editor to split the file into smaller sections. Another possibility is to use the upload directory mechanism, which accesses the directory defined in $cfg['UploadDir']. In recent phpMyAdmin versions, the Partial import feature can also solve this file size problem. By selecting the Allow interrupt... checkbox, the import process will interrupt itself if it detects that it's close to the time limit. We can also specify a number of queries to skip from the start, in case we successfully import a number of rows and wish to continue from that point. Uploading into a temporary directory On a server, a PHP security feature called open_basedir (which limits the files that can be opened by PHP to the specified directory tree) can impede the upload mechanism. In this case, or if uploads are problematic for any other reason, the $cfg['TempDir'] parameter can be set with the value of a temporary directory. This is probably a subdirectory of phpMyAdmin's main directory, into which the web server is allowed to put the uploaded file. Importing SQL files Any file containing MySQL statements can be imported via this mechanism. This format is more commonly used for backup/restore purposes. The relevant dialog is available in the Database view or the Table view, via the Import subpage, or in the Query window. There is no relation between the currently-selected table (here author) and the actual contents of the SQL file that will be importeAll of the contents of the SQL file will be imported, and it's those contents that determine which tables or databases are affected. However, if the imported file does not contain any SQL statementsto select a database, all statements in the imported file will be executed on the currently selected database. Let's try an import exercise. First, we make sure that we have a current SQL export of the book table (as explained in Chapter 6, Exporting Structure and Data (Backup)). This export file must contain the structure and the data. Then we drop the book table—yes, really! We could also simply rename it. (See Chapter 9, Performing Table and Database Operations, for the procedure.) Now it's time to import the file back. We should be on the Import subpage, where we can see the Location of the text file dialog. We just have to hit the Browse button and choose our file. phpMyAdmin is able to detect which compression method (if any) has been applied to the file. Depending on the phpMyAdmin version, and the extensions that are available in the PHP component of the web server, there is variation in the format that the program can decompress. However, to import successfully, phpMyAdmin must be informed of the character set of the file to be imported. The default value is utf8. However, if we know that the import file was created with another character set, we should specify it here. A SQL compatibility mode selector is available at import time. This mode should be adjusted to match the actual data that we are about to import, according to the type of the server where the data was previously exported. Another option, Do not use AUTO_INCREMENT for zero values, is selected by default. If we have a value of zero in a primary key and we want it to stay zero instead of being auto-incremented, we should use this option. To start the import, we click on Go. The import procedure continues and we receive a message: Import has been successfully finished, 2 queries executed. We can browse our newly-created tables to confirm the success of the import operation. The file could be imported for testing in a different database or even on another MySQL server. Importing CSV files In this section, we will examine how to import CSV files. There are two possible methods—CSV and CSV using LOAD DATA. The first method is implemented internally by phpMyAdmin and is the recommended one for its simplicity. With the second method, phpMyAdmin receives the file to be loaded and passes it to MySQL. In theory, this method should be faster. However, it has more requirements due to MySQL itself (see the Requirements subsection of the CSV using LOAD DATA section). Differences between SQL and CSV formats There are some differences between the SQL and CSV formats. The CSV file format contains data only, so we must already have an existing table in place. This table does not need to have the same structure as the original table (from which the data comes); the Column names dialog enables us to choose which columns are affected in the target table. Because the table must exist prior to the import, the CSV import dialog is available only from the Import subpage in the Table view, and not in the Database view. Exporting a test file Before trying an import, let's generate an author.csv export file from the author table. We use the default values in the CSV export options. We can then Empty the author table—we should avoid dropping this table because we still need the table structure. CSV From the author table menu, we select Import and then CSV: We can influence the behavior of the import in a number of ways. By default, importing does not modify existing data (based on primary or unique keys). However, the Replace table data with file option instructs phpMyAdmin to use the REPLACE statement instead of the INSERT statement, so that existing rows are replaced with the imported data. Using Ignore duplicate rows, INSERT IGNORE statements are generated. These cause MySQL to ignore any duplicate key problems during insertion. A duplicate key from the import file does not replace existing data, and the procedure continues for the next line of CSV data. We can also specify the character that terminates each field, the character that encloses data, and the character that escapes the enclosing character. Usually, this is. For example, for a double quote enclosing character, if the data field contains a double quote, it must be expressed as "some data " some other data". For Lines terminated by, recent versions of phpMyAdmin offer the auto choice, which should be tried first as it detects the end-of-line character automatically. We can also specify manually which characters terminate the lines. The usual choice is n for UNIX-based systems, rn for DOS or Windows systems, and r for Mac-based system (up to Mac OS 9). If in doubt, we can use a hexadecimal file editor on our client computer (not part of phpMyAdmin) to examine the exact codes. By default, phpMyAdmin expects a CSV file with the same number of fields and the same field order as the target table. However, this can be changed by entering a comma-separated list of column names in Column names, respecting the source file format. For example, let's say our source file contains only the author ID and the author name information: "1","John Smith" "2","Maria Sunshine" We'd have to put id, name in Column names in order to match the source file. When we click on Go, the import is executed and we receive a confirmation. We might also see the actual INSERT queries generated if the total size of the file is not too big. Import has been successfully finished, 2 queries executed. INSERT INTO `author` VALUES ('1', 'John Smith', '+01 445 789-1234')# 1 row(s) affected. INSERT INTO `author` VALUES ('2', 'Maria Sunshine', '333-3333')# 1 row(s) affected.
Read more
  • 0
  • 0
  • 2765

article-image-installing-phpmyadmin
Packt
12 Oct 2010
10 min read
Save for later

Installing phpMyAdmin

Packt
12 Oct 2010
10 min read
  Mastering phpMyAdmin 3.3.x for Effective MySQL Management Intoduction It's time to install the product and configure it minimally for first-time use. Our reason for installing phpMyAdmin could be one of the following: Our host provider did not install a central copy Our provider installed it, but the version installed is not current We are working directly on our enterprise's web server Required information Some host providers offer an integrated web panel where we can manage accounts, including MySQL accounts, and also a file manager that can be used to upload web content. Depending on this, the mechanism that we use to transfer phpMyAdmin source files to our webspace may vary. We will need some specific information (listed below) before starting the installation: The web server's name or address: Here, we will assume it to be www.mydomain.com Our web server's account information (username, password): This information will be used for either FTP or SFTP transfer, SSH login, or web control panel login. The MySQL server's name or IP address: If this information is not available, a good alternative choice is localhost, which means that the MySQL server is located on the same machine as the web server. We will assume this to be localhost. Our MySQL server's account information (username, password). System requirements The up-to-date requirements for a specific phpMyAdmin version are always stated in the accompanying Documentation.html file. For phpMyAdmin 3.3, the minimum PHP version required is PHP 5.2 with session support and the Standard PHP Library (SPL). Moreover, the web server must have access to a MySQL server (version 5.0 or later)—either locally or on a remote machine. It is strongly recommended that the PHP mcrypt extension be present for improved performance in cookie authentication mode. In fact, on a 64-bit server, this extension is required. On the browser side, cookie support must be activated, whatever authentication mode we use. Downloading the files There are various files available in the Download section of http://www.phpmyadmin.net. There might be more than one version offered here, and it is always a good idea to download the latest stable version. We only need to download one file, which works regardless of the platform (browser, web server, MySQL, or PHP version). For version 3.3, there are two groups of files—english and all-languages. If we need only the English interface, we can download a file whose name contains "english"—for example, phpMyAdmin-3.3.2-english.zip. On the other hand, if we have the need for at least one other language, choosing all-languages would be appropriate. If we are using a server supporting only PHP4—for which the PHP team has discontinued support since December 31, 2007—the latest stable version of phpMyAdmin is not a good choice for download. I recommend using version 2.11.x, which is the latest branch that supports PHP4. The files offered have various extensions: .zip, .tar.bz2, .tar.gz, .7z. Download a file having an extension for which you have the corresponding extractor. In the Windows world, .zip is the most universal file format, although the files are bigger than .gz or .bz2 files (which are common in the Linux/Unix world). The .7z extension denotes a 7-Zip file, which is a format that achieves a higher compression ratio than the other formats offered—an extractor is available at http://www.7-zip.org. In the following examples, we will assume that the chosen file was phpMyAdmin-3.3.2-all-languages.zip. After clicking on the appropriate file, the nearest mirror site will be chosen by Sourceforge.net. The file will start to download, and we can save it on our computer. Installing on different platforms The next step depends on the platform you are using. The following sections detail the procedures for some common platforms. You may proceed directly to the relevant section. Installing on a remote server using a Windows machine Using the File explorer, we double-click the phpMyAdmin-3.3.2-all-languages.zip file that we just downloaded on the Windows machine. A file extractor will start, showing us all of the scripts and directories inside a main phpMyAdmin-3.3.2-all-languages directory. Use whatever mechanism your file extractor offers to save all the files, including subdirectories, to some location on your workstation. Here, we have chosen C:. Therefore, a C:phpMyAdmin-3.3.2-all-languages directory has been created for extraction. Now, it's time to transfer the entire directory structure C:phpMyAdmin-3.3.2-alllanguages to the web server in our webspace. We use our favorite SFTP or FTP software, or the web control panel, for the transfer. The exact directory under which we transfer phpMyAdmin may vary. It could be our public_html directory or another directory to which we usually transfer web documents. For further instructions about the exact directory to be used, or the best way to transfer the directory structure, we can consult our host provider's help desk. After the transfer is complete, these files can be removed from our Windows machine as they are no longer needed. Installing on a local Linux server Let's say we chose phpMyAdmin-3.3.2-all-languages.tar.gz and downloaded it directly to some directory on the Linux server. We move it to our web server's document root directory (for example, /var/www/html) or to one of its subdirectories (for example, /var/www/html/utilities). We then extract it by issuing the following shell command or by using any graphical file extractor that our window manager offers: tar -xzvf phpMyAdmin-3.3.2-all-languages.tar.gz We must ensure that the permissions and ownership of the directory and files are appropriate for our web server. The web server user or group must be able to read them. Installing on local Windows servers (Apache, IIS) The procedure here is similar to that described in the Installation on a remote server using a Windows machine section, except that the target directory will be under our DocumentRoot (for Apache) or our wwwroot (for IIS). Of course, we do not need to transfer anything after modifications are made to config.inc.php (described in the next section), as the directory is already on the webspace. Apache is usually run as a service. Hence, we have to ensure that the user under whom the service is running has normal read privileges to access our newly-created directory. The same principle applies to IIS, which uses the IUSR_machinename user. This user must have read access to the directory. You can adjust permissions in the Security/permissions tab of the directory's properties. Configuring phpMyAdmin Here, we will learn how to prepare and use the configuration file containing the parameters to connect to MySQL, and which can be customized as per our requirements. Before configuring, we can rename the directory phpMyAdmin-3.3.2-all-languages to something like phpMyAdmin or just something easier to remember. This way, we and our users can visit an easily-remembered URL to start phpMyAdmin. On most servers, the directory part of URLs is case-sensitive, so we should communicate the exact URL to our users. We can also use a symbolic link if our server supports this feature. The config.inc.php file This file contains valid PHP code that defines the majority of the parameters (expressed by PHP variables) that we can change in order to tune phpMyAdmin to our own needs. There are also normal PHP comments in it, and we can comment our changes. Be careful not to add any blank lines at the beginning or end of the file; doing so would hamper the execution of phpMyAdmin. Note that phpMyAdmin looks for this file in the first level directory—the same one where index.php is located. In versions before 2.8.0, a generic config.inc.php file was included in the downloaded kit. Since 2.8.0, this file is no longer present in the directory structure. Since version 2.9.0, a config.sample.inc.php file is included, and this can be copied and renamed to config.inc.php to act as a starting point. However, it is recommended that you use the web-based setup script instead, for a more comfortable configuration interface. There is another file—layout.inc.php—that contains some configuration information. Because phpMyAdmin offers theme management, this file contains the theme-specific colors and settings. There is one layout.inc.php file per theme, located in themes/themename, for example, themes/original. Avoiding false error messages about permissions on config.inc.php In its normal behavior, phpMyAdmin verifies that the permissions on this file do not allow everyone to modify it. This means that the file should not be writable to the world. Also, it displays a warning if the permissions are not correct. However, in some situations (for example, an NTFS file system mounted on a non-Windows server), the permission detection fails. In these cases, you should set the followingparameter to false: $cfg['CheckConfigurationPermissions'] = false; The following sections explain various methods for adding or changing a parameter in the config.inc.php file. Configuration principles phpMyAdmin's behavior, given that no configuration file is present, has changed in version 3.1.0. In versions 3.0 and earlier, the application used its default settings as defined in libraries/config.default.php and tried to connect to a MySQL server on localhost—the same machine where the web server is running—with user as root and no password. This is the default set-up produced by most MySQL installation procedures, even though it is not really secure. Therefore, if our freshly installed MySQL server were still to have the default root account, we would have logged on automatically and would have seen a warning given by phpMyAdmin about such lack of security. If the notion of a MySQL root user eludes you, now might be the time to browse http://dev.mysql.com/doc/refman/5.1/en/privilege-system.html, in order to learn the basics about MySQL's privilege system. Since version 3.1.0, the development team has wanted to promote a more flexible login panel. This is why, with the lack of a configuration file, phpMyAdmin displays the cookie-based login panel by default We can verify this fact by visiting http://www.mydomain.com/phpMyAdmin and substituting the appropriate values for the domain part and the directory part. If we are able to log in, it means that there is a MySQL server running on the same host as the web server (localhost) and we've just made a connection to it. However, not having created a configuration file means that we would not be able to manage other hosts via our installation of phpMyAdmin. Moreover, many advanced phpMyAdmin features (for example, query bookmarks, full relational support, column transformation, and so on) would not be activated. The cookie-based authentication method uses Blowfish encryption for storing credentials in browser cookies. If no configuration file exists, a Blowfish secret key is generated and stored in session data, which can open the door to security issues. This is why the following warning message is displayed: The configuration file now needs a secret passphrase (blowfish_secret). At this point, we have some choices: Use phpMyAdmin without a configuration file Use the web-based setup script to generate a config.inc.php file Create a config.inc.php file manually These options are presented in the following sections. We should note that even if we use the web-based setup script, we should familiarize ourselves with the config.inc.php file format, because the setup script does not cover all of the possible configuration options.
Read more
  • 0
  • 0
  • 2323
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-performing-table-and-database-operations-phpmyadmin-33x-effective-mysql-management
Packt
12 Oct 2010
4 min read
Save for later

Performing Table and Database Operations in phpMyAdmin 3.3.x for Effective MySQL Management

Packt
12 Oct 2010
4 min read
  Mastering phpMyAdmin 3.3.x for Effective MySQL Management A complete guide to get started with phpMyAdmin 3.3 and master its features The best introduction to phpMyAdmin available Written by the project leader of phpMyAdmin, and improved over several editions A step-by-step tutorial for manipulating data with phpMyAdmin Learn to do things with your MySQL database and phpMyAdmin that you didn't know were possible!        Introduction Various links that enable table operations have been put together on the Operations subpage of the Table view. Here is an overview of this subpage: Maintaining a table During its lifetime, a table repeatedly gets modified and is therefore continually growing and shrinking. In addition, outages may occur on the server, leaving some tables in a damaged state. Using the Operations subpage, we can perform various operations, which are listed next. However, not every operation is available for every storage engine: Check table: Scans all rows to verify that deleted links are correct. A checksum is also calculated to verify the integrity of the keys. If everything is alright, we will obtain a message stating OK or Table is already up to date; if any other message shows up, it's time to repair this table (see the third bullet). Analyze table: Analyzes and stores the key distribution; this will be used on subsequent JOIN operations to determine the order in which the tables should be joined. This operation should be performed periodically (in case data has changed in the table), in order to improve JOIN efficiency. Repair table: Repairs any corrupted data for tables in the MyISAM and ARCHIVE engines. Note that a table might be so corrupted that we cannot even go into Table view for it! In such a case, refer to the Multi-table operations section for the procedure to repair it. Optimize table: This is useful when the table contains overheads. After massive deletions of rows or length changes for VARCHAR fields, lost bytes remain in the table. phpMyAdmin warns us in various places (for example, in the Structure view) if it feels the table should be optimized. This operation reclaims unused space in the table. In the case of MySQL 5.x, the relevant tables that can be optimized use the MyISAM, InnoDB, and ARCHIVE engines. Flush table: This must be done when there have been many connection errors and the MySQL server blocks further connections. Flushing will clear some internal caches and allow normal operations to resume. Defragment table: Random insertions or deletions in an InnoDB table fragment its index. The table should be periodically defragmented for faster data retrieval. This operation causes MySQL to rebuild the table, and only applies to InnoDB. The operations are based on the available underlying MySQL queries—phpMyAdmin only calls those queries. Changing table attributes Table attributes are the various properties of a table. This section discusses the settings for some of them. Table storage engine The first attribute that we can change is called Storage Engine. This controls the whole behavior of the table—its location (on-disk or in-memory), the index structure, and whether it supports transactions and foreign keys. The drop-down list varies depending on the storage engines supported by our MySQL server. Changing a table's storage engine may be a long operation if the number of rows is large. Table comments This allows us to enter comments for the table: These comments will be shown at appropriate places—for example, in the navigation panel, next to the table name in the Table view, and in the export file. Here is what the navigation panel looks like when the $cfg['ShowTooltip'] parameter is set to its default value of TRUE: The default value (FALSE) of $cfg['ShowTooltipAliasDB'] and $cfg['ShowTooltipAliasTB'] produces the behavior we saw earlier—the true database and table names are displayed in the navigation panel and in the Database view for the Structure subpage. Comments appear when the cursor is moved over a table name. If one of these parameters is set to TRUE, the corresponding item (database names for DB and table names for TB) will be shown as a tooltip instead of the names. This time, the mouseover box shows the true name for the item. This is convenient when the real table names are not meaningful. There is another possibility for $cfg['ShowTooltipAliasTB']—the 'nested' value. Here is what happens if we use this feature: The true table name is displayed in the navigation panel The table comment (for example project__) is interpreted as the project name and is displayed as it is
Read more
  • 0
  • 0
  • 2642

article-image-sessions-and-users-php-5-cms
Packt
17 Aug 2010
14 min read
Save for later

Sessions and Users in PHP 5 CMS

Packt
17 Aug 2010
14 min read
(For more resources on PHP, see here.) The problem Dealing with sessions can be confusing, and is also a source of security loopholes. So we want our CMS framework to provide basic mechanisms that are robust. We want them to be easy to use by more application-oriented software. To achieve these aims, we need to consider: The need for sessions and their working The pitfalls that can introduce vulnerabilities Efficiency and scalability considerations Discussion and considerations To see what is required for our session handling, we shall first review the need for them and consider how they work in a PHP environment. Then the vulnerabilities that can arise through session handling will be considered. Web crawlers for search engines and more nefarious activities can place a heavy and unnecessary load on session handling, so we shall look at ways to avoid this load. Finally, the question of how best to store session data is studied. Why sessions? The need for continuity was mentioned when we first discussed users. But it is worth reviewing the requirement in a little more detail. If Tim Berners-Lee and his colleagues had known all the developments that would eventually occur in the internet world, maybe the Web would have been designed differently. In particular, the basic web transport protocol HTTP might not have treated each request in isolation. But that is hindsight, and the Web was originally designed to present information in a computer-independent way. Simple password schemes were sufficient to control access to specific pages. Nowadays, we need to cater for complex user management, or to handle things like shopping carts, and for these we need continuity. Many people have recognized this, and introduced the idea of sessions. The basic idea is that a session is a series of requests from an individual website visitor, and the session provides access to enduring information that is available throughout the session. The shopping cart is an obvious example of information being retained across the requests that make up a session. PHP has its own implementation of sessions, and there is no point reinventing the wheel, so PHP sessions are the obvious tool for us to use to provide continuity. How sessions work There are three main choices which have been available for handling continuity: Adding extra information to the URI Using cookies Using hidden fields in the form sent to the browser All of them can be used at times. Which of them is most suitable for handling sessions? PHP uses either of the first two alternatives. Web software often makes use of hidden variables, but they do not offer a neat way to provide an unobtrusive general mechanism for maintaining continuity. In fact, whenever hidden variables are used, it is worth considering whether session data would be a better alternative. For reasons discussed in detail later, we shall consider only the use of cookies, and reject the URI alternative. There was a time when there were lots of scary stories about cookies, and people were inclined to block them. While there will always be security issues associated with web browsing, the situation has changed, and the majority of sites now rely on cookies. It is generally considered acceptable for a site to demand the use of cookies for operations such as user login or for shopping carts and purchase checkout. The PHP cookie-based session mechanism can seem obscure, so it is worth explaining how it works. First we need to review the working of cookies. A cookie is simply a named piece of data, usually limited to around 4,000 bytes, which is stored by the browser in order to help the web server to retain information about a user. More strictly, the connection is with the browser, not the user. Any cookie is tied to a specific website, and optionally to a particular part of the website, indicated by a path. It also has a life time that can be specified explicitly as a duration; a zero duration means that the cookie will be kept for as long as the browser is kept open, and then discarded. The browser does nothing with cookies, except to save and then return them to the server along with requests. Every cookie that relates to the particular website will be sent if either the cookie is for the site as a whole, or the optional path matches the path to which the request is being sent. So cookies are entirely the responsibility of the server, but the browser helps by storing and returning them. Note that, since the cookies are only ever sent back to the site that originated them, there are constraints on access to information about other sites that were visited using the same browser. In a PHP program, cookies can be written by calling the set_cookie function, or implicitly through session handling. The name of the cookie is a string, and the value to be stored is also a string, although the serialize function can be used to make more structured data into a string for storage as a cookie. Take care to keep the cookies within the size limit. PHP makes available the cookies that have been sent back by the browser in the $_COOKIES super-global, keyed by their names. Apart from any cookies explicitly written by code, PHP may also write a session cookie. It will do so either as a result of calls to session handling functions, or because the system has been configured to automatically start or resume a session for each request. By default, session cookies do not use the option of setting an expiry time, but can be deleted when the browser is closed down. Commonly, browsers keep this type of cookie in memory so that they are automatically lost on shutdown. Before looking at what PHP is doing with the session cookie, let's note that there is an important general consideration for writing cookies. In the construction of messages between the server and the browser, cookies are part of the header. That means rules about headers must be obeyed. Headers must be sent before anything else, and once anything else has been sent, it is not permitted to send more headers. So, in the case of server to browser communication, the moment any part of the XHTML has been written by the PHP program, it is too late to send a header, and therefore too late to write a cookie. For this reason, a PHP session is best started early in the processing. The only purpose PHP has in writing a session cookie is to allocate a unique key to the session, and retrieve it again on the next request. So the session cookie is given an identifying name, and its value is the session's unique key. The session key is usually called the session ID, and is used by PHP to pick out the correct set of persistent values that belong to the session. By default, the session name is PHPSESSID but it can, in most circumstances, be changed by calling the PHP function session_name prior to starting the session. Starting, or more often restarting, a session is done by calling session_start, which returns the session ID. In a simple situation, you do not need the session ID, as PHP places any existing session data in another superglobal, $_SESSION. In fact, we will have a use for the session ID as you will soon see. The $_SESSION super-global is available once session_start has been called, and the PHP program can store whatever data it chooses in it. It is an array, initially empty, and naturally the subscripts need to be chosen carefully in a complex system to avoid any clashes. The neat part of the PHP session is that provided it is restarted each time with session_start, the $_SESSION superglobal will retain any values assigned during the handling of previous requests. The data is thus preserved until the program decides to remove it. The only exception to this would be if the session expired, but in a default configuration, sessions do not expire automatically. Later in this article, we will look at ways to deliberately kill sessions after a determinate period of inactivity. As it is only the session ID that is stored in the cookie, rules about the timing of output do not apply to $_SESSION, which can be read or written at any time after session_start has been called. PHP stores the contents of $_SESSION at the end of processing or on request using the PHP function session_write_close. By default, PHP puts the data in a temporary file whose name includes the session ID. Whenever the session data is stored, PHP retrieves it again at the next session_start. Session data does not have to be stored in temporary files, and PHP permits the program to provide its own handling routines. We will look at a scheme for storing the session data in a database later in the article. Avoiding session vulnerabilities So far, the option to pass the session ID as part of the URI instead of as a cookie has not been considered. Looking at security will show why. The main security issue with sessions is that a cracker may find out the session ID for a user, and then hijack that user's session. Session handling should do its best to guard against that happening. PHP can pass the session ID as part of the URI. This makes it especially vulnerable to disclosure, since URIs can be stored in all kinds of places that may not be as inaccessible as we would like. As a result, secure systems avoid the URI option. It is also undesirable to find links appearing in search engines that include a session ID as part of the URI. These two points are enough to rule out the URI option for passing session ID. It can be prevented by the following PHP calls: ini_set('session.use_cookies', 1);ini_set('session.use_only_cookies', 1); These calls force PHP to use cookies for session handling, an option that is now considered acceptable. The extent to which the site will function without cookies depends on what a visitor can do with no continuity of data—user login will not stick, and anything like a shopping cart will not be remembered. It is best to avoid the default name of PHPSESSID for the session cookie, since that is something that a cracker could look for in the network traffic. One step that can be taken is to create a session name that is the MD5 hash of various items of internal information. This makes it harder but not impossible to sniff messages to find out a session ID, since it is no longer obvious what to seek—the well known name of PHPSESSID is not used. It is important for the session ID to be unpredictable, but we rely on PHP to achieve that. It is also desirable that the ID be long, since otherwise it might be possible for an attacker to try out all possible values within the life of a session. PHP uses 32 hexadecimal digits, which is a reasonable defense for most purposes. The other main vulnerability apart from session hijacking is called session fixation. This is typically implemented by a cracker setting up a link that takes the user to your site with a session already established, and known to the cracker. An important security step that is employed by robust systems is to change the session ID at significant points. So, although a session may be created as soon as a visitor arrives at the site, the session ID is changed at login. This technique is used by Amazon among others so that people can browse for items and build up a shopping cart, but on purchase a fresh login is required. Doing this reduces the available window for a cracker to obtain, and use, the session ID. It also blocks session fixation, since the original session is abandoned at critical points. It is also advisable to change the ID on logout, so although the session is continued, its data is lost and the ID is not the same. It is highly desirable to provide logout as an option, but this needs to be supplemented by time limits on inactive sessions. A significant part of session handling is devoted to keeping enough information to be able to expire sessions that have not been used for some time. It also makes sense to revoke a session that seems to have been used for any suspicious activity. Ideally, the session ID is never transmitted unencrypted, but achieving this requires the use of SSL, and is not always practical. It should certainly be considered for high security applications. Search engine bots One aspect of website building is, perhaps unexpectedly, the importance of handling the bots that crawl the web. They are often gathering data for search engines, although some have more dubious goals, such as trawling for e-mail addresses to add to spam lists. The load they place on a site can be substantial. Sometimes, search engines account for half or more of the bandwidth being used by a site, which certainly seems excessive. If no action is taken, these bots can consume significant resources, often for very little advantage to the site owner. They can also distort information about the site, such as when the number of current visitors is displayed but includes bots in the counts. Matters are made worse by the fact that bots will normally fail to handle cookies. After all, they are not browsers and have no need to implement support for cookies. This means that every request by a bot is separate from every other, as our standard mechanism for linking requests together will not work. If the system starts a new session, it will have to do this for every new request from a bot. There will never be a logout from the bot to terminate the session, so each bot-related session will last for the time set for automatic expiry. Clearly it is inadvisable to bar bots, since most sites are anxious to gain search engine exposure. But it is possible to build session handling so as to limit the workload created by visitors who do not permit cookies, which will mostly be bots. When we move into implementation techniques, the mechanisms will be demonstrated. Session data and scalability We could simply let PHP take care of session data. It does that by writing a serialized version of any data placed into $_SESSION into a file in a temporary directory. Each session has its own file. But PHP also allows us to implement our own session data handling mechanism. There are a couple of good reasons for using that facility, and storing the information in the database. One is that we can analyze and manage the data better, and especially limit the overhead of dealing with search engine bots. The other is that by storing session data in the database, we make it feasible for the site to be run across multiple servers. There may well be other issues before that can be achieved, but providing session continuity is an essential requirement if load sharing is to be fully effective. Storing session data in a database is a reliable solution to this issue. Arguments against storing session data in a database include questions about the overhead involved, constraints on database performance, or the possibility of a single point of failure. While these are real issues, they can certainly be mitigated. Most database engines, including MySQL, have many options for building scalable and robust systems. If necessary, the database can be spread across multiple computers linked by a high speed network, although this should never be done unless it is really needed. Design of such a system is outside the scope of this article, but the key point is that the arguments against storing session data in a database are not particularly strong.
Read more
  • 0
  • 0
  • 2320

article-image-error-handling-php-5-cms
Packt
17 Aug 2010
10 min read
Save for later

Error Handling in PHP 5 CMS

Packt
17 Aug 2010
10 min read
(For more resources on PHP, see here.) The problem Errors will happen whether we like it or not. Ideally the framework can help in their discovery, recording, and handling by: Trapping different kinds of errors Making a record of errors with sufficient detail to aid analysis Supporting a structure that mitigates the effect of errors Discussion There are three main kinds of errors that can arise. Many possible situations can crop up within PHP code that count as errors, such as an attempt to use a method on a variable that turns out not to be an object, or is an object but does not implement the specified method. The database will sometimes report errors, such as an attempt to retrieve information from a non-existent table, or to ask for a field that has not been defined for a table. And the logic of applications can often lead to situations that can only be described as errors. What resources do we have to handle these error situations? PHP error handling If nothing else is done, PHP has its own error handler. But developers are free to build their own handlers. So that is the first item on our to do list. Consistently with our generally object oriented approach, the natural thing to do is to build an error recording class, and then to tell PHP that one of its methods is to be called whenever PHP detects an error. Once that is done, the error handler must deal with whatever PHP passes, as it has taken over full responsibility for error handling. It has been a common practice to suppress the lowest levels of PHP error such as notices and warnings, but this is not really a good idea. Even these relatively unimportant messages can reveal more serious problems. It is not difficult to write code to avoid them, so that if a warning or notice does arise, it will indicate something unexpected and therefore worth investigation. For example, the PHP foreach statement expects to work on something iterable and will generate a warning if it is given, say, a null value. But this is easily avoided, either by making sure that methods which return arrays will always return an array, even if it is an array of zero items, rather than a null value. Failing that, the foreach can be protected by a preceding test. So it is safest to assume that a low level error may be a symptom of a bigger problem, and have our error handler record every error that is passed to it. The database is the obvious place to put the error, and the handler receives enough information to make it possible to save only the latest occurrence of the same error, thus avoiding a bloated table of many more or less identical errors. The other important mechanism offered by PHP is new to version 5 and is the try, catch, and throw construct. A section of code can be put within a try and followed by one or more catch specifications that define what is to be done if a particular kind of problem arises. The problems are triggered by using throw. This is a valuable mechanism for errors that need to break the flow of program execution, and is particularly helpful for dealing with database errors. It also has the advantage that the try sections can be nested, so if a large area of code, such as an entire component, is covered by a try it is still possible to write a try of narrower scope within that code. In general, it is better to be cautious about giving information about errors to users. For one thing, ordinary users are simply irritated by technically oriented error messages that mean nothing to them. Equally important is the issue of cracking, and the need to avoid displaying any weaknesses too clearly. It is bad enough that an error has occurred, without giving away details of what is going wrong. So a design assumption for error handling should be that the detail of errors is recorded for later analysis, but that only a very simple indication of the presence of an error is given to the user with a message that it has been noted for rectification. Database errors Errors in database operations are a particular problem for developers. Within the actual database handling code, it would be negligent to ignore the error indications that are available through the PHP interfaces to database systems. Yet within applications, it is hard to know what to do with such errors. SQL is very flexible, and a developer has no reason to expect any errors, so in the nature of things, any error that does arise is unexpected, and therefore difficult to handle. Furthermore, if there have to be several lines of error handling code every time the database is accessed, then the overhead in code size and loss of clarity is considerable. The best solution therefore seems to be to utilize the PHP try, catch, and throw structure. A special database error exception can be created by writing a suitable class, and the database handling code will then deal with an error situation by "throwing" a new error with an exception of that class. The CMS framework can have a default try and catch in place around most of its operation, so that individual applications within the CMS are not obliged to take any action. But if an application developer wants to handle database errors, it is always possible to do so by coding a nested try and catch within the application. One thing that must still be remembered by developers is that SQL easily allows some kinds of error situation to go unnoticed. For example, a DELETE or UPDATE SQL statement will not generate any error if nothing is deleted or updated. It is up to the developer to check how many rows, if any, were affected. This may not be worth doing, but issues of this kind need to be kept in mind when considering how software will work. A good error handling framework makes it easier for a developer to choose between different checking options. Application errors Even without there being a PHP or database error, an application may decide that an error situation has arisen. For some reason, normal processing is impossible, and the user cannot be expected to solve the problem. There are two main choices that will fit with the error handling framework we are considering. One is to use the PHP trigger_error statement. It raises a user error, and allows an error message to be specified. The error that is created will be trapped and passed to the error handler, since we have decided to have our own handler. This mechanism is best used for wholly unexpected errors that nonetheless could arise out of the logic of the application. The other choice is to use a complete try, catch, and throw structure within the application. This is most useful when there are a number of fatal errors that can arise, and are somewhat expected. The CMS extension installer uses this approach to deal with the various possible fatal errors that can occur during an attempt to install an extension. They are mostly related to errors in the XML packaging file, or in problems with accessing the file system. These are errors that need to be reported to help the user in resolving the problem, but they also involve abandoning the installation process. Whenever a situation of this kind arises, try, catch, and throw is a good way to deal with it. Exploring PHP—Error handling PHP provides quite a lot of control over error handling in its configuration. One question to be decided is whether to allow PHP to send any errors to the browser. This is determined by setting the value of display_errors in the php.ini configuration file. It is also possible to determine whether errors will be logged by setting log_errors and to decide where they should be logged by setting error_log. (Often there are several copies of this file, and it is important to find the one that is actually used by the system.) The case against sending errors is that it may give away information useful to crackers. Or it may look bad to users. On the other hand, it makes development and bug fixing harder if errors have to be looked up in a log file rather than being visible on the screen. And if errors are not sent to the screen, then in the event of a fatal error, the user will simply see a blank screen. This is not a good outcome either. Although the general advice is that errors should not be displayed on production systems, I am still rather inclined to show them. It seems to me that an error message, even if it is a technical one that is meaningless to the user, is rather better than a totally blank screen. The information given is only a bare description of the error, with the name and line number for the file having the error. It is unlikely to be a great deal of use to a cracker, especially since the PHP script just terminates on a fatal error, not leaving any clear opportunity for intrusion. You should make your own decision on which approach is preferable. Without any special action in the PHP code, an error will be reported by PHP giving details of where it occurred. Providing our own error handler by using the PHP set_error_handler function gives us far more flexibility to decide what information will be recorded and what will be shown to the user. A limitation on this is that PHP will still immediately terminate on a fatal error, such as attempting a method on something that is not an object. Termination also occurs whenever a parsing error is found, that is to say when the PHP program code is badly formed. It is not possible to have control transferred to a user provided error handler, which is an unfortunate limitation on what can be achieved. However, an error handler can take advantage of knowledge of the framework to capture relevant information. Quite apart from special information on the framework, the handler can make use of the useful PHP debug_backtrace function to find out the route that was followed before the error was reached. This will give information about what called the current code. It can then be used again to find what called that, and so on until no further trace information is available. A trace greatly increases the value of error reporting as it makes it much easier to find out the route that led to the error. When an error is trapped using PHP's try and catch, then it is best to trace the route to the error at the point the exception is thrown. Otherwise, the error trace will only show the chain of events from the exception to the error handler. There are a number of other PHP options that can further refine how errors are handled, but those just described form the primary tool box that we need for building a solid framework.
Read more
  • 0
  • 0
  • 1540

article-image-drupal-7-preview
Packt
11 Aug 2010
3 min read
Save for later

Drupal 7 Preview

Packt
11 Aug 2010
3 min read
You'll need a localhost LAMP or XAMPP environment to follow along with the examples here. If you don't have one set up, I recommend using the Acquia Stack Drupal Installer: http://acquia.com/downloads. Once your testing environment is configured, download Drupal 7: http://drupal.org/drupal-7.0-alpha6. Installing D7 Save the installer to your localhost Drupal /sites folder and extract it. Set up your MySQL database using your preferred method. Note to developers: D7's new database abstraction layer will theoretically support multiple database types including SQLite, PostgreSQL, MSSQL and Oracle. So if you are running Oracle you may be able to use D7. Now load the installer page in your browser (note I renamed my extracted folder to drupal7): http://localhost:8082/drupal7/install.php. The install process is about the same as D6 - you're still going to need to copy your /sites/default/default.settings.php file and re-name it to settings.php. Also make sure to create your /files folder. Make sure the file has write permissions for the install process. Once you do this and have your db created, it's time to run the installer. One immediate difference with the installer is that D7 now offers you a Standard or Minimal install profile. Standard will install D7 with common Drupal functionality and features that you are familiar with. Minimal is the choice for developers who want only the core Drupal functionality enabled. I'll leave it set for Standard profile. Navigate through the installer screens choosing language; and adding your database information. Enhancements With D7 installed what are the immediate noticeable enhancements? The overall look and feel of the administrative interface now uses overlay windows to present links to sections and content. Navigation in the admin interface now runs horizontally along the top of the site. Directly under the toolbar navigation is a shortcut link navigation. You can customize this by adding your own shortcuts pointing to various admin functionality. In the toolbar, Content points to your content lists. Structure contains links to Blocks, Content types, Menus and Taxonomy. CCK is now built into Drupal 7 so you can create custom content types and manage custom fields without having to install modules. If you want to restore the user interface to look more like D6 you can do this by disabling the Overlay module or tweaking role permissions for the Overlay module. Content Types Two content types are enabled with Drupal 7 core. Article replaces the D6 Story type. Basic Page replaces the D6 Page type. Developers hope these more accurate names will help new Drupal users understand how to add content easily to their site.
Read more
  • 0
  • 0
  • 2613
article-image-django-debugging-overview
Packt
28 Apr 2010
9 min read
Save for later

Django Debugging Overview

Packt
28 Apr 2010
9 min read
Django debug settings Django has a number of settings that control the collection and presentation of debug information. The primary one is named DEBUG; it broadly controls whether the server operates in development (if DEBUG is True) or production mode. In development mode, the end-user is expected to be a site developer. Thus, if an error arises during processing of a request, it is useful to include specific technical information about the error in the response sent to the web browser. This is not useful in production mode, when the user is expected to be simply a general site user. This section describes three Django settings that are useful for debugging during development. Additional settings are used during production to control what errors should be reported, and where error reports should be sent. These additional settings will be discussed in the section on handling problems in production. The DEBUG and TEMPLATE_DEBUG settings DEBUG is the main debug setting. One of the most obvious effects of setting this to True is that Django will generate fancy error page responses in the case of serious code problems, such as exceptions raised during processing of a request. If TEMPLATE_DEBUG is also True, and the exception raised is related to a template error, then the fancy error page will also include information about where in the template the error occurred. The default value for both of these settings is False, but the settings.py file created by manage.py startproject turns both of them on by including these lines at the top of the file: DEBUG = True TEMPLATE_DEBUG = DEBUG Note that setting TEMPLATE_DEBUG to True when DEBUG is False isn't useful. The additional information collected with TEMPLATE_DEBUG turned on will never be displayed if the fancy error pages, controlled by the DEBUG setting, are not displayed. Similarly, setting TEMPLATE_DEBUG to False when DEBUG is True isn't very useful. In this case, for template errors, the fancy debug page will be lacking helpful information. Thus, it makes sense to keep these settings tied to each other, as previously shown. Details on the fancy error pages and when they are generated will be covered in the next section. Besides generating these special pages, turning DEBUG on has several other effects. Specifically, when DEBUG is on: A record is kept of all queries sent to the database. Details of what is recorded and how to access it will be covered in a subsequent section. For the MySQL database backend, warnings issued by the database will be turned into Python Exceptions. These MySQL warnings may indicate a serious problem, but a warning (which only results in a message printed to stderr) may pass unnoticed. Since most development is done with DEBUG turned on, raising exceptions for MySQL warnings then ensures that the developer is aware of the possible issue. The admin application performs extensive validation of the configuration of all registered models and raises an ImproperlyConfigured exception on the first attempt to access any admin page if an error is found in the configuration. This extensive validation is fairly expensive and not something you'd generally want done during production server start-up, when the admin configuration likely has not changed since the last start-up. When running with DEBUG on, though, it is possible that the admin configuration has changed, and thus it is useful and worth the cost to do the explicit validation and provide a specific error message about what is wrong if a problem is detected. Finally, there are several places in Django code where an error will occur while DEBUG is on, and the generated response will contain specific information about the cause of the error, whereas when DEBUG is off the generated response will be a generic error page. The TEMPLATE_STRING_IF_INVALID setting A third setting that can be useful for debugging during development is TEMPLATE_STRING_IF_INVALID. The default value for this setting is the empty string. This setting is used to control what gets inserted into a template in place of a reference to an invalid (for example, non-existent in the template context) variable. The default value of an empty string results in nothing visible taking the place of such invalid references, which can make them hard to notice. Setting TEMPLATE_STRING_IF_INVALID to some value can make tracking down such invalid references easier. However, some code that ships with Django (the admin application, in particular), relies on the default behavior of invalid references being replaced with an empty string. Running code like this with a non-empty TEMPLATE_STRING_IF_INVALID setting can produce unexpected results, so this setting is only useful when you are specifically trying to track down something like a misspelled template variable in code that always ensures that variables, even empty ones, are set in the template context. Debug error pages With DEBUG on, Django generates fancy debug error pages in two circumstances: When a django.http.Http404 exception is raised When any other exception is raised and not handled by the regular view processing code In the latter case, the debug page contains a tremendous amount of information about the error, the request that caused it, and the environment at the time it occurred. The debug pages for Http404 exceptions are considerably simpler. To see examples of the Http404 debug pages, consider the survey_detail view def survey_detail(request, pk): survey = get_object_or_404(Survey, pk=pk) today = datetime.date.today() if survey.closes < today: return display_completed_survey(request, survey) elif survey.opens > today: raise Http404 else: return display_active_survey(request, survey) There are two cases where this view may raise an Http404 exception: when the requested survey is not found in the database, and when it is found but has not yet opened. Thus, we can see the debug 404 page by attempting to access the survey detail for a survey that does not exist, say survey number 24. The result will be as follows: Notice there is a message in the middle of the page that describes the cause of the page not found response: No Survey matches the given query. This message was generated automatically by the get_object_or_404 function. By contrast, the bare raise Http404 in the case where the survey is found but not yet open does not look like it will have any descriptive message. To confirm this, add a survey that has an opens date in the future, and try to access its detail page. The result will be something like the following: That is not a very helpful debug page, since it lacks any information about what was being searched for and why it could not be displayed. To make this page more useful, include a message when raising the Http404 exception. For example: raise Http404("%s does not open until %s; it is only %s" % (survey.title, survey.opens, today)) Then an attempt to access this page will be a little more helpful: Note that the error message supplied with the Http404 exception is only displayed on the debug 404 page; it would not appear on a standard 404 page. So you can make such messages as descriptive as you like and not worry that they will leak private or sensitive information to general users. Another thing to note is that a debug 404 page is only generated when an Http404 exception is raised. If you manually construct an HttpResponse with a 404 status code, it will be returned, not the debug 404 page. Consider this code: return HttpResponse("%s does not open until %s; it is only %s" % (survey.title, survey.opens, today), status=404) If that code were used in place of the raise Http404 variant, then the browser will simply display the passed message: Without the prominent Page not found message and distinctive error page formatting, this page isn't even obviously an error report. Note also that some browsers by default will replace the server-provided content with a supposedly "friendly" error page that tends to be even less informative. Thus, it is both easier and more useful to use the Http404 exception instead of manually building HttpResponse objects with status code 404. A final example of the debug 404 page that is very useful is the one that is generated when URL resolution fails. For example, if we add an extra space before the survey number in the URL, the debug 404 page generated will be as follows: The message on this page includes all of the information necessary to figure out why URL resolution failed. It includes the current URL, the name of the base URLConf used for resolution, and all patterns that were tried, in order, for matching. If you do any significant amount of Django application programming, it's highly likely that at some time this page will appear and you will be convinced that one of the listed patterns should match the given URL. You would be wrong. Do not waste energy trying to figure out how Django could be so broken. Rather, trust the error message, and focus your energies on figuring out why the pattern you think should match doesn't in fact match. Look carefully at each element of the pattern and compare it to the actual element in the current URL: there will be something that doesn't match. In this case, you might think the third listed pattern should match the current URL. The first element in the pattern is the capture of the primary key value, and the actual URL value does contain a number that could be a primary key. However, the capture is done using the pattern d+. An attempt to match this against the actual URL characters—a space followed by 2—fails because d only matches numeric digits and the space character is not a numeric digit. There will always be something like this to explain why the URL resolution failed. For now, we will leave the subject of debug pages and learn about accessing the history of database queries that is maintained when DEBUG is on.
Read more
  • 0
  • 0
  • 3497

article-image-tracking-sql-queries-request-using-django
Packt
22 Apr 2010
13 min read
Save for later

Tracking SQL Queries for a Request using Django

Packt
22 Apr 2010
13 min read
For a typical Django application, database interactions are of key importance. Ensuring that the database queries being made are correct helps to ensure that the application results are correct. Further, ensuring that the database queries produced for the application are efficient helps to make sure that the application will be able to support the desired number of concurrent users. Django provides support in this area by making the database query history available for examination. This type of access is useful to see the SQL that is issued as a result of calling a particular model method. However, it is not helpful in learning about the bigger picture of what SQL queries are made during the processing of a particular request. This section will show how to include information about the SQL queries needed for production of a page in the page itself. We will alter our existing survey application templates to include query information, and examine the query history for some of the existing survey application views. Though we are not aware of any problems with the existing views, we may learn something in the process of verifying that they issue the queries we expect. Settings for accessing query history in templates Before the query history can be accessed from a template, we need to ensure some required settings are configured properly. Three settings are needed in order for the SQL query information to be available in a template. First, the debug context processor, django.core.context_processors.debug, must be included in the TEMPLATE_CONTEXT_PROCESSORS setting. This context processor is included in the default value for TEMPLATE_CONTEXT_PROCESSORS. We have not changed that setting; therefore we do not need to do anything to enable this context processor in our project. Second, the IP address of the machine sending the request must be listed in the INTERNAL_IPS setting. This is not a setting we have used before, and it is empty by default, so we will need to add it to the settings file. When testing using the same machine as where the development server runs, setting INTERNAL_IPS to include the loopback address is sufficient: # Addresses for internal machines that can see potentially sensitive # information such as the query history for a request. INTERNAL_IPS = ('127.0.0.1', ) If you also test from other machines, you will need to include their IP addresses in this setting as well. Third and finally, DEBUG must be True in order for the SQL query history to be available in templates. When those three settings conditions are met, the SQL query history may be available in templates via a template variable named sql_queries. This variable contains a list of dictionaries. Each dictionary contains two keys: sql and time. The value for sql is the SQL query itself, and the value for time is the number of seconds the query took to execute. Note that the sql_queries context variable is set by the debug context processor. Context processors are only called during template rendering when a RequestContext is used to render the template. Up until now, we have not used RequestContexts in our survey application views, since they were not necessary for the code so far. But in order to access the query history from the template, we will need to start using RequestContexts. Therefore, in addition to modifying the templates, we will need to change the view code slightly in order to include query history in the generated pages for the survey application. SQL queries for the home page Let's start by seeing what queries are issued in order to generate the survey application home page. Recall that the home page view code is: def home(request): today = datetime.date.today() active = Survey.objects.active() completed = Survey.objects.completed().filter(closes__gte=today- datetime.timedelta(14)) upcoming = Survey.objects.upcoming().filter( opens__lte=today+datetime.timedelta(7)) return render_to_response('survey/home.html', {'active_surveys': active, 'completed_surveys': completed, 'upcoming_surveys': upcoming, }) There are three QuerySets rendered in the template, so we would expect to see that this view generates three SQL queries. In order to check that, we must first change the view to use a RequestContext: from django.template import RequestContext def home(request): today = datetime.date.today() active = Survey.objects.active() completed = Survey.objects.completed().filter(closes__gte=today- datetime.timedelta(14)) upcoming = Survey.objects.upcoming().filter( opens__lte=today+datetime.timedelta(7)) return render_to_response('survey/home.html', {'active_surveys': active, 'completed_surveys': completed, 'upcoming_surveys': upcoming,}, RequestContext(request)) The only change here is to add the RequestContext(request) as a third parameter to render_to_response, after adding an import for it earlier in the file. When we make this change, we may as well also change the render_to_response lines for the other views to use RequestContexts as well. That way when we get to the point of examining the SQL queries for each, we will not get tripped up by having forgotten to make this small change. Second, we'll need to display the information from sql_queries somewhere in our survey/home.html template. But where? We don't necessarily want this information displayed in the browser along with the genuine application data, since that could get confusing. One way to include it in the response but not have it be automatically visible on the browser page is to put it in an HTML comment. Then the browser will not display it on the page, but it can be seen by viewing the HTML source for the displayed page. As a first attempt at implementing this, we might change the top of survey/home.html to look like this: {% extends "survey/base.html" %} {% block content %} <!-- {{ sql_queries|length }} queries {% for qdict in sql_queries %} {{ qdict.sql }} ({{ qdict.time }} seconds) {% endfor %} --> This template code prints out the contents of sql_queries within an HTML comment at the very beginning of the content block supplied by survey/home. html. First, the number of queries is noted by filtering the list through the length filter. Then the code iterates through each dictionary in the sql_queries list and displays sql, followed by a note in parentheses about the time taken for each query. How well does that work? If we try it out by retrieving the survey home page (after ensuring the development server is running), and use the browser's menu item for viewing the HTML source for the page, we might see that the comment block contains something like: <!-- 1 queries SELECT `django_session`.`session_key`, `django_session`.`session_data`, `django_session`.`expire_date` FROM `django_session` WHERE (`django_ session`.`session_key` = d538f13c423c2fe1e7f8d8147b0f6887 AND `django_ session`.`expire_date` &gt; 2009-10-24 17:24:49 ) (0.001 seconds) --> Note that the exact number of queries displayed here will depend on the version of Django you are running. This result is from Django 1.1.1; later versions of Django may not show any queries displayed here. Furthermore, the history of the browser's interaction with the site will affect the queries issued. This result is from a browser that had been used to access the admin application, and the last interaction with the admin application was to log out. You may see additional queries if the browser had been used to access the admin application but the user had not logged out. Finally, the database in use can also affect the specific queries issued and their exact formatting. This result is from a MySQL database. That's not exactly what we expected. First, a minor annoyance, but 1 queries is wrong, it should be 1 query. Perhaps that wouldn't annoy you, particularly just in internal or debug information, but it would annoy me. I would change the template code that displays the query count to use correct pluralization: {% with sql_queries|length as qcount %} {{ qcount }} quer{{ qcount|pluralize:"y,ies" }} {% endwith %} Here, since the template needs to use the length result multiple times, it is first cached in the qcount variable by using a {% with %} block. Then it is displayed, and it is used as the variable input to the pluralize filter that will put the correct letters on the end of quer depending on the qcount value. Now the comment block will show 0 queries, 1 query, 2 queries, and so on. With that minor annoyance out of the way, we can concentrate on the next, larger, issue, which is that the displayed query is not a query we were expecting. Furthermore, the three queries we were expecting, to retrieve the lists of completed, active, and upcoming surveys, are nowhere to be seen. What's going on? We'll take each of these in turn. The query that is shown is accessing the django_session table. This table is used by the django.contrib.sessions application. Even though the survey application does not use this application, it is listed in our INSTALLED_APPS, since it is included in the settings.py file that startproject generates. Also, the middleware that the sessions application uses is listed in MIDDLEWARE_CLASSES. The sessions application stores the session identifier in a cookie, named sessionid by default, that is sent to the browser as soon as any application uses a session. The browser will return the cookie in all requests to the same server. If the cookie is present in a request, the session middleware will use it to retrieve the session data. This is the query we see previously listed: the session middleware is retrieving the data for the session identified by the session cookie sent by the browser. But the survey application does not use sessions, so how did the browser get a session cookie in the first place? The answer is that the admin application uses sessions, and this browser had previously been used to access the admin application. At that time, the sessionid cookie was set in a response, and the browser faithfully returns it on all subsequent requests. Thus, it seems likely that this django_session table query is due to a sessionid cookie set as a side-effect of using the admin application. Can we confirm that? If we find and delete the cookie from the browser and reload the page, we should see that this SQL query is no longer listed. Without the cookie in the request, whatever code was triggering access to the session data won't have anything to look up. And since the survey application does not use sessions, none of its responses should include a new session cookie, which would cause subsequent requests to include a session lookup. Is this reasoning correct? If we try it, we will see that the comment block changes to: <!-- 0 queries --> Thus, we seem to have confirmed, to some extent, what happened to cause a django_session table query during processing of a survey application response. We did not track down what exact code accessed the session identified by the cookie—it could have been middleware or a context processor, but we probably don't need to know the details. It's enough to keep in mind that there are other applications running in our project besides the one we are working on, and they may cause database interactions independent of our own code. If we observe behavior which looks like it might cause a problem for our code, we can investigate further, but for this particular case we will just avoid using the admin application for now, as we would like to focus attention on the queries our own code is generating. Now that we understand the query that was listed, what about the expected ones that were not listed? The missing queries are due to a combination of the lazy evaluation property of QuerySets and the exact placement of the comment block that lists the contents of sql_queries. We put the comment block at the top of the content block in the home page, to make it easy to find the SQL query information when looking at the page source. The template is rendered after the three QuerySets are created by the view, so it might seem that the comment placed at the top should show the SQL queries for the three QuerySets. However, QuerySets are lazy; simply creating a QuerySet does not immediately cause interaction with the database. Rather, sending the SQL to the database is delayed until the QuerySet results are actually accessed. For the survey home page, that does not happen until the parts of the template that loop through each QuerySet are rendered. Those parts are all below where we placed the sql_queries information, so the corresponding SQL queries had not yet been issued. The fix for this is to move the placement of the comment block to the very bottom of the content block. When we do that we should also fix two other issues with the query display. First, notice that the query displayed above has &gt; shown instead of the > symbol that would actually have been in the query sent to the database. Furthermore, if the database in use is one (such as PostgreSQL) that uses straight quotes instead of back quotes for quoting, all of the back quotes in the query would be shown as &quot;. This is due to Django's automatic escaping of HTML markup characters. This is unnecessary and hard to read in our HTML comment, so we can suppress it by sending the sql query value through the safe filter. Second, the query is very long. In order to avoid needing to scroll to the right in order to see the entire query, we can also filter the sql value through wordwrap to introduce some line breaks and make the output more readable. To make these changes, remove the added comment block from the top of the content block in the survey/home.html template and instead change the bottom of this template to be: {% endif %} <!-- {% with sql_queries|length as qcount %} {{ qcount }} quer{{ qcount|pluralize:"y,ies" }} {% endwith %} {% for qdict in sql_queries %} {{ qdict.sql|safe|wordwrap:60 }} ({{ qdict.time }} seconds) {% endfor %} --> {% endblock content %} Now, if we again reload the survey home page and view the source for the returned page, we will see the queries listed in a comment at the bottom: <!-- 3 queries SELECT `survey_survey`.`id`, `survey_survey`.`title`, `survey_survey`.`opens`, `survey_survey`.`closes` FROM `survey_survey` WHERE (`survey_survey`.`opens` <= 2009-10-25 AND `survey_survey`.`closes` >= 2009-10-25 ) (0.000 seconds) SELECT `survey_survey`.`id`, `survey_survey`.`title`, `survey_survey`.`opens`, `survey_survey`.`closes` FROM `survey_survey` WHERE (`survey_survey`.`closes` < 2009-10-25 AND `survey_survey`.`closes` >= 2009-10-11 ) (0.000 seconds) SELECT `survey_survey`.`id`, `survey_survey`.`title`, `survey_survey`.`opens`, `survey_survey`.`closes` FROM `survey_survey` WHERE (`survey_survey`.`opens` > 2009-10-25 AND `survey_survey`.`opens` <= 2009-11-01 ) (0.000 seconds) --> That is good, those look like exactly what we expect to see for queries for the home page. Now that we seem to have some working template code to show queries, we will consider packaging up this snippet so that it can easily be reused elsewhere.
Read more
  • 0
  • 0
  • 6207

article-image-handling-invalid-survey-submissions-django
Packt
20 Apr 2010
5 min read
Save for later

Handling Invalid Survey Submissions with Django

Packt
20 Apr 2010
5 min read
What would make a survey submission invalid? The only likely error case for our QuestionVoteForm is if no answer is chosen. What happens, then, if we attempt to submit a survey with missing answers? If we try it, we see that the result is not ideal: There are at least two problems here. First, the placement of the error messages, above the survey questions, is confusing. It is hard to know what the first error message on the page is referring to, and the second error looks like it is associated with the first question. It would be better to move the error messages closer to where the selection is actually made, such as between the question and answer choice list. Second, the text of the error message is not very good for this particular form. Technically the list of answer choices is a single form field, but to a general user the word field in reference to a list of choices sounds odd. We will correct both of these errors next. Coding custom error message and placement Changing the error message is easy, since Django provides a hook for this. To override the value of the error message issued when a required field is not supplied, we can specify the message we would like as the value for the required key in an error_messages dictionary we pass as an argument in the field declaration. Thus, this new definition for the answer field in QuestionVoteForm will change the error message to Please select an answer below: class QuestionVoteForm(forms.Form): answer = forms.ModelChoiceField(widget=forms.RadioSelect, queryset=None, empty_label=None, error_messages={'required': 'Please select an answer below:'}) Changing the placement of the error message requires changing the template. Instead of using the as_p convenience method, we will try displaying the label for the answer field, errors for the answer field, and then the answer field itself, which displays the choices. The {% for %} block that displays the survey forms in the survey/active_survey.html template then becomes: {% for qform in qforms %} {{ qform.answer.label }} {{ qform.answer.errors }} {{ qform.answer }}{% endfor %} How does that work? Better than before. If we try submitting invalid forms now, we see: While the error message itself is improved, and the placement is better, the exact form of the display is not ideal. By default, the errors are shown as an HTML unordered list. We could use CSS styling to remove the bullet that is appearing (as we will eventually do for the list of choices), but Django also provides an easy way to implement custom error display, so we could try that instead. To override the error message display, we can specify an alternate error_class attribute for QuestionVoteForm, and in that class, implement a __unicode__ method that returns the error messages with our desired formatting. An initial implementation of this change to QuestionVoteForm and the new class might be: class QuestionVoteForm(forms.Form): answer = forms.ModelChoiceField(widget=forms.RadioSelect, queryset=None, empty_label=None, error_messages={'required': 'Please select an answer below:'}) def __init__(self, question, *args, **kwargs): super(QuestionVoteForm, self).__init__(*args, **kwargs) self.fields['answer'].queryset = question.answer_set.all() self.fields['answer'].label = question.question self.error_class = PlainErrorListfrom django.forms.util import ErrorListclass PlainErrorList(ErrorList): def __unicode__(self): return u'%s' % ' '.join([e for e in sefl]) The only change to QuestionVoteForm is the addition of setting its error_class attribute to PlainErrorList in its __init__ method. The PlainErrorList class is based on the django.form.util.ErrorList class and simply overrides the __unicode__ method to return the errors as a string with no special HTML formatting. The implementation here makes use of the fact that the base ErrorList class inherits from list, so iterating over the instance itself returns the individual errors in turn. These are then joined together with spaces in between, and the whole string is returned. Note that we're only expecting there to ever be one error here, but just in case we are wrong in that assumption, it is safest to code for multiple errors existing. Although our assumption may never be wrong in this case, it's possible we might decide to re-use this custom error class in other situations where the single possible error expectation doesn't hold. If we code to our assumption and simply return the first error in the list, this may result in confusing error displays in some situations where there are multiple errors, since we will have prevented reporting all but the first error. If and when we get to that point, we may also find that formatting a list of errors with just spaces intervening is not a good presentation, but we can deal with that later. First, we'd like to simply verify that our customization of the error list display is used.
Read more
  • 0
  • 0
  • 2296
article-image-django-debug-toolbar
Packt
20 Apr 2010
9 min read
Save for later

The Django Debug Toolbar

Packt
20 Apr 2010
9 min read
The debug toolbar has a far more advanced way of displaying the information than simply embedding it in HTML comments. The capabilities are best shown by example, so we will immediately proceed with installing the toolbar. Installing the Django Debug Toolbar The toolbar can be found on the Python package index site: https://pypi.python.org/pypi/django-debug-toolbar. Once installed, activating the debug toolbar in a Django project is accomplished with the addition of just a couple of settings. First, the debug toolbar middleware, debug_toolbar.middleware.DebugToolbarMiddleware, must be added to the MIDDLEWARE_CLASSES setting. The documentation for the toolbar notes that it should be placed after any other middleware that encodes the response content, so it is best to place it last in the middleware sequence. Second, the debug_toolbar application needs to be added to INSTALLED_APPS. The debug_toolbar application uses Django templates to render its information, thus it needs to be listed in INSTALLED_APPS so that its templates will be found by the application template loader. Third, the debug toolbar requires that the requesting IP address be listed in INTERNAL_IPS. Finally, the debug toolbar is displayed only when DEBUG is True. We've been running with debug turned on, so again we don't have to make any changes here. Note also that the debug toolbar allows you to customize under what conditions the debug toolbar is displayed. It's possible, then, to set things up so that the toolbar will be displayed for requesting IP addresses not in INTERNAL_IPS or when debug is not turned on, but for our purposes the default configuration is fine so we will not change anything. One thing that is not required is for the application itself to use a RequestContext in order for things such as the SQL query information to be available in the toolbar. The debug toolbar runs as middleware, and thus is not dependent on the application using a RequestContext in order for it to generate its information. Thus, the changes made to the survey views to specify RequestContexts on render_to_response calls would not have been needed if we started off first with the Django Debug Toolbar. Debug toolbar appearance Once the debug toolbar is added to the middleware and installed applications settings, we can see what it looks like by simply visiting any page in the survey application. Let's start with the home page. The returned page should now look something like this: Note this screenshot shows the appearance of the 0.8.0 version of the debug toolbar. Earlier versions looked considerably different, so if your results do not look like this you may be using a different version than 0.8.0. The version that you have will most likely be newer than what was available when this was written, and there may be additional toolbar panels or functions that are not covered here. As you can see, the debug toolbar appears on the right-hand side of the browser window. It consists of a series of panels that can be individually enabled or disabled by changing the toolbar configuration. The ones shown here are the ones that are enabled by default. Before taking a closer look at some of the individual panels, notice that the toolbar contains an option to hide it at the top. If Hide is selected, the toolbar reduces itself to a small tab-like indication to show that it is present: This can be very useful for cases where the expanded version of the toolbar obscures application content on the page. All of the information provided by the toolbar is still accessible, after clicking again on the DjDT tab; it is just out of the way for the moment. Most of the panels will provide detailed information when they are clicked. A few also provide summary information in the main toolbar display. As of debug toolbar version 0.8.0, the first panel listed, Django Version, only provides summary information. There is no more detailed information available by clicking on it. As you can see in the screenshot, Django 1.1.1 is the version in use here. Note that the current latest source version of the debug toolbar already provides more information for this panel than the 0.8.0 release. Since 0.8.0, this panel has been renamed to Versions, and can be clicked to provide more details. These additional details include version information for the toolbar itself and for any other installed Django applications that provide version information. The other three panels that show summary information are the Time, SQL, and Logging panels. Thus, we can see at a glance from the first appearance of the page that 60 milliseconds of CPU time were used to produce this page (111 milliseconds total elapsed time), that the page required four queries, which took 1.95 milliseconds, and that zero messages were logged during the request. In the following sections, we will dig into exactly what information is provided by each of the panels when clicked. We'll start first with the SQL panel, since it is one of the most interesting and provides the same information (in addition to a lot more). The SQL panel If we click on the SQL section of the debug toolbar, the page will change to: At a glance, this is a much nicer display of the SQL queries for the page than what we came up with earlier. The queries themselves are highlighted so that SQL keywords stand out, making them easier to read. Also, since they are not embedded inside an HTML comment, their content does not need to be altered in any way—there was no need to change the content of the query containing the double dash in order to avoid it causing display problems. (Now would probably be a good time to remove that added query, before we forget why we added it.) Notice also that the times listed for each query are more specific than what was available in Django's default query history. The debug toolbar replaces Django's query recording with its own, and provides timings in units of milliseconds instead of seconds. The display also includes a graphical representation of how long each query took, in the form of horizontal bars that appear above each query. This representation makes it easy to see when there are one or more queries that are much more expensive than the others. In fact, if a query takes an excessive amount of time, its bar will be colored red. In this case, there is not a great deal of difference in the query times, and none took particularly long, so all the bars are of similar length, and are colored gray. Digging deeper, some of the information we had to manually figure out earlier in this article is just a click away on this SQL query display. Specifically, the answer to the question of what line of our code triggered a particular SQL query to be issued. Each of the displayed queries has a Toggle Stacktrace option, which when clicked will show the stack trace associated with the query: Here we can see that all queries are made by the home method in the survey views. py file. Note that the toolbar filters out levels in the stack trace that are within Django itself, which explains why each of these has only one level shown. The first query is triggered by Line 61, which contains the filter call added to test what will happen if a query containing two dashes in a row was logged. The remaining queries are all attributed to Line 66, which is the last line of the render_to_response call in the home view. These queries, as we figured out earlier, are all made during the rendering of the template. (Your line numbers may vary from those shown here, depending on where in the file various functions were placed.) Finally, this SQL query display makes available information that we had not even gotten around to wanting yet. Under the Action column are links to SELECT, EXPLAIN, and PROFILE each query. Clicking on the SELECT link shows what the database returns when the query is actually executed. For example: Similarly, clicking on EXPLAIN and PROFILE displays what the database reports when asked to explain or profile the selected query, respectively. The exact display, and how to interpret the results, will differ from database to database. (In fact, the PROFILE option is not available with all databases—it happens to be supported by the database in use here, MySQL.) Interpreting the results from EXPLAIN and PROFILE is beyond the scope of what's covered here, but it is useful to know that if you ever need to dig deep into the performance characteristics of a query, the debug toolbar makes it easy to do so. We've now gotten a couple of pages deep into the SQL query display. How do we get back to the actual application page? Clicking on the circled >> at the upper-right of the main page display will return to the previous SQL query page, and the circled >> will turn into a circled X. Clicking the circled X on any panel detail page closes the details and returns to displaying the application data. Alternatively, clicking again on the panel area on the toolbar for the currently displayed panel will have the same effect as clicking on the circled symbol in the display area. Finally, if you prefer using the keyboard to the mouse, pressing Esc has the same effect as clicking the circled symbol. Now that we have completely explored the SQL panel, let's take a brief look at each of the other panels provided by the debug toolbar. The Time panel Clicking on the Time panel brings up more detailed information on where time was spent during production of the page: The total CPU time is split between user and system time, the total elapsed (wall clock) time is listed, and the number of voluntary and involuntary context switches are displayed. For a page that is taking too long to generate, these additional details about where the time is being spent can help point towards a cause. Note that the detailed information provided by this panel comes from the Python resource module. This is a Unix-specific Python module that is not available on non-Unix-type systems. Thus on Windows, for example, the debug toolbar time panel will only show summary information, and no further details will be available.
Read more
  • 0
  • 0
  • 4536

article-image-reports-and-statistics-openx-ad-server
Packt
29 Mar 2010
2 min read
Save for later

Reports and Statistics in OpenX Ad Server

Packt
29 Mar 2010
2 min read
OpenX provides a very detailed and useful statistics and reports mechanism to evaluate the progress as well as performance of any campaign. OpenX reports and statistics tools have the ability to retrieve data on several groupings such as advertisers, campaigns, banners, websites, and zones. Advertisers and campaigns statistics It is the default OpenX statistics type. It provides current statistics of all advertisers under an account. It is also possible to retrieve statistics according to start and end dates. Time for action – getting advertisers and campaign statistics Now, let's learn how to get the statistics for a certain advertiser and the campaigns under it. We will then see how to analyze them on a spreadsheet. Click on Statistics menu item. Advertisers & Campaigns screen should automatically open. If not, click on Advertisers & Campaigns link. Now, we should be able to see current date's statistics as a default for all the advertisers under an account. Note that following screenshots are from a real OpenX Ad Server implementation and do not necessarily have to match with your screens. Let's view all the statistics for the advertiser, starting from the installation of OpenX Ad Server. Select All statistics option from the date list. We can now see all of the statistics for the advertiser. Click on tiny arrow near the advertiser name. We can now see the all the campaigns statistics under the advertiser. Click on the Clicks column. You can note that the campaign's statistics are sorted accordingly: Click on the tiny arrow near one of the campaign names.
Read more
  • 0
  • 0
  • 2203