Building a reusable, maintainable environment

It's worth spending time getting the environment right at the very start of your implementation. This pays dividends in the long run as it can help reduce errors when moving code between environments.

Background

Whether you have just one QlikView Server or two, three, or more servers, you can make life easier and less error-prone by following a few simple rules right from the start. Most installations have their environments structured along the lines of development, test, and production. These could all either be on one machine or spread across several machines. Moving code between environments commonly causes problems because the environments are inconsistent. Hardcoded paths, differently-spelt folders, and different folder structures all have the potential to cause time-consuming and avoidable errors.

How to do it

Let's use Borchester Models as an example here. The company has been incredibly profitable and has purchased three physical servers and three QlikView Server licenses. The development and test machines are fairly small and run QlikView Small Business Server with a handful of users on each machine. The production machine is much bigger and runs QlikView Enterprise Server. The development machine has two disks and all the QlikView work is done on D:\. The test server has only a C:\ drive, so all the QlikView work resides there. The production server has three drives with the QlikView work residing on E:\. The first two developments are underway for Sales and Finance.

We can immediately see that the hardcoded pathnames need to be changed every time a document is moved between environments. We need to design a solution that ensures that code can be developed, tested, and deployed to production without the need to change it every time it moves between environments.

In this example, we must do two things to ensure that code can be moved easily and without changes. Firstly, the folder structures need to be identical on all three machines, but they don't have to start at the same place on each machine. Secondly, we always use relative paths in our load script.

The folder structures might look something similar to this:

Development machine:

D:\
  D:\OtherStuff
  D:\QlikView
    D:\QlikView\Sales (sales.qvw lives here)
      D:\QlikView\Sales\Data
    D:\QlikView\Finance (finance.qvw lives here)
      D:\QlikView\Finance\Data

Test machine:

C:\
  C:\OtherStuff
  C:\BI
    C:\BI\QlikView
      C:\BI\QlikView\Sales (sales.qvw lives here)
        C:\BI\QlikView\Sales\Data
      C:\BI\QlikView\Finance (finance.qvw lives here)
        C:\BI\QlikView\Finance\Data 

Production machine:

E:\
  E:\OtherStuff
  E:\BI
    E:\BI\Sales (sales.qvw lives here)
      E:\BI\Sales\Data
    E:\BI\Finance (finance.qvw lives here)
      E:\BI\Finance\Data 

We normally want to have further separation between various file types, so we might consider a more complete structure, such as this:

E:\BI\Sales\Includes 
E:\BI\Sales\Data\QVD
E:\BI\Sales\Data\Excel

In the preceding folder structures, where the QlikView documents reside is different on each machine, but this doesn't matter. As long as they're identical to where the QlikView document resides downwards, this will work. Furthermore, if sales.qvw needs to access a file called ForSales.qvd that resides in Finance\Data, we can code the relative path in our script, [..\Finance\Data\ForSales.qvd]. Thus, we'll pick up this file no matter which environment we happen to be running in, always assuming the file is there, of course!

There are many variations on this theme. If all the environments were on a single server, we would probably want to use folder permissions and a structure similar to this:

Everything machine:

E:\
  E:\BI
  E:\BI\Development
    E:\BI\Development\Sales (sales.qvw lives here)
E:\BI\Development\Sales\Data
    E:\BI\Development\Finance (finance.qvw lives here)
E:\BI\Development\Finance\Data
  E:\OtherStuff
  E:\BI
    E:\BI\Test
      E:\BI\Test\Sales (sales.qvw lives here)
        E:\BI\Test\Sales\Data
      E:\BI\Test\Finance (finance.qvw lives here)
        E:\BI\Test\Finance\Data
    E:\BI\Production
      E:\BI\Production\Sales (sales.qvw lives here)
        E:\BI\Production\Sales\Data
      E:\BI\Production\Finance (finance.qvw lives here)
        E:\BI\Production\Finance\Data

Once again, this will work because the relative paths are still the same.

Obviously, this principle can be applied to a PC if that's where you do your development. Things get a little more complicated if the data resides on multiple disks or is not on locally attached storage. We'll discuss solving this with variables in INCLUDE files in Chapter 6, Make It Easy on Yourself–Some QlikView Tips and Tricks.

It's not uncommon to see folder structures that have a number prefix that gives some sense of sequencing to the folders. Here's an example:

DocumentName (for example, Sales)

1. Includes: These are any INCLUDE files needed by any of the QVWs in this application.
2. Source Data: This is the original source data; this folder could have subfolders for Excel and text if required.
3. ETL: These are all QVWs that convert source data to QVD files. Multiple stages could have the QVWs prefixed by a sequence number.
4. QVD: This includes all the files generated in 2.
5. Application: sales.qvw resides here.
6. Archive: This contains old copies of data or documents for reference but is probably best avoided in a production environment.
7. Documentation: This includes any specifications or developer notes, along with version control information, such as release notes.

You can devise any kind of folder structure you like to suit your working methods, but the golden rule is to keep it consistent between environments.