Understanding how Python works to interpret a script, and then executes the commands within, is as important as understanding the Python language itself. Hours of debugging and error checking can be avoided by taking the time to set up Python correctly. The interpretive nature of Python means that a script will first have to be converted into bytecode before it can be executed. We will cover the steps that Python takes to achieve our goal of automating GIS analysis.
How Python executes a script
What is a Python script?
Let's start with the very basics of writing and executing a Python script. What is a Python script? It is a simple text file that contains a series of organized commands, written in a formalized language. The text file has the extension .py, but other than that, there is nothing to distinguish it from any other text file. It can be opened using a text editor such as Notepad or WordPad, but the "magic" that Python does is that it does not reside in a Python script. Without the Python interpreter, a Python script cannot run, and its commands cannot be executed.
Python interpreter
The Python interpreter, in a Windows environment, is a program that has been 'compiled' from the Python source code into a Windows executable and has the extension .exe. The Python interpreter, python.exe, is written in C, an older and extensively used programming language with a more complex syntax.
The Python interpreter, as its name implies, interprets the commands contained within a Python script. When a Python script is run, or executed, the syntax is first checked to make sure that it conforms to the rules of Python (for example, indentation rules are followed, and that the variables follow naming conventions). Then, if the script is valid, the commands contained within are converted into bytecode, a specialized code that is executed by the bytecode interpreter, a virtual machine written in C. The bytecode interpreter further converts the bytecode (which is contained within files that end with the extension .pyc) into the correct machine code for the computer being used, and then the CPU executes the script. This is a complex process, which allows Python to maintain a semblance of simplicity.
Where is the Python interpreter located?
The location of the Python interpreter within the folder structure of a computer is an important detail to master. Python is often downloaded directly from https://www.python.org/, and installed separately from ArcGIS. However, each ArcGIS version will require a specific version of Python; given this requirement, the inclusion of Python within the ArcGIS installation package is helpful. For this book, we will be using ArcGIS 10.5, and this will require Python 2.7.
On a Windows machine, the Python folder structure is placed directly in the C:\ drive unless it is explicitly loaded on another drive. The installation process for ArcGIS 10.5 will create a folder at C:\Python27, which will contain another folder called either ArcGIS10.5 or ArcGIS10.5x64 depending on the version of ArcGIS that has been installed. For this book, we will be using the 32-bit version of ArcGIS, so, the final folder path will be C:\Python27\ArcGIS10.5.
Within that folder are a number of subfolders as well as python.exe, which is the Python interpreter itself. Also included is a second version of the interpreter called pythonw.exe; this version is also very important, as it will execute a script without causing a terminal window to appear. Both python.exe and pythonw.exe contain complete copies of all Python commands, and can be used to execute a script.
Which Python interpreter should be used?
The general rule for executing a script directly using the Python interpreters is to use pythonw.exe, as no terminal window will appear. When there is a need to test code snippets, or to see output within a terminal window, then start python.exe by double-clicking the executable file.
When python.exe is started, a Python interpreter console will appear as seen in the following screenshot:
Note the distinctive three chevrons >>> that appear below the header explaining version information. That is the Python "prompt" where code is entered to be executed line by line, instead of in a completed script. This direct access to the interpreter is useful for testing code snippets and understanding syntax. A version of this interpreter, the Python Window, has been built into ArcMap and ArcCatalog since ArcGIS 10. It will be discussed further in later chapters.
How does the machine know where the interpreter is?
To be able to execute Python scripts directly (that is, to make the scripts run by double-clicking on them), the computer will also need to know where the interpreter sits within its folder structure. To accomplish this requires both administrative account access, and advanced knowledge of how Windows searches for a program. If you have this, you can adjust an environment variable within the advanced system settings dialogue to register the interpreter with the system path.
On a Windows 7/10 machine, click on the Start menu, and right-click on Computer. Then select Properties from the menu. On a Windows 8 machine, open up Windows explorer, right click on This PC, and select Properties from the menu. These commands are shortcuts to get to the Control Panel's System and Security/System menu. Select Advanced system settings from the panel on the left. Click on the Environment Variables button at the bottom of the System Properties menu that appears. In the lower portion of the Environment Variables menu, scroll in the System variables window until the Path variable appears. Select it by clicking on it, and click on the Edit button. The Edit System Variable window will appear like this:
This variable has two components: Variable name (Path) and Variable value. The value is a series of folder paths separated by semicolons. This is the path that is searched when Windows looks for specific executables that have been associated with a file extension. In our case, we will add in the folder path that contains the Python interpreter. Type C:\Python27\ArcGIS10.5 (or the equivalent on your machine) into the Variable value field, making sure to separate it from the value before it with a semi-colon. Press OK to exit the Edit dialogue, OK to exit the Environment Variables menu, and OK to exit the System Properties menu. The machine will now know where the Python interpreter is, as it will search all folders contained within the Path variable to look for an executable called Python. To test that the path adjustment worked correctly, open up a command window (Start Menu/Run, and type "cmd"), and type python.
The interpreter should start directly in the command window:
If the Python header with version information and the triple chevron appears, the path adjustment has worked correctly.
If there is no admin access available, there is a workaround. In a command-line window, pass the entire path to the Python interpreter (for example, C:\Python27\ArcGIS10.5\python.exe) to start the interpreter.
Make Python scripts executable when clicked
The final step to make the scripts run when clicked (which also means they can run outside of the ArcGIS environment, saving lots of memory overhead) is to associate files with the .py extension with the Python interpreter. If the scripts have not already been associated with the interpreter, they will appear as files of an unknown type or as a text file.
To change this, right-click on a Python script. Select Open With, and then select Choose Default Program. If python.exe or pythonw.exe does not appear as a choice, navigate to the folder that holds them (C:\Python27\ArcGIS10.5 in this case), and select either python.exe or pythonw.exe. Again, the difference between the two is the appearance of a terminal window when the scripts are run using python.exe, which will contain any output from the script (but this window will disappear when the script is done). I recommend using pythonw.exe when executing scripts, and python.exe for testing out code. Python scripts can also explicitly call pythonw.exe by saving the script with the extension .pyw instead of .py.
