Proxmox High Availability

Chapter 1. Basic Concepts of a Proxmox Virtual Environment

Have you ever imagined how good it would be if you don't need to stop your services during a system upgrade operation? Do you have a powerful server but haven't used all of its resources? Do you want to set up a server platform quickly using a system template? If the answer to all these questions is yes, then you will be very happy to work with Proxmox Virtual Environment (also called Proxmox VE).

This chapter will show you some basic concepts of Proxmox VE before actually using it, including the technology used, basic administration, and some options available during set up.

The following topics are going to be covered in this chapter:

An explanation of server virtualization used by Proxmox VE
An introduction of basic administrative tools available in Proxmox VE
An explanation of different virtualization modes and storage options

Introduction to Proxmox Virtual Environment

So what is Proxmox Virtual Environment actually used for? Proxmox VE is an open source bare metal environment based on the Debian Linux distribution (also called hypervisor or Virtual Machine Monitor (VMM)) for server virtualization. It allows a user to install different operating systems (for example, Windows, Linux, Unix, and others) on a single computer or a cluster built by grouping computers together. It consists of powerful Kernel-based virtual machines and lightweight OpenVZ containers as an alternative.

The main features for Proxmox VE can be summarized as follows:

Open source: It is fully open source under General Public License, version 3 (GNU AGPL, v3), which means you can freely view, alter, and remove the source code, and distribute your own version as long as you are compliant with the license.
Live migration: This allows moving a running virtual machine from one physical server to another without downtime.
High availability: In Proxmox HA cluster mode, when one node fails, the remaining virtual machines will be moved to a healthy node to make sure there is minimal service interruption.
Bridged networking: Proxmox VE allows a user to build a private network between the virtual machines. VLAN options are also available.
Flexible storage: A wide range of storage options are available, including both local and network-based storage technologies such as LVM, iSCSI, NFS, the Gluster filesystem, and the CEPH filesystem.
OS template: Proxmox VE allows users to build their own OS template for further deployment. Of course, it is also possible for users to download a template file from the Internet and import that file into their system.
Scheduled backup: A user interface is provided to users so that they can set up their own backup strategy. The backup files can be stored locally or on any supported storage option that you have configured.
Command-line (CLI) tool: Proxmox VE provides different CLI management tools allowing users to access the virtual machine container, manage resources, and so on.

Tip

You can try Proxmox for free at http://pve.proxmox.com/wiki/Downloads.

Introduction to server virtualization

Have you ever heard about cloud computing? It is a hot topic in the IT industry and claims that you can allocate nearly unlimited computer resources on a pay-as-you-go basis. Are you not curious to know how they are able to provide such a service? The underlying technology that allows them to be able to provide such a service is hardware virtualization. If you don't understand how virtualization works, it could be difficult for you to imagine how they can add or remove resources instantly. Let's talk about how server virtualization works and what it offers us.

Why should we use server virtualization? Since the new generation of server class machines are becoming much more powerful, it becomes quite difficult to use up all the system resources if we only install one Operating System (OS) on it. Also, renting multiple server racks in a data center is expensive. Centralizing multiple servers into a limited set of powerful servers seems to be a more cost-effective solution, thus virtualization appears. Server virtualization allows users to create multiple system objects called virtual machines (VMs) that act like normal computers. Virtualization of physical devices implies that each virtual machine has its own CPU units (called vCPU), memory, hard disk, and network card, according to a user's allocation. When the user turns on the virtual machines, different OSes can be installed on them. Therefore, better system resources utilization can be achieved. The following figure shows the difference between a physical machine and a virtual machine:

Depending on the kind of processor used, there are three different types of virtualizations available: full virtualization, para-virtualization, and hardware-assisted virtualization. In order to state the difference between these virtualization methods, we need to know how a Control Processing Unit (CPU) executes code that a user has passed.

During any process execution, a CPU is the computing unit that executes predefined instruction sets to generate the results that the program had defined. But, it is dangerous if we give full access to all applications on our devices. For example, if there is no restriction on hardware access, a web page will be able to inject suspicious code into local memory and it may further damage the data stored in our computers. Therefore, a term called protection ring ranking, which ranges from 0 to 3 under the x86 architecture is used to protect our hardware. Normally, Ring 0 (also called supervisor mode) is used in the OS to monitor and control system resources. Ring 3 (also called user mode) is used for a user application, and if we would like to have access to hardware, a system call provided from the supervisor mode must be made. The following figure shows the system ring structure for the x86 platform:

Based on the levels of virtualization, we have the following different types of virtualization types: full virtualization, para-virtualization, and hardware-assisted virtualization:

Full virtualization: In this the VMM is placed under Ring 0 while the virtualized guest OS is installed under Ring 1. However, some system calls can only be executed under Ring 0. Therefore, a process called binary translation is used to translate such system calls, and thus, the performance is degraded. In this mode, the guest OS does not know it is being virtualized, so it does not require kernel modification. Here is a simple structure for this type of virtualization:
Para-virtualization: This is very similar to full virtualization, but custom drivers are installed on the guest OS in order to access CPU resources without downgrading to Ring 1. So, the performance of the guest OS is near to that of the physical machine because the translation process is not needed, but the guest OS requires a modified kernel. Thus, the guest cannot run a different operating system from the host operation system. The following diagram shows the structure of this virtualization:
Hardware-assisted virtualization: CPU manufacturers introduce a new functionality for a virtualized platform, Intel VT-x and AMD-V. The ring level 0 to 3 is categorized into non-root modes, and a new level, -1, is introduced as the root mode. The guest OS is now installed to Ring 0, which means it can access hardware directly. Because it does not need a custom API to make system calls under Ring 0, no kernel modification is needed. The following diagram shows you the structure of the virtualization mode:

Comparing types of server virtualization software

We have discussed why we need to learn server virtualization and how virtualization works, so are you curious about how many types of major virtualization software are on the market? What are the differences between them? Let's take a deep look at it:

Proxmox VE: As mentioned before, Proxmox VE is an open source hypervisor based on GNU/Linux (Debian-based) with a RHEL-based kernel and published under GNU AGPL v3. It differs from the alternative virtualization software as Proxmox provides a central web-based management without further installation. The underlying technologies used are Open Virtuozzo (OpenVZ) and Kernel-based Virtual Machine (KVM), which will be described in Version 1.6. Subscription plans are available for accessing enterprise repository, software updates, and user support.
XenServer: This is a native hypervisor based on GNU/Linux developed by the Xen Project and published under GNU AGPL v2 as open source. For XenServer, a concept of domain is used for all virtual machines, and the most privileged domain (for example, a domain that allows direct access to hardware)—dom0, is used by the hypervisor to manage other domU virtual machines. It supports para-virtualization, which allows a user to run virtualized guests on a CPU without support for virtualization; for example, no Intel VT-x or AMD-V is needed. Amazon Web Service (AWS) is a production example of using XenServer.
VMware ESX/ESXi: This is a bare-metal hypervisor developed by VMware based on a customized kernel called vmkernel, which is a microkernel developed by VMware.
The difference between ESX and ESXi is that ESXi is a free version of ESX with some resource limitations. ESX has a hardware compatibility list that includes many drivers for network cards and SCSI cards. An extra hardware driver can be added to the base installation if needed. On top of the para-virtualization and hardware-assisted virtualization, ESX provides full virtualization as another option.
There are two management tools available: vSphere client and vCenter server. VSphere client is enough for normal administration operation on one ESX while vCenter server allows the user to manage multiple ESXs, including the configuration of advanced features such as high availability and live migration.
Hyper-V server: This is a proprietary virtualization platform produced by Microsoft Corporation running under the Windows platform starting from Windows Server 2008. If you are mainly using a Windows platform as your virtualized guest, it is recommended that you use Hyper-V, especially if you have enabled an Active Directory domain services.
Hyper-V provides better migration options to users; it not only provides live migration, but also provides unlimited guest movements between hosts. The benefit of the features of an Active Directory domain is that Hyper-V provides replica on virtual machines, which allows a user to copy a specific VM from the source's site to the target site asynchronously via a WAN or a secure VPN.

Comparison table of hypervisors

We have learned that there are many virtualization platforms on the market, so what are the differences between them? Let's take a look at the following table:

Virtualization platform	Proxmox	XenServer	VMware ESX/ESXi	Hyper-V server
Latest version	3.2	6.1	5.5	Server 2012 R2
License	GNU GPL v3	GNU GPL v2	Proprietary	Free
Open Source	Yes	Yes	No	No
Base OS	Linux	Linux	Vmkernel	Windows
Console OS	No	Yes	Yes	No
Management tools	Web GUI	XenCenter	vSphere Client vCenter	Hyper-V Manager
Host HA	Yes	Yes	Yes (vCenter)	Yes
Guest HA	Yes	Yes, (XenMotion)	Yes, (vMotion)	Yes (Live Migration)
Supported storage	LVM groupNFS, iSCSI, RBD, NAS, and SAS	NFS, iSCSI, fiber channel, NAS, SAS, and CIFS	Fibre Channel, iSCSI, NAS, and SAS	SMB, iSCSI, Fibre channel, NAS, and SAS
Supported virtualization technology*	FV (KVM) OpenVZ	PV HV	FV PV HV	PV HV
License model	Per CPU	Per CPU	Per CPU	Per VM and host

In the previous table, FV stands for full virtualization, PV stands for para-virtualization, and HV stands for hardware-assisted virtualization.

Basic administration on Proxmox VE

After we have talked about so many concepts, let's learn how to configure Proxmox! I assume you have downloaded and installed your own Proxmox. If you face any difficulty during the installation, you can refer to the following link:

https://pve.proxmox.com/wiki/Quick_installation

The following steps will help you perform basic administration in Proxmox VE:

After the installation, you should be able to see the following screen:
In this step, access the Proxmox VE using a web browser, and make sure you have network access and you are connected in the same subnet as Proxmox. Then, type in a URL in the following format in your browser: https://<Proxmox_IP>:8006. When you first visit the page, your browser will warn you that the SSL certificate is not a trusted one. It is normal if you have experience in building your self-signed SSL certificate on your web server. If you have a domain name for your Proxmox server and would like to have a SSL certificate signed by an authorized CA, please take a look at the following sites:
- Apache CSR creation using OpenSSL http://www.digicert.com/csr-creation-apache.htm
- HTTPS certificate configuration http://pve.proxmox.com/wiki/HTTPSCertificateConfiguration
Next, simply choosing Continue to this website in Internet Explorer adds to the exception. Similarly, choose I understand the risk in Firefox and Chrome. Next, just log in to the system with the root account since we haven't created our own user account. Being logged in, the following warning message indicates that you do not have a valid subscription plan. You can ignore this warning message if you currently don't need to have technical support.
Within the web GUI, you will be able to manage most features of the Proxmox platform. For example, you can create new virtual machines by using the buttons located in the upper-right corner, as shown in the following screenshot:
There are two ways to create new virtual machines: Create VM and Create CT.
By clicking on the Create CT button, you can create a virtualized Linux platform under the OpenVZ container. It is recommended for GNU/Linux guests because the Linux kernel is shared with the host server, which means that the used memory on the host server will be reduced. When you go through the wizard after clicking on the Create CT button, a virtualized OS is created based on the OS template you used. An OS template is an operation system (for example, CentOS, Debian, and others) with customized packages installed and predefined system parameters. The templates are available in the form of a brand new system or installed with a web server or an application like Drupal. Typically, no graphical user interface (GUI) is preinstalled in the virtualized guests if you use this type of installation.

Uploading the OS template or the ISO file to Proxmox

Before we can start trying to build our own virtual machines with OpenVZ, we have to first upload either an OS template or an ISO file that contains the operating system installation files to Proxmox. The following steps will help us understand how to do this:

Double-click on the icon that is named as the hostname of Proxmox, then choose local, click on Content, and choose Upload from the panel on the right-hand side, as shown in the following screenshot:
After we have clicked on the Upload button, three options are available, as shown in the following screenshot:
The three options available are as follows:
- ISO image: This is used as the installation source file for Kernel-based virtualization
- VZDump backup file: This is used to upload backup files created by Proxmox for restoration
- OpenVZ template: This is used for the installation source for OpenVZ virtualization
Then, we can simply browse the file that we would like to upload. Pay attention when uploading the OpenVZ template file; you need to use the following naming format:
```
{OSName}-{OSVersion}-{OSName}_{OSType}_{amd64/i386}.tar.gz
```
This format is shown in the following example:
```
centos-6-CentOS_standard_amd64.tar.gz
```
The output can be seen in the following screenshot:

We are now ready to create our own virtual machines; let's see how we can create an OpenVZ-based virtual machine first. For example, the OpenVZ template file will be stored inside /var/lib/vz/template/cache/; you can also manually upload it to that directory if you don't wish to use the web interface.

Creating an OpenVZ-based virtual machine

To create an OpenVZ-based virtual machine, please follow these steps:

Click on the Create CT button, and the following window pops up; a system hostname and a password for the root account can be defined, as shown in the following screenshot:
Next, the system will ask you for the template you want to use; for example, you can choose centOS-6-CentOS_devel_amd64.tar.gz:
Then, you will be asked to assign resources to the VM; we can accept the default setting for testing purposes:
After we have allocated system resources to the new virtual machine, we reach an important part: networking. Here, you have the following two options:
- Routed mode: This uses the default network interface used by Proxmox, and simply specifies an IP address. Only select routed mode if you create a guest system that lies in the same IP subnet as the host system. It would be slightly easier during the system configuration than in the Bridged mode.
- Bridged mode: This allows you to choose a network adapter other than the default one (including a VLAN-enabled interface). It requires a DHCP server for IP assignment or is manually configured under VM. A demonstration on how to configure the Bridged mode will be shown in Chapter 4, Configuring a Proxmox VE Cluster.
In the following screenshot, we choose the Routed mode and specify an IP address for demonstration:
The following settings regarding the DNS parameters are saved to the /etc/resolv.conf file in the virtualized guest:
A summary for the configuration is displayed before the VM is created.
After the VM creation is completed, you can view its status in the Summary tab, as shown in the following screenshot:

Creating a kernel-based virtual machine

We have seen how to create OpenVZ-based virtual machines; now, let's learn how we can create a kernel-based virtual machine:

Click on the Create VM button in the top-right corner, as shown in the following screenshot:
Specify a name for identification, but please note that the name is not directly applied to the VM's hostname, as shown in the following screenshot:
Then, we need to choose the type of platform we are going to create, shown as follows:
Next, the OS installation source is specified, shown as follows:
After that, we have to allocate the hard disk for the virtual machine. Pay attention to the hard disk's Format you have chosen; the raw format will instantly allocate the space you have defined (that is, if you defined a 20 GB hard disk, a disk image with 20 GB will be created under the host server) while the QEMU format will only allocate the space based on the current usage of the guest OS. If the guest OS takes up 8 GB, then a disk image with 8 GB in size will be created. For more details, refer to the Virtual disk options under Proxmox VE section. The hard disk window is shown in the following screenshot:
The number of cores/sockets and the CPU types have to be specified for the guest, as shown in the following screenshot:
Of course, we need to allocate the amount of memory for the guest platform, shown as follows:
For the network part, similar to the OpenVZ creation, we have bridged mode and NAT mode. Here, we can specify Model for the virtualized VM; generally, choosing Intel E1000 and Realtek RTL8139 should be fine. The VirtIO option provides better performance, but the driver must be installed on the guest OS before it can be used. This is shown in the following screenshot:
A summary of the configurations is displayed for confirmation.
Just as it appears for OpenVZ, we can find our created VM under the menu, as follows:

Isn't it easy to create a virtual machine with a web interface? By the way, you can identify whether a virtual machine is an OpenVZ container or KVM by its associated icons, as shown in the following screenshot:

Accessing the new virtual machine

Congratulations! You have just created your own virtual machine! Wait, how can we access the system? That's a good question; here, we have two options to achieve our goal. One option is to access it from the web browser. What? Access an operating system with a web browser? Yes, let's see how it works:

Right-click on the virtual machine you would like to access from the panel on the left-hand side, and then choose console.
Prior to Proxmox 3.2, a new console mode, SPICE, was introduced, which provides a better usage performance especially in a KVM machine. You can refer to https://pve.proxmox.com/wiki/SPICE for more details.
Make sure you have the latest JAVA runtime (JRE), preferably with JAVA 7 on your browser, and allow it to run if there is any prompt message. You can download it from http://www.oracle.com/technetwork/java/javase/downloads/index.html. If you cannot run the applet, refer to Chapter 8, Troubleshooting on a Proxmox Cluster, for troubleshooting.

Pay attention to OpenVZ-based VM; you will see that it cannot be accessed from the console because you have to configure a getty service inside the VM. So, we need to use another method for this purpose, for example, the OpenVZ management CLI:

Access your Proxmox console with the root account, as shown in the following screenshot:
You can check the existing running OpenVZ containers with the vzlist command as shown in the following screenshot:
Then, access it with vzctl enter <CTID>, for example, vzctl enter 102.
Create a new file under /etc/init.d/tty.conf with vi or any text editor you like.

Paste the following code in the tty.conf file:

# This service maintains agetty on tty1 from the point the 
# system is started until it is shut down again.
start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]
respawn
exec /sbin/agetty -8 tty1 38400

Reboot the guest VM or type start tty to reboot.

Virtualization options in Proxmox VE

There are two types of virtualizations available in Proxmox: OpenVZ and KVM. What are the differences between them?

OpenVZ is an operating-system-level virtualization based on the GNU/Linux kernel and the host operation system. Theoretically, OpenVZ is not a type of virtualization but more like the jail concept in Linux. Since a patched Linux kernel is needed, only Linux guests can be created. All guests are called containers that share the same kernel and architecture as long as the host OS, while each container reserves a separate user space.

There is no overhead for OpenVZ as containers can call hardware resources directly. However, since all containers share the system kernel of the host OS, a system-related problem might appear during the host OS kernel upgrade. Besides, OpenVZ stores container files as normal files in the host OS, so it is not recommended to use OpenVZ if there are confidential files stored in the virtual machine. Kernel-based Virtual Machine (KVM) is basically a hardware-assisted virtualization with the modified Linux kernel built with the KVM module. KVM itself does not perform any emulation or virtualization. Instead, it simply exposes the /dev/kvm interface. QEMU is chosen as a software-based emulator to simulate hardware for the virtualized environment. The structure of KVM is shown as follows:

As we can see, overheads on frequent requests appear in QEMU-emulated devices. Thus, an improved version for KVM is published with VirtIO drivers. VirtIO creates a buffer for both the guest system and QEMU, which speeds up the I/O performance and reduces the overhead. To enjoy the performance burst, a VirtIO driver must be installed separately on each emulated hardware device. In the following diagram, we have demonstrated the new structure of the KVM machines with the VirtIO drivers installed:

The following table shows the supported operating systems provided by OpenVZ and KVM:

Virtualization method	Supported operating system
OpenVZ	CentOS, Debian, Fedora, Scientific Linux, SUSE, and Ubuntu
KVM	FreeBSD, Windows Server 2000/XP/2003/2008, Windows 7/8, and all OS supported by OpenVZ

Tom Jan 14, 2015

Proxmox High Availability is maybe not a book for beginners who are testing their first virtualization setups (although it covers some basics), but for admins that already have some experience in this environment.It definitely gives good advice as well as really good hints and examples in setting up clustered servers.What I do like most about the book is that it does not only explain Proxmox itself but also how to install drdb, glusterfs, (clustered filesystems) etc..So everything you need to build a complex and robust environment is included here.

Amazon Verified review

Matteo Jan 19, 2015

This is a good book to start using and planning/designing Proxmox clusters, giving you all the basics on storage, networking and compute setup for creating a safe and reliable enviroment, avoiding design and implementation errors if you are not a Proxmox expert. This book is also a good starting point to evaluate virtualization at low cost for home/SME enviroments without the CAPEX of VMware or Hyper-V. There are some minor faults with some diagrams but nothing major, use the book as a starting point for your lab and grow your skills over time

Juan Vicente Herrera Ruiz de Alejo Jan 13, 2015

This book, Proxmox High Availability, is recommended for people that have expertise working with ProxmoxYou can find a detailed explanation about how to getting started to build a Proxmox HA environment with Proxmox VE Cluster, a useful explanation about how to migrate an existing system to a VE Cluster, disaster recovery of the cluster and a last section about troubleshooting.It is a well written book with practical and real examples and definitely worth a shot.

k0d3g3ar Mar 12, 2015

Good book. Definitely a must have for anyone doing more than 1 Proxmox server installation. However if you are just new to the topic, this is probably way too much detail. Also I found it lacking when dealing with troubleshooting clustering in PM. This is a big topic that most admins need some help with. A really good approach to troubleshooting in the book would go a long way to help here. Otherwise its solid, complete and well worth investing in.

Proxmox High Availability: Discover how to introduce, design, and implement high availability clusters for your business without hassle

What do you get with Print?

Proxmox High Availability

Chapter 1. Basic Concepts of a Proxmox Virtual Environment

Introduction to Proxmox Virtual Environment

Tip

Introduction to server virtualization

Server virtualization basics – guest versus host

Comparing types of server virtualization software

Comparison table of hypervisors

Basic administration on Proxmox VE

Uploading the OS template or the ISO file to Proxmox

Creating an OpenVZ-based virtual machine

Creating a kernel-based virtual machine

Accessing the new virtual machine

Virtualization options in Proxmox VE

Virtual disk options under Proxmox VE

Introducing the OpenVZ template

Summary

Page 1 of 10

Description

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs

Proxmox High Availability: Discover how to introduce, design, and implement high availability clusters for your business without hassle

What do you get with Print?

Description

What you will learn

Product Details

What do you get with Print?

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs