Why is cloud computing so popular?

Depending on who you ask, some estimates peg the global cloud computing market at around USD 370 billion in 2020, growing to about USD 830 billion by 2025. This implies a Compound Annual Growth Rate (CAGR) of around 18% for the period.

There are multiple reasons why the cloud market is growing so fast. Some of them are listed here:

• Elasticity
• Security
• Availability
• Faster hardware cycles
• System administration staff
• Faster time to market

Let's look at the most important one first.

Elasticity

Elasticity may be one of the most important reasons for the cloud's popularity. Let's first understand what it is.

Do you remember the feeling of going to a toy store as a kid? There is no feeling like it in the world. Puzzles, action figures, games, and toy cars are all at your fingertips, ready for you to play with them. There was only one problem: you could not take the toys out of the store. Your mom or dad always told you that you could only buy one toy. You always had to decide which one you wanted and invariably, after one or two weeks of playing with that toy, you got bored with it and the toy ended up in a corner collecting dust, and you were left longing for the toy you didn't choose.

What if I told you about a special, almost magical, toy store where you could rent toys for as long or as little as you wanted, and the second you got tired with the toy you could return it, change it for another toy, and stop any rental charges for the first toy? Would you be interested?

The difference between the first traditional store and the second magical store is what differentiates on-premises environments and cloud environments.

The first toy store is like having to set up infrastructure on your own premises. Once you purchase a piece of hardware, you are committed to it and will have to use it until you decommission it or sell it at a fraction of what you paid for it.

The second toy store is analogous to a cloud environment. If you make a mistake and provision a resource that's too small or too big for your needs, you can transfer your data to a new instance, shut down the old instance, and importantly, stop paying for that instance.

More formally defined, elasticity is the ability of a computing environment to adapt to changes in workload by automatically provisioning or shutting down computing resources to match the capacity needed by the current workload.

In AWS as well as with the main cloud providers, resources can be shut down without having to completely terminate them, and the billing for resources will stop if the resources are shut down.

This distinction cannot be emphasized enough. Computing costs in a cloud environment on a per-unit basis may even be higher when comparing them with on-premises prices, but the ability to shut resources down and stop getting charged for them makes cloud architectures cheaper in the long run, often in a quite significant way. The only time when absolute on-premises costs may be lower than cloud costs is if workloads are extremely predictable and consistent. Let's look at exactly what this means by reviewing a few examples.

Web storefront

A popular use case for cloud services is to use it to run an online storefront. Website traffic in this scenario will be highly variable depending on the day of the week, whether it's a holiday, the time of day, and other factors.

This kind of scenario is ideally suited for a cloud deployment. In this case, we can set up resource load balancers that automatically start and shut down compute resources as needed. Additionally, we can set up policies that allow database storage to grow as needed.

Apache Spark and Hadoop workloads

The popularity of Apache Spark and Hadoop continues to increase. Many Spark clusters don't necessarily need to run consistently. They perform heavy batch computing for a period and then can be idle until the next batch of input data comes in. A specific example would be a cluster that runs every night for 3 or 4 hours and only during the working week.

In this instance, the shutdown of resources may be best managed on a schedule rather than by using demand thresholds. Or, we could set up triggers that automatically shut down resources once the batch jobs are completed.

Online storage

Another common use case in technology is file and object storage. Some storage services may grow organically and consistently. The traffic patterns can also be consistent.

This may be one example where using an on-premises architecture may make sense economically. In this case, the usage pattern is consistent and predictable.

Elasticity is by no means the only reason that the cloud is growing in leaps and bounds. Having the ability to easily enable world-class security for even the simplest applications is another reason why the cloud is becoming pervasive. Let's understand this at a deeper level.

Security

The perception of on-premises environments being more secure than cloud environments was a common reason that companies big and small would not migrate to the cloud. More and more enterprises are now realizing that it is extremely hard and expensive to replicate the security features that are provided by cloud providers such as AWS. Let's look at a few of the measures that AWS takes to ensure the security of its systems.

Physical security

You probably have a better chance of getting into the Pentagon without a badge than getting into an Amazon data center.

AWS data centers are continuously upgraded with the latest surveillance technology. Amazon has had decades to perfect the design, construction, and operation of its data centers.

AWS has been providing cloud services for over 15 years and it literally has an army of technologists, solution architects, and some of the brightest minds in the business. It leverages this experience and expertise to create state-of-the-art data centers. These centers are in nondescript facilities. You could drive by one and never know what it is. If you do find out where one is, it will be extremely difficult to get in. Perimeter access is heavily guarded. Visitor access is strictly limited, and they always must be accompanied by an Amazon employee.

Every corner of the facility is monitored by video surveillance, motion detectors, intrusion detection systems, and other electronic equipment.

Amazon employees with access to the building must authenticate themselves four times to step on the data center floor.

Only Amazon employees and contractors that have a legitimate right to be in a data center can enter. Any other employee is restricted. Whenever an employee does not have a business need to enter a data center, their access is immediately revoked, even if they are only moved to another Amazon department and stay with the company.

Lastly, audits are routinely performed and part of the normal business process.

Encryption

AWS makes it extremely simple to encrypt data at rest and data in transit. It also offers a variety of options for encryption. For example, for encryption at rest, data can be encrypted on the server side, or it can be encrypted on the client side. Additionally, the encryption keys can be managed by AWS, or you can use keys that are managed by you.

Compliance standards supported by AWS

AWS has robust controls in place to allow users to maintain security and data protection. We'll be talking more about how AWS shares security responsibilities with its customers, but the same is true with how AWS supports compliance. AWS provides many attributes and features that enable compliance with many standards established by different countries and standards organizations. By providing these features, AWS simplifies compliance audits. AWS enables the implementation of security best practices and many security standards, such as these:

• ITAR
• SOC 1/SSAE 16/ISAE 3402 (formerly SAS 70)
• SOC 2
• SOC 3
• FISMA, DIACAP, and FedRAMP
• PCI DSS Level 1
• DOD CSM Levels 1-5
• ISO 9001/ISO 27001/ISO 27017/ISO 27018
• MTCS Level 3
• FIPS 140-2
• HITRUST

In addition, AWS enables the implementation of solutions that can meet many industry-specific standards, such as these:

• Criminal Justice Information Services (CJIS)
• Family Educational Rights and Privacy Act (FERPA)
• Cloud Security Alliance (CSA)
• Motion Picture Association of America (MPAA)
• Health Insurance Portability and Accountability Act (HIPAA)

Another important thing that can explain the meteoric rise of the cloud is how you can stand up high-availability applications without having to pay for the additional infrastructure needed to provide these applications. Architectures can be crafted in such a way that additional resources are started when other resources fail. This ensures that we only bring additional resources when they are necessary, keeping costs down. Let's analyze this important property of the cloud in a deeper fashion.

Availability

When we deploy infrastructure in an on-premises environment, we have two choices. We can purchase just enough hardware to service the current workload, or we can make sure that there is enough excess capacity to account for any failures that may occur. This extra capacity and the elimination of single points of failure is not as simple as it may first seem. There are many places where single points of failure may exist and need to be eliminated:

• Compute instances can go down, so we need to have a few on standby.
• Databases can get corrupted.
• Network connections can be broken.
• Data centers can flood or burn down.

This last one may seem like a hypothetical example but there was a fire reported in the suburb of Tama in Tokyo, Japan, that apparently was at an AWS data center under construction. Here is a clip of the incident:

https://www.datacenterdynamics.com/en/news/aws-building-site-burns-in-fatal-tokyo-fire-reports-say/

Using the cloud simplifies the "single point of failure" problem. We already determined that provisioning software in an on-premises data center can be a long and arduous process. In a cloud environment, spinning up new resources can take just a few minutes. So, we can configure minimal environments knowing that additional resources are a click away.

AWS data centers are built in different regions across the world. All data centers are "always on" and delivering services to customers. AWS does not have "cold" data centers. Their systems are extremely sophisticated and automatically route traffic to other resources if a failure occurs. Core services are always installed in an N+1 configuration. In the case of a complete data center failure, there should be the capacity to handle traffic using the remaining available data centers without disruption.

AWS enables customers to deploy instances and persist data in more than one geographic region and across various data centers within a region.

Data centers are deployed in fully independent zones. Each data center is constructed with enough separation between them such that the likelihood of a natural disaster affecting two of them at the same time is very low. Additionally, data centers are not built in flood zones.

To increase resilience, data centers have discrete Uninterruptable Power Supplies (UPSes) and onsite backup generators. They are also connected to multiple electric grids from multiple independent utility providers. Data centers are connected redundantly to multiple tier-1 transit providers. Doing all this minimizes single points of failure.

Faster hardware cycles

When hardware is provisioned on-premises, from the instant that it is purchased it starts becoming obsolete. Hardware prices have been on an exponential downtrend since the first computer was invented, so the server you bought a few months ago may now be cheaper, or a new version of the server may be out that's faster and still costs the same. However, waiting until hardware improves or becomes cheaper is not an option. At some point, a decision needs to be made and a purchase needs to be made.

Using a cloud provider instead eliminates all these problems. For example, whenever AWS offers new and more powerful processor types, using them is as simple as stopping an instance, changing the processor type, and starting the instance again. In many cases, AWS may keep the price the same even when better and faster processors and technology become available.

System administration staff

An on-premises implementation may require full-time system administration staff and a process to ensure that the team remains fully staffed. By using cloud services, many of these tasks can be handled by the cloud providers, allowing you to focus on core application maintenance and functionality and not have to worry about infrastructure upgrades, patches, and maintenance.

By offloading this task to the cloud provider, costs can come down because instead of having a dedicated staff, the administrative duties can be shared with other cloud customers.