Author Posts - Cloud & Networking

The definition of DevOps is a hotly disputed topic among amateur practitioners and experienced engineers alike. Ironically, DevOps was actually supposed to bring some order into the messy and chaotic environment of IT software development. In DevOps Paradox, DevOps expert Viktor Farcic talks to fellow industry figures who reveal their perspectives on the trend and what it means to them. In this article, we’ll see what some prominent people in the DevOps community have to say about DevOps. The quotes in this article are taken directly from the book. So, how are we supposed to incorporate DevOps into our organizations if we don't even know what it is? Let’s hear Viktor’s thoughts about what DevOps is and what are it’s trends and future aspects: What is DevOps and why do we need it? What is DevOps and why do we need it? What is the most important thing DevOps helps us achieve? What are the factors that drive the development of DevOps? Viktor Farcic: Almost everyone gives a different answer to the question “What is DevOps?”—there is a huge discrepancy between the idea and the implementation. I believe that the main objective of DevOps is to enable self-sufficient product-oriented teams capable of having a full control of their products. That is in stark contrast with the way many companies operate today. Normally, a lifecycle of an application is split between many teams. Business analysts define requirements, architects work on guidelines that must be followed and frameworks that must be used, developers write code, testers are in charge of validations, operators deploy new releases, and so on and so forth. The problem is that each of those groups belong to different departments and have different and often opposing objectives. Instead of fostering collaboration towards a common goal, different teams (departments) are looking for their own shortsighted interests. DevOps tries to remove organization based on the type of tasks performed and unite all the expertise required for the whole lifecycle of an application into a single team reporting to a single person. It forces us to work together and it builds empathy. Is DevOps a process? Or a set of technologies? What's your perspective on this area of debate? Viktor: DevOps is neither of the two. Unlike some agile frameworks (e.g. Scrum), there is no prescribed process to follow. Similarly, there is no technology we can adapt that will convert us into “DevOps” teams. It is only an idea that developers, operators, and everyone else needs to work together instead of being isolated in different silos. That does not mean that technology is not important; it is, but often for other than obvious reasons. Every new technology is created by a group of people that worked together to create it. As such, it always reflects processes of those involved in creating it. Those processes, on the other hand, are a result of the culture of the people following it. In other words, every tech is a result of certain processes created as a result of the culture of the team that worked on it. So, even though we use an end result, it is a product of a process created in a specific culture. If we adopt a technology that does not match our own processes and culture, it will produce suboptimal results at best. So, we must either adopt technology that matches our processes and culture or use it to change them. One cannot work without the other. All in all, DevOps is an idea, not much more than that. It’s up to us to figure out which processes and technology will help us make it reality. What does a DevOps engineer do? Is it even a real job role? What are the core roles of DevOps Engineers in terms of development and Infrastructure? Viktor: I don't think there is such a thing as a “DevOps Engineer.” The term was invented by people who were not ready to apply the changes DevOps leads to. Most of the time, a “DevOps Engineer” is just a different name for someone working in shared services, operations, infrastructure, or whichever department was first to be renamed into DevOps. Do you think DevOps is experiencing an identity crisis? Viktor: DevOps was never defined as a process. Agile, for example, got quite a few implementations that tell people what to do. Among others, we got Scrum that clearly defines what to do. We could even argue that Scrum, as being a set of practices that must be followed, is against the spirit of Agile, but that’s a conversation for some other time. What matters is that no one defined the process behind DevOps. There is no such thing as a set of steps that must be followed daily or weekly. It is just an idea that we should work together and not throw things over department walls. As such, the way to accomplish that is open to interpretation. So, DevOps never had a clear identity, so it cannot have an identity crisis either. It’s just an idea, and it’s up to each one of us to try to figure out how to make it reality. The biggest challenges in DevOps today What are the biggest challenges in DevOps at the moment? Viktor: Currently, DevOps is mostly misunderstood. More often than not, companies just rename a department. In some companies, shared services become DevOps teams; in others it is infrastructure, operations, or any other department. It’s as if it was a race and the first department to change their name into “DevOps” was a winner. Logically, changing the name means nothing and does not result in any tangible improvement. The key challenges are related to people and culture. DevOps is not easy because it challenges current organizational structure, it restructures power within an organization, and it questions the need for the existence of many departments. As such, middle management is often against it because it is perceived as a risk to their position. At the same time, people who spent many years doing the same thing over and over again feel that their credibility is at risk if the structure that allowed them to climb company ladder is removed. Congratulations on the release of DevOps Paradox. Could you talk a little bit about the idea behind it and what you hope it achieves? Viktor: I go to a lot of conferences and I realized that scheduled talks are not the main takeaway from them. True, I learned things by listening to them, but the primary reason I continue attending are “corridor talks.” Conferences are a great opportunity for me to find interesting people and have amazing discussions. Unlike scheduled talks, those conversations are not structured. I do not prepare a list of questions for the next person I’ll meet in between talks or at a party. Instead, we’d just start talking about a random thing that happens to be interesting. I wanted to bring those types of conversations to people who cannot travel the world and be every moth in a different conference in a different country. So, I did not have any real goals for this book, other than speaking with people about any topic, as long as it is related to DevOps. Since DevOps can be anything related to software development, you can say that the scope of the book is as broad as it can be. My true goal was to enjoy having conversations with people. I did not prepare questions in advance. Instead, I just gathered people I’d like to speak with if I’d meet them in a conference and say, “Let’s have a coffee and see what you were up to since the last time we met”. Some of those I interviewed are my friends, while others I met for the first time. Some work for huge enterprises, while others work in startups. Some worked in software industry for many years, while others are young up-and-coming experts. I wanted to make sure that the book gives as many different opinions as possible. Find Viktor Farcic's DevOps Paradox on the Packt store. Read the first chapter for free on the Packt subscription platform.

API management is a discipline that has evolved to deliver the processes and tools required to discover, design, implement, use, or operate enterprise-grade APIs. The discipline bisects two distinct communities and deserves the attention of both: developers who build APIs and business and IT leaders looking at APIs to drive growth. In Enterprise API Management, Luis Weir shows how to define the right architecture, implement the right patterns, and define the right organization model for business-driven APIs. The book explores architectural decisions, implementation patterns, and management practices for successful enterprise APIs. It also gives clear, actionable advice on choosing and executing the right API strategy in your enterprise. Let’s see what Luis has to say about API management and key principles to improve API design for enterprise organizations. What API management involves What does API management mean and involve? Luis Weir: In simple terms, it’s the discipline that aligns tools with processes and people in order to realize the value from implementing enterprise-grade APIs throughout their full cycle. By enterprise-grade, I mean APIs that comply with a minimum set of quality standards, not just in the actual API itself (e.g. use of normalize semantics, well-documented interfaces, and good user experience), but also in the engineering processes behind their delivery (e.g. CICD pipelines and robust automation at all levels, different levels of testing, and so on). Guiding principles for API design What are the some guiding principles that can improve API design? LW: First and foremost is the identification of APIs themselves. It’s not just about building an API for the sake of it and value will just come. Without adopting a process (e.g. ideation) that can help identify APIs that can truly add value, there is real risk that an API might just end up being a DOA (dead on arrival), as there might not even be a need for it. Assuming such a process has taken place and APIs that have real potential to add value have been identified, the next step is to conceptualize a design. It is at this point that disciplines such as domain-driven design can help produce such a design in a way that both business and IT people can relate to it. This design should capture things such as consuming applications, producing applications (data sources), data entities, and services involved in the concept. It should clearly and simply define the relationship between the components and define boundaries (bounded contexts) as these will be key not just in the actual implementation of the API or APIs (as it may end up being more than just one), but also in the creation of the API specifications themselves thought IDLs (e.g. an OAS file, API blueprint, GraphQL schema, .proto file in gRPC to name a few). The next and very important step for producing a good API is to follow an API design-first process. This process ensures that the API specifications and API mocks (produced from the API specifications themselves) undergo a series of validations by potential consumers of the API themselves as well as other relevant parties. The idea is to obtain as much feedback as possible through multiple iterations (or feedback-loops) to ensure that the API is fit for purpose but that it also delivers a good user experience. For more details, please refer to the API Life cycle section in my book. Testing APIs What are different API testing approaches? LW: At the very minimum, API testing should involve the following testing approaches: Interface testing Functional testing Performance testing Security testing Interface testing is used to validate that an API implementation conforms to the API specification. Functional testing is used to validate that the API delivers the functionality that it is meant to deliver and with the expected behavior. Performance testing ensures that APIs can actually handle the expected volume and scale as required. Security testing ensures that the API is not vulnerable to common threads such as those described in the OWASP top 10 projects. Other more sophisticated testing approaches may include A/B testing and Chaos testing. A/B testing dynamically tests new API features against a subset of the API audience and in a running environment (even production). Chaos testing (e.g. randomly shutting down components of the solution in production to ensure the API is resilient) should be considered as the API initiative matures. Understanding API gateways What are the key features of an API gateway? LW: There are many capabilities expected of an API gateway and these are all well described in the API exposure section in my book. However, in addition to such capabilities, which in my view are all essential, there are some key features that put modern API gateways (3rd generation) apart from more traditional ones (1st and 2nd gen). These are: Lightweight: Requires minimum disk space, CPU, and RAM to run. Hybrid: Can run on-premise, on cloud, and on multiple cloud platforms (e.g. AWS, Azure, Google, Oracle, etc). Kubernetes ready: k8s has become the most popular runtime platform for microservices. Modern APIs should be easily deployed into the K8s runtime and support many of the patterns as described in my book. Common Control Plane: If the management of APIs deployed on gateways isn’t centralized in some way, shape, or form, then allowing enterprise users to discover and (re)use already built (or being built) APIs will be extremely difficult and will lead to a lot of duplication. We’ve already seen this in the SOA days. Modern API Gateways should, therefore, be pluggable to control planes that take care of things like API lifecycle management and gateway infra management. Phone-home: This is a key feature and one that still not many modern gateways support. The ability for an API gateway to stablish the communication to the management tier via the control plane (Phone-home) using standard ports is key in hybrid architectures to avoid networking and other security constraints. Enterprise API Management, I think, provides a pretty comprehensive overview of what modern API platforms look like and how to differentiate them with more traditional ones. Common mistakes in API management What are the common mistakes people make in API management? LW: Throughout my time as an API strategist and practitioner I’ve seen many mistakes and also made some myself. The important thing is being able to recognize what they are and learnt from them. The top 3 that come to my mind: Thinking that API management is just about implementing a product or tools without having business and customer value at the epicentre of the API strategy. (Sometimes there even isn’t an API strategy.) This is perhaps the most common one, and one that happened a lot in the old SOA days…unfortunately still occurs in the modern API-led era. My book, Enterprise API Management, can be used as the guideline on how to avoid making an API management initiative less about tools, and more about business/customer value, people, and processes. Thinking that all APIs are the same and therefore treating them all the same way. In some cases, this just happens accidentally, in other cases this happens to avoid ‘layering’ APIs because ‘microservices architectures and practitioners say so’. The matter of fact is, that an API that is built specifically in support of a given mobile application will be less generic and less suited for its used outside of the ‘context’ on which it was built, as compare to an API that was built without any specific consuming application in mind (and thus is not coupled to any application lifecycle). Adopting the wrong organizational model to provide API capabilities across the enterprise. Foor example, this could be a model that centralizes all API efforts and capabilities thus becoming a bottleneck and eventually becoming slow (aka traditional IT). Modern API initiatives should think about adopting platforms models with self-service at the epicentre. In addition to the above 3, there are many common pitfalls when it comes to API architecture and design. However, to cover these I strongly recommend my talk on the 7 deadly sins of API design... https://www.youtube.com/watch?v=Sx2_etbb9JA API management and DevOps What are your thoughts about 3rd generation API management having huge impact on DevOps? LW: Succeeding in modern API management and microservices architectures requires changes beyond technology and also requires diving deep into the organization and its culture. It means moving away from traditional project-based deliveries wherein teams assemble just for the duration of a project and hand over the delivered software (e.g. an API and related services) to different support teams. Instead, move towards a product-based organization wherein teams are assemble around business capabilities and retain accountability and ownership through the entire life cycle of the product. This fundamental change of approach in delivering software means that there is no longer a split between development and operation teams, as a product team has full ownership and accountability over its product. With that said, in order to avoid (re)building these product teams and maintaining core IT capabilities from scratch (e.g. API platforms and service runtimes), a platform operating model can be adopted. This model can offer common IT capabilities, although in a decentralized, on-demand, and self-service way. And for me accomplishing the above is true DevOps. It is at this point that organizations can become more agile and can truly increase their time to markets. What were your goals and objectives in this book, and how well do you feel you achieved them? LW: When I started defining and implementing API and microservices strategies in large enterprises (many of them Fortune 500), although there was plentiful of content around to get inspiration from (much of this content referenced in my book), I had to literally go through several articles, books, videos, and others in order to conceive a top-down, business-led approach towards delivering end-to-end API and microservices strategies. When I say end to end, it doesn’t mean just defining PowerPoints and lengthy Word documents explaining how to deliver API/Microservices strategies and then just walking away. Or worst, sitting on the side with an opinion but no accountability (unfortunately, only too common in the consulting world - lots of senior consultants with strong opinions but who have little or no real practical knowledge and experience). Rather it means walking the talk, defining the strategy, and also delivering it with all of its implications. With this book, I, therefore, wanted to share to the community an approach that I created, evolved through the years, and have seen working. It’s not just theory, but a mix of theory with practice. It’s not just ideas, but ideas that I have put into practice. This book is about sharing my real-life experiences and approach in delivering API and microservices strategies. Therefore, I think (or hope) that I have accomplished my goals with this book. I felt that there is great stuff out there focused on specific things of the “end to end” but not the actual “end to end,” which is what I wanted to cover in this book. I didn’t want to be too high level or too detailed. I wanted to give something to multiple audiences, as it requires multiple audiences (technical and non-technical) working together in order to successfully deliver API management. Ultimately, the readers will be the judge, but I think I have accomplished my goals with this book. Find Enterprise API Management on the Packt store. Read the first chapter for free on Packt's subscription platform.

Cloud-native applications are an assembly of independent services used to build new applications, optimize existing ones, and connect them in such a way that the applications can skillfully deliver the desired result. More specifically, they are employed to build scalable and fault-tolerant applications in public, private, or hybrid clouds.  Launched in March this year, Quarkus, a new Kubernetes-native framework launched its first stable version, Quarkus 1.0 last month. Quarkus allows Java developers to combine the power of containers, microservices, and cloud-native to build reliable applications. To get a more clear understanding of Cloud-Native Applications with Java and Quarkus, we interviewed Francesco Marchioni, a Red Hat Certified JBoss Administrator (RHCJA) and Sun Certified Enterprise Architect (SCEA) working at Red Hat. Francesco is the author of the book ‘Hands-On Cloud-Native Applications with Java and Quarkus’.  Francesco on Quarkus 1.0 and how Quarkus is bringing Java into the modern microservices and serverless modes of developing Quarkus is coming up with its first stable version Quarkus 1.0 at the end of this month. It is expected to have features like a new reactive core based on Vert.x, a non-blocking security layer, and a new Quarkus ecosystem called ‘universe’. What are you most excited about in Quarkus 1.0? What are your favorite features in Quarkus? One of my favorite features of Quarkus is the reactive core ecosystem which supports both reactive and imperative programming models, letting Quarkus handle the execution model switch for you. This is one of the biggest gains you will enjoy when moving from a monolithic core, which is inherently based on synchronous executions, to a reactive environment that follows events and not just a loop of instructions. I also consider of immense value that the foundation of Quarkus API is a well-known set of APIs that I was already skilled with, therefore I could ramp up and write a book about it in less than one year! How does the Quarkus Java framework compare with Spring? How do you think the Spring API compatibility in Quarkus 1.0 will help developers? Both Quarkus and Spring boot offer a powerful stack of technologies and tools to build Java applications. In general terms, Quarkus inherits its core features from the Java EE, with CDI and JAX-RS being the most evident example. On the other hand, Spring boot follows an alternative modular architecture based on the Spring core. In terms of Microservices, they also differ as Quarkus leverages the Microprofile API while Spring Boot relies on Spring Boot Actuator and Netflix Hystrix. Besides the different stacks, Quarkus has some unique features available out of the box such as Build time class initialization, Kubernetes resources generation and GraalVM native images support. Although there are no official benchmarks, in the typical case of a REST Service built with Quarkus, you can observe an RSS memory reduction to half and a 5x increase in boot speed. In terms of compatibility, it's worth mentioning that, while users are encouraged to use CDI annotations for your applications, Quarkus provides a compatibility layer for Spring dependency injection (e.g. @Autowired) in the form of the spring-di extension. Quarkus is tailored for GraalVM and crafted by best-of-breed Java libraries and standards. How do you think Quarkus brings Java into the modern microservices and serverless modes of developing? Also, why do you think Java continues to be a top programming language for back-end enterprise developers? Although native code execution, in combination with GraalVM, Quarkus is an amazing opportunity for Java. I mean I wouldn't say Quarkus is just native centric as it immediately buys to Java developers an RSS memory reduction to about half, an increase in boot speed, top Garbage Collector performance, plus a set of libraries that are tailored for the JDK. This makes Java a first-class citizen in the microservices ecosystem and I bet it will continue to be one of the top programming languages still for many years. On how his book will benefit Java developers and architects In your book “Hands-On Cloud-Native Applications with Java and Quarkus” you have demonstrated advanced application development techniques such as Reactive Programming, Message Streaming, Advanced configuration hacks. Apart from these, what are the other techniques that can be used for managing advanced application development in Quarkus? Also, apart from the use cases in your book, what other areas/domains can you use Quarkus? In terms of configuration, a whole chapter of the book explores the advanced configuration options which are derived from the MicroProfile config API and the Applications’ profile management, which is a convenient way to shift the configuration options from one environment to another- think for example how easy can be with Quarkus to switch from a Production DB to a Development or Test Database. Besides the use cases discussed in the book, I’d say Quarkus is rather polyvalent, based on the number of extensions that are already available. For example, you can easily extend the example provided in the last chapter, which is about Streaming Data, with advanced transformation patterns and routes provided by the camel extension, thus leveraging the most common integration scenarios. What does your book aim to share with readers? Who will benefit the most from your book? How will your book help Java developers and architects in understanding the microservice architecture? This book is a log of my journey through the Quarkus Land which started exactly one year ago, at its very first internal preview by our engineers. Therefore my first aim is to ignite the same passion to the readers, whatever is their "maturity level" in the IT. I believe developers and architects from the Java Enterprise trenches will enjoy the fastest path to learning Quarkus as many extensions are pretty much the same they have been using for years. Nevertheless, I believe any young developer with a passion for learning can quickly get on board and become proficient with Quarkus by the end of this book. One advantage of younger developers over seasoned ones, like me, is that it will be easier for them to start thinking in terms of services instead of building up monolithic giant applications like we used to do for years. Although microservices patterns are not the main focus of this book, a lot of work has been done to demonstrate how to connect services and not just how to build them up. On how Red Hat uses Quarkus in its products and service Red Hat is already using Quarkus in their products and services. How is it helping Red Hat in increasing the efficiency of your Cloud-Native applications? To be precise, Quarkus is not yet a Red Hat supported Product, but it has already reached an important milestone with the release Quarkus 1.0 final, so it will definitely be included in the list of our supported products, according to our internal productization road-map. That being said, Red Hat is working in increasing the efficiency of your Cloud-Native applications in several ways through a combination of practices, technologies, processes that can be summarized in the following steps that will eventually lead to cloud-native application success: Evolve a DevOps culture and practices to embrace new technology through tighter collaboration. Speed up existing, monolithic applications with simple migration processes that will eventually lead to microservices or mini services. Use ready-to-use developer tools such as application services, to speed up the development of business logic. Openshift tools (web and CLI) is an example of it. Choose the right tool for the right application by using a container-based application platform that supports a large mix of frameworks, languages, and architectures. Provide self-service, on-demand infrastructure for developers using containers and container orchestration technology to simplify access to the underlying infrastructure, give control and visibility to IT operations, and provide application lifecycle management across environments. Automate IT to accelerate application delivery using clear service requirements definition, self-service catalogs that empower users (such as the Container catalog) and metering, monitoring of runtime processes. Implement continuous delivery and advanced deployment techniques to accelerate the delivery of your cloud-native applications. Evolve your applications into a modular architecture by choosing a design that fits your specific needs, such as microservices, a monolith-first approach, or mini services. On Quarkus’ cloud-native security and its competitors Cloud-native applications provide customers with a better time-to-market strategy and also allows them to build, more robust, resilient, scalable, and cost-effective applications. However, they also come with a big risk of potential security breaches. What is your take on cloud-native security for cloud-native applications? Also, what are your thoughts on future-proofing cloud applications? Traditionally, IT security was focused on hardening and the datacenter perimeter—but today, with Cloud applications, that perimeter is fading out. Public and hybrid clouds are shifting responsibility for security and regulatory compliance across the vendors. The adoption of containers at scale requires the adoption of new methods of analyzing, securing, and updating the delivery of applications. As a result, static security policies don’t scale well for containers in the enterprise but need to move to a new concept of security called "continuous container security". This includes some key aspects such as securing the container pipeline and the application, securing the container deployment environment(s) and infrastructure, integrating with enterprise security tools and meeting or enhancing existing security policies. About future-proofing of cloud applications, I believe proper planning and diligence can ensure that a company’s cloud investments withstand future change or become future-proof. It needs to be understood that new generation applications (such as apps for social, gaming and generally mobile apps) have different requirements and generate different workloads. This new generation of applications requires a substantial amount of dynamic scaling and elasticity that would be quite expensive or impossible to achieve with traditional architectures based on old data centers and bare-metal machines. Micronaut and Helidon, the other two frameworks that support GraalVM native images and target cloud-native microservices are often compared to Quarkus. In what aspects are they similar? And in what ways is Quarkus better than and/or different from the other two?   Although it is challenging to compare a set of cutting edge frameworks as some factors might vary in a middle/long term perspective, in general terms I'd say that Quarkus provides the highest level of flexibility especially if you want to combine reactive programming model with the imperative programming model. Also, Quarkus builds on the top of well-known APIs such as CDI, JAX-RS, and Microprofile API, and uses the standard "javax" namespaces to access them. Hence, the transition from the former Enterprise application is quite smooth compared with competitive products. Micronaut too has some interesting features such as support for multiple programming languages (Java, Kotlin, and Groovy the latter being exclusive of Micronaut) and a powerful Command Line Interface (CLI) to generate projects. (A CLI is not yet available in Quarkus, although there are plans to include it in the upcoming versions of it). On the other hand, Helidon is the less polyglot alternative (supports only Java right now) yet, it features a clean and simple approach to Container by providing a self-contained Dockerfile that can be built by simply calling docker build, not requiring anything locally (except the Docker tool of course). Also, the fact that Helidon plays well with GraalVM should be acknowledged as they are both official Oracle products. So, although for new projects the decision is often a matter of personal preferences and individual skills in your team, I'd say that Quarkus leverages existing Java Enterprise experience for faster results. If you want to become an expert in building Cloud-Native applications with Java and Quarkus, learn the end-to-end development guide presented in the book “Hands-On Cloud-Native Applications with Java and Quarkus”. This book will also help you in understanding a wider range of distributed application architectures to use a full-stack framework and give you a headsup on the new features in Quarkus 1.0. About the author Francesco Marchioni is a Red Hat Certified JBoss Administrator (RHCJA) and Sun Certified Enterprise Architect (SCEA) working at Red Hat in Rome, Italy. He started learning Java in 1997, and since then he has followed all the newest application program interfaces released by Sun. In 2000, he joined the JBoss community, when the application server was running the 2.X release. He has spent years as a software consultant, where he has enabled many successful software migrations from vendor platforms to open source products, such as JBoss AS, fulfilling the tight budget requirements necessitated by the current economy. Francesco also manages a blog on 'WildFly Application Server, Openshift, JBoss Projects and Enterprise Applications' focused on Java and JBoss technologies. You can reach him on Twitter and LinkedIn. RedHat’s Quarkus announces plans for Quarkus 1.0, releases its rc1  How Quarkus brings Java into the modern world of enterprise tech Introducing ‘Quarkus’, a Kubernetes native Java framework for GraalVM & OpenJDK HotSpot OpenJDK Project Valhalla’s head shares how they plan to enhance the Java language and JVM with value types, and more Snyk’s JavaScript frameworks security report 2019 shares the state of security for React, Angular, and other frontend projects

Serverless computing is a growing trend that lets software developers focus more on code than the back-end processes. While there are a lot of serverless computing platforms, in this article we will focus on Microsoft’s Azure serverless computing platform, which provides its users with  fully managed, end-to-end Azure serverless solutions to boost developer productivity, optimise resources and expedite the development processes. To understand the nitty-gritties of Azure Serverless, we got in touch with Lorenzo Barbieri, a cloud-native application specialist who works at Microsoft’s One Commercial Partner Technical Organization and, Massimo Bonanni, an Azure Technical trainer at Microsoft. In their recently published book, Mastering Azure Serverless Computing, they explain how developers with Microsoft’s Azure Serverless platform can build scalable systems and also deploy serverless applications with Azure Functions. Sharing their thoughts about Azure serverless and its security the authors said that although security is one of the most important topics while designing a complex solution, security depends both on the cloud infrastructure as well as the code. They further shared how Powershell in Azure Functions allows you to combine the best language for automation with one of the best services. Sharing their experiences working at Microsoft, they also talked about how their recently published book will help developers master various processes in Azure serverless. On how Microsoft ensures complete security within the Serverless Computing process Every architecture should guarantee a secure environment for the user. Also, the security of any Serverless functions depends on the cloud provider's infrastructure, which may or may not be secure. What are the certain security checks that Microsoft ensures for complete security within the Serverless Computing processes? Lorenzo: Security of Serverless functions depends both on the cloud provider’s infrastructure and the application code. For example,  SQL Injections depends on how the application code is written; you should check all the inputs (depending on the trigger) to avoid these types of attacks. Many other types of attacks depend on application code and third party dependencies. On its side, Microsoft is responsible for managing and patching servers and application frameworks, and keeps them updated when security updates are released. .” Massimo: Security is one of the most important topics when you design a complex solution, and in particular, when it will run on a cloud provider. You must think about it from the beginning of your design. Azure provides a series of ot-of-the-box services to ensure the security of the solutions that you deploy on it. For example, Azure DDoS Protection Service is an Azure service you have for free on every solution you deploy, and especially if you are developing Azure Functions triggered by HTTP trigger. On the other hand, you must guarantee that your code is safe and that your third party dependencies are secure too. If one of the actors of your solution chain is unsafe, all your solution becomes potentially not secure. On general availability of PowerShell in Azure Functions V2 The Microsoft team recently announced the general availability of PowerShell in Azure Functions V2. Azure Functions is known for its speed and PowerShell for its automation; how will this feature enhance serverless computing on Azure Cloud? What benefits can users or organizations expect with this feature? What does this mean for Azure developers? Lorenzo: GA of PowerShell in Azure Functions is a great news for cloud administrators and developers that can use them connected for example with Azure Monitor alerts, to create custom auto-scale rules or to implement mitigation for problems that could arise. Massimo: Serverless architecture gives its best for event-driven solutions. Automation in Azure is, generally, driven by events generated by the platform. For example, you have to do something when someone creates a storage, or you have to execute a task every hour. Using Powershell in an azure function allows you to combine the best language for automation with one of the best services to react to events. On why developers should prefer Azure Serverless computing Can you tell us some of the pre-requisites expected before reading your book? How does your book prepare its readers to master Azure Serverless Computing and to be industry ready? Lorenzo: A working knowledge of .NET or other programming languages is expected, together with basic understanding of Cloud architectures. For Chapter 7 [Serverless and Containers], basic knowledge of containers and Kubernetes is expected. The book covers all the advanced features of Azure Serverless Computing, not only Azure Functions. After reading the book, one can decide which technology to use. Massimo: The book supposes that you have a basic knowledge of programming language (e.g. C# or Node.js) and a basic knowledge of Cloud topics and architecture. Moreover, for some chapters (e.g., Chapter 7), you need some other knowledge like containers and Kubernetes. In your book, ‘Mastering Azure Serverless Computing’, you have said that Containers and Orchestrators are the main competitors of Serverless in terms of Architecture. What makes Serverless architecture better than the other two? How does one decide while migrating from a monolith, which architecture to adopt? What are some real-world success stories of serverless migration? Lorenzo: In Chapter 7 we’ve seen that it’s possible to create Containers and run them inside Azure Functions, and that’s also possible to run Azure Functions inside Kubernetes, AKS or OpenShift together with KEDA. The two worlds are not mutually exclusive, but most of the times you choose one route or another. Which one you should use? Serverless is more productive, it’s really easy to scale and it’s better suited for event-driven architectures. With Orchestrators like Kubernetes you can customize every aspect of your infrastructure, you can create complex service connections and dependencies, and you can deploy them everywhere. Stylelabs, a leading Belgium/US-based marketing software company, successfully integrated Azure Functions into its cloud architecture to benefit from serverless in addition to traditional solutions like VMs and App Services. Massimo: I think that there isn't a better tool to implement something. As I always say during my technical sessions (even if I seem repetitive and boring), when you choose an architecture (e.g. microservices or serverless), you choose it because that architecture meets the requirements of the solution you are designing. If you choose an architecture because it is popular or "fashionable", you are making a serious mistake that you will pay when your solution will be deployed. In particular, Microservice architecture (that you can implement using Container and Orchestrator) and Serverless architecture meet different requirements (e.g. Serverless is the best solution when you need an event-driven architecture while one of the most important characteristics of the microservices architecture is high availability and orchestration), so I think they can be used together. A few highlights of Microsoft Azure Functions What are the top 5 highlights of Azure Functions that make it a go-to serverless platform for newbies and professionals? Massimo: For the Azure Functions, the five best features are, in my opinion: Support for a number of programming languages and also has the possibility to support any other programming languages, which are not currently available; Extensibility of triggers and bindings to support your custom data sources; Availability of a number of tools available to implement Azure Functions (Visual Studio, Visual Studio Code, Azure Functions Tools, etc., etc.); Use of the open-source approach for runtime and tools; Capability to easily use Azure Functions with other Azure services such as Event Grid or Azure Key Vault. Lorenzo and Massimo on their personal experiences working with Microsoft Azure services Lorenzo, you have a specialization in Cloud Native Applications and Application Modernization. Can you share your experience and the challenges you faced with the Cloud-native learning curve? You have also been using Azure Functions since the first previews. How has it grown from the first preview? In the beginning it was difficult. Azure includes many services and it’s growing even faster. In the beginning, I simply tried to understand the big picture of the services and their relationship. Then I started going deeper in the services that I needed to use. I’m thankful to many highly skilled colleagues, who started this journey before me. I can say that two years of working with Azure and the experience you gain is the minimum time to master the parts that you need. Speaking of Azure Functions, the first preview was interesting, but limited. Azure Functions v2 and the upcoming v3 are great platforms, both in terms of features and in terms of scalability, and configuration. Massimo, you are an Azure Technical Trainer at Microsoft, can you share with us your journey with Microsoft. What were the projects you enjoyed being involved in? Where do you see microservice and serverless architecture in the next five years? During my career, I have always worked with Microsoft technologies and have always wanted to be a Microsoft employee. For several years I was a Microsoft MVP, and, finally, three years ago, I was hired. Initially, I worked for the business unit that provides consulting to customers and partners for implementing solutions (not only Cloud oriented). In almost three years of consulting, I worked on various projects for different customers and partners with different Azure technologies, specially Microservice architecture, and during the last year, serverless. I think that these two architectures will be the most important in the next years specially for enterprise solutions. When you are a consultant, you are involved in a lot of projects, and every project has its peculiarity and its problems to solve, and it isn't simple to remember all of them. The most important thing that I learned during these years, is that those who design solutions for the Cloud must be like a Chef: you can use different ingredients (the various services offered by the Cloud) but must mix them in the right way to get the right recipe. Since three months, I am an Azure Technical Trainer, and I help our customers to better understand Azure services and use the right one in their solutions. About the Authors Lorenzo Barbieri Lorenzo Barbieri works for Microsoft, in the One Commercial Partner Technical Organization, helping partners, developers, communities, and customers across Western Europe, supporting software development on Microsoft and OSS technologies. He specializes in cloud-native applications and application modernization on Azure and Office 365, Windows and cross-platform applications, Visual Studio, and DevOps, and likes to talk with people and communities about technology, food, and funny things. He is also a speaker, trainer, and a public speaking coach and has helped many students, developers, and other professionals, as well as many of his colleagues, to improve their stage presence with a view to delivering exceptional presentations. Massimo Bonanni Massimo Bonanni is an Azure technical trainer in Microsoft and his goal is to help customers utilize their Azure skills to achieve more and leverage the power of Azure in their solutions. He specializes in cloud application development and, in particular, in Azure compute technologies. Over the last 3 years, he has worked with important Italian and European customers to implement distributed applications using Service Fabric and microservices architecture. Massimo is also a technical speaker at national and international conferences, a Microsoft Certified Trainer, a former MVP (for 6 years in Visual Studio and Development Technologies and Windows Development), an Intel Software Innovator, and an Intel Black Belt. About the book Mastering Azure Serverless Computing will guide you through using Microsoft's Azure Functions to process data, integrate systems, and build simple APIs and microservices. You will also discover how to apply serverless computing to speed up deployment and reduce downtime. You'll also explore Azure Functions, including its core functionalities and essential tools, along with understanding how to debug and even customize Azure Functions. “Microservices require a high-level vision to shape the direction of the system in the long term,” says Jaime Buelta Glen Singh on why Kali Linux is an arsenal for any cybersecurity professional [Interview] Why become an advanced Salesforce administrator: Enrico Murru, Salesforce MVP, Solution and Technical Architect [Interview]

The Packt library provides you with all the tools you need to stay relevant in tech, whether you’re looking to brush up your PHP skills or take advantage of our learning paths to start from scratch. Here’s our top five reasons to begin a Packt subscription.

Last month, Fastly, a provider of an edge cloud platform, introduced a collection of resources to help developers learn the ins and outs of popular cloud solutions. The collection consists of step-by-step tutorials and ready-to-deploy code that developers can customize, and deploy to their Fastly configuration. We had the opportunity to interview Adam Denenberg, Fastly’s SVP of Customer Solutions, to get more insight into this particular project and other initiatives Fastly is taking to empower developers. We also grabbed this opportunity to talk to Denenberg about the emergence and growth of edge computing and fog computing and what it all means for the industry. What are the advantages of edge computing over cloud? Cloud computing is a centralized service that provides computing resources including servers, storage, databases, networking, software, analytics, and intelligence on demand. It is flexible, scalable, enables faster innovation, and has revolutionized the way people store and interact with data. However, because it is a centralized system, it can cause issues such as higher latency, limited bandwidth, security issues, and the requirement of high-speed internet connectivity. This is where edge computing comes in - to address these limitations. In essence, it’s a decentralized cloud. “Edge computing is the move to put compute power and logic as close to the end-user as possible. The edge cloud uses the emerging cloud computing serverless paradigm in which the cloud provider runs the server and dynamically manages the allocation of machine resources,” Denenberg explains. When it comes to making real-time decisions edge computing, can be very effective. He adds, “The average consumer expects speedy online experiences, so when milliseconds matter, the advantage of processing at the edge is that it is an ideal way to handle highly dynamic and time-sensitive data quickly. “In contrast, running modern applications from a central cloud poses challenges related to latency, ability to pre-scale, and cost-efficiency.” What is the difference between fog computing and edge computing? Fog computing and edge computing can appear very similar. They both involve pushing intelligence and processing capabilities closer to the origin of data. However, the difference lies in where the location of intelligence and compute power is placed. Explaining the difference between the two, Denenberg said, “fog computing, a term invented by Cisco, shares some similar design goals as edge computing, such as reducing latency to the end-user request and providing access to compute resources in a decentralized model. After that, things begin to differ.” He adds, “On the one hand, fog computing has a focus on use cases like IoT and sensors. This allows enterprises to extend their network from a central cloud closer to their devices and sensors, while maintaining a reliance on the central cloud. “Edge computing, on the other hand, is also about moving compute closer to the end-user, but doing so in a way that removes the dependency on the central cloud as much as possible. By collocating compute and storage (cache) on Fastly’s edge cloud, our customers are able to build very complex, global-scale applications and digital experiences without any dependency on a centralized compute resources.” Will edge computing replace cloud computing? A short answer to this question would be “not really.” “I don’t think anything at this moment will fully replace the central cloud,” Denenberg explains. “People said data centers were dead as soon as AWS took off, and, while we certainly saw a dramatic shift in where workloads were being run over the last decade, plenty of organizations still operate very large data centers. “There will continue to be certain workloads such as large-scale offline data processing, data warehouses, and the building of machine learning models that are much more suited to an environment that requires high compute density and long and complex processing times that operate on extremely massive data sets with no time sensitivity.” What is Fastly? Fastly’s story started back in 2008 when Artur Bergman, its founder, was working at Wikia. Three years later, he founded Fastly, headquartered in San Francisco, with its branches in four cities including London, Tokyo, New York, and Denver. Denenberg shared that Fastly’s edge cloud platform was built to address the limitations in content delivery networks (CDNs). “Fastly is an edge cloud platform built by developers, to empower developers. It came about as a result of our founder Artur Bergman's experience leading engineering at Wikia, where his passion for delivering fast, reliable, and secure online experiences for communities around the world was born. So he saw firsthand that CDNs -- which were supposed to address this problem -- weren't equipped to enable the global, real-time experiences needed in the modern era.” He further said, “To ensure a fast, reliable, and secure online experience, Fastly developed an edge cloud platform designed to provide unprecedented, real-time control, and visibility that removes traditional barriers to innovation. Knowing that developers are at the heart of building the online experience, Fastly was built to empower other developers to write and deploy code at the edge. We did this by making the platform extremely accessible, self-service, and API-first.” Fastly’s new edge cloud resources Coming to Fastly’s new edge cloud resources, Denenberg shared the motivation behind this launch. He said, “We’re here to serve the developer community and allow them to dream bigger at the edge, where we believe the future of the web will be built. This new collection of recipes and tutorials was born out of countless collaborations and problem-solving discussions with Fastly's global community of customers. Fastly's new collection of edge cloud resources make it faster and safer for developers to discover, test, customize, and deploy edge cloud solutions.” Currently, Fastly has shared 66 code-based edge cloud solutions covering aspects like authentication, image optimization, logging, and more. It plans to add more solutions to the list in the near future. Denenberg shared, “Our initial launch of 66 recipes and four solution patterns were created from some of the most common and valuable solutions we’ve seen when working with our global customer base. However, this is just the beginning - many more solutions are on our radar to launch on a regular cadence. This is what has us really excited-- as we expose more of these solutions to customers, the more inspiration they have to go even further in their work, which creates a remarkable flywheel of innovation on our edge cloud.” Challenges when developing on the edge When asked about what edge cloud solutions Denenberg thinks developers often find difficult, he said, “I think difficulty is a tricky thing to address because engineering is a lot of times about tradeoffs. Those tradeoffs are most often realized when pursuing instant scalability, being able to run edge functions everywhere, and achieving low latency and microsecond boot time. He adds, “NoSQL saw tremendous growth because it presented the ability to achieve scale with very reasonable trade-offs based on the types of applications people were building that traditional SQL databases made very difficult, from an architectural perspective, like scaling writes linearly to a cluster easily, for example. So for me, given the wide variety of applications our customers can build, I think it’s about taking advantage of our platform in a way that improves the overall user experience, which sometimes just requires a shifting of the mindset in how those applications are architected.” We asked Denenberg whether other developers will be able to pitch in to expand this collection of resources. “We are already talking with customers who are excited to share what they have built on our platform that might allow others to achieve enhanced online experiences for their end users,” he told us. “Fastly has an internal team dedicated to reviewing the solutions customers are interested in sharing to ensure they have the same consistency and coding style that mirrors how we would publish them internally. We welcome the sharing of innovation from our customer base that continues to inspire us through their work on the edge.” Other initiatives by Fastly to empower developers Fastly is continuously contributing towards making the internet more trustworthy and safer by getting involved in projects like QUIC, Encrypted SNI, and WebAssembly. Last year, Fastly made three of its projects available on Fastly Labs: Terrarium, Fiddle, and Insights. Read also: Mozilla introduces Neqo, Rust implementation for QUIC, new http protocol Denenberg shared that there are many ways Fastly is contributing to the open source community. “Yes, empowering developers is at the forefront of what we do. As developers are familiar with the open-source caching software that we use, it makes adopting our platform easier. We give away free Fastly services to open source and nonprofit projects. We also continue to work on open source projects, which empower developers to build applications in multiple languages and run them faster and more securely at our edge.” Fastly also constantly tries to improve its edge cloud platform to meet its customers’ needs and empower them to innovate. “As an ongoing priority, we work to ensure that developers have the control and insight into our edge platform they need. To this end, our programmable edge provides developers with real-time visibility and control, where they can write and deploy code to push application logic to the edge. This supports modern application delivery processes and, just as importantly, frees developers to innovate without constraints,” Denenberg adds. He concludes, “Finally, we believe our values empower our community in several ways. At Fastly, we have chosen to grow with a focus on transparency, integrity, and inclusion. To do this, we are building a kind, ethical, and inclusive team that reflects our diverse customer base and the diversity of the developers that are creating online experiences. The more diverse our workforce, the easier it is to attract diverse talent and build technology that provides true value for our developer community across the world.” Follow Adam Denenberg on Twitter: @denen Learn more about Fastly and its edge-cloud platform at Fastly’s official website. More on cloud computing Cloud Next 2019 Tokyo: Google announces new security capabilities for enterprise users Google Cloud and Nvidia Tesla set new AI training records with MLPerf benchmark results How do AWS developers manage Web apps?

“The service control platform is the next-gen of traditional API management,” Kong CTO and co-founder Marco Palladino tells me. “It’s not about APIs any more, it’s about services.” This shift in the industry is what makes Kong so interesting. It’s one of the reasons I wanted to speak to Palladino. Its success is an index of businesses’ technological priorities today, and useful as an indicator of the way the world is going - one, it’s safe to say, that’s increasingly cloud-native and highly distributed. As part of a broad and growing cloud-native ecosystem, Kong is playing an important role in the digital transformation of thousands of companies around the world. Furthermore, the fact that it follows an open core model, with an open source version of Kong made available in 2015, underlines the way in which the platform occupies a valuable position in the valley between developer enablement and managerial control. This isn’t always an easy place to be. 'Digital transformation' is a well-worn phrase, but behind it is the messy truth about the reality of how companies use technology: at their own pace and often shaped by necessity rather than best practice. So, with Kong a useful proxy for the state of the software industry, I wanted to dive deeper into Kong’s challenges, as well as the opportunities the platform can potentially unlock for its users. What is Kong? Before going any further it’s probably worth explaining what Kong actually is. Essentially, Kong is an API management platform - it allows teams to manage how services interact and move within their architecture. [caption id="attachment_29326" align="alignright" width="248"] via konghq.com[/caption] “APIs allow information to be in flight within our systems,” Palladino explains. Information can, he continues, either be “at rest in a database” or “in use by a monolith or microservice.” Naturally then, it follows that “the more we decouple - the more we distribute our applications - the more information will be… in flight.” This is why Palladino believes Kong is so valuable today. The “flight” of information (he never uses the word “data”) necessarily implies a network and, as anyone familiar with L. Peter Deutsch’s 7 Fallacies of Distributed Computing will know, “the network is unreliable.” “So how do we protect that communication? How do we secure it? How do we connect it? How do we route that transmission?” Palladino says. “The more we decouple, the more we distribute, the more those problems become critical, if not essential, for a successful microservice organization… what Kong provides is a platform that allows us to intelligently broker the flow of information across the organization.” Why does the world need Kong? Do we really need another API management solution? The short answer to this is relatively straightforward: the world is moving toward (micro)services and Kong provides you with a way of managing them. This control is crucial, moreover, because “in microservices, being slow is the new down - if you’re slow, you’re down.” But that’s only half of the picture. This “new world” is still in development and transition with each organization following its own technological path. Kong is necessary because it supports and facilitates these unique transitions, all of them happening in different ways around the world. “Kong is a platform agnostic system that can run across different architectures, but most importantly it can run across different platforms,” Palladino says. “While we do work very well with Kubernetes, we also support… traditional legacy virtual machines or bare metal infrastructures. And the reason we do support both the modern and the old is that we’re working with enterprise organizations… [who] might be deploying new greenfield applications in Kubernetes or OpenShift but… still have a significant part of their software running in traditional virtual machines.” One of Kong’s strengths, Palladino suggests, is its pragmatism and the way in which the company is alive to their customer’s respective levels of technological maturity. “I’m very proud to say we’re a very pragmatic company. While we do work with developers to make sure that Kong is a leader in what we do in microservices and traditional API management, we’re also very pragmatic - we understand that’s the end goal, it's not necessarily the current state of affairs in our enterprise organizations.” Read next: It’s Black Friday: But what’s the business (and developer) cost of downtime? “We’re not just a vendor. We don’t give you the platform and then let you figure it out. We want to be a strategic technology partner with our customers.” Kong sees itself as a 'strategic technology partner' However, while every organization has its own timeline when it comes to technology, its CTO describes Kong as a platform that is paving a way for the future rather than simply catering to the needs of its customers. “We’re not an industry follower, we’re an industry leader,” says Palladino. “We’re looking at these large scale systems that organizations are creating and we’re thinking how can we make that better from a security standpoint, from a discoverability standpoint, from a documentation standpoint?” This isn’t just Silicon Valley posturing. As the software world moves toward cloud and microservices, the landscape shifts at a much faster rate. That makes it essential for organizations like Kong to pave the way for the future rather than react to the needs and demands of their customers. In turn, this means the scope of Kong’s work is growing. “We’re not just a vendor. We don’t give you the platform and then let you figure it out. We want to be a strategic technology partner with our customers,” says Palladino. “We engage with them, not just from a low-level standpoint with the teams, but we also engage... from a higher level executive standpoint, because we want to enable not just the technology but the business itself to be successful.” This is something Palladino is well aware of. Kong’s customers aren’t, after all, needlessly indulging in “an exercise in adopting new technologies,” but are rather doing so in response to business requirements. Having a more extensive relationship - or partnership, as Palladino puts it - ensures that digital transformation is both impactful and relatively risk free. "You simply can’t afford to have a black box at the center of your infrastructure. You need to know what’s happening and how services are interacting with one another - the way of achieving this is through open source software." Open source and the rise of bottom-up software adoption However, although Kong positions itself as a company attuned to the business needs of their customers, it’s also clear that it understands the developer’s power in today’s technology ecosystem. Palladino sees open source as playing a big part in this. And as an open core platform, Kong is able to build a community of creative and innovative developers around the wider product ecosystem. But Palladino is also keen to point out that you can’t really separate open source and the API and microservices revolutions. “10 years ago APIs used to be a nice-to-have” Palladino says. The approach was, he explains, little more than a kind of curiosity or a desire for a small community around a platform: “let’s open up some APIs, let’s open up this monolithic black box and see what happens.” However, “it’s not like that any more." If “APIs are the core business of every organization,” as Palladino puts it to me, “then you simply can’t afford to have a black box at the center of your infrastructure. You need to know what’s happening and how services are interacting with one another - the way of achieving this is through open source software.” “When we look at the microservices transition, we look at Docker, we look at Kubernetes, we look at Elastic, we look at Zipkin… Kafka… Kong, what’s the baseline? Open source. Each one of these products is open source at their core. Open source is driving this new transformation within the enterprise,” says Palladino. Palladino continues on this, offering a compelling narrative of why open source has become the dominant form of software. He begins with the problems posed by traditional central IT, “an ivory tower far from the business, far from real usage” which consequently “were not able to iterate fast enough to be able to answer those business requirements.” “The teams building the apps were closer to the business, closer to the customer, and they had to pick the right solution in order to be successful. And so what these… teams did was to go into self-service ecosystems - like... CNCF [Cloud Native Computing Foundation] - and pick and choose open source technologies they could adopt without having to go through an enterprise process… that’s why open source became more important - because it allowed them to be in production and get business value without having to deal with the bureaucracy of central IT - so it’s a bottom-up adoption from the teams all the way up as opposed from central IT to all the teams.” Developer freedom and organizational control Palladino refers to ‘bottom-up’ adoption a number of times throughout our conversation. He argues that it’s an industry shift that has been initiated by microservices. “With the emergence of microservices something happened in the industry - software, is not being sold top down anymore as much as it used to be - it’s more bottom-up adoption.” He also explains that having an open source element to the Kong offering is actually helping the company to grow. It’s a useful onboarding route. “Sometimes - often actually - Kong is being adopted just because the organization happens to already be running Kong in production, and you need enterprise features and enterprise support,” says Palladino. But while developer power seems to be part of this new post-central IT world, it also makes Kong even more valuable for those in leadership positions. Taking the example of multi-cloud, Palladino explains saying that “it’s very rare to see a CIO saying we would like to be multi cloud. Sometimes it happens, [but] most likely the organization is already multi-cloud because it naturally evolved to be multi-cloud. Different teams, different products using different clouds, different services.” With the wealth of tools, platforms and environments being used by forward-thinking developers trying to solve the problems in their immediate vicinity, it makes sense that the “C-Level Executives” who express an interest in Kong are looking for “a way to consolidate and standardize how their APIs and microservices are being managed and secured across multiple clouds, across multiple platforms.” A big concern for the leadership of the top Global 5000 organizations we’re working with… [is] making sure they can consolidate how security is being done, how monitoring is being done, how observability and enablement is being done across multiple clouds,” Palladino says. Read next: Honeycomb CEO Charity Majors discusses observability and dealing with “the coming armageddon of complexity” [Interview] The future of Kong and API management The future for Kong looks bright. The two new features released by the platform - Kong Brain and Kong Immunity - launched earlier this year, signal what the broader trends might be in the software infrastructure and systems engineering space. Both are backed by artificial intelligence, and provide incredibly cutting edge ways to manage the reliability and security of the services inside your infrastructure. Kong Brain, Palladino explains, lets you “listen to… runtime traffic to auto generate documentation for APIs… services, and monoliths” that organizations have no visibility on “after 20 years of running them in production.” Essentially then, it’s a tool that will prove incredibly useful in working with legacy software; it will certainly encourage the ‘lift and shift’ mentality that we’re starting to see emerge. Kong Immunity, meanwhile, is a security tool that uses machine learning to detect anomalies in traffic - allowing users to identify security threats and breaches within their system. “Traditional web application firewalls… don’t work within east-west traffic [server to server],” Palladino says. “They work perhaps in north-south traffic [client to server], but they’re slow, they’re very heavy weight.” Kong, then “tries to take away that security concern by providing a machine learning platform that can asynchronously, with no performance loss, learn from existing traffic across every version of every microservice.” With releases like these, it’s hard to dispute Palladino’s assertion that Kong is indeed an ‘industry leader.’ However, as Palladino also appears to be aware of, to be truly successful, it’s not enough to just lead the industry - you have to make sure you can bring people with you. Learn more about Kong here, and follow Marco Palladino on Twitter.

We've been talking about DevOps a lot on the Packt Podcast. The reason for that is simple: it's a critical part of how we actually build software from both a technical and an organizational perspective. And anything that draws us closer to the relationship between people and software can only be a good thing right? For this edition of the Packt Podcast we spoke to Nigel Kersten, who's the VP of Ecosystem Engineering at Puppet. With Puppet playing an important role in the evolution of DevOps over the last decade or so, we thought he would be a great person to give an insight not only on how Puppet has been adapting to industry trends (yes, we're waving at you, Kubernetes). Listen to the episode: https://soundcloud.com/packt-podcasts/puppets-vp-of-engineering-nigel-kersten-on-the-organizational-challenges-of-devops Nigel Kersten talks DevOps We covered a diverse range of topics in the episode. From Nigel's move from Google to Puppet (which, he tells us, slightly upset his mom...), through to the challenges - and pitfalls - engineering teams face when trying to implement DevOps. Read next: DevOps engineering and full-stack development – 2 sides of the same agile coin Key quotes from this podcast episode How to automate workflows effectively “One thing we definitely tell people to do is… don’t automate one service from end to end. Don’t pick one complicated three tier web application put a small team on it and say “your job is to puppetize all of this infrastructure. What, instead, is a more powerful way to work is you go what are those low level building blocks that are across all of your infrastructure...? What are the things that are common across all of your infrastructure? Automate those things because they’re often really simple to do, and the rewards are huge.”  “Look at the things that are causing you pain in production. If you go and talk to the people who are on call, in charge of deployments, any of those parts of your infrastructure and ask them what would be the one thing that you would fix that would make your infrastructure more reliable, they will always have a shortlist of things… and when you do this, you start building trust across the whole organization.” The fear of automation “There’s always fear about adopting automation. There’s always fear about people’s jobs changing and adopting new tools and disciplines - sort of in an endless cycle of new tool adoption, people being told that they have to learn new things - the more you can actually show value across the whole organization that this thing’s relatively easy, a small investment for large returns, the more powerful an effect you're actually going to have.” DevOps challenges “I think it’s a huge mistake if people think they’re embarking on a DevOps journey and they’re not willing to actually make some of the cultural and organizational changes - it’s about creating more cross-functional teams, it’s about giving them more autonomy, and it’s about actually letting people work across organizational boundaries without having to go up and down the hierarchy of the organization.” “Most people are actually struggling pre-DevOps in many ways… the people who we’ve seen fail are the ones who have gone, look we’re going to jump exactly from where we are now and try to move to an incredibly automated environment without putting a lot of the ground work in place  - like building up trust within the org, giving teams more autonomy, allowing service owners to configure monitoring themselves - I think all of those sorts of things are really prerequisites for a whole organization succeeding at DevOps.”

As software systems become more distributed, reliability and resiliency have become more and more important. This is one of the reasons why we've seen the emergence of chaos engineering - unreliability causes downtime which, in turn, also causes downtime. And downtime costs money. The impact of downtime is particularly significant for huge organizations that depend on the resilience and reliability of their platforms and applications. Take Uber - not only does the simplicity of the user experience hide its astonishing complexity, but it also has to ensure it can manage that complexity in a way that's reliable. A ride-hailing app couldn't be anywhere near as successful as Uber if it didn't work even if it had 1% downtime. Building resilient software is difficult But actually building resilient systems is difficult. We've recently seen how Uber uses distributed tracing to build more observable systems which can help improve reliability and resiliency in the last podcast episode with Yuri Shkuro but in this week's podcast we're diving even deeper into resiliency with Vilas Veeraraghavan, who's Director of Engineering at Walmart Labs. Vilas has experience at Netflix, the company where chaos engineering originated, but at Walmart, he's been playing a central role in bringing a more evolved version of chaos engineering - which Vilas calls resiliency engineering - to the organization. In this episode we discuss: Whether chaos engineering and resiliency engineering are for everyone Cultural challenges How to get buy-in Getting tooling right https://soundcloud.com/packt-podcasts/walmart-labs-director-of-engineering-vilas-veeraraghavan-on-chaos-engineering-resiliency   “You do not want to get up in the middle of the night get on the call with the VP of engineering and blurt out saying I have no idea what happened. Your answer should be I know exactly what happened because we have tested this exact scenario multiple times. We developed a recipe for it, and here is what we can do… that gives you as an engineer, the power to be able to stand up and say I know exactly what’s going on, I’ll fix it, don’t worry, we’re not going to cause an outage.”

We've been talking a lot about observability on the Packt Hub over the last few months. Back in March we spoke to Honeycomb CEO Charity Majors who told us why observability is so important and why it can be so challenging for engineering teams to implement. It's clear it's a big topic with plenty of perspectives - but one that could have a ripple effect across the software industry. To get a further perspective on the topic, we spoke to Yuri Shkuro, who's an engineer at Uber and author of Mastering Distributed Tracing (which was published in February) to talk about how distributed tracing can help engineers build more observable systems. Yuri spoke in detail in the podcast about the value of observability in the context of complex distributed systems, as well as some of the challenges in implementing distributed tracing. As one of the creators of Jaeger, an open source tool built specifically for distributed tracing, he's well-placed to comment on how the ecosystem is evolving and how organizations can start thinking more seriously about observability. Read an extract from Yuri's book here. The episode covers: The difference between monitoring and observability Some of the misconceptions around distributed tracing Who can benefit from distributed tracing - from DevOps to SREs Practical advice for getting started with distributed tracing Listen on SoundCloud: https://soundcloud.com/packt-podcasts/if-youre-on-call-you-need-observability-tools-uber-engineer-yuri-shkuro-on-distributed-tracing “Tracing is conceptually a white box instrumentation technique. You cannot do tracing in an application by purely observing it from the outside, because that feature of context propagation is simply not possible - if you have 10 incoming requests into an application concurrently, and it does 100 outbound requests then how do you know which ones correlate to the incoming requests? That’s what context propagation allows us to achieve, it allows us to establish causality within events.”

Transparency is underrated in the tech industry. But as software systems grow in complexity and their relationship with the real world becomes increasingly fraught, it nevertheless remains a value worth fighting for. But to effectively fight for it, it’s essential to remember that transparency is a technological issue, not just a communication one. Decisions about how software is built and why it’s built in the way that it is lie at the heart of what it means to work in software engineering. Indeed, the industry is in trouble if we can’t see just how important those questions are in relation to everything from system reliability to our collective mental health. Observability, transparency, and humility One term has recently emerged as a potential solution to these challenges: observability (or o11y as it's known in the community). This is a word that has been around for some time, but it’s starting to find real purchase in the infrastructure engineering world. There are many reasons for this, but a good deal of credit needs to go to observability platform Honeycomb and its CEO Charity Majors. [caption id="attachment_26599" align="alignleft" width="225"] Charity Majors[/caption] Majors has been a passionate advocate for observability for years. You might even say Honeycomb evolved from that passion and her genuine belief that there is a better way for software engineers to work. With a career history spanning Parse and Facebook (who acquired Parse in 2011), Majors is well placed to understand, diagnose, and solve the challenges the software industry faces in terms of managing and maintaining complex distributed systems designed to work at scale. “It’s way easier to build a complex system than it is to run one or to understand one,” she told me when I spoke to her in January. “We’re unleashing all these poorly understood complex systems on the world, and later having to scramble to make sense of it.” Majors is talking primarily about her work as a systems engineer, but it’s clear (to me at least) that this is true in lots of ways across tech, from the reliability of mobile apps to the accuracy of algorithms. And ultimately, impenetrable complexity can be damaging. Unreliable systems, after all, cost money. The first step, Majors suggests, to counteracting the challenges of distributed systems, is an acceptance of a certain degree of impotence. We need humility. She talks of “a shift from an era when you could feel like your systems were up and working to one where you have to be comfortable with the fact that it never is.” While this can be “uncomfortable and unsettling for people in the beginning,” in reality it’s a positive step. It moves us towards a world where we build better software with better processes. And, most importantly, it cultivates more respect for people on all sides - engineers and users. Charity Majors’ (personal) history of observability Observability is central to Charity Majors’ and Honeycomb’s purpose. But it isn’t a straightforward concept, and it’s also one that has drawn considerable debate in recent months. Ironically, although the term is all about clarity, it has been mired in confusion, with the waters of its specific meaning being more than a little muddied. “There are a lot of people in this space who are still invested in ‘oh observability is a generic synonym for telemetry,’” Majors complains. However, she believes that “engineers are hungry for more technical terminology,” because the feeling of having to deal with problems for which you are not equipped - quite literally - is not uncommon in today’s industry. With all the debate around what observability is, and its importance to Honeycomb, Majors is keen to ensure its definition remains clear. “When Honeycomb started up… observability was around as a term, but it was just being used as a generic synonym for telemetry… when we started… the hardest thing was trying to think about how to talk about it... because we knew what we were doing was different,” Majors explains. Experimentation at Parse The route to uncovering the very specific - but arguably more useful - definition of observability was through a period of sustained experimentation while at Parse. “Around the time we got acquired... I was coming to this horrifying realisation that we had built a system that was basically un-debuggable by some of the best engineers in the world.” The key challenge for Parse was dealing with the scale of mobile applications. Parse customers would tell Majors and her team that the service was down for them, underlining Parse’s monitoring tools’ lack of capability to pick up these tiny pockets of failure (“Behold my wall of dashboards! They’re all green, everything is fine!” Majors would tell them). Scuba: The “butt-ugly” tool that formed the foundations of Honeycomb The monitoring tools Parse was using at the time weren’t that helpful because they couldn’t deal with high-cardinality dimensions. Put simply, if you wanted to look at things on a granular, user by user basis, you just couldn’t do it. “I tried everything out there… the one thing that helped us get a handle on this problem was this butt-ugly tool inside Facebook that was aggressively hostile to users and seemed very limited in its functionality, but did one thing really well… it let you slice and dice in real time on dimensions of arbitrarily high cardinality.” Despite its shortcomings, this set it apart from other monitoring tools which are “geared towards low cardinality dimensions,” Majors explains. [caption id="attachment_26601" align="alignright" width="225"] More than just a quick fix (Credit: Charity Majors)[/caption] So, when you’re looking for “needles in a haystack,” as Parse engineers often were, the level of cardinality is essential. “It was like night and day. It went from hours, days, or impossible, to seconds. Maybe a minute.” Observability: more than just a platform problem This experience was significant for Majors and set the tone for Honeycomb. Her experience of working with Scuba became a frame for how she would approach all software problems. “It’s not even just about, oh the site is down, debug it, it’s, like, how do I decide what to build?” It had, she says, “become core to how I experienced the world.” Over the course of developing Honeycomb, it became clear to Majors that the problems the product was trying to address were actually deep: “a pure function of complexity.” “Modern infrastructure has become so ephemeral you may not even have servers, and all of our services are far flung and loosely coupled. Some of them are someone else’s service,” Majors says. “So I realise that everyone is running into this problem and they just don’t have the language for it. All we have is the language of monitoring and metrics when... this is inherently a distributed systems problem, and the reason we can’t fix them is because we don’t have distributed systems tools.” Towards a definition of observability Looking over my notes, I realised that we didn’t actually talk that much about the definition of observability. At first I was annoyed, but in reality this is probably a good thing. Observability, I realised, is only important insofar as it produces real world effects on how people work. From the tools they use to the way they work together, observability, like other tech terms such as DevOps, only really have value to the extent that they are applied and used by engineers. [caption id="attachment_26606" align="alignleft" width="225"] It's not always easy to tell exactly what you're looking at (Credit: Charity Majors)[/caption] “Every single term is overloaded in the data space - every term has been used - and I was reading the dictionary definition of the word ‘observability’ and... it’s from control systems and it’s about how much can you understand and reason about the inner workings of these systems just by observing them from the outside. I was like oh fuck, that’s what we need to talk about!” In reality, then, observability is a pretty simple concept: how much can you understand and reason about the inner workings of these systems just by observing them from the outside. Read next: How Gremlin is making chaos engineering accessible [Interview] But things, as you might expect, get complicated when you try and actually apply the concept. It isn’t easy. Indeed, that’s one of the reasons Majors is so passionate about Honeycomb. Putting observability into practice Although Majors is a passionate advocate for Honeycomb, and arguably one of its most valuable salespeople, she warns against the tendency for tooling to be viewed as silver bullet solutions to problems. “A lot of people have been sold this magic spell idea which is that you don’t have to think about instrumentation or explaining your code back to yourself” Majors says. Erroneously, some people will think they “can just buy this tool for millions of dollars that will do it for you… it’s like write code, buy tool, get magic… and it doesn’t actually work, it never has and it never will.” This means that while observability is undoubtedly a tooling issue, it’s just as much a cultural issue too. With this in mind, you definitely shouldn’t make the mistake of viewing Honeycomb as magic. “It asks more of you up front,” Majors says. “There is no magic. At no point in the future are you going to get to just write code and lob it over the wall for ops to deal with. Those days are over, and anyone who is telling you anything else is selling you some very expensive magic beans. The systems of the future do require more of developers. They ask you to care a little bit more up front, in terms of instrumentation and operability, but over the lifetime of your code you reap that investment back hundreds or thousands of times over. We're asking you, and helping you, make the changes you need to deal with the coming Armageddon of complexity.” Observability is important, but it’s a means to an end: the end goal is to empower software engineers to practice software ownership. They need to own the full lifecycle of their code. How transparency can improve accountability Because Honeycomb demands more ‘up front’ from its users, this requires engineering teams to be transparent (with one another) and fully aligned. Think of it this way: if there’s no transparency about what’s happening and why, and little accountability for making sure things do or don’t happen inside your software, Honeycomb is going to be pretty impotent. We can only really get to this world when everyone starts to care properly about their code, and more specifically, how their code runs in production. “Code isn’t even interesting on its own… code is interesting when users interact with it,” Majors says. “it has to be in production.” That’s all well and good (if a little idealistic), but Majors recognises there’s another problem we still need to contend with. “We have a very underdeveloped set of tools and best practices for software ownership in production… we’ve leaned on ops to… be just this like repository of intuition… so you can’t put a software engineer on call immediately and have them be productive…” Observability as a force for developer well-being This is obviously a problem that Honeycomb isn’t going to fix. And yes, while it’s a problem the Honeycomb marketing team would love to fix, it’s not just about Honeycomb’s profits. It’s also about people’s well being. [caption id="attachment_26602" align="alignright" width="300"] The Honeycomb team (Credit: Charity Majors)[/caption] “You should want to have ownership. Ownership is empowering. Ownership gives you the power to fix the thing you know you need to fix and the power to do a good job… People who find ownership is something to be avoided - that’s a terrible sign of a toxic culture.” The impact of this ‘toxic culture’ manifests itself in a number of ways. The first is the all too common issue of developer burnout. This is because a working environment that doesn’t actively promote code ownership and accountability, leads to people having to work on code they don’t understand. They might, for example, be working in production environments they haven’t been trained to adequately work with. "You can’t just ship your code and go home for the night and let ops deal with it," Majors asserts. "If you ship a change and it does something weird, the best person to find that problem is you. You understand your intent, you have all the context loaded in your head. It might take you 10 minutes to find a problem that would take anyone else hours and hours." Superhero hackers The second issue is one that many developers will recognise: the concept of the 'superhero hacker'. Read next: Don’t call us ninjas or rockstars, say developers “I remember the days of like… something isn’t working, and we’d sit around just trying random things or guessing... it turns out that is incredibly inefficient. It leads to all these cultural distortions like the superhero hacker who does the best guessing. When you have good tooling, you don’t have to guess. You just look and see.” Majors continues on this idea: “the source of truth about your systems can’t live in one guy’s head. It has to live in a tool where everyone has access to the same information about the system, one single source of truth... Otherwise you’re gonna have that one guy who can’t go on vacation ever.” While a cynic might say well she would say that - it’s a product pitch for Honeycomb, they’d ultimately be missing the point. This is undoubtedly a serious issue that’s having a severe impact on our working lives. It leads directly to mental health problems and can even facilitate discrimination based on gender, race, age, and sexuality. At first glance, that might seem like a stretch. But when you’re not empowered - by the right tools and the right support - you quite literally have less power. That makes it much easier for you to be marginalized or discriminated against. Complexity stops us from challenging the status quo The problem really lies with complexity. Complexity has a habit of entrenching problems. It stops us from challenging the status quo by virtue of the fact that we simply don’t know how to. This is something Majors takes aim at. In particular, she criticises "the incorrect application of complexity to the business problem it solves." She goes on to say that “when this happens, humans end up plugging the dikes with their thumbs in a continuous state of emergency. And that is terrible for us as humans." How Honeycomb practices what it preaches Majors’ passion for what she believes is evidenced in Honeycomb's ethos and values. It’s an organization that is quite deliberately doing things differently from both a technical and cultural perspective. [caption id="attachment_26604" align="alignright" width="300"] Inside the Honeycomb HQ (Credit: Charity Majors)[/caption] Majors tells me that when Honeycomb started, the intention was to build a team that didn’t rely upon superstar engineers: “We made the very specific intention to not build a team of just super-senior expert engineers - we could have, they wanted to come work with us, but we wanted to hire some kids out of bootcamp, we wanted to hire a very well rounded team of lots of juniors and intermediates... This was a decision that I made for moral reasons, but I honestly didn’t know if I believed that it would be better, full disclosure - I honestly didn’t have full confidence that it would become the kind of high powered team that I felt so proud to work on earlier in my career. And yet... I am humbled to say this has been the most consistent high-performing engineering team that I have ever had the honor to work with. Because we empower them to collaborate and own the full lifecycle of their own code.” Breaking open the black boxes that sustain internal power structures This kind of workplace, where "the team is the unit you care about" is one that creates a positive and empowering environment, which is a vital foundation for a product like Honeycomb. In fact, the relationship between the product and the way the team works behind it is almost mimetic, as if one reflects the other. Majors says that "we’re baking" Honeycomb's organizational culture “into the product in interesting ways." [caption id="attachment_26603" align="alignleft" width="300"] Teamwork (Credit: Charity Majors)[/caption] She says that what’s important isn’t just the question of “how do we teach people to use Honeycomb, but how do we teach people to feel safe and understand their giant sprawling distributed systems. How do we help them feel oriented? How do we even help them feel a sense of safety and security?"   Honeycomb is, according to Majors, like an "outsourced brain." It’s a product that means you no longer need to worry about information about your software being locked in a single person’s brain, as that information should be available and accessible inside the product. This gives individuals safety and security because it means that typical power structures, often based on experience or being "the guy who’s been there the longest" become weaker. Black boxes might be mysterious but they're also pretty powerful. With a product like Honeycomb, or, indeed, the principles of observability more broadly, that mystery begins to lift, and the black box becomes ineffective. Honeycomb: building a better way of developing software and developing together In this context, Liz Fong-Jones’ move to Honeycomb seems fitting. Fong-Jones (who you can find on Twitter @lizthegrey) was a Staff SRE at Google and a high profile critic of the company over product ethics and discrimination. She announced her departure at the beginning of 2019 (in fact, Fong-Jones started at Honeycomb in the last week of February). By subsequently joining Honeycomb, she left an environment where power was being routinely exploited, for one where the redistribution of power is at the very center of the product vision. Honeycomb is clearly a product and a company that offers solutions to problems far more extensive and important than it initially thought it would. Perhaps we’re now living in a world where the problems it’s trying to tackle are more profound than they first appear. You certainly wouldn’t want to bet against its success with Charity Majors at the helm. Follow Charity Majors on Twitter: @mipsytipsy Learn more about Honeycomb and observability at honeycomb.io. You can try Honeycomb for yourself with a free trial.

This week I published a post that explored some of the key trends in software infrastructure that security engineers, SREs, and SysAdmins should be paying attention to in 2019. There was clearly a lot to discuss - which is why I sat down with my colleague Stacy Matthews to discuss some of the topics explored in the post in a little more. Enjoy! https://soundcloud.com/packt-podcasts/why-observability-and-chaos-engineering-will-be-vital-in-2019 What do you think? Is chaos engineering too immature for widespread adoption? And how easy will it be to begin building for observability?

At a time when software systems are growing in complexity, and when the expectations and demands from users have never been more critical, it's easy to forget that just making things work can be a huge challenge. That's where site reliability engineering (SRE) comes in; it's one of the reasons we're starting to see it grow as a discipline and job role. The central philosophy behind site reliability engineering can be seen in trends like chaos engineering. As Gremlin CTO Matt Fornaciari said, speaking to us in June, "chaos engineering is simply part of the SRE toolkit." For site reliability engineers, software resilience isn't an optional extra - it's critical. In crude terms, downtime for a retail site means real monetary losses, but the example extends beyond that. Because people and software systems are so interdependent, SRE is a useful way for thinking about how we build software more broadly. To get to the heart of what site reliability engineering is, I spoke to Nat Welch, an SRE currently working at First Look Media, whose experience includes time at Google and Hillary Clinton's 2016 presidential campaign. Nat has just published a book with Packt called Real-World SRE. You can find it here. Follow Nat on Twitter: @icco What is site reliability engineering? Nat Welch: The idea [of site reliability engineering] is to write and modify software to improve the reliability of a website or system. As a term and field, it was founded by Google in the early 2000s, and has slowly spread across the rest of the industry. Having engineers dedicated to global system health and reliability, working with every layer of the business to improving reliability for systems. "By building teams of engineers focused exclusively on reliability, there can be someone arguing for and focusing on reliability in a way to improve the speed and efficiency of product teams." Why do we need site reliability engineering? Nat Welch: Customers get mad if your website is down. Engineers often were having trouble weighing system reliability work versus new feature work. Because of this, product feature work often takes priority, and reliability decisions are made by guess work. By building teams of engineers focused exclusively on reliability, there can be someone arguing for and focusing on reliability in a way to improve the speed and efficiency of product teams. Why do we need SRE now, in 2018? Nat Welch: Part of it is that people are finally starting to build systems more like how Google has been building for years (heavy use of containers, lots of services, heavily distributed). The other part is a marketing effort by Google so that they can make it easier to hire. What are the core responsibilities of an SRE? How do they sit within a team? Nat Welch: SRE is just a specialization of a developer. They sit on equal footing with the rest of the developers on the team, because the system is everyone's responsbility. But while some engineers will focus primarily on new features, SRE will primarily focus on system reliability. This does not mean either side does not work on the other (SRE often write features, product devs often write code to make the system more reliable, etc), it just means their primary focus when defining priorities is different. What are the biggest challenges for site reliability engineers? Nat Welch: Communication with everyone (product, finance, executive team, etc.), and focus - it's very easy to get lost in fire fighting. What are the 3 key skills you need to be a good SRE? Nat Welch: Communication skills, software development skills, system design skills. You need to be able to write code, review code, work with others, break large projects into small pieces and distribute the work among people, but you also need to be able to take a system (working or broken) and figure out how it is designed and how it works. Thanks Nat! Site reliability engineering, then, is a response to a broader change in the types of software infrastructure we are building and using today. It's certainly a role that offers a lot of scope for ambitious and curious developers interested in a range of problems in software development, from UX to security. If you want to learn more, take a look at Nat's book.

In the past few years, Agile software development has seen tremendous growth. There is a huge demand for software delivery solutions that are fast, yet flexible to numerous amendments. As a result, Continuous Integration (CI) and Continuous Delivery (CD) methodologies are gaining popularity. They are considered to be the cornerstones of DevOps and drive the possibilities of modern architectures like microservices and cloud native. Author’s Bio Nikhil Pathania, a DevOps practitioner at Siemens Gamesa Renewable Energy, started his career as an SCM engineer and later moved on to learn various tools and technologies in the fields of automation and DevOps. Throughout his career, Nikhil has promoted and implemented Continuous Integration and Continuous Delivery solutions across diverse IT projects. He is the author of Learning Continuous Integration with Jenkins. In this exclusive interview, Nikhil gives us a sneak peek into the trends and challenges of Continuous Integration in DevOps. Key Takeaways The main function of Continuous Integration is to provide feedback on integration issues. When practicing DevOps, a continuous learning attitude, sharp debugging skills, and an urge to improvise processes is needed Pipeline as a code is a way of describing a Continuous Integration pipeline in a pre-defined syntax One of the main reasons for Jenkin’s popularity is it’s growing support via plugins Making yourself familiar with a scripting language like Shell or Python will help you accomplish difficult tasks related to CI/CD Continuous Integration is built on Agile and requires a fair understanding of the 12 principles. Full Interview On the popularity of DevOps DevOps as a concept and culture is gaining a lot of traction these days. What is the reason for this rise in popularity? What role does Continuous Integration have to play in DevOps? To understand this, we need to look back at the history of software development. For a long period, the Waterfall model was the predominant software development methodology in practice. Later, when there was a sudden surge in the usage and development of software applications, the Waterfall model proved to be inefficient, thus giving rise to the Agile model. This new model proposed coding, building, testing, packaging, and releasing software in a quick and incremental fashion. As the Agile model gained momentum, more and more teams wanted to ship their applications faster and more frequently. This added a huge pressure on the release management process. To cope up with this pressure, engineers came up with new processes and techniques (collectively bundled as DevOps), such as the usage of improved branching strategies, Continuous Integration, Continuous Delivery, Automated environment provisioning, monitoring, and configuration. Continuous Integration involves continuous building and testing of your integrated code; it’s an integral part of DevOps, dealing with automated builds, testing, and more. Its core function is to provide a quick feedback on the integration issues. On your journey as a DevOps engineer You have been associated with DevOps for quite some time now and hold vast experience as a DevOps engineer and consultant. How and when did your journey start? Which tools did you master to help you with your day-to-day tasks? I started my career as a Software Configuration Engineer and was trained in SCM and IBM Rational Clearcase. After working as a Build and Release Engineer for a while, I turned towards new VCS tools such as Git, automation, and scripting. This is when I was introduced to Jenkins followed by a large number of other DevOps tools such as SonarQube, Artifactory, Chef, Teamcity, and more. It’s hard to spell out the list of tools that you are required to master since the list keeps increasing as the days pass by. There is always a new tool in the DevOps tool chain replacing the old one. A DevOps tool itself changes a lot in its usage and working over a period of time. A continuous learning attitude, sharp debugging skills, and an urge to improvise processes is what is needed, I’ll say. On the challenges of implementing Continuous Integration What are some of the common challenges faced by engineers in implementing Continuous Integration? Building the right mind-set in your organization: By this I mean preparing teams in your organisation to get Agile. Surprised! 50% of the time we spend at work is on migrating teams from old ways of working to the new ones. Implementing CI is one thing, while making the team, the project, the development process, and the release process ready for CI is another. Choosing the right VCS tool and CI tool: This is an important factor that will decide where your team will stand a few years down the line—rejoicing in the benefits of CI or shedding tears in distress. On how the book helps overcome these challenges How does your book 'Learning Continuous Integration with Jenkins' help DevOps professionals overcome the aforementioned challenges? This is why I have a whole chapter (Concepts of Continuous Integration) explaining how Continuous Integration came into existence and why projects need it. It also talks a little bit about the software development methodologies that gave rise to it. The whole book is based on implementing CI using Jenkins, Git, Artifactory, SonarQube, and more. About Pipeline as a Code Pipeline as a Code was a great introduction in Jenkins 2. How does it simplify Continuous Integration? Pipeline as a code is a way of describing your Continuous Integration pipeline in a pre-defined syntax. Since it’s in the form of code, it can be version-controlled along with your source code and there are endless possibilities of programming it, which is something you cannot get with GUI pipelines. On the future of Jenkins and competition Of late, tools such as TravisCI and CircleCI have got a lot of positive recognition. Do you foresee them going toe to toe with Jenkins in the near future? Over the past few years Jenkins has grown into a versatile CI/CD tool. What makes Jenkins interesting is its huge library of plugins that keeps growing. Whenever there is a new tool or technology in the software arena, you have a respective plugin in Jenkins for it. Jenkins is an open source tool backed by a large community of developers, which makes it ever-evolving. On the other hand, tools like TravisCI and CircleCI are cloud-based tools that are easy to start with, limited to CI in their functionality, and work with GitHub projects. They are gaining popularity mostly in teams and projects that are new. While it’s difficult to predict the future, what I can say for sure is that Jenkins will adapt to the ever-changing needs and demands of the software community. On key takeaways from the book Learning Continuous Integration with Jenkins Coming back to your book, what are the 3 key takeaways from it that readers will find to be particularly useful? In-depth coverage of the concepts of Continuous Integration. A step-by-step guide to implementing Continuous Integration, Continuous Delivery with Jenkins 2 using all the new features. A practical usage guide to Jenkins's future, the Blue Ocean. On the learning path for readers Finally, what learning path would you recommend for someone who wants to start practicing DevOps and, specifically, Continuous Integration? What are the tools one must learn? Are there any specific certifications to take in order to form a solid resume? To begin with, I would recommend learning a VCS tool (say Git), a CI/CD tool (Jenkins), a configuration management tool (Chef or Puppet, for example), a static code analysis tool, a cloud tool like AWS or Digital Ocean, and an artifactory management tool (say Artifactory). Learn Docker. Build a solid foundation in the Build, Release and Deployment processes. Learn lots of scripting languages (Python, Ruby, Groovy, Perl, PowerShell, and Shell to name a few), because the real nasty tasks are always accomplished by scripts. A good knowhow of the software development process and methodologies (Agile) is always nice to have. Linux and Windows administration will always come in handy. And above all, a continuous learning attitude, an urge to improvise the processes, and sharp debugging skills is what is needed. If you enjoyed reading this interview, check out Nikhil’s latest edition Learning Continuous Integration with Jenkins. Top 7 DevOps Tools in 2018 Everything you need to know about Jenkins X 5 things to remember when implementing DevOps

Golang is currently one of the fastest growing programming languages in the software industry, finding its way into almost every nook and cranny of application development. Its speed, simplicity and reliability make it the perfect choice for all kinds of developers. We recently interviewed Mina Andrawos, an experienced Go engineer and the author of the book, Cloud Native programming with Golang.  Mina explains why Go is being rapidly adopted in various development areas and by leading projects like Docker and Ethereum, how it is evolving as a language and what makes it great for cloud development. He shares expert insights into Go’s adoption for mobile development, embedded systems and the serverless web. He has also thrown light on the new directions of cloud computing and how Go makes development a piece of cake. Author’s Bio Mina Andrawos is an experienced engineer who has developed deep expertise in Go by using it personally and professionally. He has written numerous Go applications with varying degrees of complexity. Other than Go, he has skills in Java, C#, Python, and C++. He has worked with various databases and software architectures and is skilled with the agile methodology for software development. Besides software development, he has working experience of scrum mastering, sales engineering, and software product management. Key Takeaways The 3 most notable features of Go are its concurrency model that sets it apart from mainstream languages, the fairly mature standard package which covers a wide range of use cases and its ease of deployment. Go is designed to be simple and intuitive, yet reliable and robust for application development. There are currently several mature tools to write Go programs, like VSCode, Vim, Atom or Sublime text. Mina’s book Cloud native programming with Golang helps you build production level cloud native microservices and covers a wide range of important topics in the space such as types of message queues, docker containers, how to monitor microservices, perform continuous integration and much more. Go can be viewed as a hybrid between mainstream statically typed languages like Java, and popular dynamic scripting languages like Javascript. Go was built with the goal of being fully cross-platform in mind, and it can work in smaller mobile processors like ARM. Full Interview Go is one of the most popular and fast growing programming languages. What according to you, are the 3 notable features of Go? Go is a very remarkable programming language. Numerous articles were written about the advantages of the language. Trying to gather notable features in Go can actually produce enough material to fill a number of white papers. However, having said that, let’s try to squeeze three out of them: 1. Concurrency: Go’s unique concurrency features are legendary. The language offers a concurrency model that stands apart from most mainstream programming languages. Go advocates a different way of thinking about concurrency problems in modern software. In one of the articles I wrote, I have described what concurrency means in the Go language. 2.  The Standard package: Go has the advantage of being coupled with a fairly mature standard package, which covers tons of key features for building modern software. This means that once you install Go, you can build production level software that can cover a wide range of use cases from Restful web APIs to encryption software, before needing to consider any third party packages. 3.  Ease of deployment: A program written in pure Go code typically compiles to a single native binary, which basically makes deploying an application written in Go as easy as copying the application file to the destination server. In other words, there is no special software needed to run Go applications in production servers like language runtimes \ virtual machines (As an example, for Java programs, we need to install the Java runtime environment in our production servers to run our programs) . Go is also cross platform, so you can target an operating system of your choice when compiling a piece of code. You have been developing software for quite some time now. What tools do you use on a day-to-day basis? Programming is a very fun craft, and the tools we use in our development are integral to making the environment enjoyable. For me, because I work with multiple programming languages, I use different tools based on the project. My current tool of choice for the Go programming language is VSCode, combined with its Go Plugin by lukehoban.  This is just my preference however. There are lots of other tools that could be used to write Go programs. Some developers prefer Vim with all it’s popular features, while others prefer Atom or sublime text. There is also a Go plugin for the IntelliJ IDE, which I had used in the past and really liked. What kind of learning plan would you suggest to web developers who are interested in using Go as their main development tool to build Cloud Native Applications? What aspects do you feel are tricky to get past? The plan would include three steps: Get comfortable with Go. Learn the design patterns, the software tools, and the technologies of cloud native applications. Get familiar with a cloud service provider (like AWS, Azure, or Google cloud) Go is designed from the grounds up to be simple and intuitive. This makes learning Go a better and more straight forward experience compared to many other languages. For developers new to Go, one of the best resources to start learning Go, is the Go tour. Once the developer is familiar with Go, then they are ready to move to the next step of the learning plan, which is to learn the design patterns of cloud native applications, as well as the software technologies needed to build and deploy such applications. A good way to start is to check out my newly published book: Cloud Native Programming with Go. One major advantage of the book is that it not only covers the technologies and design patterns associated with cloud native applications, but it also connects these technologies and design patterns with Go, which makes it an excellent resource for Go developers looking to build cloud native software. This, in my opinion, is the trickiest aspect a software professional needs to get past to acquire the necessary skills to build cloud native applications. For the third step, the execution will depend on the cloud service provider that you or your business would like to work with. Some enterprises like to utilize their own private clouds, while others are tied to a mainstream cloud provider due to existing contracts or executive preferences. For AWS, my book should provide enough insights into  how to write Go cloud native applications, that are capable of making use of the cloud platform. In the context of all the above, how does your book, Cloud Native programming with Golang, prepare its readers to be industry ready? What are the key takeaways for readers from your title and how does your book help with the learning curve? The book was the product of great amount of research, sleepless nights, and focused effort. I am a coauthor of the book with Martin Helmich, who I enjoyed working with immensely. The book was designed from the get go to expose the reader to the practical experience needed to build production level cloud native microservices in Go, with the least amount of fat. It takes the reader into an expanding learning journey, which starts from the ten thousands foot view of cloud native microservices, then dives deep down into all the different aspects that need to work together in harmony in order to produce production level cloud native applications. It will prepare you to be industry ready by covering a wide array of topics that are vital in a production environment. Examples include: Different types of message queues found in production environments, docker containers, monitoring microservices via Prometheus, continuous integration, Restful APIs design, security and authentication, AWS Go APIs, NoSQL databases, ReactJS, and more. What makes it so special that it doesn’t shy away from covering sophisticated and diverse topics from scratch. For example, if you look at the Restful API chapter, we don’t assume that you already have knowledge of the HTTP protocol or web services design. Instead, we build the concepts with you from point zero up. The only knowledge you need before reading the book is some familiarity with the Go language. Another example is our message queues chapter, you can start reading the chapter knowing nothing about message queues, but then finish the chapter with more than enough knowledge to be very effective in utilizing message queues in your applications. The book is perfect for readers who want to begin learning how to build cloud native microservice applications. It will carry the reader from a beginner level to a point where they become capable of tackling advanced tools and design patterns in that space. You've been working with several other languages like Java, C++, C# and Python. How does Go compare to the other languages you've worked with? Go, in my opinion, could be viewed as a hybrid between mainstream statically typed languages (like Java), and popular dynamic scripting languages (like Javascript). That is because Go doesn’t require the same level of verbosity that you would need in a Java program. However it’s still a bit more verbose than an equivalent Javascript or Python implementation, luckily, Go makes up for this extra verbosity compared to dynamic languages, by delivering software that is much faster than the equivalent Python or Javascript implementation. One very hotly debated feature that is missing in Go is generics. Some people in the community believe it’s a good thing Go doesn’t have generics, while others can’t wait till Go maintainers are convinced that generics need to be added. From my personal experience, I have come across situations where it would have been nice to have generics, however it never got to the point where I couldn’t complete the task at hand. Having said that, there are some situations where you can argue that a piece of Go code might be a bit more verbose than an equivalent piece of Java code that makes use of generics. As mentioned earlier, Go’s concurrency model is different than almost all mainstream programming languages. Once you master the building blocks of Go’s concurrency model (namely, Go channels and goroutines), you can build very powerful concurrent software with relative ease. I always find writing concurrent software in Go to be a much more smooth experience for me than writing concurrent software in other languages. Also another mention from earlier was the ease of deployment. I never tire from enjoying how easy it is to deploy my Go programs to production compared to other languages. One last notable mention is the tooling. Since Go is a relatively new programming language, the tooling is not yet as fancy as what’s available for older languages like C# or Java for example. However, having said that, the Go ecosystem is maturing nicely every day, and we have more than enough tools right now to build fairly sophisticated software in Go. There is no more proof of this fact than the uprise of advanced software projects written in Go like Docker and Ethereum. You've worked with JavaScript as well. What's your take on using Go for full-stack web development / Isomorphic web development, over JS? That is a very interesting question. For people not familiar with the term ‘isomorphic web development’, it basically means using the same programming language for most of the front-end and the back-end components of the web application (combined with CSS or LESS or some other front-end styling technology). There is an important distinction to make between ‘Isomorphic web development’, and ‘full-stack web development’. You can be a full-stack web developer, while using Javascript for the front-end in addition to another language like Go or Ruby for the backend. However, if you are building an ‘Isomorphic’ web application, the idea is that you make use of one language for almost all your code, whether it’s on the front-end or the backend. I think Go enjoys being in the sweet spot where simplicity meets performance. That is because, Go comes included with out-of-the-box packages, that make web development relatively smooth. Not to mention a growing third party ecosystem, that complements the standard package and further facilitates writing web applications in the Go programming language. Having said that, Javascript was built initially for the sole purpose of front-end web pages, but then grew in scope after the Node.Js project came into existence, which made Javascript a more than capable backend language as well. So for the sake of being neutral and impartial, I would like to cover some advantages and disadvantages of using Go for web development vs Javascript. Let’s start with the disadvantages of using Go for web development compared to Javascript: Javascript is a language that could natively be used in the frontend and the backend components of web applications, this will always be an advantage of using Javascript over any other programming language, when it comes to web development. However, in case of Go, this disadvantage is countered to some extent, by the existence of GopherJS . GopherJS converts Go code to Javascript code. This means that you can write front-end code in Go, then have it converted to Javascript in order to work on the browser, which will get you very close to the isomorphic web development experience you obtain from using Javascript on the frontend combined with Node.JS on the backend. GopherJS is a very popular project, with more than 6000 stars on Github. People use it and it delivers them results. Having said that, the disadvantage of GopherJS is that it’s not native, since it converts your Go code to Javascript code, which means that when tricky issues happen, you may need to troubleshoot the auto-generated Javascript code, which is not always a fun experience, especially if your reason for using GopherJS is to avoid Javascript in the first place. Your experience will vary based on your projects, and the goals you are trying to achieve. Where do you see the future of Go's development going? What changes or improvements can the community expect in future releases? Go is growing in popularity every day. I see an immensely positive outlook for the future of development in Go. I think the sky's the limit. Go currently powers some of the most exciting projects in the industry, like Docker, Kubernetes, and Ethereum, among many others. Not only that, but Go also became integral to the operations of major players in the software industry Like Google and Uber, among many others as well. All of this richness of the user base, provides Go unprecedented opportunities for growth and adoption. Engineers and maintainers who experienced Go first hand, tend to use it in their future endeavors, further enriching the ecosystem.  The language had been fairly stable and consistent for a while now, and no substantial language changes are to be expected in the near future. So if you start learning Go now, you skills will stay relevant for a long time. Most of the improvements currently getting added to Go are more related to it’s runtime performance as well as standard package enhancements. Are there any interesting areas of implementation you've noticed Go finding its way into? Do you think the language would be best fit for any specific kind of development? One interesting area for me that Go is starting to find it’s way into is mobile development. Since Go was built with the goal of being fully cross-platform in mind, it can work in smaller mobile processors like ARM for example. This means that programs written in Go not only can work in server and desktop operating systems -like Linux, MacOS, and Windows- but they also can function in mobile environments like Android and IOS.  Having said that, it is important to mention that the ecosystem for developing Go apps on mobile devices is still young and maturing. If curious, you can check https://github.com/golang/mobile for Go’s mobile tools. There is also an interesting Go framework that is still in early development but looks extremely promising as a tool to write mobile applications in Go, you can find it here: https://gomatcha.io Regarding best fit use cases for the language, I see Go as a powerhouse for backend software development. Especially the kind of modern backend that relies on microservices and distributed architectures. The power that Go gives you in the world of the server backend is indisputable. Can you give developers 3 reasons why they should pick up your book? This book Cloud Native programming with Golang covers a diverse set of practical topics from scratch, that can help the reader build production level cloud native microservices. We did a lot of research to put all these topics together. I honestly doubt you would find another resource that would cover all those topics in one place. Example of topics covered are: Restful APIs, Secure microservices, message queues (Kafka, RabbitMQ, and AWS SQS), ReactJS, MongoDB, DynamoDB, Docker, Kubernetes, AWS, microservices monitoring with Prometheus, and continuous delivery, among others. Additionally it covers the topics in a logical top-down order, which solidifies the learning process. So we start the journey by covering the 10,000 foot view about how a cloud native architecture looks like, the design, the thinking process, the scalability, and more. From there, we take satisfying deep dives into the different aspects of cloud native applications. Towards the end of the learning journey, we don’t just leave the reader with no direction. Instead, we offer a path forward to where they should take their learning journey to the next level. Amazon recently added Lambda support for Go. What's your opinion on Serverless Go? Would it go hand in hand with Cloud Native development? It was a very exciting announcement indeed. I believe serverless support is a powerful tool in the developer’s toolbox to build cloud native applications in Go. The option to include a serverless component in your application, allows you to automate very focused triggered tasks that are not supposed to run forever. This ability helps you build better cloud native applications in the long run. Microservices, on the other hand, are better suited for tasks and operations that are expected to run continuously. If you enjoyed this interview, do head over to check out Mina’s book, Cloud Native programming with Golang.