Tech News - Cloud & Networking

Yesterday, the team at GitLab released GitLab 12.3, a DevOps lifecycle tool that provides a Git-repository manager. This release comes with Web Application Firewall, Productivity Analytics, new Environments section and much more. What’s new in GitLab 12.3? Web Application Firewall In GitLab 12.3, the team has shipped the first iteration of the Web Application Firewall that is built in the GitLab SDLC platform. The Web Application Firewall focuses on monitoring and reporting the security concerns related to Kubernetes clusters.  Productivity Analytics  From GitLab 12.3, the team has started releasing Productivity Analytics that will help teams and their leaders in discovering the best practices for better productivity. This release will help in drilling into the data and learning insights for improvements in future. Group level analytics workspace can be used to provide performance insight, productivity, and visibility across multiple projects. Environments section This release comes with “Environments” section in the cluster page that gives an overview of all the projects that are making use of the Kubernetes cluster. License compliance  License Compliance feature can be used to disallow a merger when a blacklisted license is found in a merge request.  Keyboard shortcuts This release comes with the new ‘n’ and ‘p’ keyboard shortcuts that can be used to move to the next and previous unresolved discussions in Merge Requests. System hooks System hooks allow automation by triggering requests whenever a variety of events in GitLab take place. Multiple IP subnets This release introduces the ability to specify multiple IP subnets so instead of specifying a single range, it is now possible for large organizations to restrict incoming traffic to their specific needs. GitLab Runner 12.3 Yesterday, the team also released GitLab Runner 12.3, an open-source project that is used for running CI/CD jobs and sending the results back to GitLab. Audit logs In this release, the audit logs for push events are disabled by default for preventing performance degradation on GitLab instances. Few GitLab users are unhappy as some of the features of this release including Productivity Analytics are available to Premium or Ultimate users only. https://twitter.com/gav_taylor/status/1175798696769916932 To know more about this news, check out the official page. Other interesting news in cloud and networking Kubernetes 1.16 releases with Endpoint Slices, general availability of Custom Resources, and other enhancements DevOps platform for coding, GitLab reached more than double valuation of $2.75 billion than its last funding and way ahead of its IPO in 2020 Istio 1.3 releases with traffic management, improved security, and more!    

Yesterday, the Kubernetes team announced the availability of Kubernetes 1.16, which consists of 31 enhancements: 8 moving to stable, 8 is beta, and 15 in alpha. This release contains a new feature called Endpoint Slices in alpha to be used as a scalable alternative to Endpoint resources. Kubernetes 1.16 also contains major enhancements like custom resources, overhauled metrics and volume extension. It also brings additional improvements like the general availability of custom resources and more. Extensions like extensions/v1beta1, apps/v1beta1, and apps/v1beta2 APIs are deprecated in this version. This is Kubernetes' third release this year. The previous version Kubernetes 1.15 released three months ago. It accorded features like extensibility around core Kubernetes APIs and cluster lifecycle stability and usability improvements. Introducing Endpoint Slices in Kubernetes 1.16 The main goal of Endpoint Slices is to increase the scalability for Kubernetes Services. With the existing Endpoints, a single resource had to include all the network endpoints making the corresponding Endpoints resources large and costly. Also, when an Endpoints resource is updated, all the pieces of code watching the Endpoints required a full copy of the resource. This became a tedious process when dealing with a big cluster. With Endpoint Slices, the network endpoints for a Service are split into multiple resources by decreasing the amount of data required for updates. The Endpoint Slices are restricted to 100 endpoints each, by default. The other goal of Endpoint Slices is to provide extensible and useful resources for a variety of implementations. Endpoint Slices will also provide flexibility for address types. The blog post states, “An initial use case for multiple addresses would be to support dual stack endpoints with both IPv4 and IPv6 addresses.”  As the feature is available in alpha only, it is not enabled by default in Kubernetes 1.16. Major enhancements in Kubernetes 1.16 General availability of Custom Resources With Kubernetes 1.16, CustomResourceDefinition (CRDs) is generally available, with apiextensions.k8s.io/v1, as it contains the integration of API evolution in Kubernetes. CRDs were previously available in beta. It is widely used as a Kubernetes extensibility mechanism. In the CRD.v1, the API evolution has a ‘defaulting’ support by default. When defaulting is  combined with the CRD conversion mechanism, it will be possible to build stable APIs over time. The blog post adds, “Updates to the CRD API won’t end here. We have ideas for features like arbitrary subresources, API group migration, and maybe a more efficient serialization protocol, but the changes from here are expected to be optional and complementary in nature to what’s already here in the GA API.” Overhauled metrics In the earlier versions, the global metrics registry was extensively used by the Kubernetes to register exposed metrics. In this latest version, the metrics registry has been implemented, thus making the Kubernetes metrics more stable and transparent. Volume Extension This release contains many enhancements to volumes and volume modifications. The volume resizing support in (Container Storage Interface) CSI specs has moved to beta, allowing the CSI spec volume plugin to be resizable. Additional Windows Enhancements in Kubernetes 1.16 Workload identity option for Windows containers has moved to beta. It can now gain exclusive access to external resources. New alpha support is added for kubeadm which can be used to prepare and add a Windows node to cluster. New plugin support is introduced for CSI in alpha. Interested users can download Kubernetes 1.16 on GitHub. Check out the Kubernetes blog page for more information. Other interesting news in Kubernetes The Continuous Intelligence report by Sumo Logic highlights the rise of Multi-Cloud adoption and open source technologies like Kubernetes Kubernetes releases etcd v3.4 with better backend storage, improved raft voting process, new raft non-voting member and more CNCF-led open source Kubernetes security audit reveals 37 flaws in Kubernetes cluster; recommendations proposed

Yesterday, GitLab, a San Francisco based start-up, raised $268 million in a Series E funding round valuing the company at $2.75 billion, more than double of its last valuation. In the Series D round funding of $100 million the company was valued at $1.1 billion; and with today’s announcement, the valuation has more than doubled in less than a year. GitLab provides a DevOps platform for developing and collaborating on code and offers a single application for companies to draft, develop and release code. The product is used by companies like Delta Air Lines Inc., Ticketmaster Entertainment Inc. and Goldman Sachs Group Inc etc. The Series E funding round was led by investors including Adage Capital Management, Alkeon Capital, Altimeter Capital, Capital Group, Coatue Management, D1 Capital Partners, Franklin Templeton, Light Street Capital, Tiger Management Corp. and Two Sigma Investments. GitLab plans to go public in November 2020 According to Forbes, GitLab has already set November 18, 2020 as the date for going public. The company seems to be primed and ready for the eventual IPO. As for the $268 million, it gives the company considerable time ahead of the planned event and also gives the flexibility to choose how to take the company public. “One other consideration is that there are two options to go public. You can do an IPO or direct listing. We wanted to preserve the optionality of doing a direct listing next year. So if we do a direct listing, we’re not going to raise any additional money, and we wanted to make sure that this is enough in that case,” Sid Sijbrandij, Gitlab co-founder and CEO explained in an interview for TechCrunch. He further adds, that the new funds will be used to add monitoring and security to GitLab’s offering, and to increase the company’s staff to more than 1,000 employees this year from 400 employee strength currently. GitLab is able to add workers at a rapid rate, since it has an all-remote workforce. GitLab wants to be independent and chooses transparency for community Sijbrandij says that the company made a deliberate decision to be transparent early on. Being based on an open-source project, it’s sometimes tricky to make the transition to a commercial company, and sometimes that has a negative impact on the community and the number of contributions. Transparency was a way to combat that, and it seems to be working. He reports that the community contributes 200 improvements to the GitLab open-source products every month, and that’s double the amount of just a year ago, so the community is still highly active. He did not ignore the fact that Microsoft acquired GitHub last year for $7.5 billion. And GitLab is a similar kind of company that helps developers manage and distribute code in a DevOps environment. He claims in spite of that eye-popping number, his goal is to remain an independent company and take this through to the next phase. “Our ambition is to stay an independent company. And that’s why we put out the ambition early to become a listed company. That’s not totally in our control as the majority of the company is owned by investors, but as long as we’re more positive about the future than the people around us, I think we can we have a shot at not getting acquired,” he said. Community is happy with GitLab’s products and services Overall the community is happy with this news and GitLab’s products and services. One of the comments on Hacker News reads, “Congrats, GitLab team. Way to build an impressive business. When anybody tells you there are rules to venture capital — like it’s impossible to take on massive incumbents that have network effects — ignore them. The GitLab team is doing something phenomenal here. Enjoy your success! You’ve earned it.” Another user comments, “We’ve been using Gitlab for 4 years now. What got us initially was the free private repos before github had that. We are now a paying customer. Their integrated CICD is amazing. It works perfectly for all our needs and integrates really easily with AWS and GCP. Also their customer service is really damn good. If I ever have an issue, it’s dealt with so fast and with so much detail. Honestly one of the best customer service I’ve experienced. Their product is feature rich, priced right and is easy. I’m amazed at how the operate. Kudos to the team” Other interesting news in programming Microsoft open-sources its C++ Standard Library (STL) used by MSVC tool-chain and Visual Studio Linux 5.3 releases with support for AMD Navi GPUs, Zhaoxin x86 CPUs and power usage improvements NVIM v0.4.0 releases with new API functions, Lua library, UI events and more!

Last week, the team behind Istio, an open-source service mesh platform, announced Istio 1.3. This release makes using the service mesh platform easier for users. What’s new in Istio 1.3? Traffic management In this release, automatic determination of HTTP or TCP has been added for outbound traffic when ports are not correctly named as per Istio’s conventions. The team has added a mode to the Gateway API that is used for mutual TLS operation. Envoy proxy has been improved,  it now checks Envoy’s readiness status. The team has improved the load balancing for directing the traffic to the same region and zone by default. And the Redis load balancer has now defaulted to MAGLEV while using the Redis proxy. Improved security This release comes with trust domain validation for services that use mutual TLS. By default, the server only authenticates the requests from the same trust domain. The team has added SDS (Software Defined Security) support for delivering the private key and certificates to each of the Istio control plane services. The team implemented major security policies including RBAC, directly into Envoy.  Experimental telemetry  In this release, the team has improved the Istio proxy to emit HTTP metrics directly to Prometheus, without the need of istio-telemetry service.  Handles inbound traffic securely Istio 1.3 secures and handles all inbound traffic on any port without the need of containerPort declarations. The team has eliminated the infinite loops that are caused in the IP tables rules when workload instances send traffic to themselves. Enhanced EnvoyFilter API The team has enhanced the EnvoyFilter API so that users can fully customize HTTP/TCP listeners, their filter chains returned by LDS (Listener discovery service ), Envoy HTTP route configuration that is returned by RDS (Route Discovery Service) and much more. Improved control plane monitoring The team has enhanced control plane monitoring by adding new metrics to monitor configuration state, metrics for sidecar injector and a new Grafana dashboard for Citadel. Users all over seem to be excited about this release.  https://twitter.com/HamzaZ21823474/status/1172235176438575105 https://twitter.com/vijaykodam/status/1172237003506798594 To know more about this news, check out the release notes. Other interesting news in Cloud & networking StackRox App integrates into the Sumo Logic Dashboard  for improved Kubernetes security The Continuous Intelligence report by Sumo Logic highlights the rise of Multi-Cloud adoption and open source technologies like Kubernetes Kong announces Kuma, an open-source project to overcome the limitations of first-generation service mesh technologies        

Today, StackRox, a company providing threat protection for containers and Kubernetes, announced the availability of the StackRox App for the Sumo Logic Continuous Intelligence Platform. The StackRox App for Sumo Logic provides customers with critical insights into misconfigurations and security events for their container and Kubernetes environments directly within their Sumo Logic Dashboard. Using this app, different security teams can view StackRox data regarding vulnerabilities, misconfigurations, runtime threats, and other policy violations within Sumo Logic and streamline their remediation efforts. John Coyle, vice president of business development for Sumo Logic, said, "We're excited to launch our Kubernetes security integration with StackRox since it will enable customers to gain unparalleled insights and operational metrics in a single dashboard to ensure their cloud-native environments are continuously protected.” "The StackRox Kubernetes-native container security platform provides unique context on misconfigurations, risk profiling, and runtime incidents that will enable our joint customers to more quickly identify and address security issues," Coyle further added. The StackRox App for Sumo Logic provides several key metrics such as vulnerabilities, runtime threats, and compliance violations across container and Kubernetes environments through the following dashboards: StackRox Overview:  This offers a snapshot of key metrics about an organization’s overall Kubernetes and container security posture StackRox Image Violations: These display information from StackRox’s image scanning and vulnerability management capabilities and prioritizes security issues in container images based on rich context derived from Kubernetes StackRox Kubernetes Violations: These highlight prioritized list of misconfigurations of Kubernetes components based on more than 70 DevOps and Security best practices StackRox Runtime Violations: These provide insights into threats and other suspicious activity at runtime based on continuous monitoring of every single container within Kubernetes environments Richard Reinders, manager of security operations for Looker, a joint StackRox and Sumo Logic customer said, “StackRox gives us a Kubernetes-centric single pane of glass view into the security posture of our multi-cloud infrastructure. Having StackRox’s unique Kubernetes security insights available directly on our Sumo Logic Dashboard provides us with a single place to view security and compliance details alongside our operational analytics for our cloud-native infrastructure. This integration also allows us to use a single, consistent, security event detection and response pipeline.” To more about the StackRox App for Sumo Logic head over to its official website. Other interesting news in security CNCF-led open-source Kubernetes security audit reveals 37 flaws in Kubernetes cluster; recommendations proposed Over 47K Supermicro servers’ BMCs are prone to USBAnywhere, a remote virtual media vulnerability Espressif IoT devices susceptible to WiFi vulnerabilities can allow hijackers to crash devices connected to enterprise networks

Today, Sumo Logic revealed the fourth edition of their “Continuous Intelligence Report: The State of Modern Applications and DevSecOps in the Cloud.” The primary goal of this report is to present data-driven insights, best practices and the latest trends by analyzing technology adoption among Sumo Logic customers. The data in the report is derived from 2000+ Sumo Logic customers running applications on cloud platforms like AWS, Azure, Google Cloud Platform, as well as, on-premise environments. This year, the Continuous Intelligence report finds that, with an increase of 50% in enterprise adoption and deployments of multi-cloud, Multi-cloud is growing faster than any other modern infrastructure category. In a statement, Kalyan Ramanathan, vice president of product marketing for Sumo Logic says, “the increased adoption of services to enable and secure a multi-cloud strategy are adding more complexity and noise,  which current legacy analytics solutions can’t handle. To address this complexity, companies will need a continuous intelligence strategy that consolidates all of their data into a single pane of glass to close the intelligence gap. Sumo Logic provides this strategy as a cloud-native, continuous intelligence platform, delivered as a service.” Key findings of the Modern App Report 2019 Kubernetes highly prevalent in multi-cloud environments Kubernetes offers broad multi-cloud support and can be used by many organizations to run applications across cloud environments. The 2019 Modern App survey reveals that 1 in 5 AWS customers use Kubernetes. Image Source: The Continuous Intelligence Report The report states, “Enterprises are betting on Kubernetes to drive their multi-cloud strategies. It is imperative that enterprises deploy apps on Kubernetes to easily orchestrate/manage/scale apps and also retain the flexibility to port apps across different clouds.” Open source has disrupted the modern application stack Open source has disrupted the modern application stack with open source solutions for containers like orchestration, infrastructure and application services leading in majority. 4 out of 6 application infrastructure platforms are dominated by open source now. One of the open source solution called the orchestration technologies are used to not only automate the deployment and scaling of containers, but also to ensure reliability of applications and workloads which are running on containers. Image Source: The Continuous Intelligence Report Adoption of individual IaaS services suggests enterprises are trying to avoid vendor lock-in The Modern App 2019 survey finds that typical enterprises are only using 15 out of 150+ discrete services marketed and available for consumption in AWS. The adoption of AWS services demonstrates that basic compute, storage, database, network, and identity services are some of the top 10 adopted services in AWS. It is also found that services like management, tooling, and advanced security services are adopted at a lower rate than the core infrastructure services (50% or less). Image Source: The Continuous Intelligence Report Serverless technology mainly AWS Lambda continue to rise Serverless technologies like AWS Lambda continues to grow steeply as it is a cost-effective option to speed cloud and DevOps deployment automation. The Modern App Report 2019 reveals that AWS Lambda adoption grew to 36% in 2019, up 24% from 2017. It is also being used in several non-production use cases. AWS Lambda continues to increase their cloud migration and digital transformation efforts which makes it one of the top 10 AWS services by adoption. “Lambda usage for application or deployment automation technology should be considered for every production application,” reads the report. Image Source: The Continuous Intelligence Report The 2019 Continuous Intelligence Report is the first industry report to quantitatively define the state of the Modern Application Stack and its implication to the growing technology. Professionals like cloud architects, Site Reliability Engineers (SREs), data engineers, operations teams, DevOps and Chief Information Security Officers (CISOs) can learn how to build, run and secure modern applications and cloud infrastructures by leveraging information from this report. If you are interested to know more, you can check out the full report at the Sumo Logic blog. Other news in Cloud and Networking Containous introduces Maesh, a lightweight and simple Service Mesh to ease microservices adoption Amazon announces improved VPC networking for AWS Lambda functions Kubernetes releases etcd v3.4 with better backend storage, improved raft voting process, new raft non-voting member and more

Today, the team at Kong, the creators of the API and service lifecycle management platform for modern architectures announced the release of Kuma, a new open-source project.  Kuma is based on the open-source Envoy proxy that addresses limitations of first-generation service mesh technologies by seamlessly managing services on the network. The first-generation meshes didn't have a mature control plane, and later on, when they provided a control plane, it wasn’t easy to use them as they were hard to deploy. Kuma is easy to use and enables rapid adoption of mesh. Also Read: Kong CTO Marco Palladino on how the platform is paving the way for microservices adoption [Interview] Features of Kuma Runs on all the platforms Kuma can run on any platform including Kubernetes, containers, virtual machines, and legacy environments. It also includes a fast data plane as well as an advanced control plane that makes it easier to use.  It is reliable The initial service mesh solutions were not flexible and it was difficult to use them. Kuma ensures reliability by automating the process of securing the underlying network.  Support for all the environments Kuma has support for all the environments in the organization, so the existing applications can still be used in their traditional environments. This provides comprehensive coverage across an organization. Couples a fast data plane using control plane Kuma couples a fast data plane with a control plane that helps users to set permissions, routing rules and expose metrics with just a few commands. Tracing and logging Kuma helps users to implement tracing and logging and analyze metrics for rapid debugging. Routing and Control  Kuma provides traffic control capabilities including circuit breakers and health checks in order to enhance L4 (Layer 4) routing. Marco Palladino, CTO and co-founder of Kong, said, “We now have more microservices talking to each other and connectivity between them is the most unreliable piece: prone to failures, insecure and hard to observe.”  Palladino further added, “It was important for us to make Kuma very easy to get started with on both Kubernetes and VM environments, so developers can start using service mesh immediately even if their organization hasn’t fully moved to Kubernetes yet, providing a smooth path to containerized applications and to Kubernetes itself. We are thrilled to be open-sourcing Kuma and extending the adoption of Envoy, and we will continue to contribute back to the Envoy project like we have done in the past. Just as Kong transformed and modernized API Gateways with open-source Kong, we are now doing that for service mesh with Kuma.” The Kuma platform will be on display during the second annual Kong Summit, which is to be held on October 2-3, 2019. Other interesting news in Cloud and Networking  Kubernetes releases etcd v3.4 with better backend storage, improved raft voting process, new raft non-voting member and more VMworld 2019: VMware Tanzu on Kubernetes, new hybrid cloud offerings, collaboration with multi cloud platforms and more! The Accelerate State of DevOps 2019 Report: Key findings, scaling strategies and proposed performance & productivity models

Yesterday, Containous, a cloud-native networking company, announced Maesh, a lightweight and simple Service Mesh. Maesh is aimed at making service-to-service communications simpler for developers building modern, cloud-native applications. It is easy to use and fully featured to help developers connect, secure and monitor traffic to and from their microservices-based applications. Mesh also supports the latest Service Mesh Interface specification (SMI), a standard specification for service mesh interoperability in Kubernetes. Maesh allows developers to adopt microservices thus, improving the service mesh experience by offering an easy way to connect, secure and monitor the network traffic in any Kubernetes environment. It helps developers optimize internal traffic, visualize traffic patterns, and secure communication channels, all while improving application performance. Also Read: Red Hat announces the general availability of Red Hat OpenShift Service Mesh Maesh is designed to be completely non-invasive, allowing development teams across the organization to incrementally “opt-in” applications progressively over time. It is backed by Traefik’s rich feature-set thus, providing OpenTracing, load balancing for HTTP, gRPC, WebSocket, TCP, rich routing rules, retries and fail-overs, not to mention access controls, rate limits, and circuit breakers. Maesh can run in both TCP and HTTP mode. “In HTTP mode, Maesh leverages Traefik’s feature set to enable rich routing on virtual-host, path, headers, cookies. Using TCP mode allows seamless and easy integration with SNI routing support,” Containous team reports. It also enables critical features across any Kubernetes environment including observability, Multi-Protocol Support, Traffic Management, Security and Safety. Also Read: Mapbox introduces MARTINI, a client-side terrain mesh generation code In an email statement to us, Emile Vauge, CEO, Containous said, “With Maesh, Containous continues to innovate with the mission to drastically simplify cloud-native adoption for all enterprises. We’ve been proud of how popular Traefik has been for developers as a critical open source solution, and we’re excited to now bring them Maesh.” https://twitter.com/resouer/status/1169310994490748928 To know more about Maesh in detail, read the Containous’ Medium blog post. Other interesting news in Networking Amazon announces improved VPC networking for AWS Lambda functions Pivotal open sources kpack, a Kubernetes-native image build service Kubernetes releases etcd v3.4 with better backend storage, improved raft voting process, new raft non-voting member and more

Yesterday, the team at Amazon announced improved VPC (Virtual Private Cloud) networking for AWS Lambda functions. It is a major improvement on how AWS Lambda function will work with Amazon VPC networks.  In case a Lambda function is not configured to connect to your VPCs then the function can access anything available on the public internet including other AWS services, HTTPS endpoints for APIs, or endpoints and services outside AWS. So, the function has no way to connect to your private resources that are inside your VPC. When the Lambda function is configured to connect to your own VPC, it creates an elastic network interface within the VPC and does a cross-account attachment. Image Source: Amazon These Lambda functions run inside the Lambda service’s VPC but they can only access resources over the network with the help of your VPC. But in this case, the user still won’t have direct network access to the execution environment where the functions run. What has changed in the new model? AWS Hyperplane for providing NAT (Network Address Translation) capabilities  The team is using AWS Hyperplane, the Network Function Virtualization platform that is used for Network Load Balancer and NAT Gateway. It also has supported inter-VPC connectivity for AWS PrivateLink. With the help of Hyperplane the team will provide NAT capabilities from the Lambda VPC to customer VPCs. Network interfaces within VPC are mapped to the Hyperplane ENI The Hyperplane ENI (Elastic Network Interfaces), a network resource controlled by the Lambda service, allows multiple execution environments to securely access resources within the VPCs in your account. So, in the previous model, the network interfaces in your VPC were directly mapped to Lambda execution environments. But in this case, the network interfaces within your VPC are mapped to the Hyperplane ENI. Image Source: Amazon How is Hyperplane useful? To reduce latency When a function is invoked, the execution environment now uses the pre-created network interface and establishes a network tunnel to it which reduces the latency. To reuse network interface cross functions Each of the unique security group:subnet combination across functions in your account needs a distinct network interface. If such a combination is shared across multiple functions in your account, it is now possible to reuse the same network interface across functions. What remains unchanged? AWS Lambda functions will still need the IAM permissions for creating and deleting network interfaces in your VPC. Users can still control the subnet and security group configurations of the network interfaces.  Users still need to use a NAT device(for example VPC NAT Gateway) for giving a function internet access or for using VPC endpoints to connect to services outside of their VPC. The types of resources that your functions can access within the VPCs still remain the same. The official post reads, “These changes in how we connect with your VPCs improve the performance and scale for your Lambda functions. They enable you to harness the full power of serverless architectures.” To know more about this news, check out the official post. What’s new in cloud & networking this week? Kubernetes releases etcd v3.4 with better backend storage, improved raft voting process, new raft non-voting member and more VMworld 2019: VMware Tanzu on Kubernetes, new hybrid cloud offerings, collaboration with multi cloud platforms and more! The Accelerate State of DevOps 2019 Report: Key findings, scaling strategies and proposed performance & productivity models  

Last Friday, a team at Kubernetes announced the release of etcd 3.4 version. etcd 3.4 focuses on stability, performance and ease of operation. It includes features like pre-vote and non-voting member and improvements to storage backend and client balancer. Key features and improvements in etcd v3.4 Better backend storage etcd v3.4 includes a number of performance improvements for large scale Kubernetes workloads. In particular, etcd experienced performance issues with a large number of concurrent read transactions even when there is no write (e.g. “read-only range request ... took too long to execute”). Previously, the storage backend commit operation on pending writes, blocks incoming read transactions, even when there was no pending write. Now, the commit does not block reads which improve long-running read transaction performance. The team has further made backend read transactions fully concurrent. Previously, ongoing long-running read transactions block writes and upcoming reads. With this change, write throughput is increased by 70% and P99 write latency is reduced by 90% in the presence of long-running reads. They also ran Kubernetes 5000-node scalability test on GCE with this change and observed similar improvements. Improved raft voting process etcd server implements Raft consensus algorithm for data replication. Raft is a leader-based protocol. Data is replicated from leader to follower; a follower forwards proposals to a leader, and the leader decides what to commit or not. Leader persists and replicates an entry, once it has been agreed by the quorum of cluster. The cluster members elect a single leader, and all other members become followers. The elected leader periodically sends heartbeats to its followers to maintain its leadership, and expects responses from each follower to keep track of its progress. In its simplest form, a Raft leader steps down to a follower when it receives a message with higher terms without any further cluster-wide health checks. This behavior can affect the overall cluster availability. For instance, a flaky (or rejoining) member drops in and out, and starts campaign. This member ends up with higher terms, ignores all incoming messages with lower terms, and sends out messages with higher terms. When the leader receives this message of a higher term, it reverts back to follower. This becomes more disruptive when there’s a network partition. Whenever the partitioned node regains its connectivity, it can possibly trigger the leader re-election. To address this issue, etcd Raft introduces a new node state pre-candidate with the pre-vote feature. The pre-candidate first asks other servers whether it’s up-to-date enough to get votes. Only if it can get votes from the majority, it increments its term and starts an election. This extra phase improves the robustness of leader election in general. And helps the leader remain stable as long as it maintains its connectivity with the quorum of its peers. Introducing a new raft non-voting member, “Learner” The challenge with membership reconfiguration is that it often leads to quorum size changes, which are prone to cluster unavailabilities. Even if it does not alter the quorum, clusters with membership change are more likely to experience other underlying problems. In order to address failure modes, etcd introduced a new node state “Learner”, which joins the cluster as a non-voting member until it catches up to leader’s logs. This means the learner still receives all updates from leader, while it does not count towards the quorum, which is used by the leader to evaluate peer activeness. The learner only serves as a standby node until promoted. This relaxed requirements for quorum provides the better availability during membership reconfiguration and operational safety. Improvements to client balancer failover logic etcd is designed to tolerate various system and network faults. By design, even if one node goes down, the cluster “appears” to be working normally, by providing one logical cluster view of multiple servers. But, this does not guarantee the liveness of the client. Thus, etcd client has implemented a different set of intricate protocols to guarantee its correctness and high availability under faulty conditions. Historically, etcd client balancer heavily relied on old gRPC interface: every gRPC dependency upgrade broke client behavior. A majority of development and debugging efforts were devoted to fixing those client behavior changes. As a result, its implementation has become overly complicated with bad assumptions on server connectivity. The primary goal in this release was to simplify balancer failover logic in etcd v3.4 client; instead of maintaining a list of unhealthy endpoints, whenever client gets disconnected from the current endpoint. To know more about this release, check out the Changelog page on GitHub. What’s new in cloud and networking this week? VMworld 2019: VMware Tanzu on Kubernetes, new hybrid cloud offerings, collaboration with multi cloud platforms and more! The Accelerate State of DevOps 2019 Report: Key findings, scaling strategies and proposed performance & productivity models Pivotal open sources kpack, a Kubernetes-native image build service

VMware kicked off its VMworld 2019 US in San Francisco last week on 25th August and ended yesterday with a series of updates, spanning Kubernetes, Azure, security and more. This year’s event theme was “Make Your Mark” aimed at empowering VMworld 2019 attendees to learn, connect and innovate in the world of IT and business. 20,000 attendees from more than 100 countries descended to San Francisco for VMworld 2019. VMware CEO Pat Gelsinger took the stage, and articulated VMware’s commitment and support for TechSoup, a one-stop IT shop for global nonprofits. Gelsinger also put emphasis on the company's 'any cloud, any application, any device, with intrinsic security' strategy. “VMware is committed to providing software solutions to enable customers to build, run, manage, connect and protect any app, on any cloud and any device,” said Pat Gelsinger, chief executive officer, VMware. “We are passionate about our ability to drive positive global impact across our people, products and the planet.” Let us take a look at the key highlights of the show: VMworld 2019: CEO's take on shaping tech as a force for good The opening keynote from Pat Gelsinger had everything one would expect; customer success stories, product announcements and the need for ethical fix in tech. "As technologists, we can't afford to think of technology as someone else's problem," Gelsinger told attendees, adding “VMware puts tremendous energy into shaping tech as a force for good.” Gelsinger cited three benefits of technology which ended up opening the Pandora's Box. Free apps and services led to severely altered privacy expectations; ubiquitous online communities led to a crisis in misinformation; while the promise of blockchain has led to illicit uses of cryptocurrencies. "Bitcoin today is not okay, but the underlying technology is extremely powerful," said Gelsinger, who has previously gone on record regarding the detrimental environmental impact of crypto. This prism of engineering for good, alongside good engineering, can be seen in how emerging technologies are being utilised. With edge, AI and 5G, and cloud as the "foundation... we're about to redefine the application experience," as the VMware CEO put it. Read also: VMware reaches the goal of using 100% renewable energy in its operations, a year ahead of their 2020 vision Gelsinger’s 2018 keynote was about the theme of tech 'superpowers'. Cloud, mobile, AI, and edge. This time, more focus was given to how the edge was developing. Whether it was a thin edge, containing a few devices and an SD-WAN connection, a thick edge of a remote data centre with NFV, or something in between, VMware aims to have it all covered. "Telcos will play a bigger role in the cloud universe than ever before," said Gelsinger, referring to the rise of 5G. "The shift from hardware to software [in telco] is a great opportunity for US industry to step in and play a great role in the development of 5G." VMworld 2019 introduces Tanzu to build, run and manage software on Kubernetes VMware is moving away from virtual machines to containerized applications. On the product side VMware Tanzu was introduced, a new product portfolio that aims to enable enterprise-class building, running, and management of software on Kubernetes. In Swahili, ’tanzu’ means the growing branch of a tree and in Japanese, ’tansu’ refers to a modular form of cabinetry. For VMware, Tanzu is their growing portfolio of solutions that help build, run and manage modern apps. Included in this is Project Pacific, which is a tech preview focused on transforming VMware vSphere into a Kubernetes native platform. "With project Pacific, we're bringing the largest infrastructure community, the largest set of operators, the largest set of customers directly to the Kubernetes. We will be the leading enabler of Kubernetes," Gelsinger said. Read also: VMware Essential PKS: Use upstream Kubernetes to build a flexible, cost-effective cloud-native platform Other product launches included an update to collaboration program Workspace ONE, including an AI-powered virtual assistant, as well as the launch of CloudHealth Hybrid by VMware. The latter, built on cloud cost management tool CloudHealth, aims to help organisations save costs across an entire multi-cloud landscape and will be available by the end of Q3. Collaboration, not compete with major cloud providers - Google Cloud, AWS & Microsoft Azure At VMworld 2019 VMware announced an extended partnership with Google Cloud earlier this month led the industry to consider the company's positioning amid the hyperscalers. VMware Cloud on AWS continues to gain traction - Gelsinger said Outposts, the hybrid tool announced at re:Invent last year, is being delivered upon - and the company also has partnerships in place with IBM and Alibaba Cloud. Further, VMware in Microsoft Azure is now generally available, with the facility to gradually switch across Azure data centres. By the first quarter of 2020, the plan is to make it available across nine global areas. Read also: Cloud Next 2019 Tokyo: Google announces new security capabilities for enterprise users The company's decision not to compete, but collaborate with the biggest public clouds has paid off. Gelsinger also admitted that the company may have contributed to some confusion over what hybrid cloud and multi-cloud truly meant. But the explanation from Gelsinger was pretty interesting. Increasingly, with organisations opting for different clouds for different workloads, and changing environments, Gelsinger described a frequent customer pain point for those nearer the start of their journeys. Do they migrate their applications or do they modernise? Increasingly, customers want both - the hybrid option. "We believe we have a unique opportunity for both of these," he said. "Moving to the hybrid cloud enables live migration, no downtime, no refactoring... this is the path to deliver cloud migration and cloud modernisation." As far as multi-cloud was concerned, Gelsinger argued: "We believe technologists who master the multi-cloud generation will own it for the next decade." Collaboration with NVIDIA to accelerate GPU services on AWS NVIDIA and VMware today announced their intent to deliver accelerated GPU services for VMware Cloud on AWS to power modern enterprise applications, including AI, machine learning and data analytics workflows. These services will enable customers to seamlessly migrate VMware vSphere-based applications and containers to the cloud, unchanged, where they can be modernized to take advantage of high-performance computing, machine learning, data analytics and video processing applications. Through this partnership, VMware Cloud on AWS customers will gain access to a new, highly scalable and secure cloud service consisting of Amazon EC2 bare metal instances to be accelerated by NVIDIA T4 GPUs, and new NVIDIA Virtual Compute Server (vComputeServer) software. “From operational intelligence to artificial intelligence, businesses rely on GPU-accelerated computing to make fast, accurate predictions that directly impact their bottom line,” said Jensen Huang, founder and CEO, NVIDIA. “Together with VMware, we’re designing the most advanced GPU infrastructure to foster innovation across the enterprise, from virtualization, to hybrid cloud, to VMware's new Bitfusion data center disaggregation.” Read also: NVIDIA’s latest breakthroughs in conversational AI: Trains BERT in under an hour, launches Project Megatron to train transformer based models at scale Apart from this, Gelsinger made special note to mention VMware's most recent acquisitions, with Pivotal and Carbon Black and discussed about where they fit in the VMware stack at the back. VMware’s hybrid cloud platform for Next-gen Hybrid IT VMware introduced new and expanded cloud offerings to help customers meet the unique needs of traditional and modern applications. VMware empowers IT operators, developers, desktop administrators, and security professionals with the company’s hybrid cloud platform to build, run, and manage workloads on a consistent infrastructure across their data center, public cloud, or edge infrastructure of choice. VMware uniquely enables a consistent hybrid cloud platform spanning all major public clouds – AWS, Azure, Google Cloud, IBM Cloud – and more than 60 VMware Cloud Verified partners worldwide. More than 70 million workloads run on VMware. Of these, 10 million are in the cloud. These are running in more than 10,000 data centers run by VMware Cloud providers. Take a look at the full list of VMworld 2019 announcements here. What’s new in cloud and virtualization this week? VMware signs definitive agreement to acquire Pivotal Software and Carbon Black Pivotal open sources kpack, a Kubernetes-native image build service Oracle directors support billion dollar lawsuit against Larry Ellison and Safra Catz for NetSuite deal

On Tuesday, Reuters reported that Oracle directors gave a go ahead for a million dollar lawsuit filed against Larry Ellison and Safra Catz in a NetSuite deal in 2016. This was made possible by several board members who wrote an extraordinary letter to the Delaware Court. According to Reuters, in 2017, shareholders led by the Firemen’s Retirement System of St. Louis alleged that Oracle directors breached their duties when they approved a $9.3 billion acquisition of NetSuite – a company controlled by Oracle chair Larry Ellison – at a huge premium above NetSuite’s trading price. Shareholders alleged that Oracle directors sanctioned Ellison’s self-dealing - and also claimed that Oracle’s board members were too entwined with Ellison to be entrusted with the decision of whether the company should sue him and other directors over the NetSuite deal. In an opinion published in Reuters in May 2018, Vice-Chancellor Sam Glasscock of Delaware Chancery Court agreed that shareholders had shown it would have been futile for them to demand action from the board itself. Three years after closing a $9.3 billion deal to acquire NetSuite, three board members, including former U.S. Defense Secretary Leon Panetta, sent a letter on August 15th to Sam Glasscock III, Vice Chancellor for the Court of Chancery in Georgetown, Delaware, approving the lawsuit as members of a special board of directors entity known as the Special Litigation Committee. This lawsuit in legal parlance is known as a derivative suit. According to Justia, this type of suit is filed in cases like this. “Since shareholders are generally allowed to file a lawsuit in the event that a corporation has refused to file one on its own behalf, many derivative suits are brought against a particular officer or director of the corporation for breach of contract or breach of fiduciary duty,” the Justia site explained. The letter went on to say there was an attempt to settle this suit, which was originally launched in 2017, through negotiation outside of court, but when that attempt failed, the directors wrote this letter to the court stating that the suit should be allowed to proceed. As per the letter, the lawsuit, which was originally filed by the Firemen’s Retirement System of St. Louis, could be worth billions. It reads, “One of the lead lawyers for the Firemen’s fund, Joel Friedlander of Friedlander & Gorris, said at a hearing in June that shareholders believe the breach-of-duty claims against Oracle and NetSuite executives are worth billions of dollars. So in last week’s letter, Oracle’s board effectively unleashed plaintiffs’ lawyers to seek ten-figure damages against its own members.” Oracle directors struggled with its cloud footing and ended up buying NetSuite TechCrunch noted that Larry Ellison was involved in setting up NetSuite in the late 1990s and was a major shareholder of NetSuite at the time of the acquisition. Oracle directors were struggling to find its cloud footing in 2016, and it was believed that by buying an established SaaS player, like NetSuite, it could begin to build out its cloud business much faster than trying to develop something like it internally. On Hacker News, a few users commented saying Oracle directors overpaid NetSuite and enriched Larry Ellison. One comment reads, “As you know people, as you learn about things, you realize that these generalizations we have are, virtually to a generalization, false. Well, except for this one, as it turns out. What you think of Oracle, is even truer than you think it is. There has been no entity in human history with less complexity or nuance to it than Oracle. And I gotta say, as someone who has seen that complexity for my entire life, it's very hard to get used to that idea. It's like, 'surely this is more complicated!' but it's like: Wow, this is really simple! This company is very straightforward, in its defense. This company is about one man, his alter-ego, and what he wants to inflict upon humanity -- that's it! ...Ship mediocrity, inflict misery, lie our asses off, screw our customers, and make a whole shitload of money. Yeah... you talk to Oracle, it's like, 'no, we don't fucking make dreams happen -- we make money!' ...You need to think of Larry Ellison the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower hates me' -- lawnmower doesn't give a shit about you, lawnmower can't hate you. Don't anthropomorphize the lawnmower. Don't fall into that trap about Oracle.” Oracle does “organizational restructuring” by laying off 100s of employees IBM, Oracle under the scanner again for questionable hiring and firing policies The tug of war between Google and Oracle over API copyright issue has the future of software development in the crossfires

Yesterday, VMware announced in a press release that they entered a conclusive agreement to acquire Carbon Black, a cloud-native endpoint security software developer. According to the agreement, “VMware will acquire Carbon Black in an all cash transaction for $26 per share, representing an enterprise value of $2.1 billion.”  VMware intends to use Carbon Black’s big data and behavioral analytics to offer customers advanced threat detection and behavioral insight to defend against experienced attacks. Consequently, they aspire to protect clients through big data, behavioral analytics, and AI. Pat Gelsinger, the CEO of VMware says, “By bringing Carbon Black into the VMware family, we are now taking a huge step forward in security and delivering an enterprise-grade platform to administer and protect workloads, applications, and networks.” He adds, “With this acquisition, we will also take a significant leadership position in security for the new age of modern applications delivered from any cloud to any device.” Yesterday, after much speculation, VMware also announced that they have acquired Pivotal Software, a cloud-native platform provider, for an enterprise value of $2.7 billion. Dell technologies is a major stakeholder in both companies. Lately, VMware has been heavily investing in Kubernetes. Last year, it also launched a VMware Kubernetes Engine (VKE) to offer Kubernetes-as-a-Service. This year, Pivotal also teamed up with the Heroku team to create Cloud Native Buildpacks for Kubernetes and recently, also launched a Pivotal Spring Runtime for Kubernetes. With Pivotal, VMware plans to “deliver a comprehensive portfolio of products, tools and services necessary to build, run and manage modern applications on Kubernetes infrastructure with velocity and efficiency.” Read More: VMware’s plan to acquire Pivotal Software reflects a rise in Pivotal’s shares Gelsinger told ZDNet that both these “acquisitions address two critical technology priorities of all businesses today — building modern, enterprise-grade applications and protecting enterprise workloads and clients.” Gelsinger also pointed out that multi-cloud, digital transformation, and the increasing trend of moving “applications to the cloud and access it over distributed networks and from a diversity of endpoints” are significant reasons for placing high stakes on security. It is clear that by acquiring Carbon Black and Pivotal Software, the cloud computing and virtualization software company is seeking to expand its range of products and services with an ultimate focus on security in Kubernetes. A user on Hacker News comments, “I'm not surprised at the Pivotal acquisition. VMware is determined to succeed at Kubernetes. There is already a lot of integration with Pivotal's Kubernetes distribution both at a technical as well as a business level.” Also, developers around the world are excited to see what the future holds for VMware, Carbon Black, and Pivotal Software. https://twitter.com/rkagal1/status/1164852719594680321 https://twitter.com/CyberFavourite/status/1164656928913596417 https://twitter.com/arashg_/status/1164785525120618498 https://twitter.com/jambay/status/1164683358128857088 https://twitter.com/AnnoyedMerican/status/1164646153389875200 Per the press release, both the transaction payments are expected to be concluded in the second half of VMware’s fiscal year i.e., January 31, 2020. Interested users can read the VMware acquiring Carbon Black and Pivotal Software press releases for more information. VMware reaches the goal of using 100% renewable energy in its operations, a year ahead of their 2020 vision VMware Essential PKS: Use upstream Kubernetes to build a flexible, cost-effective cloud-native platform VMware Kubernetes Engine (VKE) launched to offer Kubernetes-as-a-Service

In April, Pivotal and Heroku teamed up to create Cloud Native Buildpacks for Kubernetes. Cloud-Native Buildpacks turn source code into production-ready Docker images that are OCI image compatible and is based around the popular Buildpack model. Yesterday, they open-sourced kpack, which is a set of experimental build service Kubernetes resource controllers. Basically, kpack is Kubernetes’ native way to build and update containers. It automates the creation and update of container images that can be run anywhere. Pivotal’s commercial implementation of kpack comes via Pivotal Build Service. Users can use it atop Kubernetes to boost developer productivity. The Build Service integrates kpack with buildpacks and the Kubernetes permissions model. kpack presents a CRD as its interface, and users can interact with all Kubernetes API tooling including kubectl. Pivotal has open-sourced kpack for two reasons, as mentioned in their blog post. “First, to provide Build Service’s container building functionality and declarative logic as a consumable component that can be used by the community in other great products. Second, to provide a first-class interface, to create and modify image resources for those who desire more granular control.” Many companies and communities have announced that they will be using Kpack in their projects. Project riff will use kpack to build functions to handle events. The Cloud Foundry community plans to feature kpack as the new app staging mechanism in the Cloud Foundry Application Runtime. Check out the kpack repo for more details. You can also request alpha access to Build Service. In other news, Pivotal and VMware, the former’s parent company are negotiating a deal for VMware to acquire Pivotal as per a recent regulatory filing from Dell. VMware, Pivotal, and Dell have jointly filed the document informing the government regulators about the potential transaction. Pivotal and Heroku team up to create Cloud Native Buildpacks for Kubernetes VMware’s plan to acquire Pivotal Software reflects a rise in Pivotal’s shares Introducing ‘Pivotal Function Service’ (alpha): an open, Kubernetes based, multi-cloud serverless framework for developer workloads.

Yesterday, Puppet announced a vulnerability remediation solution called Puppet Remediate which aims to reduce the time taken by IT teams to identify, prioritize and rectify mission-critical vulnerabilities. Matt Waxman, head of product at Puppet said, “There is a major gap between sophisticated scanning tools that identify vulnerabilities and the fragmented and manual, error-prone approach of fixing these vulnerabilities.” He adds, “Puppet Remediate closes this gap giving IT the insight they need to end the current soul-crushing work associated with vulnerability remediation to ensure they are keeping their organization safe.” Puppet Remediate will produce faster remedial solution by taking support from security partners who have access to potentially sensitive vulnerability data. It will discover vulnerabilities depending on the type of infrastructure resources affected by them. Next, Puppet Remediate will render instant action “to remediate vulnerable packages without requiring any agent technology on the vulnerable systems on both Linux and Windows through SSH and WinRM”, says Puppet. Key features in Puppet Remediate Shared vulnerability data between security and IT Ops Puppet Remediate unifies infrastructure data and vulnerability data, to help IT Ops get access to vulnerability data in real-time, thus reducing delays and eliminating risks associated to manual handover of data. Risk-based prioritization It will assist IT teams to prioritize critical systems and identify vulnerabilities within the organization's systems based on infrastructure context. It will give IT teams more clarity on what to fix first. Agentless remediation IT teams will be able to take immediate action to rectify a vulnerability without requiring to leave the application or without the need of requiring any agent technology on the vulnerable systems. Channel partners will provide Puppet an established infrastructure and InfoSec practices Puppet have selected initial channel partners depending on their established infrastructure and InfoSec practices. The channel partners will help Puppet Remediate to bridge the gap between security and IT practices in enterprises. Fishtech, a cybersecurity solutions provider and Bitbone, a Germany based computer software store are the initial channel partners for Puppet Remediate. Sebastian Scheuring, CEO of Bitbone AG says, “Puppet Remediate offers real added value with its new functions to our customers. It drastically automates the workflow of vulnerability remediation through taking out the manual, mundane and error-prone steps that are required to remediate vulnerabilities. Continuous scans, remediation tasks and short cycles of update processes significantly increase the security level of IT environments.” Check out the website to know more about Puppet Remediate. Listen: Puppet’s VP of Ecosystem Engineering Nigel Kersten talks about key DevOps challenges [Podcast] Puppet announces updates in a bid to help organizations manage their “automation footprint” “This is John. He literally wrote the book on Puppet” – An Interview with John Arundel