Tech Guides - High Performance

Whenever I think of performance engineering, I am reminded of Amazon’s CEO Jeff Bezos’ statement, “Focusing on the customer makes a company more resilient.” Any company which follows this consumer-focused approach has a performance engineering domain in it, though in varying capacity and form. The connection is simple. More and more businesses are becoming web-based, so they are interacting with their customers digitally. In such a scenario, if they have to provide exceptional customer experience, they have to build resilient, stable, user-centric and high performing web-systems and applications. And to do that, they need performance engineering. What is Performance Engineering? Let me explain performance engineering with an example. Suppose, your team is building an online shopping portal. The developers will build a system that allows people to access products and buy them. They will ensure that the entire transaction is smooth, uncomplicated for the user and can be done quickly. Now imagine that to promote the portal, you do a flash sale, and 1000 users come on the platform and start doing transactions simultaneously. And your system, under this load, starts performing slower, a lot of transactions fail and your users are dejected. This will directly affect your brand image, customer loyalty, and revenue. How about we fix this before such a situation occurs? That is exactly what performance engineering entails. A performance engineer would essentially take into account such scenarios and conduct load tests and check the system’s performance in the development phase itself. Load tests check the behavior of your system in particular situations. A ‘load’ is a possible scenario that can affect the system, for instance, sale offers or peak times. If the system is able to handle the load, it will check if it is scalable. If the system is unable to handle it, they will analyze the result, find the possible bottleneck by checking the code and try to rectify it. So, for the above example, a performance engineer would have tested the system for 100 transactions at a time, then 500, and then 1000 and would have even gone up to one hundred thousand. Hence, performance engineering ensures crash-free operation of a system, software or application. Using processes and systematic techniques, practices, and activities, a performance engineer ensures that the performance requirements are met during the development cycle. However, this is not a blanket role. It would vary with your field of operation. The work of a performance engineer working on a web application will be a lot different than that of a database performance engineer or that of a streaming performance engineer. For each of these, your “load” would vary but your goal is the same, ensuring that your system is resilient enough to shoulder that load. Before I dive deeper into the role of a performance engineer, I’d like to clarify the difference between a performance tester and a performance engineer. (Yes, they are not the same!) Performance Tester versus Performance Engineer Well, many people think that 2-3 years of experience as a performance tester can easily land you a performance engineering job. Well, no. It is a long journey, which requires much more knowledge than what a tester has. A performance tester would have testing knowledge and would know about performance analysis and performance monitoring concepts across different applications. They would essentially conduct a “load test” to check the performance, stability, and scalability of a system, and produce reports to share with the developer to work on. Their work ends here. But this is not the case for a performance engineer. A performance engineer will look for the root cause of the performance issue, work towards finding a possible solution for it and then tune and optimize the system to sort the said issue until the performance parameters are met. Simply put, performance testing can be considered as a part of performance engineering but not as the same thing. Roles and Responsibilities of a Performance Engineer Designing Effective Tests As a performance engineer, your first task is to design an effective test to check the system. I found this checklist on Dzone that is really helpful for designing tests: Identify your goals, requirements, desires, workload model and your stakeholders. Understand how to test concurrency, arrival rates, and scheduling. Understand the roles of scalability, capacity, and reliability as quality attributes and requirements. Understand how to setup/create test data and data management. Scripting, Running Tests and Interpreting Results There are several performance testing tools available in the market. But you would have to work with different languages based on the tool you use. For instance, you’d have to build your testing in C and Javascript while working with Microfocus Loadrunner. Similarly, you’d script in Java and Javascript for Apache JMeter. Once your test is ready, you’d run that test on your system. Make sure you use consistent metrics while running these tests or else your results would be inaccurate. Finally, you will interpret those results. In this, you’d have to figure out what the bottlenecks are and where they are occurring. For that, you would have to read results and analyze graphs that your performance testing tool has produced and draw conclusions. Fine Tuning And Performance Optimisation Once you know what the bottleneck is and where it is occurring, you would have to find a solution to overcome it to enhance the performance of the system you are testing. (Something a performance tester won’t do!) Your task is to ensure that the system/application is optimized to the level where it works optimally at the maximum load possible. Of course, you can seek aid from a developer (backend, frontend or full-stack) working on the project to figure this out. But as a performance engineer, you’d have to be involved actively in this fine-tuning and optimization process. There are four major skills/attributes that differentiate an exceptional performance engineer from an average one. Proves that their load results are scalable If you are a good performance engineer, you will not serve a half-cooked meal. First of all, take all possibilities into account. For instance, take the example of the same online shopping portal. If you are considering a load test for 1000 simultaneous transactions, consider it for both scenarios wherein the transactions are happening for different products or when it is happening for the same product. If your portal does a launch sale for an exclusive product that is available for a limited period, you may have too many people trying to buy it at the same time. Ask yourself if your system could withstand that load? Proves that their load results are sustainable Not just this, you should also consider whether your results are sustainable over a defined period of time. The system should operate without crashing. It is often recommended that a load test runs for 30 mins. While thirty minutes will be enough to detect most new performance changes as they are introduced, in order to make these tests legitimate, it is necessary to prove they can run for at least two hours at the same load. These time durations may vary for different programs/systems/applications. Uses Benchmarks A benchmark essentially is a point of reference based on which you can compare and assess the performance of your system. It is a set standard against which you can check the quality of your product/application/system. For some systems, like databases, standard benchmarks are readily available for you to test on. As a performance engineer, you must be aware of the performance benchmarks in your field/domain. For example, you’d find benchmarks for testing firewalls, databases, and end-to-end IT systems. The most commonly used benchmarking frameworks are Benchmark Framework 2.0 & TechEmpower. Understands User Behavior If you don’t have an understanding of user reactions in different situations, you cannot design an effective load test. A good performance engineer knows their user demographics, understands their key behavior and knows how the user would interact with the system. While it is impossible to predict user behavior entirely, for instance, a sale may result in 100,000 transactions per hour to barely 100 per hour, you should check user statistics, analyze user activity and conduct and prepare your system for optimum usage. All in all, besides strong technical skills, as a performance engineer, you must always be far-sighted. You must be able to see beyond what meets the eye and catch what others might miss. The role, invariably, requires a lot of technical expertise. But it also requires non-technical skills like problem-solving, attention-to-detail and insightfulness. About the Author Dr Sandeep Deshmukh is the founder and CEO at Workship. He holds a  PhD from IIT Bombay, and has worked in Big Data, Hadoop ecosystem, Distributed Systems, AI/ML, etc for 12+ yrs. He has been an Engineering Manager at DataTorrent and Data Scientist with Reliance Industries.

Salesforce has always been proactive in developing and bringing to market new features and functionality in all of its products. Throughout the lifetime of the Salesforce CRM product, there have been several upgrades to the user interface. In 2015, Salesforce began promoting its new platform – Salesforce Lightning. Although long time users and Salesforce developers may have grown accustomed to the classic user interface, Salesforce Lightning may just covert them. It brings in a modern UI with new features, increased productivity, faster deployments, and a seamless transition across desktop and mobile environments. Recently, Salesforce has been actively encouraging its developers, admins and users to migrate from the classic Salesforce user interface to the new Lightning Experience. Andrew Fawcett, currently VP Product Management and a Salesforce Certified Platform Developer II at Salesforce, writes in his book, Salesforce Lightning Enterprise Architecture, “One of the great things about developing applications on the Salesforce Lightning Platform is the support you get from the platform beyond the core engineering phase of the production process.” This book is a comprehensive guide filled with best practices and tailor-made examples developed in the Salesforce Lightning. It is a must-read for all Lightning Platform architects! Why should you consider migrating to Salesforce Lightning Earlier this year, Forrester Consulting published a study quantifying the total economic impact and benefits of Salesforce Lightning for Service Cloud. In the study, Forrester found that a composite service organization deploying Lightning Experience obtained a return on investment (ROI) of 475% over 3 years. Among the other potential benefits, Forrester found that over 3 years organizations using Lighting platform: Saved more than $2.5 million by reducing support handling time; Saved $1.1 million by avoiding documentation time; and Increased customer satisfaction by 8.5% Apart from this, the Salesforce Lightning platform allows organizations to leverage the latest cloud-based features. It includes responsive and visually attractive user interfaces which is not available within the Classic themes. Salesforce Lightning provides stupendous business process improvements and new technological advances over Classic for Salesforce developers. How does the Salesforce Lightning architecture look like While using the Salesforce Lightning platform, developers and users interact with a user interface backed by a robust application layer. This layer runs on the Lightning Component Framework which comprises of services like the navigation, Lightning Data Service, and Lightning Design System. Source: Salesforce website As part of this application layer, Base Components and Standard Components are the building blocks that enable Salesforce developers to configure their user interfaces via the App Builder and Community Builder. Standard Components are typically built up from one or more Base Components, which are also known as Lightning Components. Developers can build Lightning Components using two programming models: the Lightning Web Components model, and the Aura Components model. The Lightning platform is critical for a range of services and experiences in Salesforce: Navigation Service: The navigation service is supported for Lightning Experience and the Salesforce app. It is built with extensive routing, deep linking, and login redirection, Salesforce's navigation service powers app navigation, state changes, and refreshes. Lightning Data Service: Lightning Data Service is built on top of the User Interface API, It enables developers to load, create, edit, or delete a record in your component without requiring Apex code. Lightning Data Service improves performance and data consistency across components. Lightning Design System: With Lightning Design System, developers can build user interfaces easily including the component blueprints, markup, CSS, icons, and fonts. Base Lightning Components: Base Lightning Components are the building blocks for all UI across the platform. Components range from a simple button to a highly functional data table and can be written as an Aura component or a Lightning web component. Standard Components: Lightning pages are made up of Standard Components, which in turn are composed of Base Lightning Components. Salesforce developers or admins can drag-and-drop Standard Components in tools like Lightning App Builder and Community Builder. Lightning App Builder: Lightning App Builder will let developers build and customize interfaces for Lightning Experience, the Salesforce App, Outlook Integration, and Gmail Integration. Community Builder: For Communities, developers can use the Community Builder to build and customize communities easily. Apart from the above there are other services available within the Salesforce Lightning platform, like the Lightning security measures and record detail pages on the platform and Salesforce app. How to plan transitioning from Classic to Lightning Experience As Salesforce admins/developers prepare for the transition to Lightning Experience, they will need to evaluate three things: how does the change benefit the company, what work is needed to prepare for the change, and how much will it cost. This is the stage to make the case for moving to Lightning Experience by calculating the return on investment of the company and defining what a Lightning Experience implementation will look like. First they will need to analyze how prepared the organization is for the transition to Lightning Experience. Salesforce admins/developers can use the Lightning Experience Readiness Check, it is a tool that produces a personalized Readiness Report and shows which users will benefit right away, and how to adjust the implementation for Lightning Experience. Further Salesforce developers/admins can make the case to their leadership team by showing how migrating to Lightning Experience can realize business goals and improve the company's bottom line. Finally, by using the results of the activities carried out to assess the impact of the migration, understand the level of change required and decide on a suitable approach. If the changes required are relatively small, consider migrating all users and all areas of functionality at the same time. However, if the Salesforce environment is more complex and the amount of change is far greater, consider implementing the migration in phases or as an initial pilot to start with. Overall, the Salesforce Lightning Platform is being increasingly adopted by admins, business analysts, consultants, architects, and especially Salesforce developers. If you want to deliver packaged applications using Salesforce Lightning that cater to enterprise business needs, read this book, Salesforce Lightning Platform Enterprise Architecture, written by Andrew Fawcatt.  This book will take you through the architecture of building an application on the Lightning platform and help you understand its features and best practices. It will also help you ensure that the app keeps up with the increasing customers’ and business requirements. What are the challenges of adopting AI-powered tools in Sales? How Salesforce can help Salesforce open sources ‘Lightning Web Components framework’ “Facebook is the new Cigarettes”, says Marc Benioff, Salesforce Co-CEO Build a custom Admin Home page in Salesforce CRM Lightning Experience How to create and prepare your first dataset in Salesforce Einstein  

Quantum computing uses quantum mechanics in quantum computers to solve a diverse set of complex problems. It uses qubits to store information in parallel dimensions. Quantum computers can work through a solution involving large parameters with far fewer operations than a standard computer. What is so special about Quantum Computing? As they have potential to work through and solve complex problems of tomorrow, research and work on this area is attracting funding from everywhere. But these computers need a lot of physical space right now, kind of like the very first computers in the twentieth century. Quantum computers also pose a security threat since they are good at calculating large items/numbers. Quantum encryption anyone? Quantum computing is even available on the Cloud from different companies. There is even a dedicated language called Q# by Microsoft. Using concepts like entanglement to speed up computation, quantum computing can solve complex problems and is a tricky one, but I call it a treat. What about the security threat? Well, Dr. Alan Turing built a better computer to decrypt messages from another machine, we’ll let you think now.

Einstein described it as “Spooky action at a distance”. Quantum entanglement is a phenomenon observed in photons where particles share information of their state - even if separated by a huge distance. This state sharing phenomenon happens almost instantaneously. Quantum particles can be in any possible state until their state is measured by an observer. These states are called Eigen-Values. In case of quantum entanglement, two particles separated by several miles of distance, when observed, change into the same state. Quantum entanglement is hugely important for modern day computation tasks. The reason is that the state information between photons travel sometimes at speeds like 10k times the speed of light. This if implemented in physical systems, like quantum computers, can be a huge boost. Source: picoquant One important concept for us to understand this idea is ‘Qubit’. What is a Qubit? It’s the unit of information in Quantum computing. Like ‘Bit’ in case of normal computers. A bit can be represented by two states - ‘0’ or ‘1’. Qbits are also like ‘bits’, but they are governed by the weirder rules of Quantum Computing. Qubits don’t just contain pure states like ‘0’ and ‘1’, but they can also exist as superposition of these two states like {|0>,|1>},{ |1>,|0>}, {|0>,|0>}, {|1>,|1>}. This particular style of writing particle states is called the Dirac Notation. Because of these unique superposition of states, the quantum particles get entangled and share their state related information. A recent research experiment by a Chinese group has claimed to have packed 18 Qubits of information in just 6 entangled photons. This is revolutionary. What this basically means is that if one bit can pack in three times the information that it can carry presently, then our computers would become three times faster and smoother to work with. The reasons which make this a great start for future implementation of faster and practical quantum computers are: It’s very difficult to entangle so many electrons There are instances of more than 18 qubits getting packed into a larger number of photons, however the degree of entanglement has been much simpler Entanglement of each new particle takes increasingly more computer simulation time Introducing each new qubit creates a separate simulation taking up more processing time. The possible reason why this experiment has worked might be credited to the multiple degrees of freedom that photons can have. This particular experiment has been performed using Photons in a networking system. The fact that such a system allows multiple degrees of freedom for the Photon meant that this result is specific to this particular quantum system. It would be difficult to replicate the results in other systems like a Superconducting Network. Still this result means a great deal for the progress of quantum computing systems and how they can evolve to be a practical solution and not just remain in theory forever. Quantum Computing is poised to take a quantum leap with industries and governments on. PyCon US 2018 Highlights: Quantum computing, blockchains and serverless rule! Q# 101: Getting to know the basics of Microsoft’s new quantum computing language  

A thread can be defined as an ordered stream of instructions that can be scheduled to run as such by operating systems. These threads, typically, live within processes, and consist of a program counter, a stack, and a set of registers as well as an identifier. These threads are the smallest unit of execution to which a processor can allocate time. Threads are able to interact with shared resources, and communication is possible between multiple threads. They are also able to share memory, and read and write different memory addresses, but therein lies an issue. When two threads start sharing memory, and you have no way to guarantee the order of a thread's execution, you could start seeing issues or minor bugs that give you the wrong values or crash your system altogether. These issues are, primarily, caused by race conditions, an important topic for another post. The following figure shows how multiple threads can exist on multiple different CPUs: Types of threads Within a typical operating system, we, typically, have two distinct types of threads: User-level threads: Threads that we can actively create, run, and kill for all of our various tasks Kernel-level threads: Very low-level threads acting on behalf of the operating system Python works at the user-level, and thus, everything we cover here will be, primarily, focused on these user-level threads. What is multithreading? When people talk about multithreaded processors, they are typically referring to a processor that can run multiple threads simultaneously, which they are able to do by utilizing a single core that is able to very quickly switch context between multiple threads. This switching context takes place in such a small amount of time that we could be forgiven for thinking that multiple threads are running in parallel when, in fact, they are not. When trying to understand multithreading, it's best if you think of a multithreaded program as an office. In a single-threaded program, there would only be one person working in this office at all times, handling all of the work in a sequential manner. This would become an issue if we consider what happens when this solitary worker becomes bogged down with administrative paperwork, and is unable to move on to different work. They would be unable to cope, and wouldn't be able to deal with new incoming sales, thus costing our metaphorical business money. With multithreading, our single solitary worker becomes an excellent multi-tasker, and is able to work on multiple things at different times. They can make progress on some paperwork, and then switch context to a new task when something starts preventing them from doing further work on said paperwork. By being able to switch context when something is blocking them, they are able to do far more work in a shorter period of time, and thus make our business more money. In this example, it's important to note that we are still limited to only one worker or processing core. If we wanted to try and improve the amount of work that the business could do and complete work in parallel, then we would have to employ other workers or processes as we would call them in Python. Let's see a few advantages of threading: Multiple threads are excellent for speeding up blocking I/O bound programs They are lightweight in terms of memory footprint when compared to processes Threads share resources, and thus communication between them is easier There are some disadvantages too, which are as follows: CPython threads are hamstrung by the limitations of the global interpreter lock (GIL), about which we'll go into more depth in the next chapter. While communication between threads may be easier, you must be very careful not to implement code that is subject to race conditions It's computationally expensive to switch context between multiple threads. By adding multiple threads, you could see a degradation in your program's overall performance. This is an excerpt from the book, Learning Concurrency in Python by Elliot Forbes. To know how to deal with issues such as deadlocks and race conditions that go hand in hand with concurrent programming be sure to check out the book.     

Declarative programming and imperative programming are two different approaches that offer a different way of working on a given project or application. But what is the difference between declarative and imperative programming? And when should you use one over the other? What is declarative programming? Let us first start with declarative programming. This is the form or style of programming where we are most concerned with what we want as the answer, or what would be returned. Here, we as developers are not concerned with how we get there, simply concerned with the answer that is received. What is imperative programming? Next, let's take a look at imperative programming. This is the form and style of programming in which we care about how we get to an answer, step by step. We want the same result ultimately, but we are telling the complier to do things a certain way in order to achieve that correct answer we are looking for. An analogy If you are still confused, hopefully this analogy will clear things up for you. The analogy is comparing wanting to learn a topic or outsourcing work to have someone else do it. If you are outsourcing work, you might not care about how the work is completed; rather you care just what the final product or result of the work looks like. This is likened to declarative programming. You know exactly what you want and you program a function to give you just that. Let us say you did not want to outsource work for your business or project. Instead, you tell yourself that you want to learn this skill for the long term. Now, when attempting to complete the task, you care about how it is actually done. You need to know the individual steps along the way in order to get it working properly. This is similar to imperative programming. Why you should use declarative programming Reusability Since the way the result is achieved does not necessarily matter here, it allows for the functions you build to be more general and could potentially be used for multiple purposes and not just one. Not rewriting code can speed up the program you are currently writing and any others that use the same functionality in the future. Reducing Errors Given that in declarative programming you tend to write functions that do not change state as you would in functional programming, the chances of errors arising are smaller and it allows for your application to become more stable. The removal of side effects from your functions allows you to know exactly what comes in and what comes out, allows for a more predictable program. Potential drawbacks of declarative programming Lack of Control In declarative programming, you may use functions that someone else created, in order to achieve the desired results. But you may need specific things to be completed behind the scenes to make your result come out properly. You do not have this control in declarative programming as you would in imperative programming. Inefficiency When the implementation is controlled by something else, you may have problems making your code efficient. In applications where there may be a time constraint, you will need to program the individual steps in order to make sure your program is running as efficient as possible. There are benefits and disadvantages to both forms. Overall, it is entirely up to you, the programmer, to decide which implementation you would like to follow in your code. If you are solely focused on the data, perhaps consider using the declarative programming style. If you care more about the implementation and how something works, maybe stick to an imperative programming approach. More importantly, you can have a mix of both styles. It is extremely flexible for you. You are in charge here.

There are two very popular programming paradigms in software development that developers design and program to. They are known as object oriented programming and functional programming. You've probably heard of these terms before, but what exactly are they and what is the difference between functional and object oriented programming? Let's take a look. What is object oriented programming? Object oriented programming is a programming paradigm in which you program using objects to represent things you are programming about (sometimes real world things). These objects could be data structures. The objects hold data about them in attributes. The attributes in the objects are manipulated through methods or functions that are given to the object. For instance, we might have a Person object that represents all of the data a person would have: weight, height, skin color, hair color, hair length, and so on. Those would be the attributes. Then the person object would also have things that it can do such as: pick box up, put box down, eat, sleep, etc. These would be the functions that play with the data the object stores. Engineers who program using object oriented design say that it is a style of programming that allows you to model real world scenarios much simpler. This allows for a good transition from requirements to code that works like the customer or user wants it to. Some examples of object oriented languages include C++, Java, Python, C#, Objective-C, and Swift. Want to learn object oriented programming? We recommend you start with Learning Object Oriented Programming. What is functional programming? Functional programming is the form of programming that attempts to avoid changing state and mutable data. In a functional program, the output of a function should always be the same, given the same exact inputs to the function. This is because the outputs of a function in functional programming purely relies on arguments of the function, and there is no magic that is happening behind the scenes. This is called eliminating side effects in your code. For example, if you call function getSum() it calculates the sum of two inputs and returns the sum. Given the same inputs for x and y, we will always get the same output for sum. This allows the function of a program to be extremely predictable. Each small function does its part and only its part. It allows for very modular and clean code that all works together in harmony. This is also easier when it comes to unit testing. Some examples of Functional Programming Languages include Lisp, Clojure, and F#. Problems with object oriented programming There are a few problems with object oriented programing. Firstly, it is known to be not as reusable. Because some of your functions depend on the class that is using them, it is hard to use some functions with another class. It is also known to be typically less efficient and more complex to deal with. Plenty of times, some object oriented designs are made to model large architectures and can be extremely complicated. Problems with functional programming Functional programming is not without its flaws either. It really takes a different mindset to approach your code from a functional standpoint. It's easy to think in object oriented terms, because it is similar to how the object being modeled happens in the real world. Functional programming is all about data manipulation. Converting a real world scenario to just data can take some extra thinking. Due to its difficulty when learning to program this way, there are fewer people that program using this style, which could make it hard to collaborate with someone else or learn from others because there will naturally be less information on the topic. A comparison between functional and object oriented programming Both programming concepts have a goal of wanting to create easily understandable programs that are free of bugs and can be developed fast. Both concepts have different methods for storing the data and how to manipulate the data. In object oriented programming, you store the data in attributes of objects and have functions that work for that object and do the manipulation. In functional programming, we view everything as data transformation. Data is not stored in objects, it is transformed by creating new versions of that data and manipulating it using one of the many functions. I hope you have a clearer picture of what the difference between functional and object oriented programming. They can both be used separately or can be mixed to some degree to suite your needs. Ultimately you should take into the consideration the advantages and disadvantages of using both before making that decision. Antonio Cucciniello is a Software Engineer with a background in C, C++ and JavaScript (Node.js) from New Jersey. His most recent project called Edit Docs is an Amazon Echo skill that allows users to edit Google Drive files using your voice. He loves building cool things with software, reading books on self-help and improvement, finance, and entrepreneurship. Follow him on Twitter @antocucciniello, and follow him on GitHub here.