Author Posts

This week I published a post that explored some of the key trends in software infrastructure that security engineers, SREs, and SysAdmins should be paying attention to in 2019. There was clearly a lot to discuss - which is why I sat down with my colleague Stacy Matthews to discuss some of the topics explored in the post in a little more. Enjoy! https://soundcloud.com/packt-podcasts/why-observability-and-chaos-engineering-will-be-vital-in-2019 What do you think? Is chaos engineering too immature for widespread adoption? And how easy will it be to begin building for observability?

Over the past few years, we have seen some advanced technologies in artificial intelligence shaping human life. Deep learning (DL) has become the main driving force in bringing new innovations in almost every industry. We are sure to continue to see DL everywhere. Most of the companies including startups are already integrating deep learning into their own day-to-day process. Deep learning techniques and algorithms have made building advanced neural networks practically feasible, thanks to high-level open source libraries such as TensorFlow, Keras, PyTorch and more. We recently interviewed Valentino Zocca, a deep learning expert and the author of the book, Python Deep Learning. Valentino explains why deep learning is getting so much hype, and what's the roadmap ahead in terms of new technologies and libraries. He will also talks about how major vendors and tech-savvy startups adopt deep learning within their organization. Being a consultant and an active developer he is expecting a better approach than back-propagation for carrying out various deep learning tasks. Author’s Bio Valentino Zocca graduated with a Ph.D. in mathematics from the University of Maryland, USA, with a dissertation in symplectic geometry, after having graduated with a laurel in mathematics from the University of Rome. He spent a semester at the University of Warwick. After a post-doc in Paris, Valentino started working on high-tech projects in the Washington, D.C. area and played a central role in the design, development, and realization of an advanced stereo 3D Earth visualization software with head tracking at Autometric, a company later bought by Boeing. At Boeing, he developed many mathematical algorithms and predictive models, and using Hadoop, he has also automated several satellite-imagery visualization programs. He has since become an expert on machine learning and deep learning and has worked at the U.S. Census Bureau and as an independent consultant both in the US and in Italy. He has also held seminars on the subject of machine learning and deep learning in Milan and New York. Currently, Valentino lives in New York and works as an independent consultant to a large financial company, where he develops econometric models and uses machine learning and deep learning to create predictive models. But he often travels back to Rome and Milan to visit his family and friends. Key Takeaways Deep learning is one of the most adopted techniques used in image and speech recognition and anomaly detection research and development areas. Deep learning is not the optimum solution for every problem faced. Based on the complexity of the challenge, the neural network building can be tricky. Open-source tools will continue to be in the race when compared to enterprise software. More and more features are expected to improve on providing efficient and powerful deep learning solutions. Deep learning is used as a tool rather than a solution across organizations. The tool usage can differ based on the problem faced. Emerging specialized chips expected to bring more developments in deep learning to mobile, IoT and security domain. Valentino Zocca states We have a quantity vs. quality problem. We will be requiring better paradigms and approaches in the future which can be improved through research driven innovative solutions instead of relying on hardware solutions. We can make faster machines, but our goal is really to make more intelligent machines for performing accelerated deep learning and distributed training. Full Interview Deep learning is as much infamous as it is famous in the machine learning community with camps supporting and opposing the use of DL passionately. Where do you fall on this spectrum? If you were given a chance to convince the rival camp with 5-10 points on your stand about DL, what would your pitch be like? The reality is that Deep Learning techniques have their own advantages and disadvantages. The areas where Deep Learning clearly outperforms most other machine learning techniques are in image and speech recognition and anomaly detection. One of the reasons why Deep Learning does so much better is that these problems can be decomposed into a hierarchical set of increasingly complex structures, and, in multi-layer neural nets, each layer learns these structures at different levels of complexity. For example, an image recognition, the first layers will learn about the lines and edges in the image. The subsequent layers will learn how these lines and edges get together to form more complex shapes, like the eyes of an animal, and finally the last layers will learn how these more complex shapes form the final image. However, not every problem can suitably be decomposed using this hierarchical approach. Another issue with Deep Learning is that it is not yet completely understood how it works, and some areas, for example, banking, that are heavily regulated, may not be able to easily justify their predictions. Finally, many neural nets may require a heavier computational load than other classical machine learning techniques. Therefore, the reality is that one still needs a proficient machine learning expert who deeply understands the functioning of each approach and can make the best decision depending on each problem. Deep Learning is not, at the moment, a complete solution to any problem, and, in general, there can be no definite side to pick, and it really depends on the problem at hand. Deep learning can conquer tough challenges, no doubt. However, there are many common myths and realities around deep learning. Would you like to give your supporting reasoning on whether the following statements are myth or fact? You need to be a machine learning expert or a math geek to build deep learning models We need powerful hardware resources to use deep learning Deep learning models are always learning, they improve with new data automagically Deep learning is a black box, so we should avoid using it in production environments or in real-world applications. Deep learning is doomed to fail. It will be replaced eventually by data sparse, resource economic learning methods like meta-learning or reinforcement learning. Deep learning is going to be central to the progress of AGI (artificial general intelligence) research Deep Learning has become almost a buzzword, therefore a lot of people are talking about it, sometimes misunderstanding how it works. People hear the word DL together with "it beats the best player at go", "it can recognize things better than humans" etc., and people think that deep learning is a mature technology that can solve any problem. In actuality, deep learning is a mature technology only for some specific problems, you do not solve everything with deep learning and yet at times, whatever the problem, I hear people asking me "can't you use deep learning for it?" The truth is that we have lots of libraries ready to use for deep learning. For example, you don’t need to be a machine learning expert or a math geek to build simple deep learning models for run-of-the-mill problems, but in order to solve for some of the challenges that less common issues may present, a good understanding of how a neural network works may indeed be very helpful. Like everything, you can find a grain of truth in each of those statements, but they should not be taken at face value. With MLaaS being provided by many vendors from Google to AWS to Microsoft, deep learning is gaining widespread adoption not just within large organizations but also by data-savvy startups. How do you view this trend? More specifically, is deep learning being used differently by these two types of organizations? If so, what could be some key reasons? Deep Learning is not a monolithic approach. We have different types of networks, ANNs, CNNs, LSTMs, RNNs, etc. Honestly, it makes little sense to ask if DL is being used differently by different organizations. Deep Learning is a tool, not a solution, and like all tools it should be used differently depending on the problem at hand, not depending on who is using it. There are many open source tools and enterprise software (especially the ones which claim you don't need to code much) in the race. Do you think this can be the future where more and more people will opt for ready-to-use (MLaaS) enterprise backed cognitive tools like IBM Watson rather than open-source tools? This holds true for everything. At the beginning of the internet, people would write their own HTML code for their web pages, now we use tools who do most of the work for us. But if we want something to stand-out we need a professional designer. The more a technology matures, the more ready-to-use tools will be available, but that does not mean that we will never need professional experts to improve on those tools and provide specialized solutions. Deep learning is now making inroads to mobile, IoT and security domain as well. What makes DL great for these areas? What are some challenges you see while applying DL in these new domains? I do not have much experience with DL in mobiles, but that is clearly a direction that is becoming increasingly important. I believe we can address these new domains by building specialized chips. Deep learning is a deeply researched topic within machine learning and AI communities. Every year brings us new techniques from neural nets to GANs, to capsule networks that then get widely adopted both in research and in real-world applications. What are some cutting-edge techniques you foresee getting public attention in deep learning in 2018 and in the near future? And why? I am not sure we will see anything new in 2018, but I am a big supporter of the idea that we need a better paradigm that can excel more at inductive reasoning rather than just deductive reasoning. At the end of last year, even DL pioneer Geoff Hinton admitted that we need a better approach than back-propagation, however, I doubt we will see anything new coming out this year, it will take some time. We keep hearing noteworthy developments in AI and deep learning by DeepMind and OpenAI. Do you think they have the required armory to revolutionize how deep learning is performed? What are some key challenges for such deep learning innovators? As I mentioned before, we need a better paradigm, but what this paradigm is, nobody knows. Gary Marcus is a strong proponent of introducing more structure in our networks, and I do concur with him, however, it is not easy to define what that should be. Many people want to use the brain as a model, but computers are not biological structures, and if we had tried to build airplanes by mimicking how a bird flies we would not have gone very far. I think we need a clean break and a new approach, I do not think we can go very far by simply refining and improving what we have. Improvement in processing capabilities and the availability of custom hardware have propelled deep learning into production-ready environments in recent years. Can we expect more chips and other hardware improvements in the coming years for GPU accelerated deep learning and distributed training? What other supporting factors will facilitate the growth of deep learning? Once again, foreseeing the future is not easy, however, as these questions are related, I think only so much can be gained by improving chips and GPUs. We have a quantity vs. quality problem. We can improve quantity (of speed, memory, etc.) through hardware improvements, but the real problem is that we need a real quality improvement, better paradigms, and approaches, that needs to be achieved through research and not with hardware solutions. We can make faster machines, but our goal is really to make more intelligent machines. A child can learn by seeing just a few examples, we should be able to create an approach that allows a machine to also learn from few examples, not by cramming millions of examples in a short time. Would you like to add anything more to our readers? Deep Learning is a fascinating discipline, and I would encourage anyone who wanted to learn more about it to approach it as a research project, without underestimating his or her own creativity and intuition. We need new ideas. If you found this interview to be interesting, make sure you check out other insightful interviews on a range of topics: Blockchain can solve tech’s trust issues – Imran Bashir “Tableau is the most powerful and secure end-to-end analytics platform”: An interview with Joshua Milligan “Pandas is an effective tool to explore and analyze data”: An interview with Theodore Petrou

Data today is the world’s most important resource. However, without properly visualizing your data to discover meaningful insights, it’s useless. Creating visualizations helps in getting a clearer and concise view of the data, making it more tangible for (non-technical) audiences. To further illustrate this, below are questions aimed at giving you an idea why data visualization is so important and why Python should be your choice. In a recent interview, Tim Großmann and Mario Döbler, the authors of the course titled, ‘Data Visualization with Python’, shared with us the importance of Data visualization and why Python is the best fit to carry out Data Visualization. Key Takeaways Data visualization is a great way, and sometimes the only way, to make sense of the constantly growing mountain of data being generated today. With Python, you can create self-explanatory, concise, and engaging data visuals, and present insights that impact your business. Your data visualizations will make information more tangible for the stakeholders while telling them an interesting story. Visualizations are a great tool to transfer your understanding of the data to a less technical co-worker. This builds a faster and better understanding of data. Python is the most requested and used language in the industry. Its ease of use and the speed at which you can manipulate and visualize data, combined with the number of available libraries makes Python the best choice. Full Interview Why is Data Visualization important? What problem is it solving? As the amount of data grows, the need for developers with knowledge of data analytics and especially data visualization spikes. In recent years we have experienced an exponential growth of data. Currently, the amount of data doubles every two years. For example, more than eight thousand tweets are sent per second; and more than eight hundred photos are uploaded to Instagram per second. To cope with the large amounts of data, visualization is essential to make it more accessible and understandable. Everyone has heard of the saying that a picture is worth a thousand words. Humans process visual data better and faster than any other type of data. Another important point is that data is not necessarily the same as information. Often people aren’t interested in the data, but in some information hidden in the data. Data visualization is a great tool to discover the hidden patterns and reveal the relevant information. It bridges the gap between quantitative data and human reasoning, or in other words, visualization turns data into meaningful information. What other similar solutions or tools are out there? Why is Python better? Data visualizations can be created in many ways using many different tools. MATLAB and R are two of the available languages that are heavily used in the field of data science and data visualization. There are also some non-coding tools like Tableau which are used to quickly create some basic visualizations. However, Python is the most requested and used language in the industry. Its ease of use and the speed at which you can manipulate and visualize data, combined with the number of available libraries makes Python the best choice. In addition to all the mentioned perks, Python has an incredibly big ecosystem with thousands of active developers. Python really differs in a way that allows users to also build their own small additions to the tools they use, if necessary. There are examples of pretty much everything online for you to use, modify, and learn from. How can Data Visualization help developers? Give specific examples of how it can solve a problem. Working with, and especially understanding, large amounts of data can be a hard task. Without visualizations, this might even be impossible for some datasets. Especially if you need to transfer your understanding of the data to a less technical co-worker, visualizations are a great tool for a faster and better understanding of data. In general, looking at your data visualized often speaks more than a thousand words. Imagine getting a dataset which only consists of numerical columns. Getting some good insights into this data without visualizations is impossible. However, even with some simple plots, you can often improve your understanding of even the most difficult datasets. Think back to the last time you had to give a presentation about your findings and all you had was a table with numerical values in it. You understood it, but your colleagues sat there and scratched their heads. Instead had you created some simple visualizations, you would have impressed the entire team with your results. What are some best practices for learning/using Data Visualization with Python? Some of the best practices you should keep in mind while visualizing data with Python are: Start looking and experimenting with examples Start from scratch and build on it Make full use of documentation Use every opportunity you have with data to visualize it To know more about the best practices in detail, read our detailed notes on 4 tips for learning Data Visualization with Python. What are some myths/misconceptions surrounding Data Visualization with Python? Data visualizations are just for data scientists Its technologies are difficult to learn Data visualization isn’t needed for data insights Data visualization takes a lot of time Read about these myths in detail in our article, ‘Python Data Visualization myths you should know about’. Data visualization in combination with Python is an essential skill when working with data. When properly utilized, it is a powerful combination that not only enables you to get better insights into your data but also gives you the tool to communicate results better. Data nowadays is everywhere so developers of every discipline should be able to work with it and understand it. About the authors Tim Großmann Tim Großmann is a CS student with interest in diverse topics ranging from AI to IoT. He previously worked at the Bosch Center for Artificial Intelligence in Silicon Valley in the field of big data engineering. He’s highly involved in different Open Source projects and actively speaks at meetups and conferences about his projects and experiences. Mario Döbler Mario Döbler is a graduate student with a focus in deep learning and AI. He previously worked at the Bosch Center for Artificial Intelligence in Silicon Valley in the field of deep learning, using state-of-the-art algorithms to develop cutting-edge products. Currently, he dedicates himself to apply deep learning to medical data to make health care accessible to everyone. Setting up Apache Druid in Hadoop for Data visualizations [Tutorial] 8 ways to improve your data visualizations Getting started with Data Visualization in Tableau  

Machine learning has shown immense promise across domains and industries over the recent years. From helping with the diagnosis of serious ailments to powering autonomous vehicles, machine learning is finding useful applications across a spectrum of industries. However, the actual process of delivering business outcomes using machine learning currently takes too long and is too expensive, forcing some businesses to look for other less burdensome alternatives. MissingLink.ai is a recently-launched platform to fix just this problem. It enables data scientists to spend less time on the grunt work by automating and streamlining the entire machine learning cycle, giving them more time to apply actionable insights gleaned from the data. Key Takeaways Processing and managing the sheer volume of data is one of the key challenges that today’s AI tools face Yosi thinks the idea of companies creating their own machine learning infrastructure doesn’t make a lot of sense. Data professionals should be focusing on more important problems within their organizations by letting the platform take care of the grunt work. MissingLink.ai is an innovative AI platform born out of the need to simplify AI development, by taking away the common, menial data processing tasks from data scientists and allowing them to focus on the bigger data-related issues; through experiment management, data management and resource management. MissingLink is a part of the Samsung NEXT product development team that aims to help businesses automate and accelerate their projects using machine learning We had the privilege of interviewing Mr. Yosi Taguri, the founder and CEO of MissingLink, to know more about the platform and how it enables more effective deep learning. What are the biggest challenges that companies come across when trying to implement a robust Machine Learning/Deep Learning pipeline in their organization? How does it affect their business? The biggest challenge, simply put, is that today’s AI tools can’t keep up with the amount of data being produced. And it’s only going to get more challenging from here! As datasets continue to grow, they will require more and more compute power, which means we risk falling farther behind unless we change the tools we’re using. While everyone is talking about the promise of machine learning, the truth is that today, assessing data is still too time-consuming and too expensive. Engineers are spending all their time managing the sheer volume of data, rather than actually learning from it and being empowered to make changes. Let’s talk about MissingLink.ai, the platform you and your team have launched for accelerating deep learning across businesses. Why the name MissingLink? What was the motivation to launch this platform? The name is actually a funny story, and it ties pretty neatly into why we created the platform. When we were starting out three years ago, deep learning was still a relatively new concept and my team and I were working hard to master the intricacies of it. As engineers, we primarily worked with code, so to be able to solve problems with data was a fascinating new challenge for us. We quickly realized that deep learning is really hard and moves very, very slow. So we set out to solve that problem of how to build really smart machines really fast. By comparison, we thought of it through the lens of software development. Our goal was to accelerate from a glacial pace to building machine learning algorithms faster -- because we felt that there was something missing, a missing link if you will. MissingLink is a part of the growing Samsung NEXT product development team. How does it feel? What role do you think MissingLink will play in Samsung NEXT’s plans and vision going forward? Samsung NEXT’s broader mission is to help startups reach their full potential and achieve their goals. More specifically, Samsung NEXT discovers and backs the engineers, innovators, builders, and entrepreneurs who will help Samsung define the future of software and services. The Samsung NEXT product development team is focused on building software and services that take advantage of and accelerate opportunities related to some of the biggest shifts in technology including automation, supply and demand, and interfaces. This will require hardware and software to seamlessly come together. Over the past few years, nearly all startups are leveraging AI for some component of their business, yet practical progress has been slower than promised. MissingLink is a foundational tool to enable the progress of these big changes, helping entrepreneurs with great use cases for machine learning to accelerate their projects. Could you give us the key features of Missinglink.ai that make it stand out from the other AI platforms available out there? How will it help data scientists and ML engineers build robust, efficient machine learning models? First off, MissingLink.ai is the most comprehensive AI platform out there. It handles the entire deep learning lifecycle and all its elements, including code, data, experiments, and resources. I’d say that our top features include: Experiment Management: See and compare the entire history of experiments. MissingLink.ai auto-documents every aspect Data Management: A unique data store tracks data versions used in every experiment, streams data, caches it locally and only syncs changes Resources Management: Manages your resources with no extra infrastructure costs using your AWS or other cloud credentials. It grows and shrinks your cloud resources as needed. These features, together with our intuitive interface, really put data scientists and engineers in the driver's seat when creating AI models. Now they can have more control and spend less energy repeating experiments, giving them more time to focus on what is important. Your press release on the release of MissingLink states “the actual process of delivering business outcomes currently takes too long and it is too expensive. MissingLink.ai was born out of a desire to fix that.” Could you please elaborate on how MissingLink makes deep learning less expensive and more accessible? Companies are currently spending too much time and devoting too many resources to the menial tasks that are necessary for building machine learning models. The more time data scientists spend on tasks like spinning machines, copying files and DevOps, the more money that a company is wasting. MissingLink changes that through the introduction of something we’re calling DeepOps or deep learning operations, which allows data scientists to focus on data science and let the machine take care of the rest. It’s like DevOps where the role is about how to make the process of software development more efficient and productionalized, but the difference is no one has been filling this role and it’s different enough that its very specific to the task of deep learning. Today, instead of data scientists working on their models and advancing AI, they are spending their time doing this DeepOps work. MissingLink reduces load time and facilitates easy data exploration by eliminating the need to copy files through data-management in a version-aware data store. Most of the businesses are moving their operations on to the cloud these days, with AWS, Azure, GCP, etc. being their preferred cloud solutions. These platforms have sophisticated AI offerings of their own. Do you see AI platforms such as MissingLink.ai as a competition to these vendors, or can the two work collaboratively? I wouldn’t call cloud companies our competitors; we don’t provide the cloud services they do, and they don’t provide the DeepOps service that we do. Yes, we all are trying to simplify AI, but we’re going about it in very different ways. We can actually use a customer’s public cloud provider as the infrastructure to run the MissingLink.ai platform. If customers provide us with their cloud credentials, we can even manage this for them directly. Concepts such as Reinforcement Learning and Deep Learning for Mobile are getting a lot of traction these days, and have moved out of the research phase into the application/implementation phase. Soon, they might start finding extensive business applications as well. Are there plans to incorporate these tools and techniques in the platform in the near future? We support all forms of Deep Learning, including Reinforcement Learning. On the Deep Learning for Mobile side, we think the Edge is going to be a big thing as more and more developers around the world are exposed to Deep Learning. We plan to support it early next year. Currently, data privacy and AI ethics have become a focal point of every company’s AI strategy. We see tech conglomerates increasingly coming under the scanner for ignoring these key aspects in their products and services. This is giving rise to an alternate movement in AI, with privacy and ethics-centric projects like Deon, Vivaldi, and Tim Berners-Lee’s Solid. How does MissingLink approach the topics of privacy, user consent, and AI ethics? Are there processes/tools or principles in place in the MissingLink ecosystem or development teams that balance these concerns? When we started MissingLink we understood that data is the most sensitive part of Deep Learning. It is the new IP. Companies spend 80% of their time attending to data, refining it, tagging it and storing it, and therefore are reluctant to upload it to a 3rd party vendor. We have built MissingLink with that in mind - our solution allows customers to simply point us in the direction of where their data is stored internally, and without moving it or having to access it as a SaaS solution we are able to help them manage it by enabling version management as they do with code. Then we can stream it directly to the machines that need the data for processing and document which data was used for reproducibility. Finally, where do you see machine learning and deep learning heading in the near future? Do you foresee a change in the way data professionals work today? How will platforms like MissingLink.ai change the current trend of working? Right now, companies are creating their own machine learning infrastructure - and that doesn’t make sense. Data professionals can and should be focusing on more important problems within their organizations. Platforms like MissingLink.ai free data scientists from the grunt work it takes to upkeep the infrastructure, so they can focus on bigger picture issues. This is the work that is not only more rewarding for engineers to work on, but also creates the unique value that companies need to compete.  We’re excited to help empower more data professionals to focus on the work that actually matters. It was wonderful talking to you, and this was a very insightful discussion. Thanks a lot for your time, and all the best with MissingLink.ai! Read more Michelangelo PyML: Introducing Uber’s platform for rapid machine learning development Tesseract version 4.0 releases with new LSTM based engine, and an updated build system Baidu releases a new AI translation system, STACL, that can do simultaneous interpretation

Data is the new oil and is just as crude as unrefined oil. To do anything meaningful - modeling, visualization, machine learning, for predictive analysis – you first need to wrestle and wrangle with data. We recently interviewed Dr. Tirthajyoti Sarkar and Shubhadeep Roychowdhury, the authors of the course Data Wrangling with Python. They talked about their new course and discuss why do data wrangling and why use Python to do it. Key Takeaways Python boasts of a large, broad library equipped with a rich set of modules and functions, which you can use to your advantage and manipulate complex data structures NumPy, the Python library for fast numeric array computations and Pandas, a package with fast, flexible, and expressive data structures are helpful in working with “relational” or “labeled” data. Web scraping or data extraction becomes easy and intuitive with Python libraries, such as BeautifulSoup4 and html5lib. Regex, the tiny, highly specialized programming language inside Python can create patterns that help match, locate, and manage text for large data analysis and searching operations Present interesting, interactive visuals of your data with Matplotlib, the most popular graphing and data visualization module for Python. Easily and quickly separate information from a huge amount of random data using Pandas, the preferred Python tool for data wrangling and modeling. Full Interview Congratulations on your new course ‘Data wrangling with Python’. What this course is all about? Data science is the ‘sexiest job’ of 21st century’ (at least until Skynet takes over the world). But for all the emphasis on ‘Data’, it is the ‘Science’ that makes you - the practitioner - valuable. To practice high-quality science with data, first you need to make sure it is properly sourced, cleaned, formatted, and pre-processed. This course teaches you the most essential basics of this invaluable component of the data science pipeline – data wrangling. What is data wrangling and why should you learn it well? “Data is the new Oil” and it is ruling the modern way of life through incredibly smart tools and transformative technologies. But oil from the rig is far from being usable. It has to be refined through a complex processing network. Similarly, data needs to be curated, massaged and refined to become fit for use in intelligent algorithms and consumer products. This is called “wrangling” and (according to CrowdFlower) all good data scientists spend almost 60-80% of their time on this, each day, every project. It generally involves the following: Scraping the raw data from multiple sources (including web and database tables), Inputing, formatting, transforming – basically making it ready for use in the modeling process (e.g. advanced machine learning), Handling missing data gracefully, Detecting outliers, and Being able to perform quick visualizations (plotting) and basic statistical analysis to judge the quality of your formatted data This course aims to teach you all the core ideas behind this process and to equip you with the knowledge of the most popular tools and techniques in the domain. As the programming framework, we have chosen Python, the most widely used language for data science. We work through real-life examples and at the end of this course, you will be confident to handle a myriad array of sources to extract, clean, transform, and format your data for further analysis or exciting machine learning model building. Walk us through your thinking behind how you went about designing this course. What’s the flow like? How do you teach data wrangling in this course? The lessons start with a refresher on Python focusing mainly on advanced data structures, and then quickly jumping into NumPy and Panda libraries as fundamental tools for data wrangling. It emphasizes why you should stay away from traditional ways of data cleaning, as done in other languages, and take advantage of specialized pre-built routines in Python. Thereafter, it covers how using the same Python backend, one can extract and transform data from a diverse array of sources - internet, large database vaults, or Excel financial tables. Further lessons teach how to handle missing or wrong data, and reformat based on the requirement from a downstream analytics tool. The course emphasizes learning by real example and showcases the power of an inquisitive and imaginative mind primed for success. What other tools are out there? Why do data wrangling with Python? First, let us be clear that there is no substitute for the data wrangling process itself. There is no short-cut either. Data wrangling must be performed before the modeling task but there is always the debate of doing this process using an enterprise tool or by directly using a programming language and associated frameworks. There are many commercial, enterprise-level tools for data formatting and pre-processing, which does not involve coding on the part of the user. Common examples of such tools are: General purpose data analysis platforms such as Microsoft Excel (with add-ins) Statistical discovery package such as JMP (from SAS) Modeling platforms such as RapidMiner Analytics platforms from niche players focusing on data wrangling such as – Trifacta, Paxata, Alteryx At the end of the day, it really depends on the organizational approach whether to use any of these off-the-shelf tools or to have more flexibility, control, and power by using a programming language like Python to perform data wrangling. As the volume, velocity, and variety (three V’s of Big Data) of data undergo rapid changes, it is always a good idea to develop and nurture significant amount of in-house expertise in data wrangling. This is done using fundamental programming frameworks so that an organization is not betrothed to the whims and fancies of any particular enterprise platform as a basic task as data wrangling. Some of the obvious advantages of using an open-source, free programming paradigm like Python for data wrangling are: General purpose open-source paradigm putting no restriction on any of the methods you can develop for the specific problem at hand Great eco-system of fast, optimized, open-source libraries, focused on data analytics Growing support to connect Python for every conceivable data source types, Easy interface to basic statistical testing and quick visualization libraries to check data quality Seamless interface of the data wrangling output to advanced machine learning models – Python is the most popular language of choice of machine learning/artificial intelligence these days. What are some best practices to perform data wrangling with Python? Here are five best practices that will help you out in your data wrangling journey with Python. And in the end, all you’ll have is clean and ready to use data for your business needs. Learn the data structures in Python really well Learn and practice file and OS handling in Python Have a solid understanding of core data types and capabilities of Numpy and Pandas Build a good understanding of basic statistical tests and a panache for visualization Apart from Python, if you want to master one language, go for SQL What are some misconceptions about data wrangling? Though data wrangling is an important task, there are certain myths associated with data wrangling which developers should be cautious of. Myths such as: Data wrangling is all about writing SQL query Knowledge of statistics is not required for data wrangling You have to be a machine learning expert to do great data wrangling Deep knowledge of programming is not required for data wrangling Learn in detail about these misconceptions. We hope that these misconceptions would help you realize that data wrangling is not as difficult as it seems. Have fun wrangling data! About the authors Dr. Tirthajyoti Sarkar works as a Sr. Principal Engineer in the semiconductor technology domain where he applies cutting-edge data science/machine learning techniques for design automation and predictive analytics. Shubhadeep Roychowdhury works as a Sr. Software Engineer at a Paris based Cyber Security startup. He holds a Master Degree in Computer Science from West Bengal University Of Technology and certifications in Machine Learning from Stanford. 5 best practices to perform data wrangling with Python 4 misconceptions about data wrangling Data cleaning is the worst part of data analysis, say data scientists

An unreliable versioning tool makes product development a herculean task. Creating and enforcing checks and controls for the introduction, scrutiny, approval, merging, and reversal of changes in your source code, are some effective methods to ensure a secure development environment. Git and GitHub offer constructs that enable teams to conduct version control and collaborative development in an effective manner.  When properly utilized, Git and GitHub promote agility and collaboration across a team, and in doing so, enable teams to focus and deliver on their mandates and goals. We recently interviewed Joseph Muli and Alex Magana, the authors of Introduction to Git and GitHub course. They discussed the various benefits of Git and GitHub while sharing some best practices and myths. Author Bio Joseph Muli loves programming, writing, teaching, gaming, and travelling. Currently, he works as a software engineer at Andela and Fathom, and specializes in DevOps and Site Reliability. Previously, he worked as a software engineer and technical mentor at Moringa School. You can follow him on LinkedIn and Twitter. Alex Magana loves programming, music, adventure, writing, reading, architecture, and is a gastronome at heart. Currently, he works as a software engineer with BBC News and Andela. Previously, he worked as a software engineer with SuperFluid Labs and Insync Solutions. You can follow him on LinkedIn or GitHub. Key Takeaways Securing your source code with version control is effective only when you do it the right way. Understanding the best practices used in version control can make it easier for you to get the most out of Git and GitHub. GitHub is loaded with an elaborate UI. It’ll immensely help your development process to learn how to navigate the GitHub UI and install the octo tree. GitHub is a powerful tool that is equipped with useful features. Exploring the Feature Branch Workflow and other forking features, such as submodules and rebasing, will enable you to make optimum use of the many features of GitHub. The more elaborate the tools, the more time they can consume if you don’t know your way through them. Master the commands for debugging and maintaining a repository, to speed up your software development process. Keep your code updated with the latest changes using CircleCI or TravisCI, the continuous integration tools from GitHub. The struggle isn’t over unless the code is successfully released to production. With GitHub’s release management features, you can learn to complete hiccup-free software releases. Full Interview Why is Git important? What problem is it solving? Git, like all other version control tools, exists to solve for one problem, change. This has been a recurring issue, especially when coordinating work on teams, both locally and distributed, that specifically being an advantage of Git through hubs such as GitHub, BitBucket and Gitlab. The tool was created by Linus Torvalds in 2005 to aid in development and contribution on the Linux Kernel. However, this doesn’t necessarily limit Git to code any product or project that requires or exhibits characteristics such as having multiple contributors, requiring release management and versioning stands to have an improved workflow through Git. This also puts into perspective that there is no standard, it’s advisable to use what best suits your product(s). What other similar solutions or tools are out there? Why is Git better? As mentioned earlier, other tools do exist to aid in version control. There are a lot of factors to consider when choosing a version control system for your organizations, depending on product needs and workflows. Some organizations have in-house versioning tools because it suits their development. Some organizations, for reasons such as privacy and security or support, may look for an integration with third-party and in-house tools. Git primarily exists to provide for a faster and distributed version system, that is not tied to a central repository, hub or project. It is highly scalable and portable. Other VC tools include Apache SubVersion, Mercurial and Concurrent Versions System (CVS). How can Git help developers? Can you list some specific examples (real or imagined) of how it can solve a problem? A simple way to define Git’s indispensability is enabling fast, persistent and accessible storage. This implies that changes to code throughout a product’s life cycle can be viewed and updated on demand, each with simple and compact commands to enable the process. Developers can track changes from multiple contributors, blame introduced bugs and revert where necessary. Git enables multiple workflows that align to practices such as Agile e.g. feature branch workflows and others including forking workflows for distributed contribution, i.e. to open source projects. What are some best tips for using Git and GitHub? These are some of the best practices you should keep in mind while learning or using Git and GitHub. Document everything Utilize the README.MD and wikis Keep simple and concise naming conventions Adopt naming prefixes Correspond a PR and Branch to a ticket or task. Organize and track tasks using issues. Use atomic commits [box type="shadow" align="" class="" width=""]Editor’s note: To explore these tips further, read the authors’ post ‘7 tips for using Git and GitHub the right way’.[/box] What are the myths surrounding Git and GitHub? Just as every solution or tool has its own positives and negatives, Git is also surrounded by myths one should be aware of. Some of which are: Git is GitHub Backups are equivalent to version control Git is only suitable for teams To effectively use Git, you need to learn every command to work [box type="shadow" align="" class="" width=""]Editor’s note: To explore these tips further, read the authors’ post ‘4 myths about Git and GitHub you should know about’.  [/box] GitHub’s new integration for Jira Software Cloud aims to provide teams a seamless project management experience GitLab raises $100 million, Alphabet backs it to surpass Microsoft’s GitHub GitHub introduces ‘Experiments’, a platform to share live demos of their research projects  

SAP, the market-leading enterprise software, recently became the first European technology company to create an AI ethics advisory panel where they announced seven guiding principles for AI development. These guidelines revolve around recognizing AI’s significant impact on people and society. Also, last week, at the Microsoft Ignite conference, SAP, in collaboration with Microsoft and Adobe announced the Open Data Initiative. This initiative aims to help companies to better govern their data and support privacy and security initiatives. For SAP, this initiative will further bring advancements to its SAP C/4HANA and S/4HANA platforms. All of these actions emphasize SAP’s focus on transforming itself into a responsible data use company. We recently interviewed Rehan Zaidi, a senior SAP ABAP consultant. Rehan became one of the youngest authors on SAP worldwide when he was published in the prestigious SAP Professional Journal in the year 2001. He has written a number of books, and over 20 articles and professional papers for the SAP Professional Journal USA and HR Expert USA, part of the prestigious sapexperts.com library. Following are some of his views on the SAP community and products and how the SAP suite can benefit people including budding professionals, developers, and business professionals. Key takeaways SAP HANA was introduced to accelerate jobs 200 times faster while maintaining the efficiency. The introduction of SAP Leonardo brought in the next wave of AI, machine learning, and blockchain services via the SAP cloud platform and other standalone projects. Experienced ABAP developers should look forward to getting certified in one of the newest technologies such as HANA, and Fiori. SAP ERP Central Component (SAP ECC) is the on-premises version of SAP, and it is usually implemented in medium and large-sized companies. For smaller companies, SAP offers its Business One ERP platform. SAP Fiori is a line of SAP apps meant to address criticisms of SAP's user experience and UI complexity. Q.1. SAP is one of the most widely used ERP tools. How has it evolved over the past few years from the traditional on-premise model to keep up with the cloud-based trends? Yes. Let me cover the main points. SAP started in 1973 as a company and the first product SAP R/98 was launched. In 1979, SAP launched the R/2 design. It had most of the typical processes such as accounting, manufacturing processes, supply chain logistics, and human resources. Then came R/3  that brought the more efficient three-tier (Application server -  Database and the presentation (GUI)) architecture, with more new modules and functionalities added. It was a smart system fully configurable by functional consultants. This was further enhanced with Netweaver capability that brought the integration of the internet and SOA capability.  SAP introduced the ECC 5 and subsequently the ECC 6 Release. Mobility was later added that lets mobile applications running on devices to access the business processes in SAP and execute them. Both display and updation of SAP data was possible. HANA system was then introduced. It is very fast and efficient - allows you to do 200 times faster jobs than before Cloud systems then became available that let customers connect to SAP Cloud Platform via their on-premise systems and then get access to services such as Mobile Service for app protection, Mobile Service for SAP Fiori, among others. SAP Leonardo was finally introduced, as a way of bringing in next-gen AI, machine learning and blockchain services via standalone projects and the SAP cloud platform. Q.2. Being a Senior ABAP Programming Analyst, how does your typical day look like? Ahh. Well, a typical day! No two days are the same for us. Each morning we find ourselves confronting a problem whose solution is to be devised. A different problem every day- followed by a unique solution. We spend hours and hours finding issues in custom developed programs. We learn about making custom programs run faster. We get requirements of a wide variety of users. They may be in the Human Resource, Materials Management, Sales and Distribution or Finance, and so on. This requirement may be pertaining to an entirely new report or a dialog program having a set of screens. We even do Fiori ( using Javascript based library) applications that may be accessible from the PC or a mobile device. I even get requirements of teaching junior or trainee SAP developers on a wide variety of technologies. Q.3. Can you tell us about the learning curve for SAP? There are different job profiles related to SAP which range from executives to consultants and managers. How do each of them learn or update themselves on SAP? Yes, this is a very important question. A simple answer to this question is that “there is no end to learning and at any stage, learning is never enough,” no matter to which field within SAP you belong to. Things are constantly changing. The more you read and the more you work, you feel that there is a lot to be done. You need to constantly update yourself and learn about new technologies. There is plenty of material available on the internet. I usually refer to the Official SAP website for newer courses available. They even tell you for which background (managers, developers) the courses are relevant to. I also go to open.sap.com for new courses. Whether they are consultants (functional and technical), or managers, all of them need to keep themselves up-to-date. They must take new courses and learn about innovation in their technology. For example, HR must now study and try to learn about Successfactors. Even integration of SAP HANA with other software might be an interesting topic of today. There are Fiori and HANA related courses for Basis consultants and the corresponding tracks for developers. Some knowhow of newer technologies is also important for managers and executives, since your decisions may need to be adapted based on the underlying technologies running in your systems. You should know the pros and cons of all technologies in order to make the correct move for your business. Q.4. Many believe an SAP certification improves their chances of getting jobs at competitive salaries. How important are certifications? Which SAP certifications should a buddying developer look forward to obtain? When I did my Certification in October 2000, I used to think that Certifications are not important. But now I have realized, yes, it makes a difference.  Well, certifications are definitely a plus point. They enhance your CV and allow you to have an edge over those who are not certified.  I found some jobs adverts that specifically mention that certification will be required or will be advantageous. However, they are only useful when you have at least 4 years of experience. For a fresh graduate, a certification might not be very useful. A useful SAP consultant/developer is a combination of solid base/foundation of knowledge along with a touch of experience. I suggest all my juniors to go for Certifications in order to strengthen concepts, which include: C_C4C30_1711 - SAP Certified Development Associate – SAP Hybris Cloud for Customer C_CP_11 - SAP Certified Development Associate - SAP Cloud Platform C_FIORDEV_20 - SAP Certified Development Associate - SAP Fiori Application Developer C_HANADEV_13 - SAP Certified Development Associate - SAP HANA C_SMPNHB_30 - SAP Certified Development Associate - SAP Mobile Platform Application Development (SMP 3.0) C_TAW12_750 - SAP Certified Development Associate - ABAP with SAP NetWeaver 7.50 E_HANAAW_12 - SAP Certified Development Specialist - ABAP for SAP HANA For experienced ABAP developers, I suggest getting certified on the newest technologies such as HANA, and Fiori. They may help you get a project quicker and/or at a better rate than others. Q.5. The present buzz is around AI, machine learning, IoT, Big data, and many other emerging technologies. SAP Leonardo works on making it easy to create frameworks for harnessing the latest tech. What are your thoughts on SAP Leonardo? Leonardo is SAP’s response to an AI platform. It should be an important part of SAP’s offerings, mostly built on the SAP cloud platform. SAP has relaunched Leonardo as a digital innovation system. As I understand it, Leonardo allows customers to take advantage of artificial intelligence (AI), machine learning, advanced analytics and blockchain on their company’s data. SAP gives customers an efficient way of using these technologies to solve business issues. It allows you to build a system which, in conjunction with machine learning, searches for results that can be combined with SAP transactions. The benefit with SAP Leonardo is that all the company’s data is available right in the SAP system. Using Leonardo, you have access to all human resources data and any other module data residing in the ERP system. Any company from any industry can make use of Leonardo; it works equally well for retailers, food and beverage companies and medical industries, for organizations working in retail, manufacturing and automotive. An approach that works for one company in a given industry can be applied to other companies in that industry. Suppose a company operates sensors. They can link the sensor data with the data in their SAP systems and even link that with other data, and they can then use the Leonardo capabilities to solve problems or optimize performance. When a problem for one company in an industry is solved, a similar solution may be applied to the entire industry. Yes, in my opinion, Leonardo has a bright future and should be successful. For more information about Leonardo success stories, I encourage readers to check out SAP Leonardo Internet of Things Portfolio & Success Stories. Q. 6. You are currently writing a book on ABAP Objects and Design Patterns expected to be published by the end of 2018. What was your motivation behind writing it? Can you tell us more about ABAP objects? What should readers expect from this book? ABAP and ABAP Objects has gone tremendous changes since some time both on the features (and capability) as well as the syntax. It is the most unsung topic of today. It has been there for quite long but most developers are not aware of it or are not comfortable enough to use them in their day to day work. ABAP is a vast community with developers working in a variety of functional areas. The concepts covered in the book will be generic, allowing the learner to apply them to his or her particular area. This book will cover ABAP objects (the object-oriented extension of the SAP language ABAP) in the latest release of SAP NetWeaver 7.5 and explain the newest advancements. It will start with the programming of objects in general and the basics of ABAP language the developer needs to know to get started. The book will cover the most important topics needed on everyday support jobs and for succeeding in projects. The book will be goal-directed, not a collection of theoretical topics. It won’t just touch on the surface of ABAP objects, but will go in depth from building the basic foundation (e.g., classes and objects created locally and globally) to the intermediary areas (e.g., ALV programming, method chaining, polymorphism, simple and nested interfaces), and then finally into the advanced topics (e.g., shared memory, persistent Objects). The best practices for making better programs via ABAP objects will be shown at the end. No long stories, no boring theory, only pure technical concepts followed by simple examples using coding pertaining to football players. Everything will be presented in a clear, interesting manner, and readers will learn tips and tricks they can apply immediately. Learners, students, new SAP programmers and SAP developers with some experience can use this as an alternative to expensive training books. The book will also save reader’s time searching the internet for help writing new programs. Knowing ABAP objects is key for ABAP developers these days to move forward. Starting from simple ALV reporting requirements, or defining and catching exceptional situations that may occur in a program or even the enhancement technology of BAdIs that lets you enhance standard SAP applications require sound ABAP Objects understanding. In addition, Web Dynpro application development, the Business Object Processing Framework, and even OData service creation to expose data that can be used by Fiori apps all demand solid knowledge of ABAP objects. How to perform predictive forecasting in SAP Analytics Cloud Popular Data sources and models in SAP Analytics Cloud Understanding Text Search and Hierarchies in SAP HANA  

At a time when software systems are growing in complexity, and when the expectations and demands from users have never been more critical, it's easy to forget that just making things work can be a huge challenge. That's where site reliability engineering (SRE) comes in; it's one of the reasons we're starting to see it grow as a discipline and job role. The central philosophy behind site reliability engineering can be seen in trends like chaos engineering. As Gremlin CTO Matt Fornaciari said, speaking to us in June, "chaos engineering is simply part of the SRE toolkit." For site reliability engineers, software resilience isn't an optional extra - it's critical. In crude terms, downtime for a retail site means real monetary losses, but the example extends beyond that. Because people and software systems are so interdependent, SRE is a useful way for thinking about how we build software more broadly. To get to the heart of what site reliability engineering is, I spoke to Nat Welch, an SRE currently working at First Look Media, whose experience includes time at Google and Hillary Clinton's 2016 presidential campaign. Nat has just published a book with Packt called Real-World SRE. You can find it here. Follow Nat on Twitter: @icco What is site reliability engineering? Nat Welch: The idea [of site reliability engineering] is to write and modify software to improve the reliability of a website or system. As a term and field, it was founded by Google in the early 2000s, and has slowly spread across the rest of the industry. Having engineers dedicated to global system health and reliability, working with every layer of the business to improving reliability for systems. "By building teams of engineers focused exclusively on reliability, there can be someone arguing for and focusing on reliability in a way to improve the speed and efficiency of product teams." Why do we need site reliability engineering? Nat Welch: Customers get mad if your website is down. Engineers often were having trouble weighing system reliability work versus new feature work. Because of this, product feature work often takes priority, and reliability decisions are made by guess work. By building teams of engineers focused exclusively on reliability, there can be someone arguing for and focusing on reliability in a way to improve the speed and efficiency of product teams. Why do we need SRE now, in 2018? Nat Welch: Part of it is that people are finally starting to build systems more like how Google has been building for years (heavy use of containers, lots of services, heavily distributed). The other part is a marketing effort by Google so that they can make it easier to hire. What are the core responsibilities of an SRE? How do they sit within a team? Nat Welch: SRE is just a specialization of a developer. They sit on equal footing with the rest of the developers on the team, because the system is everyone's responsbility. But while some engineers will focus primarily on new features, SRE will primarily focus on system reliability. This does not mean either side does not work on the other (SRE often write features, product devs often write code to make the system more reliable, etc), it just means their primary focus when defining priorities is different. What are the biggest challenges for site reliability engineers? Nat Welch: Communication with everyone (product, finance, executive team, etc.), and focus - it's very easy to get lost in fire fighting. What are the 3 key skills you need to be a good SRE? Nat Welch: Communication skills, software development skills, system design skills. You need to be able to write code, review code, work with others, break large projects into small pieces and distribute the work among people, but you also need to be able to take a system (working or broken) and figure out how it is designed and how it works. Thanks Nat! Site reliability engineering, then, is a response to a broader change in the types of software infrastructure we are building and using today. It's certainly a role that offers a lot of scope for ambitious and curious developers interested in a range of problems in software development, from UX to security. If you want to learn more, take a look at Nat's book.

Augmented and Virtual Reality is taking on the world, one AR app at a time. Almost all tech giants have their own AR platforms, whether it be Google’s ARCore, to Apple’s ARKit, to Snapchat’s Lens Studio. Not just that, there are now various AR galleries and exhibits focused solely on AR, where designers and artists showcase their artistic talents combined with rich augmented reality experiences. While searching for designers and artists working the augmented reality space, we stumbled upon Marc-O-Matic aka Marco Ryan. We were fascinated by his artwork and wanted to interview him right away.   [embed]https://vimeo.com/219045989[/embed] As his website bio says, “Marco is a multidisciplinary Artist, Animator, Director, Storyteller and Technologist working across Augmented and Virtual Reality technologies.” He shared with us his experiences working with Augmented Reality and told us about his creation process, his current projects, tips and tricks for a budding artist venturing into the AR space and his views on merging technology and art. Key Takeaways A well-rounded creative is someone who could do everything from creating the art, the associated stories as well as execute the technical aspects of the work to make it come to life. The future belongs to these creative types who love to learn and experiment every day. An artist must consider three aspects before taking on a project. First, how to tell their story in a more engaging and interesting manner. Second, how to combine different skill sets together and third the message their work conveys. Augmented Reality has added a new level of depth to the art experience not just for artists but also for viewers. Everyone has a smartphone these days and with AR you can add many added elements to an art piece. You can add sound, motion and 3D elements to the experience, which affect more of your senses. It is easy for a beginner artist to get started with creating Augmented reality art. You need to start with learning a basic language (mostly C# or Javascript) and then you can learn more as you explore. Tools and platforms such as Adobe After Effects, Maya and Blend, are good for building shaders, materials or effects. Unity 3D is a popular choice for implementing the AR functionality, and over 91% of HoloLens experiences are made with Unity. It supports a large number of devices and also has a large community base. Unity offers highly optimized rendering pipeline and rapid iteration capabilities to create realist AR VR experiences. AI and Machine Learning are at the forefront of innovating AR/VR. Machine learning algorithms can be used in character modeling. They can map your facial movements directly to a 3D character’s face. AI can make a character emotionally react based on the tone of the audio and also automate tasks allowing an artist to focus solely on the creative aspects. Full Interview How did you get started in AR. How did you journey from being an artist to an AR artist began? I started experimenting with Immersive AR/VR tech about 2-3 years ago? At the time buzzwords like virtual and augmented reality started trending and it had me curious to see how I could implement my own existing skills into this area at the time. Prior to getting into AR and VR I came from a largely self-taught background in Art & Illustration which then evolved into exploring film and animation and then finally led to game design and programming. Exploring all these areas allowed me to become a well-rounded creative, meaning I could do everything from creating the art, the associated stories as well as execute the technical aspects of the work to make it come to life. That’s quite impressive and fascinating to know! What programming language did you learn first and why? What was your approach to learning the language? Do you also dabble in other languages? How can artists overcome the fear of learning technology? Working with the game engine Unity 3D to create my VR and AR Experiences, the engine allows you to program in either C# or Javascript. C# was the the most prevalent language of the two and for me I found it easier to pick up. As a visual learner, I initially came to understand the language through node based programming tools like Playmaker and Uscript. These plug-ins for Unity are great for beginners as they allow you to visually program behaviors and functionality into your experience by creating interconnected node trees or maps which then generates the code. I’m familiar with this form of logic building as other programs I use, such as Adobe After Effects, Maya and Blend, use similar systems for building shaders, materials or effects. Through node based programming or visual scripting, you can take a look under the hood and understand the syntax iteratively. It’s like understanding a language through reverse-engineering. By seeing the code being generated based on the node maps you create you could quickly understand how the language is structured. I try to think of learning new things as adding skills to your arsenal and the more skills you possess the greater you can expand on your ideas. I don’t think it’s so much a fear of learning new technology. I think it’s more a question of ‘Will learning this new technology be worth my time and benefit my current practice?’ The beautiful age we live in makes it so much easier to learn new skills online. So there’s so much support already available for anyone wanting to explore game design technologies in their creative practice. What tools do you use in the process and why those tools? What is the actual process you follow? What are the various steps? The Augmented Reality Art I created has two parts to them: Creating the Art Everything is drawn on paper first using your typical office ballpoint pens, inkwash and sometimes watercolours or gouache. I prefer creating my work that way. To me keeping that 'hand-crafted' style and aesthetic has its charm when bringing it to life through Augmented Reality. I think it's really important to demonstrate the importance of other traditional mediums, techniques and disciplines when working with immersive technologies as they're still very valid in creating engaging content. I see a lot of AR experiences that have a lack of charm and feel somewhat 'gamey' which is why I want to continue integrating more raw-looking aesthetics in this area. Augmented Representation & Implementation Once the art is planned and created, I then scan it and start splicing it up digitally. From the preserved textures and linework I then create a 3D representation of the artwork through 3D modeling and animation software called Blender. It's one of many 3D authoring tools out there. It not only packs a significant number of features, it's also free which makes it ideal for beginners and students. Once the 3D representation of the work is created it's then imported into a game engine called Unity3D where I implement the AR functionality. Unity 3D is a widely used game engine. It's one of many engines out there but what makes it great is its support to deploy to all manners of devices. It also has a large community base behind it should you need help. How long does a typical project take? Do you work by yourself or with a team? How long do you work? On average an Augmented Artwork may take anywhere from 3 to 4 weeks to create, which includes the Art, Animation, Modeling and AR Implementation. When I first started out it'd take much longer but overtime I've streamlined my own processes to get things done faster. My alias Marc-O-Matic tends to be mistaken as a team but I'm actually just one person. I much prefer being able to create and direct everything myself. What is your advice to a budding artist who is interested in learning tech to produce art? From my experience, don't confine yourself to one specific medium and try practising a different skill or technique everyday. As artists we should be constantly evolving not just our technical skills but also our thought processes to push the boundaries on what's achievable. 'How can I tell my story in a more engaging and interesting manner?' 'What can I create if I combine these different skill sets together?" "How do I bring an end to systematic racism and white privilege?' etc. The more knowledge you have the greater you can push the boundaries of your creative expression. How do you support yourself financially? As in do you also sell your art pieces or make art for clients etc.? Where do you sell stuff, and meet people? I work under my own Artist Entity 'Marc-O-Matic.' I much prefer working independently than within larger studios or agencies. I juggle a balance between commercial work and personal projects. There's generally a high demand for Augmented/Virtual Reality experiences and I'm really lucky and grateful that clients want content in my particular style. I work with a variety of organisations generally consulting, providing creative direction and in most cases building the experiences too. Aside from the commercial work, I'm currently touring my own Augmented Reality Art/Storytelling collection called 'Moving Marvels' where audiences get to see my illustrated works come to life right in front of them. It's also how I sell various limited edition prints of my augmented artworks. My collection is exhibited at tech/innovation conferences, symposiums, galleries and even at universities/educational institutes. It's a great way to make connections and demonstrate what immersive technologies can do in a creative capacity. No interview would be complete without a discussion on AI and machine learning. As a technophile artist, are you excited about merging AI and art in new ways? What are some of your hopes and fears about this new tech? Do you plan to try machine learning anytime soon? It’s like a double edged sword with + 5 to Creativity. Technology can enhance our creative abilities and processes in a number of ways. At the same time it can also make some of us lazy because we can become so reliant on it. Like any new technology that aims to assist in the creative process, there’s always the fear that technology will make creatives lazy or will replace professions altogether. In some ways technology has done this but from saying that it has also created new job opportunities and abilities for artists. In areas like character animation for example the assistance of machine learning algorithms means creatives can worry less about the laborious physical processes of rigging and complex animation and focus more on the storytelling. For example, we can significantly reduce production times in the areas of facial rigging through facial recognition. Through learning the behaviour and structure of your own face, machine learning algorithms can map your facial movements directly to a 3D character’s face. By simply recording your own facial movements and gestures, you’ve created an entire impression map for your 3D character to use. What’s also crazy is, on top of that, by running a voice over track on that 3D character, you can also train it to play it out to sync with the voice AS well as have the entire face emotionally react based on the tone of the audio. It’s game-changing but also terrifying stuff as this sort of technology can be used to create highly realistic fake impressions of real people. As also an animator, I’ve started experimenting with this technology for my own VR animations. What would take me animator hours or even days to animate a talking 3D character can now take mere minutes. Author Bio Marc-O-Matic is the moniker of Marco Matic Ryan. He is a multidisciplinary Artist, Animator, Director, Storyteller and Technologist working across Augmented and Virtual Reality technologies. He is based in Victoria, Australia. https://player.vimeo.com/video/125218939 Unity plugins for augmented reality application development. Google ARCore is pushing immersive computing forward. There’s another player in the advertising game: augmented reality.

Mckinsey report predicts that artificial intelligence techniques including deep learning and reinforcement learning have the potential to create between $3.5 trillion and $5.8 trillion in value annually across nine business functions in 19 industries. Reinforcement learning (RL) is an increasingly popular technique for enterprises that deal with large complex problem spaces. It enables the agents to learn from their own actions and experiences. When working in an interactive environment, they use a trial and error process to find the best-optimized result. Reinforcement learning is at the cutting-edge right now and it's finally reached a point that it can be applied to real-world industrial systems. We recently interviewed Sudharsan Ravichandiran, a data scientist at param.ai, and the author of the book, Hands-On Reinforcement Learning with Python. Sudharsan takes us on an insightful journey explaining to us why reinforcement learning is trending and becoming so popular lately. He talks about the positive contributions of RL in various research fields such as gaming industry, robotics, inventory management, manufacturing, and finance. Author’s Bio Sudharsan Ravichandiran, author of the book, Hands-On Reinforcement Learning with Python is a data scientist, researcher, and YouTuber. His area of research focuses on practical implementations of deep learning and reinforcement learning, which includes natural language processing and computer vision. He used to be a freelance web developer and designer and has designed award-winning websites. He is an open source contributor and loves answering questions on Stack Overflow. You can follow his open source contributions on GitHub. Key Takeaways Reinforcement learning adoption among the community has increased exponentially because of the augmentation of reinforcement learning with state of the art deep learning algorithms. It is extensively used in the Gaming industry, robotics, Inventory management, and Finance. You can see more and more research papers and applications leading to full-fledged self-learning agents. One of the common challenges faced in RL is safe exploration. To avoid this problem, one can use imitation learning (learning from human demonstration) to provide the best-optimized solution. Deep meta reinforcement learning will be the future of artificial intelligence where we will implement artificial general intelligence (AGI) to build a single model to master a wide variety of tasks. Thus each model will be capable to perform a wide range of complex tasks. Sudharsan suggests the readers should learn to code the algorithms from scratch instead of using libraries. It will help them understand and implement complex concepts in their research work or projects far better. Full Interview Reinforcement learning is at the cutting-edge right now, with many of the world’s best researchers working on improving the core algorithms. What do you think is the reason behind RL success and why RL is getting so popular lately? Reinforcement learning has been around for many years, the reason it is so popular right now is because it is possible to augment reinforcement learning with state of the art deep learning algorithms. With deep reinforcement learning, researchers have obtained better results. Specifically, reinforcement learning started to grow on a massive scale after the reinforcement learning agent, AlphaGo, won over the world champion in a board game called AlphaGo. Also, Deep reinforcement learning algorithms help us in taking a closer step towards artificial general intelligence which is the true AI. Reinforcement learning is a pretty complex topic to wrap your head around, what got you into RL field? What keeps you motivated to keep on working on these complex research problems? I used to be a freelance web developer during my university days. I had a paper called Artificial Intelligence on my Spring semester, it really got me intrigued and made me want to explore more about the field. Later on, I got invited to Microsoft data science conference where I met many experts and learned more about the field way better. All these got me intriguing and made me to venture into AI. The one thing which motivates and keeps me excited are the advancements happening in the field of reinforcement learning lately. DeepMind and OpenAI are doing a great job and massively contributing to the RL community. Recent advancements like human-like robot hand control to manipulate physical objects with unprecedented dexterity, imagination augmented agents which can imagine and makes decisions, world models where the agents have the ability to dream excite me and keeps me going. Can you please list down 3 popular problem areas where RL is majorly used? Also, what are the 3 most current challenges faced while implementing RL in real-life? As a developer/researcher how you are gearing up to solve them? RL is predominantly used in the Gaming industry, robotics, and Inventory management. There are several challenges in Reinforcement learning. For instance, safe exploration. Reinforcement learning is basically a trial and error process where agents try several actions to find the best and optimal action. Consider an agent learning to navigate/learning to drive a car. Agents don't know which action is better unless they try them. The agent also has to be careful in not selecting actions which are harmful to others or itself, say, for example, colliding with other vehicles. To avoid this problem, we can use imitation learning or learning from a human demonstration where the agents learn directly from the human supervisor. Apart from these, there are various evolutionary strategies used to solve the challenges faced in RL. There are few positive developments in RL happened from Open AI and DeepMind team that have got widely adopted both in research and in real-world applications. What are some cutting-edge techniques you foresee getting public attention in RL in 2018 and in the near future?   Great things are happening around RL research each and every day. Deep Meta reinforcement learning will be the future of AI where we will be so close to achieving artificial general intelligence (AGI). Instead of creating different models to perform different tasks, with AGI, a single model can master a wide variety of tasks and mimics the human intelligence. Gaming and robotics or simulations are the two popular domains where reinforcement learning is extensively used. In what other domains does RL find important use cases and how? Manufacturing In manufacturing, intelligent robots are used to place objects in the right position. If it fails or succeeds in placing the object in the right position, it remembers the action and trains itself to do this with greater accuracy. The use of intelligent agents will reduce labor costs and result in better performance. Inventory management RL is extensively used in inventory management, which is a crucial business activity. Some of these activities include supply chain management, demand forecasting, and handling several warehouse operations (such as placing products in warehouses for managing space efficiently). Infrastructure management RL is also used in infrastructure management. For an instance, Google researchers in DeepMind have developed RL algorithms for efficiently reducing the energy consumption in their own data center. Finance RL is widely used in financial portfolio management, which is the process of constant redistribution of a fund into different financial products and also in predicting and trading in commercial transactions markets. JP Morgan has successfully used RL to provide better trade execution results for large orders. Your recently published ‘Hands-On Reinforcement Learning with Python‘ has received a very positive response from the readers. What are some key challenges in learning reinforcement learning and how does your book help them? One of the key challenges in learning reinforcement learning is the lack of intuitive examples and poor understanding of RL fundamentals with required math. The book addresses all the challenges by explaining all the reinforcement learning concepts from scratch and gradually takes readers to advanced concepts by exploring them one at a time. The book also explains all the required math step by step intuitively along with plenty of examples. My intention behind adding multiple examples and code to each chapter was to help the readers understand the concepts better. This will also help them in understanding when to apply a particular algorithm. This book also works as a perfect reference for beginners who are new to reinforcement learning. Are there any prerequisites needed to get the most out of the book? What do you think they should keep in mind while developing their own self-learning agents? Readers who are familiar with machine learning and Python basics can easily follow the book. The book starts with explaining reinforcement learning fundamentals and reinforcement learning algorithms with applications and then it takes the reader in understanding deep learning algorithms followed by the book explaining advanced deep reinforcement learning algorithms. While creating self-learning agents, one should be careful in designing reward and goal functions. What in your opinion are the 3-5 major takeaways from your book? The book serves as a solid go-to place for someone who wants to venture into deep reinforcement learning. The book is completely beginner friendly and takes the readers to the advanced concepts gradually. At the end of the book, the readers can master reinforcement learning, deep learning and deep reinforcement learning along with their applications in TensorFlow and all the required math. Would you like to add anything more to our readers? I would suggest the readers code the algorithms from scratch instead of using libraries, it will help them in understanding the concepts far better. I also would like to thank each and every reader for making this book a huge success. My best wishes to them for their reinforcement learning projects. If you found this interview to be interesting, make sure you check out other insightful articles on reinforcement learning: Top 5 tools for reinforcement learning This self-driving car can drive in its imagination using deep reinforcement learning Dopamine: A Tensorflow-based framework for flexible and reproducible Reinforcement Learning research by Google OpenAI builds reinforcement learning based system giving robots human like dexterity DeepCube: A new deep reinforcement learning approach solves the Rubik’s cube with no human help

Netflix, the global poster child for streamed TV and the use of Big Data to inform the programs they develop, has shown steady customer growth for several years now. Recently, the company revealed that it would be shutting down the user reviews which have been so prominent in their media catalogue interface for so long. In the background, media and telco are merging. AT&T, the telco which undertook the biggest deal in history recently, acquired Time and wants HBO to become like Netflix. Telia, a Finnish telecommunications company bought Bonnier Broadcasting in late July 2018. The video content landscape has changed a great deal in the last decade. Everyone in the entertainment game wants to move beyond broadcast TV and to use data to develop content their users will love and which will give their customer base more variety. This means they can look to data to charge higher subscription rates per user, experiment with tiered subscriptions, decide to localize global content, globalize local content and more. These changes raise two key questions. First, are we heading for a world in which AI and ML based algorithms drive what we watch on TV? And second, are the days of human recommendation being quietly replaced by machine recommendations over which the user has no control? [caption id="attachment_21726" align="aligncenter" width="1392"] As you know, Netflix is acquiring customers fast.[/caption] Source: Statista To get an insider’s view on the answer to those questions, I sat down with Matt Jones of OVO Mobile, one of Australia’s fastest growing telecommunications companies. OVO offer their customers a unique point of difference – streaming video sports content, included in a phone plan. OVO has bought the rights to a number of niche sports in Australia which weren’t previously available and now offer free OTA (Over the Air) digital content for fans of ‘unusual’ sports like Drag Racing or Gymnastics. OTA content is anything delivered to a user’s phone over a wireless network. In OVO’s case, the data used to transport the video content they provide to their users is free. That means customers don’t have to worry about paying more for mobile data so they can watch it – a key concern for users. OVO Mobile and Netflix are in very similar businesses – and Matt has a unique point of view about how Artificial Intelligence and Machine Learning will impact the world of telco and media. Key takeaways What’s changed our media consumption habits: the ubiquitous mobile internet, the always on and connected younger generation, better mobile hardware, improved network performance and capabilities, need for control over content choices. Digitization allows new features –some of which that people have proven to love - binge watching, screening out advert breaks and time shifting. The key to understanding the value of ML and AI is not in understanding the statistical or technical models that are used to enable it, it’s the way AI is used to improve the customer experience your digital customers are having with you. The use of AI in digital/app experience has changed in a way to personalize what users can see which old media could not offer. Content producers use the information they have on us, about the programs we watch, when we watch them and for how long we watch to Contribution of AI / ML towards the delivery of online media is endless in terms of personalisation, context awareness, notification management etc. Social acceptance of media delivered to users on mobile phones is what’s driving change A number of overlapping factors are driving changes in how we engage with content. Social acceptance of the internet and mobile access to it as a core part of life is one key enabler. From a technology perspective, things have changed too. Smartphones now have bigger, higher resolution screens than ever before – and they’re with us all the time. Jones believes this change is part of a cultural evolution in how we relate to technology. He says, “There has also been a generational shift which has taken place. Younger people are used to the small screen being the primary device. They’re all about control, seeking out their interests and consuming these, as opposed to previous generations which was used to mass content distribution from traditional channels like TV.” Other factors include network performance and capability which has improved dramatically in recent years. Data speeds have grown exponentially from 3G networks – launched less than 15 years ago, which could support stuttered low resolution video to 4G and 4.5G enabled networks. These can now support live streaming of High Definition TV. Mobile data allowances in plans and offers from some phone companies to provide some content ‘data free’ (as OVO does with theirs) have also driven uptake. Finally, people want convenience and digital offers that in a way people have never experienced before. Digitization allows new features –some of which that people have proven to love - binge watching, screening out advert breaks and time shifting. What part can AI / machine learning play in the delivery of media online? Artificial Intelligence (AI) is already part of 85% of our online interactions. Gartner suggest, it will be part of every product in the future. The key to understanding the value of ML and AI is not in understanding the statistical or technical models that are used to enable it, it’s the way AI is used to improve the customer experience your digital customers are having with you. When you find a new band in Spotify, when YouTube recommends a funny video you’ll like, when Amazon show you other products that you might like to consider alongside the one you just put in to your basket, that’s AI working to improve your experience. “Over The Top content is exploding. Content owners are going direct to consumer and providing fantastic experiences for their users. What’s changing is the use of AI in digital / app experiences to personalize what users see in ways old media never could.” Says Matt. Matt’s video content recommendation app, for example, ‘learns’ not just what you like to watch but also the times you are most likely to watch it. It then prompts users with a short video to entice them to watch. And the analytics available show just how effective it is. Matt’s app can be up to 5 times more successful at encouraging customers to watch his content, than those who don’t use it. “The list of ways that AI / ML contributes to the delivery of media online is endless. Personalisation, context awareness, notification management …. Endless” By offering users recommendations on content they’ll love, producers can now engage more customers for longer. Content producers use the information they have on us, about the programs we watch, when we watch them and for how long we watch to: Personalise at volume: Apps used to deliver content can personalise what’s shown first to users, based on a number of variables known about them, including the sort of context awareness that can be relatively easy to find on mobile devices. Ultimately, every AI customer experience improvement (including the examples that follow) are all designed to automate the process of providing something special to each individual that they uniquely want. Automation means that can be done at scale, with every customer treated uniquely. Notification management: AI that tracks the success of notifications and acknowledges, critically, when they are not helpful to the user, can be employed to alert users only about things they want to know. These AI solutions provide updates to users based on their preferences and avoid the provision of irrelevant information. Content discovery & Re- engagement: AI and ML can be used in the provision of recommendations as to what users could watch, which expose customers to content they would not otherwise find, but which they are likely to value. Better / more relevant advertising: Advertising which targets a legitimately held, real, customer need is actually useful to viewers. Better analytics for AI can assist in targeting micro segments with ads which contain information customers will value. Lattice, is a Business Insights tool provider. Their ‘Lattice Engine’ product combined information held in multiple cloud based locations and uses AI to automatically assign customers to a segment which suits them. Those data are then provided to a customer’s eCommerce site and other channel interactions, and used to offer content which will help them convert better. Developing better segments: Raw data on real customers can be gathered from digital content systems to inform Above The Line marketing in the real, non digital world. Big data analytics can now be used with accurate segmentation for local area marketing and to tie together digital and retail customer experiences. McKinsey suggest that 36% of companies are actively pursuing strategies, driven from their Big Data reserves. They advise their clients that Big Data can be used to better understand and grow Customer Lifetime Values. In the future - Deep linking for calls-to-action: Some digital content is provided in a form such that viewers can find out more information about an item on screen. Providing a way to deep link from a video screen in to a shopping cart prepopulated with something just seen on screen is an exciting possibility for the future. Cutting steps out of the buying process to make it easier for eCommerce users to transact from within content apps to buying a product they’ve seen on the screen is likely to become a big business. Deep linking raises the value of the content shown to the degree it raise the sales of the products included. Bringing it all together Jones believes those that invest big in AI and machine learning, and of them, those who find a way to draw out insights and act upon them, will be the ultimate victors. “The big winners are going to be the people who connect a fan with content they love and use AI and ML to deliver the best possible experience. It’s about using all the information you have about your users and acting on them.” Said Jones. That commercial incentive is already driving behavior. AI and ML drive already provide personalized content recommendations. Progressive content companies, including Matt’s, are already working on building AI in to every facet of every Digital experience you have. As to whether AI is entirely replacing social media influence, I don’t think that’s the case. The research says people are still 4 times more likely to watch a video if it is recommended to them by a friend. Reviews have always been important to presales on the internet and that applies to TV shows, too. People want to know what real users felt when they used a product. If they can’t get reviews from Netflix, they will simply open a new tab and google for reviews in that while they are thinking of how to find something to watch on Netflix. About Matt Jones, Matt is an industry disruptor, launching the first of its kind Media and Telco brand OVO Mobile in 2015, Matt is the driving force behind convergence of new media & telco – by bringing together Telecommunications with Media Rights and digital broadcast for mass distribution. OVO is a new type of Telco, delivering content that fans are passionate about, streamed live on their mobile or tablet UNLIMITED & data free. OVO has secured exclusive 3 year+ digital broadcast and distribution rights for a range of content owners including Supercars, World Superbikes, 400 Thunder Drag Series, Audi Australia Racing & Gymnastics Australia – with a combined Australian audience estimated at over 7 Million. OVO is a multi-award winner, including winning the Money Magazine Best of the Best Award 2017 for high usage, as well as featuring on A Current Affair, Sunrise, The Today Show, Channel 7 News, Channel 9 News and multiple radio shows for their world-first kids’ mobile phone plan with built-in cyber security protection. As OVO CEO, Matt was nominated for Start-Up Executive of the Year at the CEO Magazine Awards 2017 and was awarded runner-up. The Award recognises the achievements of leaders and professionals, and the contributions they have made to their companies across industry-specific categories. Matt holds a Bachelor of Arts (BA) from the University of Tasmania and regularly speaks at Telco, Sports Marketing and Media forums and events. Matt has held executive leadership roles at leading Telecommunications brands including Telstra (Head of Strategy – Operations), Optus, Vodafone, AAPT, Telecom New Zealand as well as global Management Consulting firms including BearingPoint. Matt lives on the northern beaches of Sydney with his wife Mel and daughters Charlotte and Lucy. How to earn $1m per year? Hint: Learn machine learning We must change how we think about AI, urge AI founding fathers Alarming ways governments are using surveillance tech to watch you

Too often, as designers and developers we fail to make interfaces that are usable, fail to make software that is intuitive, and fail to make products that normal people can understand. By coating design rigour with a layer of brand fluff, and putting form over function again and again, we build products that serve nobody but the internal needs of our corporations and brands. In this interview with Will Grant, a web technology entrepreneur and veteran, we discuss ways to solve 101 UX design problems clearly and single-mindedly. We also discuss about his upcoming book 101 UX Principles, in which Will has defined and refined what it means to build products people intuitively know how to use. Author’s Bio Will Grant is a British UI/UX expert and graduate of Birmingham City University, where he studied human computer interaction and usable design. Following his degree, he trained with Jakob Nielsen and Bruce Tognazzini, pioneers in UX design. Will has been building intuitive usable software products since the birth of the consumer web over 20 years ago, through to the present day, where Will's work has reached more than a billion users. He is the co-founder and the design lead at UX-focused analytics tool Prodlytic. Key takeaways: The vast majority of UX is still about concepts, journey and the tasks we help users to achieve. The tools to deliver great UX have changed, but UX is still about familiarity, consistency and empathy. The 101 UX Principles are a shortcut for UX professionals. Designers can apply them to their products and make usable software 99% of the time for 99% of users. Over reliance on ‘brand’ and internal goals, trying to reinvent the wheel, and forgetting to put oneself in the place of the user are some common reasons why UX design fails. Many UX people forget that design – UI design in particular – isn’t art, it’s design to perform a function: to serve users. Follow Will’s 10 commandments for effective UX design to create more usable and successful products. There’s another 91 in the book 101 UX Principles too. Full Interview Of the 100+ UX design principles that you explore in your new book, if we asked you to pick the top 10, what would those be? Will’s 10 commandments for effective UX design, so to speak. Test with real users Don’t join the dark side Make your buttons look like buttons Label your icon buttons Use 2 font families, maximum Make ‘blank slates’ more than just empty views Hide ‘advanced’ settings from most users Decide if an interaction should be obvious, easy, or possible Anyone can be a UX professional Use device-native input features where possible Just following these 10 and applying them to your software design will create more usable, successful products. There’s another 91 great commandments in the book too. Will, as this book is about 101 UX Principles, what makes your principles right? Nothing is perfect, but these principles are a ‘shortcut’ for UX professionals. Instead of reinventing the wheel, designers can apply these principles to their products and make usable software 99% of the time for 99% of users. I’ve spent over 20 years, since the birth of the consumer web, building interfaces for 100s of products and over a billion users. My approach isn’t perfect, but it has been tested and proven to work at scale. This guide will help you avoid common mistakes and start with a product that’s extremely usable and intuitive - for the widest possible section of users. Why do people keep making UX mistakes? It’s usually a combination of factors; over reliance on ‘brand’ and internal goals, trying to reinvent the wheel, and forgetting to put yourself in the place of the user. Too often the internal goals of an organisation supercede the design teams who are genuinely trying to ‘fight for the user’. The CEO wants it to look a certain way (but he/she has no design background), or the marketing team decide that a certain typeface has to be used (even though it’s unreadable). The paradox is that, as UX and UI people, we’re over-exposed to components, controls, patterns and interfaces in general. It’s the curse of knowledge and we are the last people who should be designing interfaces — unless we can do the hard bit: objectivity. Name a big company that gets UX right, and one that gets it wrong This is impossible, even today after 20+ years of consumer web products, the experience people see is wildly different from product to product - regardless of the company. Generally, large companies with lots of internal bureaucracy and hierarchy produce end products that are the least usable - this is where small, nimble startups can often produce a better product: not because they are ‘better’ overall, but because they haven’t yet lost sight of the importance of UX. And, crucially, startup teams are less encumbered by legacy baggage and are more free to follow best practice in design. Who inspires you the most with the UX community? Donald Norman & Jakob Nielsen have both been hugely influential to me. Don Norman’s book “The design of everyday things”  pretty much kicked off and ‘invented’ the whole field of human-computer interaction, which these days we call ‘UX’. Nielsen & Norman are sometimes derided as ‘too purist’ but that’s what appeals to me most. Stripping back interfaces to the bare minimum, removing clutter and making things simple are things I try to do in my work every day. I worked for a boss in my early 20’s - he wasn’t a designer - but he did fly into an apoplectic rage at the slightest mistake I might make. It taught me to check, check and re-check my designs and despite him being a horrible person, my work is better for it. What was the last app that made you throw down your phone in frustration? Easy - it was the HSBC app, yesterday, with it’s dreadful ‘update’ process. Apple have gone to great lengths to build an App Store which auto-updates your apps, in the background while you’re asleep and your phone is on charge. HSBC decided that their banking app should do its own half-assed updates, whenever it feels like it, inside the app - just when you open the app and you’re about to use it. A classic example of reinventing the wheel, building a new experience that fails because nobody has thought of the user - only of their internal needs. In your more than two decades of UX design experience, how has the web evolved from a user experience perspective? What were some of the biggest surprises in UX design trends for you? What design ideas have remained unaltered by time? I think it’s remarkable how little has changed - in terms of design ideas that ‘just work’ at least. Yes, software has changed massively over that time - from basic websites and browsers on desktop computers through to web app and native apps on smartphones and tablets. However, the vast majority of UX is still about the concepts, the journey and the tasks you’re helping the user to achieve. The tools to deliver great UX have changed, but UX itself is still about familiarity, consistency and empathy. With emerging technologies like machine learning, AR, VR, IoT etc increasingly impacting how we design for the web, where do you see UX design heading in the coming years? What are some general rules worth keeping in mind when designing for the future? What are some opportunities and challenges you foresee for UX designers? It's more of a hope than a prediction, but perhaps us designers will stop doing things because we can and start asking if we should. A greater sense of social responsibility, and a reduction in sneaky 'dark pattern' UX would be great for everyone. Somewhere along the way, many UX people forgot that design – UI design in particular – isn’t art, it’s design to perform a function: to serve users. Too many designers are slavishly following the latest design trend, applying ‘flat design’ to every app, or trying to be different for the sake of it, with custom-designed interfaces and arbitrary visual metaphors. The solution is simple, too: try and be objective, fight for the user, and test with real users as you go. 101 UX Principles provides 101 ways to solve 101 UX problems clearly and single-mindedly. There are 1000s of methods to apply to each and every interaction in your product, but this book is a ‘shortcut’ to a method that works. The book is available to pre-order now and is expected to be published soon. What UX designers can teach Machine Learning Engineers? To start with: Model Interpretability Is your web design responsive? A UX strategy is worthless without a solid usability test plan    

Odoo is one of the fastest growing open source, business application development software products available. It comes with: Powerful GUI, Performance optimization, Integrated in-app purchase features Fast-growing community to transform and modernize businesses We recently interviewed Yenthe Van Ginneken, an Odoo developer, highly active in the Odoo community and recipient of Odoo best contributor of the year 2016 and Odoo community hero 2017. He spoke to us about his journey with Odoo, his thoughts on Odoo’s past, present and future, and the Odoo community. Expert's Bio Yenthe Van Ginneken, currently the technical team leader at Odoo Experts, has been an Odoo developer for over four years. He has won two awards, “Best contributor of the year 2016” and the “Odoo community hero” award in 2017. He loves improving software and teaching other people the best practices for Odoo development on his blog. You can read his Odoo blog, follow him on Twitter or reach out to him on LinkedIn. Key Takeaways Odoo is scalable and flexible to the extent that everyone, from a small startup to a giant tech company can benefit from it. It is ahead of quite a lot of ERP systems with its clean UI, advanced modules integration and the flexibility of its technical framework. Python is the preferred language of choice among most developers that want to use the Odoo framework, especially for automating and scaling tasks. The Odoo community is diverse and vast. By contributing and regularly interacting with other members, you will gain deeper insights into many different aspects of Odoo development. A great way to learn to develop in Odoo and quickly grow is actually by helping in the community. Odoo 12 will reportedly improve data processing, better report insights, and support for OCR (Optical Character Recognition) for handling documents among other exciting updates. Full Interview On who should use Odoo Odoo is more than an ERP tool. According to you, What is Odoo? Who will benefit from adopting Odoo? What made you choose Odoo?   For me, Odoo is more than an ERP. Odoo literally allows me to make any module or functionality that I can think of. Since Odoo is so flexible and scalable I believe that almost everybody can benefit from adopting Odoo. Whether you’re a small start-up or a giant tech company. The most important part to be able to benefit from adopting Odoo is adjusting the processes and mindset to use Odoo, not adjusting Odoo for the company. The projects that work the best and have the best benefit are those that don’t over-engineer and try to focus on the main company processes. I personally chose Odoo after I got an opportunity to become an Odoo developer at a company in Belgium. After the job offer I visited Odoo.com and saw the massive amount of functionalities in Odoo (while being free!) and I was genuinely amazed. After looking at the technical framework and all the default options provided by the framework I was sure that I would love to develop and implement in Odoo. Since that day I never stopped working with Odoo. On journey from OpenERP to Odoo Odoo started off as OpenERP and then in 2014, it moved beyond just ERP and was renamed Odoo. How has Odoo’s journey been so far since then? What do you think are the key milestones achieved by Odoo till date? Since the renaming from OpenERP to Odoo the company has seen a rapid growth. A bit after changing the name Odoo also introduced the enterprise version which was, in my opinion, the turning point for Odoo S.A. It allowed Odoo to keep its open source strength and market share while also gathering funds to fund the ongoing growth of the product. The big investments that are being made in the Research and Development team allow them to keep improving year after year. The main strengths and key milestones from Odoo are absolutely its flexibility, a great framework and the fact that most of the possibilities are already in Odoo by default. On the drive behind contributing to the Odoo community You are highly active in the Odoo community. How did you get into contributing for Odoo? How has this experience improved you as a developer? According to you, what are the key challenges the Odoo community is facing currently? My very first contribution started in the second half of 2014 and weren’t very significant at first. I noticed that Odoo 8, at that point the newest version, was not very well translated and had a lot of inconsistency so I started translating it in Dutch. From there on I noticed that it could have had quite a big impact and in fact could improve the ERP. It didn’t take long before I started contributing in other ways. Reporting issues, fixing bugs, maintaining bug reports and helping other people on the official help forums. By contributing to all these different subjects I got introduced to more domains and gained more insights. Thanks to my involvement with the community, I’ve learned that there is more than one side to developing and implementing projects. I believe it made me a better programmer and made me think a lot more about ways to code custom development for projects. Without being active in a community and contributing you’ll be blindsided by your own perspective. It is a great way to get challenged and you’ll see more cases by being active in the community than you could ever see on your own. The Odoo community faces a few challenges at this point. It is difficult to maintain the right balance between the enterprise version and community (free) version. There are not a lot of very active contributors to the official Odoo code and Odoo is behind on handling fixes/bug reports made by community members. This results in some community members not feeling appreciated or heard. Hiring a second community manager might be a good way to resolve these issues though. The most difficult challenge for both Odoo and the Odoo community is to make everybody feel heard and give every person the ability to contribute in the way he or she can. When there is enough help from Odoo and the community feels supported there is a possibility for a great and thriving community. On how to learn Odoo effectively As a person who has a strong hold over Odoo development, what is the typical learning curve for someone getting into Odoo, as a consultant? What is the best way to start developing in Odoo? What should one watch out for while learning? The learning curve can be quite long and can have its challenges. Usually, if you don’t have any experience with Odoo and only know basic Python it’ll take about six months before you really get to know the ins and outs of Odoo. The best way to learn to develop Odoo is probably the same as with most things in technology: dive in! Make sure you get the basics right and understand how the main functionalities work before going deeper. A great way to learn to develop in Odoo and to quickly grow is actually by helping in the community. You can get insight and help from experienced developers while also contributing to the community, it’s a win-win. Start small and build your way up to the details. It is important to find good documentation and tutorials though. At the moment there are still quite some blog posts and tutorials that are from quite a low quality. Because of this I actually started writing my own tutorials, which explain concepts step by step with samples. You can find it at https://odoo.yenthevg.com Editor’s note: Check out our collection of Odoo Books and Videos to master Odoo development. On the upcoming Odoo 12 release Odoo 12 is expected to be released later this year. What’s got you excited about this new release? Quite a lot! Every release has loads of new features that are announced and it’s an exciting time, every time. The introduction of a report designer for functional people is one of the best (known) new features. The improved reporting tools for data insight will become a great improvement too. The biggest announcements are made at Odoo Experience in October and are not publicly available yet so we’ll have to wait for that. On the future of ERP There is a lot happening in the area of ERP and BI: self-service analytics, real-time analytics, agile BI development etc. Where do you foresee the ERP market headed? We've seen ERP/CRM systems getting powerful inbuilt analytics systems, what do you think is next for the industry? What is Odoo’s role here? As with any sector in IT, a lot is becoming very data-driven. In the future integration and usage of data will only grow. I expect the combination of BI and AI to become a powerful way to process and handle data on unseen scales. Odoo itself has already hinted at improved data processing, better report insights and support for OCR (Optical Character Recognition) for handling documents. Odoo has been ahead of quite a lot of ERP systems with its clean UI, advanced modules integration and the flexibility of its technical framework for years. I expect Odoo will also be leading the way for handling all this data and getting important statistics out of it. I’m quite sure it is only a matter of time before Odoo starts working on even better BI reporting and tools. On Python and automation Automation is everywhere today and becoming an integral part of organizations and processes. Python and automation have gone hand in hand since Python’s early days. Today Python is one of the top programming languages. How do you see Python’s evolution over the years in the area of automation? What are the top ways you use Python for automation, today? It is for a reason that Python is so popular. It is flexible, quite quick to program with and the options are virtually endless. In the next years, Python will only become more popular and this will also be the case for automation projects made with Python. I personally use the Odoo framework with Python as a backbone for nearly everything that I automate (and in fact also for non-automated tasks). The projects vary from automatically handling stock moves to automatically updating remote instances to automatically getting full diagnostic reports. The combination of the programming language and the framework from Odoo allows me to automate tasks and deploy them on a big scale. ERP tool in focus: Odoo 11 How to Scaffold a New module in Odoo 11 A step by step guide to creating Odoo Addon Modules

Despite considerable hype, chaos engineering doesn’t appear to have yet completely captured the imagination of the wider software engineering world. According to this year’s Skill Up survey, when asked, only 13% of developers said they were excited about it. But that doesn’t mean we should disregard - far from it. Like many of the best trends, it might blow up when we least expect. It might find its way onto your CTOs eyes in just a few months. As site reliability engineering grows as a discipline, and as businesses start to put a value on downtime, chaos engineering is likely to become a big part of the reliability and resilience toolkit. Gremlin, chaos engineering, and the end of the age of downtime “People are expected to always be up” says Matt Fornaciari, co-founder and CTO of Gremlin, a product that offers “failure as a service” to businesses. I spoke to Fornaciari last month to get a deeper insight on Gremlin and the team and ideas behind it. He believes the world has changed in recent years, and the days of service windows when sites would just be taken down for an hour or two for an update or change is over: “that’s unacceptable to people these days.” Fornaciari isn’t an unbiased observer, of course. The success of Gremlin depends on chaos engineering’s adoption and acceptance. However, he’s not going out on a limb; there’s clear VC interest in Gremlin. At the end of 2017 the company received their first round of funding - more than 7 million USD. It’s a cliche but money does talk - and in this instance it seems to be saying that this approach might change the way we think about building our software. Arguably, chaos engineering - and by extension Gremlin - is a response to other trends in software. “I’ve seen a lot of signals that this is the way the world’s going”, Fornaciari says. He’s referring here to broader trends like cloud and microservices. He explains that because microservices is all about modularity, and breaking aspects of your software infrastructure into smaller pieces “you end up with nodes in this network” which “adds network complexity.” Consequently, this additional complexity means there is more that can go wrong - it becomes more unreliable. Gremlin’s bid to democratize chaos engineering It’s important to note here that chaos engineering has been around for some time - it’s not a radically new methodology. But it’s largely been locked away in some of the world’s biggest tech companies, like Netflix and Amazon. Many of Gremlin’s leaders actually worked at those companies - Fornaciari has worked at Salesforce and Amazon, for example. “The main goal was to democratize chaos engineering… we’ve [the Gremlin team] done it at the bigger companies and we’re like you know what, everyone can benefit from this”. That is the essential point around chaos engineering. If it’s going to catch on in the mainstream tech world, it needs to be more accessible to different businesses. Fornaciari explains that many of Gremlin’s customers are larger organizations. These are companies for whom downtime is of utmost importance, where a site outage that lasts just an hour could cost thousands of dollars. That said, from a cultural perspective, many organizations find it difficult to adopt this sort of mindset. “Proving the value of something that doesn’t happen,” Fornaciari says, is one of the biggest challenges for Gremlin. This is particularly true when selling their tool. Pager pain: How Gremlin sells chaos engineering to customers This is how Gremlin does it: “We have three qualifying questions: do you measure your downtime? Do you have somebody who’s responsible for downtime? And do you actually have a dollar amount tied to it?” Presumably, for many organizations at least one answer to these questions is “no”. That’s why customer support is so important for Gremlin. “Customer success and developer advocacy are two of our biggest initiatives… I’ve told people as we’re recruiting them that half of our goal as a company is to educate people.” Gremlin’s challenges as a product and as a business reflect the wider difficulties of managing upwards. The tension between those ‘on the ground’ and those at a more senior and managerial level is one that Gremlin is acutely aware of. This is where a lot of push back comes from, Fornaciari explains: What we’ve seen so far is just push back from top down - like, why do we need this? We use the term pager pain to define the engineer on call - the closer you are to the ground the closer you are to the on call rotation and the more you feel those pains and the more you believe in this but as you raise up a couple of levels you maybe don’t feel that as much… if you don’t have that measure on uptime - unless someone is on the hook for that at a higher level there’s oftentimes a why do we need this, why are we going to spend money on breaking things. Pager pain is a nice concept - it captures the tension between different layers of management. It highlights the conflict between ‘what do we need?’ and ‘what can we do?’ Read next: Blockchain can solve tech's trust issues  Safety, simplicity and security To successfully sell Gremlin, the way the product is designed is everything. For that reason, the Gremlin team have three tenets built into their product: safety, security, simplicity. When you’ve got a “potentially dangerous tool,” as Fornaciari himself describes it, making sure things are safe and secure is absolutely essential. Arguably, the fact that chaos engineering is so hard to do well might be something that Gremlin can use to its advantage. “One thing we hear when we talk to companies about it is ‘well we’ll go build this ourselves’ and the fact is it’s a really hard thing to do, and a hard thing to do well.” Gremlin is walking on a bit of a tightrope. On the one hand chaos engineering is for everyone, but on the other it’s difficult and dangerous. It should be accessible, but not too accessible. “One of the reasons we don’t have a free offering is because we are a little worried about protecting our customers not doing any harm to people… I mean, this is essentially giving somebody a potentially dangerous tool.. If they’re not given the proper education then that could be a problem, right?” Gremlin aren’t the only chaos engineering product out there. As with any trend, there are plenty of software platforms and tools emerging for technologically forward thinking businesses. Fornaciari doesn’t see these as a threat - he’s confident, bullish even, about Gremlin’s place in the market. “There are a lot of tools out there that people can go and use but they really lack the safety and simplicity.” Alongside its philosophy of safety, security and simplicity, a big selling point, according to Fornaciari, is the experience and expertise that is built into Gremlin’s DNA. “We’ve got fifteen years of combined expertise in this space” he says. “Being the experts on it and having built it 3 or 4 times already in different big companies, it sort of gave us this leg up to go out there in the world.” But while Fornaciari is eager to assert Gremlin’s knowledge, there’s no trace of elitism - sharing knowledge is a core part of the product offering. “We actually built out customer success tooling so we can see if particular attacks fail for them we can actually proactively reach out and be like ‘hey we saw you were trying to do this, maybe you meant to do this’” Fornaciari explains. Controlled chaos: chaos engineering and the scientific method Control is central to Gremlin’s philosophy - it’s a combination of the team’s commitment to safety, security and simplicity. In fact, this element of control that distinguishes chaos engineering today, from what went before. Central to Gremlin’s mission to make chaos engineering accessible, is also redefining how it’s done. “If you’re familiar with the netflix chaos monkey mentality of randomly terminating services, well that’s a good start, but safety is really lacking. We talked more about this controlled chaos… this idea that you start fairly small with this small blast radius and then as you become more confident you grow it out and grow it out as opposed to just like ‘cool, let’s just chuck a grenade in here and see what happens.’” Fornaciari goes on to describe this ‘controlled chaos’ in a surprising way. “It’s much more like the scientific method actually. Applying that method to your infrastructure and your reliability in general.” This approach is essential if you’re going to do chaos engineering well. How to do chaos engineering effectively When I ask Fornaciari how engineering teams and businesses can do chaos engineering well he emphasizes the importance of starting with a hypothesis: “You need to have a hypothesis that you’re trying to prove.Throwing random chaos at something is fine - it’ll sort of surface some of the unknown unknowns for you. But really having a hypothesis that you’re trying to prove is the best way to get value out of this [chaos engineering].” If you’re going to take a scientific approach to testing your infrastructure using ‘chaos experiments’, managing scale is also incredibly important. Don’t run before you can walk is the message. “Keep it very small initially, then you start to grow the blast radius. You definitely want to make sure that you’re starting off with the smallest modicum that you can.” Given the potential dangers of throwing metaphorical gremlins into your system, starting where your comfortable makes a lot of sense. “Start in staging, start where your comfortable, build your confidence. Make sure your system behaves well in front of non-customer facing traffic before you go out to the world.” That said, Gremlin have had “some pretty bold customers” who go straight ahead and start running chaos experiments in production. “That was cool. It’s a little scary, but they were confident and they’ve been using Gremlin as part of their system ever since.” Chaos engineering requires confidence and control Ultimately, if chaos engineering is going to take off - as Fornaciari believes it will - engineers will need to be incredibly confident. That’s true on a number of levels. You need confidence that you’ll be able to handle a range of experiments and deploy them wisely. But you’ll also need confidence that you can manage the expectations of those in senior management. It’s not hard to see the value of chaos engineering. As Fornaciari says “if you prevent one outage one time, you’ve saved that money to pay for the tool to make sure it doesn’t happen again.” But it might be hard to find time for it. It might be hard to get buy in and investment in the tools you need to do it. Gremlin are certainly going to play an important part in helping engineers do that. But one of its biggest challenges - and perhaps one of its most noble missions too - is transforming a culture where people don’t really appreciate ‘pager pain’. If Fornaciari and Gremlin can help solve that, good luck to them. You can follow Matt Fornaciari on Twitter: @callmeforni

The hype around blockchain has now reached fever pitch. Now the Bitcoin bubble has all but burst, it would seem that the tech world - and beyond - is starting to think more creatively about how blockchain can be applied. We've started to see blockchain being applied in a huge range of areas; that's likely to grow over the next year or so. We certainly weren't surprised to see blockchain rated highly by many developers working in a variety of fields in this year's Skill Up survey. Around 70% of all respondents believe that blockchain is going to prove to be revolutionary. Read the Skill Up report in full. Sign up to our weekly newsletter and download the PDF for free. To help us make sense of the global enthusiasm and hype for blockchain, we spoke to blockchain expert Imran Bashir. Imran is the author of Mastering blockchain, so we thought he could offer some useful insights into where blockchain is going next. He didn't disappoint. Respondents to the Skill Up survey said that blockchain would be revolutionary. Do you agree? Why? I agree. The fundamental issue that blockchain solves is that of trust. It enables two or more mutually distrusting parties to transact with each other without the need of establishing trust and a trusted third party. This phenomenon alone is enough to start a revolution. Generally, we perform transactions in a centralised and trusted environment, which is a norm and works reasonably well but think about a system where you do not need trust or a central trusted third party to do business. This paradigm fundamentally changes the way we conduct business and results in significant improvements such as cost saving, security and transparency. Why should developers learn blockchain? Do you think blockchain technology is something the average developer should be learning? Why? Any developer should learn blockchain technology because in the next year or so there will be a high demand for skilled blockchain developers/engineers. Even now there are many unfilled jobs, it is said that there are 14 jobs open for every blockchain developer. The future will be built on blockchain; every developer/technologist should strive to learn it. What most excites you about blockchain technology? It is the concept of decentralisation and its application in almost every industry ranging from finance and government to medical and law. We will see applications of this technology everywhere. It will change our lives; just the way Internet did in the 1990s. Also, smart contracts constitute a significant part of blockchain technology, and it allows you to implement Contracts that are automatically executable an enforceable. This ability of blockchain allows you drastically reduce the amount of time it takes for contract enforcement and eliminates the need for third parties and manual processes that can take a long time to come into action. Enforcement in the real world takes a long time, in blockchain world, it is reduced to few minutes, if not seconds, depending on the application and requirements. What tools do you need to learn to take advantage of blockchain? What tools do you think are essential to master in order to take advantage of blockchain? Currently, I think there are some options available. blockchain platforms such as Ethereum and Hyperledger fabric are the most commonly used for development. As such, developers should focus on at least one of these platforms. It is best to start with necessary tools and features available in a blockchain, and once you have mastered the concepts, you can move to using frameworks and APIs, which will ease the development and deployment of decentralised applications. What do you think will be the most important thing for developers to learn in the next 12 months? Learn blockchain technology and at least one related platform. Also explore how to implement business solutions using blockchain which results in bringing about benefits of blockchain such as security, cost-saving and transparency. Thanks for taking the time to talk to us Imran! You can find Imran's book on the Packt store.