Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon

Tech Guides

852 Articles
article-image-8-machine-learning-best-practices
Melisha Dsouza
02 Sep 2018
9 min read
Save for later

8 Machine learning best practices [Tutorial]

Melisha Dsouza
02 Sep 2018
9 min read
Machine Learning introduces a huge potential to reduce costs and generate new revenue in an enterprise. Application of machine learning effectively helps in solving practical problems smartly within an organization. Machine learning automates tasks that would otherwise need to be performed by a live agent. It has made drastic improvements in the past few years, but many a time, a machine needs the assistance of a human to complete its task. This is why it is necessary for organizations to learn best practices in machine learning which you will learn in this article today. This article is an excerpt from a book written by Chiheb Chebbi titled Mastering Machine Learning for Penetration Testing Feature engineering in machine learning Feature engineering and feature selection are essential to every modern data science product, especially machine learning based projects. According to research, over 50% of the time spent building the model is occupied by cleaning, processing, and selecting the data required to train the model. It is your responsibility to design, represent, and select the features. Most machine learning algorithms cannot work on raw data. They are not smart enough to do so. Thus, feature engineering is needed, to transform data in its raw status into data that can be understood and consumed by algorithms. Professor Andrew Ng once said: "Coming up with features is difficult, time-consuming, requires expert knowledge. 'Applied machine learning' is basically feature engineering." Feature engineering is a process in the data preparation phase, according to the cross-industry standard process for data mining: The term Feature Engineering itself is not a formally defined term. It groups together all of the tasks for designing features to build intelligent systems. It plays an important role in the system. If you check data science competitions, I bet you have noticed that the competitors all use the same algorithms, but the winners perform the best feature engineering. If you want to enhance your data science and machine learning skills, I highly recommend that you visit and compete at www.kaggle.com: When searching for machine learning resources, you will face many different terminologies. To avoid any confusion, we need to distinguish between feature selection and feature engineering. Feature engineering transforms raw data into suitable features, while feature selection extracts necessary features from the engineered data. Featuring engineering is selecting the subset of all features, without including redundant or irrelevant features. Machine learning best practices Feature engineering enhances the performance of our machine learning system. We discuss some tips and best practices to build robust intelligent systems. Let's explore some of the best practices in the different aspects of machine learning projects. Information security datasets Data is a vital part of every machine learning model. To train models, we need to feed them datasets. While reading the earlier chapters, you will have noticed that to build an accurate and efficient machine learning model, you need a huge volume of data, even after cleaning data. Big companies with great amounts of available data use their internal datasets to build models, but small organizations, like startups, often struggle to acquire such a volume of data. International rules and regulations are making the mission harder because data privacy is an important aspect of information security. Every modern business must protect its users' data. To solve this problem, many institutions and organizations are delivering publicly available datasets, so that others can download them and build their models for educational or commercial use. Some information security datasets are as follows: The Controller Area Network (CAN) dataset for intrusion detection (OTIDS): http://ocslab.hksecurity.net/Dataset/CAN-intrusion-dataset The car-hacking dataset for intrusion detection: http://ocslab.hksecurity.net/Datasets/CAN-intrusion-dataset The web-hacking dataset for cyber criminal profiling: http://ocslab.hksecurity.net/Datasets/web-hacking-profiling The API-based malware detection system (APIMDS) dataset: http://ocslab.hksecurity.net/apimds-dataset The intrusion detection evaluation dataset (CICIDS2017): http://www.unb.ca/cic/datasets/ids-2017.html The Tor-nonTor dataset: http://www.unb.ca/cic/datasets/tor.html The Android adware and general malware dataset: http://www.unb.ca/cic/datasets/android-adware.html Use Project Jupyter The Jupyter Notebook is an open source web application used to create and share coding documents. I highly recommend it, especially for novice data scientists, for many reasons. It will give you the ability to code and visualize output directly. It is great for discovering and playing with data; exploring data is an important step to building machine learning models. Jupyter's official website is http://jupyter.org/: To install it using pip, simply type the following: python -m pip install --upgrade pip python -m pip install jupyter Speed up training with GPUs As you know, even with good feature engineering, training in machine learning is computationally expensive. The quickest way to train learning algorithms is to use graphics processing units (GPUs). Generally, though not in all cases, using GPUs is a wise decision for training models. In order to overcome CPU performance bottlenecks, the gather/scatter GPU architecture is best, performing parallel operations to speed up computing. TensorFlow supports the use of GPUs to train machine learning models. Hence, the devices are represented as strings; following is an example: "/device:GPU:0" : Your device GPU "/device:GPU:1" : 2nd GPU device on your Machine To use a GPU device in TensorFlow, you can add the following line: with tf.device('/device:GPU:0'): <What to Do Here> You can use a single GPU or multiple GPUs. Don't forget to install the CUDA toolkit, by using the following commands: Wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.44-1_amd64.deb" sudo dpkg -i cuda-repo-ubuntu1604_8.0.44-1_amd64.deb sudo apt-get update sudo apt-get install cuda Install cuDNN as follows: sudo tar -xvf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local export PATH=/usr/local/cuda/bin:$PATH export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64" export CUDA_HOME=/usr/local/cuda Selecting models and learning curves To improve the performance of machine learning models, there are many hyper parameters to adjust. The more data that is used, the more errors that can happen. To work on these parameters, there is a method called GridSearchCV. It performs searches on predefined parameter values, through iterations. GridSearchCV uses the score() function, by default. To use it in scikit-learn, import it by using this line: from sklearn.grid_search import GridSearchCV Learning curves are used to understand the performance of a machine learning model. To use a learning curve in scikit-learn, import it to your Python project, as follows: from sklearn.learning_curve import learning_curve Machine learning architecture In the real world, data scientists do not find data to be as clean as the publicly available datasets. Real world data is stored by different means, and the data itself is shaped in different categories. Thus, machine learning practitioners need to build their own systems and pipelines to achieve their goals and train the models. A typical machine learning project respects the following architecture: Coding Good coding skills are very important to data science and machine learning. In addition to using effective linear algebra, statistics, and mathematics, data scientists should learn how to code properly. As a data scientist, you can choose from many programming languages, like Python, R, Java, and so on. Respecting coding's best practices is very helpful and highly recommended. Writing elegant, clean, and understandable code can be done through these tips: Comments are very important to understandable code. So, don't forget to comment your code, all of the time. Choose the right names for variables, functions, methods, packages, and modules. Use four spaces per indentation level. Structure your repository properly. Follow common style guidelines. If you use Python, you can follow this great aphorism, called the The Zen of Python, written by the legend, Tim Peters: "Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!" Data handling Good data handling leads to successfully building machine learning projects. After loading a dataset, please make sure that all of the data has loaded properly, and that the reading process is performing correctly. After performing any operation on the dataset, check over the resulting dataset. Business contexts An intelligent system is highly connected to business aspects because, after all, you are using data science and machine learning to solve a business issue or to build a commercial product, or for getting useful insights from the data that is acquired, to make good decisions. Identifying the right problems and asking the right questions are important when building your machine learning model, in order to solve business issues. In this tutorial, we had a look at somes tips and best practices to build intelligent systems using Machine Learning. To become a master at penetration testing using machine learning with Python,  check out this book  Mastering Machine Learning for Penetration Testing Why TensorFlow always tops machine learning and artificial intelligence tool surveys Intelligent Edge Analytics: 7 ways machine learning is driving edge computing adoption in 2018 Tackle trolls with Machine Learning bots: Filtering out inappropriate content just got easy
Read more
  • 0
  • 0
  • 11382

article-image-8-ways-artificial-intelligence-can-improve-devops
Prasad Ramesh
01 Sep 2018
6 min read
Save for later

8 ways Artificial Intelligence can improve DevOps

Prasad Ramesh
01 Sep 2018
6 min read
DevOps combines development and operations in an agile manner. ITOps refers to network infrastructure, computer operations, and device management. AIOps is artificial intelligence applied to ITOps, a term coined by Gartner. Makes us wonder what AI applied to DevOps would look like. Currently, there are some problem areas in DevOps that mainly revolve around data. Namely, accessing the large pool of data, taking actions on it, managing alerts etc. Moreover, there are errors caused by human intervention. AI works heavily with data and can help improve DevOps in numerous ways. Before we get into how AI can improve DevOps, let’s take a look at some of the problem areas in DevOps today. The trouble with DevOps Human errors: When testing or deployment is performed manually and there is an error, it is hard to repeat and fix. Many a time, the software development is outsourced in companies. In such cases, there is lack of coordination between the dev and ops teams. Environment inconsistency: Software functionality breaks when the code moves to different environments as each environment has different configurations. Teams can run around wasting a lot of time due to bugs when the software works fine on one environment but not on the other. Change management: Many companies have change management processes well in place, but they are outdated for DevOps. The time taken for reviews, passing a new module etc is manual and proves to be a bottleneck. Changes happen frequently in DevOps and the functioning suffers due to old processes. Monitoring: Monitoring is key to ensure smooth functioning in Agile. Many companies do not have the expertise to monitor the pipeline and infrastructure. Moreover monitoring only the infrastructure is not enough, there also needs to be monitoring of application performance, solutions need to be logged and analytics need to be tracked. Now let’s take a look at 8 ways AI can improve DevOps given the above context. 1. Better data access One of the most critical issues faced by DevOps teams is the lack of unregulated access to data. There is also a large amount of data, while the teams rarely view all of the data and focus on the outliers. The outliers only work as an indicator but do not give robust information. Artificial intelligence can compile and organize data from multiple sources for repeated use. Organized data is much easier to access and understand than heaps of raw data. This will help in predictive analysis and eventually a better decision making process. This is very important and enables many other ways listed below. 2. Superior implementation efficiency Artificially intelligent systems can work with minimal or no human intervention. Currently, a rules-based environment managed by humans is followed in DevOps teams. AI can transform this into self governed systems to greatly improve operational efficiency. There are limitations to the volume and complexity of analysis a human can perform. Given the large volumes of data to be analyzed and processed, AI systems being good at it, can set optimal rules to maximize operational efficiencies. 3. Root cause analysis Conducting root cause analysis is very important to fix an issue permanently. Not getting to the root cause allows for the cause to persist and affect other areas further down the line. Often, engineers don’t investigate failures in depth and are more focused on getting the release out. This is not surprising given the limited amount of time they have to work with. If fixing a superficial area gets things working, the root cause is not found. AI can take all data into account and see patterns between activity and cause to find the root cause of failure. 4 Automation Complete automation is a problem in DevOps, many tasks in DevOps are routine and need to be done by humans. An AI model can automate these repeatable tasks and speed up the process significantly. A well-trained model increases the scope of complexity of the tasks that can be automated by machines. AI can help achieve least human intervention so that developers can focus on more complex interactive problems. Complete automation also allows the errors to be reproduced and fixed promptly. 5 Reduce Operational Complexity AI can be used to simplify operations by providing a unified view. An engineer can view all the alerts and relevant data produced by the tools in a single place. This improves the current scenario where engineers have to switch between different tools to manually analyze and correlate data. Alert prioritization, root cause analysis, evaluating unusual behavior are complex time consuming tasks that depend on data. An organized singular view can greatly benefit in looking up data when required. “AI and machine learning makes it possible to get a high-level view of the tool-chain, but at the same time zoom in when it is required.” -SignifAI 6 Predicting failures A critical failure in a particular tool/area in DevOps can cripple the process and delay cycles. With enough data, machine learning models can predict when an error can occur. This goes beyond simple predictions. If an occurred fault is known to produce certain readings, AI can read patterns and predict the signs failure. AI can see indicators that humans may not be able to. As such early failure prediction and notification enable the team to fix it before it can affect the software development life cycle (SDLC). 7 Optimizing a specific metric AI can work towards solutions where the uptime is maximized. An adaptive machine learning system can learn how the system works and improve it. Improving could mean tweaking a specific metric in the workflow for optimized performance. Configurations can be changed by AI for optimal performance as required during different production phases. Real-time analysis plays a big part in this. 8 Managing Alerts DevOps systems can be flooded with alerts which are hard for humans to read and act upon. AI can analyze these alerts in real-time and categorize them. Assigning priority to alerts helps teams towards work on fixing them rather than going through a long list of alerts. The alerts can simply be tagged by a common ID for specific areas or AI can be trained for classifying good and bad alerts. Prioritizing alerts in such a way that flaws are shown first to be fixed will help smooth functioning. Conclusion As we saw, most of these areas depend heavily on data. So getting the system right to enhance data accessibility is the first step to take. Predictions work better when data is organized, performing root cause analysis is also easier. Automation can repeat mundane tasks and allow engineers to focus on more interactive problems that machines cannot handle. With machine learning, the overall operation efficiency, simplicity, and speed can be improved for smooth functioning of DevOps teams. Why Agile, DevOps and Continuous Integration are here to stay: Interview with Nikhil Pathania, DevOps practitioner Top 7 DevOps tools in 2018 GitLab’s new DevOps solution
Read more
  • 0
  • 0
  • 23911

article-image-12-ubiquitous-artificial-intelligence-powered-apps-that-are-changing-lives
Bhagyashree R
30 Aug 2018
11 min read
Save for later

12 ubiquitous artificial intelligence powered apps that are changing lives

Bhagyashree R
30 Aug 2018
11 min read
Artificial Intelligence is making it easier for people to do things every day. You can schedule your day, search for photos of loved ones, type emails on the go, or get things done with the virtual assistant. AI also provides innovative ways of tackling existing problems, from healthcare to advancing scientific discovery. According to Gartner’s Top 10 Strategic Technology Trends for 2018, the next few years will see every app, application, and service incorporating AI at some level. With major companies like Google, Amazon, IBM investing in AI and incorporating AI in their products, this statement, instead of a prediction is becoming a fact. Apple’s IPhone X comes with a Facial Recognition System, Samsung’s Bixby, Amazon’s Alexa, Google’s Google Assistant, and the recently launched Android Pie. Android Pie learns your preferences based on your usage patterns and gets better over time. It even provides you a breakdown of the time you spend on your phone. AI comes with endless possibilities, things that we used to dream of are now becoming a part of our day to day life. So, I have listed here, in no particular order, some of those innovative applications: Microsoft’s Seeing AI - Eye for the visually impaired Source: Microsoft Seeing AI is a perfect example of how technology is improving our lives. It is an intelligent camera app that uses computer vision to audibly help blind and visually impaired people to know about their surroundings. It comes with functionalities like reading out short text and documents for you, giving you description about a person, identifies currencies, colour, handwriting, light and even images in other apps using the device's camera. A data scientist named Anirudh Koul started this project (called Deep Vision earlier) to help his grandfather who was gradually losing his vision. Two breakthroughs by the Microsoft researchers facilitated him to further his idea: vision-to-language and image classification. To make the app this advance and real-time, they used the idea of making servers communicate with Microsoft Cognitive Services. This app brings in four technologies together to provide users with an array of functionalities: OCR, barcode scanner, facial recognition, and scene recognition. Check out this YouTube tutorial to understand how it works. Download App Store Ada - Healthcare in your hand Source: Digital Health Ada, with a very simple and conversational UI, helps you understand what could be wrong if you or someone you care about is not feeling well. Just like any doctor’s appointment, it starts with your basic details, then does an assessment, in which it asks several personalized questions related to the symptoms, and then gives a report. The report consists of a summary, possible causes, and less-likely causes. It also allows you to share the report as a PDF. After training over several years using real world cases, Ada has become a handy health advisor. Its platform is powered by a sophisticated Artificial Intelligence engine combined with large medical knowledge base covering many thousands of conditions, symptoms and findings. In every medical assessment, Ada takes all of a patient’s information into account, including past medical history, symptoms, risk factors and more. Using machine learning and multiple closed feedback loops, Ada becomes more intelligent. Download App Store Google Play Store Plume Air Report - An air pollution monitor Source: Plume Labs Blog Industrialization and urbanization definitely comes with their side effects, the main being air pollution. It has become inevitable to keep yourself safe from the pollution, but now at least you can be aware of the air pollution levels in your area. Plume Air Report forecasts how air quality will evolve hour by hour over the next 24 hours similar to weather forecast. You can also easily compare the air quality between cities. It gives you insight on all pollutants (PM2.5, PM10, O3, NO2), with absolute concentration levels and your local air quality scale. It uses machine learning and atmospheric sciences to deliver real-time and hourly forecast air quality data. First, latest pollution levels is collected from over 12,000 monitoring stations and 80 public agencies around the world and then filtered for errors. Local atmospheric data (wind, temperature, atmosphere, etc.) is sourced to track their influence on pollution levels in your city. A team of data scientists analyzes local specifics such as geographical features and human activities. Finally, AI algorithms and atmospheric models are developed that turn this giant amount of data into hourly forecasts. Download App Store Google Play Store Aura - Mindfulness meets AI Source: Popular Science In this fast life, slow down a little and give yourself a time out with Aura. Aura is a new kind of mindfulness app that learns about you and simplifies your learning through guided meditations. It helps in reducing stress and increases positivity through 3-minute meditations, personalized by Artificial Intelligence. Aura is an intelligent app that leverages machine learning to give you a unique experience. After every exercise, you can rate your experience and Aura will learn how to provide more tailored meditations according to your needs. You can even track your mood and learn your mood patterns. Download App Store Google Play Store Replika - An emotive chatbot as a friend for life Source: Medium Want to be friends with someone who is always there to listen to you, talk to you, and never judges you? Then Replika is for you! It helps you make a real connection with an unreal friend. The idea of building Replika came from a very tragic background. The founder of the software company, Luka, Eugenia Kudya, lost her best friend in an accident in November 2015. She used to go through their messenger texts to bring back their memories. This is how she got this idea to develop a chatbot making it learn from the sample texts sent by her best friend. In her own words, “Most of the companies try to build an app that talks, but we tried to build an app that could listen well”. The chatbot uses neural network facilitating more natural one-on-one conversation with its user, and over time, learn how to speak like them. The source code is freely available for developers under the name CakeChat. It comes with a pre-trained model that you can use as is to run a chatbot that maintains a conversation in a certain emotional state. You can also build a variety of other conversational agents by using your own dataset, for example, persona-based model, emotional chatting machine, topic-centric model. To know more about the background and evolution of Replika, check out this amazing YouTube video. Download App Store Google Play Store Google Assistant - Your personal Google Source: Google Assistant When talking of AI-powered apps, voice assistants probably come first in your mind. Google Assistant makes your life easier and helps in organizing your day better. You can manage your little tasks, plan your day, enjoy entertainment, and get answers. It can also sync to your other devices including Google Home, smart TVs, laptops, and more. To give users smart assistance, Google Assistant relies on Artificial Intelligence technologies such as natural language processing, natural language understanding, and machine learning to understand what the user is saying, and to make suggestions or act on that language input. Download App Store Google Play Store Hound - Say it, Get it Source: Android Apps In an array of virtual assistants to choose from, Hound understands your voice commands better. You do not need to give “search query” like commands and can have a more natural conversation. Hound can be used for variety of tasks, some of them are: search, discover, and play music, set alarms, timers, and reminders, call, text, navigate hands-free, get the weather forecast. Hound’s speed and accuracy comes from their powerful Houndify platform. This platform combines Speech Recognition and Natural Language Understanding into a single step, which is called Speech-to-Meaning. Download App Store Google Play Store Picai - An app that picks filters for your pics, keeping you looking your best always Source: Google Play Store Picai with the help of Artificial Intelligence, recommends picture-perfect filters by analyzing the scene. It automatically analyzes the scene and with the help of object recognition detects the type of the object, for example, a plant, a girl, etc. It then uses a proprietary deep learning model to recommend two optimum filters from 100+ filters. What makes this app stand out is the split-screen filter selection, which makes the filter selection easier for the users. When using this app be warned of the picture quality and app size (76 MB), but it is definitely worth trying! Download Google Play Store Microsoft Pix - The pro photographer Source: MSPoweuser Named one of the 50 Best Apps of the Year by Time Magazine, Microsoft Pix helps you take better photos without the extra effort! It solves the problem of “not living in the moment”. It comes with some amazing features like, hyperlapse, live images Microsoft Pix Comix, artistic styles to transform your photos, smart settings that automatically checks scene and lighting between each shutter tap, and updates settings between each shot, and more. Microsoft Pix uses Artificial Intelligence to improve the image, such as cropping edges, enhancing color and tone, and sharpening focus. It includes enhanced deep-learning capabilities around image understanding. It captures a burst of 10 frames with each shutter click and uses AI to select three best shots. Before the remaining photos are deleted, it uses data from the entire burst to remove noise. These best, enhanced images are ready in about a second. The app also detects whether your eyes are open or not using the facial recognition technology. Download App Store ELSA - Your machine learning English teacher Source: TechCrunch ELSA (English Language Speech Assistant)  helps you in learning English and bettering your pronunciation every day. It provides you a curriculum tailored just, regular feedback, progress tracking, common phrases used in daily life. You can practice in a relaxed environment and improve your speaking skills to prepare for the TOEFL, IELTS, TOEIC ELSA coaches you in improving your English pronunciations by using speech recognition, deep learning, and Artificial Intelligence. Download App Store Google Play Store Socratic - Homework in a snap Source: Google Play Store Socratic is your new helper, apart from your parents, in completing those complex Math problems. You just need to take a photo of your homework and can get explanations, videos, step-by-step help, instantly. Also, these resources are jargon-free, helping you understand the concepts better. It supports all subjects including Math (Algebra, Calculus, Statistics, Graphing, etc), Science, Chemistry, History, English, Economics, and more. Socratic uses Artificial Intelligence to figure out the concepts you need to learn in order to answer it. For this it combines cutting-edge computer vision technologies, which read questions from images, with machine learning classifiers. These classifiers are built using millions of sample homework questions, to accurately predict which concepts will help you solve your question. Download App Store Google Play Store Recent News - Stay informed Source: Recent News Recent News is an app that will provide you customized news. Some of the features that it comes with to give you the daily dose of news include one-minute news summary with very quick load time, hot news, local news, and personalized recommendations, instantly share news on Facebook, Twitter, and other social networks, and many more. It uses Artificial Intelligence to learn about your interests, suggest relevant articles, and propose topics you might like to follow. So, the more you use it the better it becomes! The app is surely innovative and saves time, but I do wish the developers applied some innovation in the app’s name as well :P Download App Store Google Play Store And that’s the end of my list. People say, “Smartphones and apps are becoming smarter, and we are becoming dumber”. But I would like to say that these apps, with the right usage, empower us to become smarter. Agree? 7 Popular Applications of Artificial Intelligence in Healthcare 5 examples of Artificial Intelligence in Web apps What Should We Watch Tonight? Ask a Robot, says Matt Jones from OVO Mobile [Interview]
Read more
  • 0
  • 0
  • 11280

article-image-nvidia-leads-the-ai-hardware-race-but-which-of-its-gpus-should-you-use-for-deep-learning
Prasad Ramesh
29 Aug 2018
8 min read
Save for later

NVIDIA leads the AI hardware race. But which of its GPUs should you use for deep learning?

Prasad Ramesh
29 Aug 2018
8 min read
For readers who are new to deep learning and who might be wondering what a GPU is, let’s start there. To make it simple, consider deep learning as nothing more than a set of calculations - complex calculations, yes, but calculations nonetheless. To run these calculations, you need hardware. Ordinarily, you might just use a normal processor like the CPU inside your laptop. However, this isn’t powerful enough to process at the speed at which deep learning computations need to happen. GPUs, however, can. This is because while a conventional CPU has only a few complex cores, a GPU can have thousands of simple cores. With a GPU, training a deep learning data set can take just hours instead of days. However, although it’s clear that GPUs have significant advantages over CPUs, there is nevertheless a range of GPUs available, each having their own individual differences. Selecting one is ultimately a matter of knowing what your needs are. Let’s dig deeper and find out how to go about shopping for GPUs… What to look for before choosing a GPU? There are a few specifications to consider before picking a GPU. Memory bandwidth: This determines the capacity of a GPU to handle large amounts of data. It is the most important performance metric, as with faster memory bandwidth more data can be processed at higher speeds. Number of cores: This indicates how fast a GPU can process data. A large number of CUDA cores can handle large datasets well. CUDA cores are parallel processors similar to cores in a CPU but their number is in thousands and are not suited for complex calculations that a CPU core can perform. Memory size: For computer vision projects, it is crucial for memory size to be as much as you can afford. But with natural language processing, memory size does not play such an important role. Our pick of GPU devices to choose from The go to choice here is NVIDIA; they have standard libraries that make it simple to set things up. Other graphics cards are not very friendly in terms of the libraries supported for deep learning. NVIDIA CUDA Deep Neural Network library also has a good development community. “Is NVIDIA Unstoppable In AI?” -Forbes “Nvidia beats forecasts as sales of graphics chips for AI keep booming” -SiliconANGLE AMD GPUs are powerful too but lack library support to get things running smoothly. It would be really nice to see some AMD libraries being developed to break the monopoly and give more options to the consumers. NVIDIA RTX 2080 Ti: The RTX line of GPUs are to be released in September 2018. The RTX 2080 Ti will be twice as fast as the 1080 Ti. Price listed on NVIDIA website for founder’s edition is $1,199. RAM: 11 GB Memory bandwidth: 616 GBs/second Cores: 4352 cores @ 1545 MHz NVIDIA RTX 2080: This is more cost efficient than the 2080 Ti at a listed price of $799 on NVIDIA website for the founder’s edition. RAM: 8 GB Memory bandwidth: 448 GBs/second Cores: 2944 cores @ 1710 MHz NVIDIA RTX 2070: This is more cost efficient than the 2080 Ti at a listed price of $599 on NVIDIA website. Note that the other versions of the RTX cards will likely be cheaper than the founder’s edition around a $100 difference. RAM: 8 GB Memory bandwidth: 448 GBs/second Cores: 2304 cores @ 1620 MHz NVIDIA GTX 1080 Ti: Priced at $650 on Amazon. This is a higher end option but offers great value for money, and can also do well in Kaggle competitions. If you need more memory but cannot afford the RTX 2080 Ti go for this. RAM: 11 GB Memory bandwidth: 484 GBs/second Cores: 3584 cores @ 1582 MHz NVIDIA GTX 1080: Priced at $584 on Amazon. This is a mid-high end option only slightly behind the 1080Ti. VRAM: 8 GB Memory bandwidth: 320 GBs/second Processing power: 2560 cores @ 1733 MHz NVIDIA GTX 1070 Ti: Priced at around $450 on Amazon. This is slightly less performant than the GTX 1080 but $100 cheaper. VRAM: 8 GB Memory bandwidth: 256 GBs/second Processing power: 2438 cores @ 1683 MHz NVIDIA GTX 1070: Priced at $380 on Amazon is currently the bestseller because of crypto miners. Somewhat slower than the 1080 GPUs but cheaper. VRAM: 8 GB Memory bandwidth: 256 GBs/second Processing power: 1920 cores @ 1683 MHz NVIDIA GTX 1060 6GB: Priced at around $290 on Amazon. Pretty cheap but the 6 GB VRAM limits you. Should be good for NLP but you’ll find the performance lacking in computer vision. VRAM: 6 GB Memory bandwidth: 216 GBs/second Processing power: 1280 cores @ 1708 MHz NVIDIA GTX 1050 Ti: Priced at around $200 on Amazon. This is the cheapest workable option. Good to get started with deep learning and explore if you’re new. VRAM: 4 GB Memory bandwidth: 112 GBs/second Processing power: 768 cores @ 1392 MHz NVIDIA Titan XP: The Titan XP is also an option but gives only a marginally better performance while being almost twice as expensive as the GTX 1080 Ti, it has 12 GB memory, 547.7 GB/s bandwidth and 3840 cores @ 1582 MHz. On a side note, NVIDIA Quadro GPUs are pretty expensive and don’t really help in deep learning they are more of use in CAD and working with heavy graphics production tasks. The graph below does a pretty good job of visualizing how all the GPUs above compare: Source: Slav Ivanov Blog, processing power is calculated as CUDA cores times the clock frequency Does the number of GPUs matter? Yes, it does. But how many do you really need? What’s going to suit the scale of your project without breaking your budget? 2 GPUs will always yield better results than just one - but it’s only really worth it if you need the extra power. There are two options you can take with multi-GPU deep learning. On the one hand, you can train several different models at once across your GPUs, or, alternatively distribute one single training model across multiple GPUs known as  “multi-GPU training”. The latter approach is compatible with TensorFlow, CNTK, and PyTorch. Both of these approaches have advantages. Ultimately, it depends on how many projects you’re working on and, again, what your needs are. Another important point to bear in mind is that if you’re using multiple GPUs, the processor and hard disk need to be fast enough to feed data continuously - otherwise the multi-GPU approach is pointless. Source: NVIDIA website It boils down to your needs and budget, GPUs aren’t exactly cheap.   Other heavy devices There are also other large machines apart from GPUs. These include the specialized supercomputer from NVIDIA, the DGX-2, and Tensor processing units (TPUs) from Google. The NVIDIA DGX-2 If you thought GPUs were expensive, let me introduce you to NVIDIA DGX-2, the successor to the NVIDIA DGX-1. It’s a highly specialized workstation; consider it a supercomputer that has been specially designed to tackle deep learning. The price of the DGX-2 is (*gasp*) $399,000. Wait, what? I could buy some new hot wheels for that, or Dual Intel Xeon Platinum 8168, 2.7 GHz, 24-cores, 16 NVIDIA GPUs, 1.5 terabytes of RAM, and nearly 32 terabytes of SSD storage! The performance here is 2 petaFLOPS. Let’s be real: many of us probably won’t be able to afford it. However, NVIDIA does have leasing options, should you choose to try it. Practically speaking, this kind of beast finds its use in research work. In fact, the first DGX-1 was gifted to OpenAI by NVIDIA to promote AI research. Visit the NVIDIA website for more on these monster machines. There are also personal solutions available like the NVIDIA DGX Workstation. TPUs Now that you’ve caught your breath after reading about AI dream machines, let’s look at TPUs. Unlike the DGX machines, TPUs run on the cloud. A TPU is what’s referred to as an application-specific integrated circuit (ASIC) that has been designed specifically for machine learning and deep learning by Google. Here’s the key stats: Cloud TPUs can provide up to 11.5 petaflops of performance in a single pod. If you want to learn more, go to Google’s website. When choosing GPUs you need to weigh up your options The GTX 1080 Ti is most commonly used by researchers and competitively for Kaggle, as it gives good value for money. Go for this if you are sure about what you want to do with deep learning. The GTX 1080 and GTX 1070 Ti are cheaper with less computing power, a more budget friendly option if you cannot afford the 1080 Ti. GTX 1070 saves you some more money but is slower. The GTX 1060 6GB and GTX 1050 Ti are good if you’re just starting off in the world of deep learning without burning a hole in your pockets. If you must have the absolute best GPU irrespective of the cost then the RTX 2080 Ti is your choice. It offers twice the performance for almost twice the cost of a 1080 Ti. Nvidia unveils a new Turing architecture: “The world’s first ray tracing GPU” Nvidia GPUs offer Kubernetes for accelerated deployments of Artificial Intelligence workloads Nvidia’s Volta Tensor Core GPU hits performance milestones. But is it the best?
Read more
  • 0
  • 0
  • 15223

article-image-a-machine-learning-roadmap-for-web-developers
Sugandha Lahoti
27 Aug 2018
7 min read
Save for later

A Machine learning roadmap for Web Developers

Sugandha Lahoti
27 Aug 2018
7 min read
Now that you’ve opened this article, I’ll assume you’re a web developer who is all excited with the prospect of building a machine learning project. You may be here for one of these reasons. Either you have been in a circle of people who find web development is dying? (Is it really dying or just unwell?). Or maybe you are stagnating in your current trajectory. And so, you want to learn something different, something trending, something like Artificial Intelligence. Or you/your employer/your client is aware of the capabilities of machine learning and want to include it in some part of your web app to make it more powerful. Or like the majority of the folks, you just want to see first hand if all the fuss about artificial intelligence is really worth all the effort to switch gears, by building a side toy ML project. Either way, there are different approaches to fulfill these needs. Learning Machine Learning for the Web with Javascript Learning machine learning coming from a web development background comes with its own constraints. You might worry about having to learn entirely different concepts from scratch - from different algorithms to programming languages like Python to mathematical concepts like linear algebra, calculus, and statistics. However, chances are you can skip learning a new language. You probably know some Javascript in some form or the other thanks to your web development experience. As such, you can learn Machine Learning in JavaScript (You don’t have to learn another programming language from scratch) and take it right to your browsers with WebGL. There are some advantages to using JavaScript for ML. Its popularity is one; while ML in JavaScript is not as popular as Python’s ML ecosystem, at the moment, the language itself is. As demand for ML applications rises, and as hardware becomes faster and cheaper, it's only natural for machine learning to become more prevalent in the JavaScript world. The JavaScript ecosystem offers a rich set of libraries suited for most Machine Learning tasks. Math: math.js Data Analysis: d3.js Server: node.js (express, koa, hapi) Performance: Tensorflow.js (e.g. GPU accelerated via WebGL API in the browser), Keras.js etc. Read also: 5 JavaScript machine learning libraries you need to know BRIIM is a good collection of materials to get you started as web developer or JavaScript enthusiast in machine learning. In case you’re interested in learning Python instead of Javascript, here are the set of libraries you should pick. Math: numpy Data Analysis: Pandas Data Mining: PySpark Server: Flask, Django Performance: TensorFlow (because it is written with a Python API over a C/C++ engine) or Keras (sits on top of TensorFlow). Using Machine Learning as a service If you don’t want to spend your time learning frameworks, tools, and languages suited for machine learning, you can adopt Machine Learning as a service or MLaaS. These services provide machine learning tools as part of cloud computing services. So basically, you can benefit from machine learning without the allied cost, time and risk of establishing an in-house internal machine learning team. All you need is sufficient knowledge of incorporating APIs. All Machine Learning tasks including data pre-processing, model training, model evaluation, and predictions can be completed through MLaaS. Read also: How machine learning as a service is transforming cloud A large number of companies provide Machine Learning as a service. Most prominent ones include: Amazon Machine Learning Amazon ML makes it easy for web developers to build smart applications using simple APIs. This includes applications for fraud detection, demand forecasting, targeted marketing, and click prediction. They provide a Developer Guide, which provides a conceptual overview of Amazon ML and includes detailed instructions for using the service. They also have a API reference, which describes all the API operations and provides sample requests and responses for supported web service protocols. Azure ML web app templates The web app templates available in the Azure Marketplace can build a custom web app that knows your web service's input data and expected results. All you need to do is give the web app access to your web service and data, and the template does the rest. There are two available templates: Azure ML Request-Response Service Web App Template Azure ML Batch Execution Service Web App Template Each template creates a sample ASP.NET application by using the API URI and key for your web service. The template then deploys the application as a website to Azure. No coding is required to use these templates. You just supply the API key and URI, and the template builds the application for you. Google Cloud based APIs Google also provides machine learning services, with pre-trained models and a service to generate your own tailored models. Google’s Cloud AutoML is a suite of machine learning products that enables developers with limited machine learning expertise to train high-quality models specific to their business needs. Cloud AutoML is used by Disney on their website shopDisney to enhance guest experience through more relevant search results, expedited discovery, and product recommendations. Building Conversational Interfaces As a web developer, another thing you might be looking into, is developing conversational interfaces or chatbots to enhance your web apps. Amazon, Google, and Microsoft provide Machine learning powered tools to help developers with building their own chatbots. Amazon Lex You can embed chatbots in your web apps with the Amazon Lex featuring ASR (Automatic Speech Recognition) and NLP (Natural Language Processing) capabilities. The API can recognize written and spoken text and the Lex interface allows you to hook the recognized inputs to various back-end solutions. Lex currently supports deploying chatbots for Facebook Messenger, Slack, and Twilio. Google Dialogflow Google’s Dialogflow can build voice and text-based conversational interfaces, such as voice apps and chatbots, powered by AI. Dialogflow incorporates Google's machine learning expertise and products such as Google Cloud Speech-to-Text. The API can be tweaked and customized for needed intents using Java, Node.js, and Python. It is also available as an enterprise edition. Microsoft Azure Cognitive Services Microsoft Cognitive Services simplify a variety of AI-based tasks, giving you a quick way to add intelligence technologies to your bots with just a few lines of code. It provides tools and APIs for aiding the development of conversational interfaces. These include: Translator Speech API Bing Speech API to convert text into speech and speech into text Speaker Recognition API for voice verification tasks Custom Speech Service to apply Azure NLP capacities using own data and models Language Understanding Intelligent Service (LUIS) is an API that analyzes intentions in text to be recognized as commands Text Analysis API for sentiment analysis and defining topics Bing Spell Check Translator Text API Web Language Model API that estimates probabilities of words combinations and supports word autocompletion Linguistic Analysis API used for sentence separation, tagging the parts of speech, and dividing texts into labeled phrases Read also: Top 4 chatbot development frameworks for developers These tools should be enough to get your feet off the ground quickly and move into the specific area of machine learning. Ultimately your choice of tool relies on the kind of application you want to build, your level of expertise, and how much time and effort you’re willing to put to learn. Obviously, depending on your area of choice, you would have to do more research and develop yourself in those areas. How should web developers learn machine learning? 5 examples of Artificial Intelligence in Web apps The most valuable skills for web developers to learn in 2018
Read more
  • 0
  • 0
  • 13313

article-image-6-artificial-intelligence-cybersecurity-tools-you-need-to-know
Savia Lobo
25 Aug 2018
7 min read
Save for later

6 artificial intelligence cybersecurity tools you need to know

Savia Lobo
25 Aug 2018
7 min read
Recently, most of the organizations experienced severe downfall due to an undetected malware, Deeplocker, which secretly evaded even the stringent cyber security mechanisms. Deeplocker leverages the AI model to attack the target host by using indicators such as facial recognition, geolocation and voice recognition. This incidence speaks volumes about the big role AI plays in the cybersecurity domain. In fact, some may even go on to say that AI for cybersecurity is no longer a nice to have tech rather a necessity. Large and small organizations and even startups are hugely investing in building AI systems to analyze the huge data trove and in turn, help their cybersecurity professionals to identify possible threats and take precautions or immediate actions to solve it. If AI can be used in getting the systems protected, it can also harm it. How? The hackers and intruders can also use it to launch an attack--this would be a much smarter attack--which would be difficult to combat. Phishing, one of the most common and simple social engineering cyber attack is now easy for attackers to master. There are a plethora of tools on the dark web that can help anyone to get their hands on phishing. In such trying conditions, it is only imperative that organizations take necessary precautions to guard their information castles. What better than AI? How 6 tools are using artificial intelligence for cybersecurity Symantec’s Targeted attack analytics (TAA) tool This tool was developed by Symantec and is used to uncover stealthy and targeted attacks. It applies AI and machine learning on the processes, knowledge, and capabilities of the Symantec’s security experts and researchers. The TAA tool was used by Symantec to counter the Dragonfly 2.0 attack last year. This attack targeted multiple energy companies and tried to gain access to operational networks. Eric Chein, Technical Director of Symantec Security says, “ With TAA, we’re taking the intelligence generated from our leading research teams and uniting it with the power of advanced machine learning to help customers automatically identify these dangerous threats and take action.” The TAA tools analyze incidents within the network against the incidents found in their Symantec threat data lake. TAA unveils suspicious activity in individual endpoints and collates that information to determine whether each action indicate hidden malicious activity. The TAA tools are now available for Symantec Advanced Threat Protection (ATP) customers. Sophos’ Intercept X tool Sophos is a British security software and hardware company. Its tool, Intercept X, uses a deep learning neural network that works similar to a human brain. In 2010, the US Defense Advanced Research Projects Agency (DARPA) created their first Cyber Genome Program to uncover the ‘DNA’ of malware and other cyber threats, which led to the creation of algorithm present in the Intercept X. Before a file executes, the Intercept X is able to extract millions of features from a file, conduct a deep analysis, and determine if a file is benign or malicious in 20 milliseconds. The model is trained on real-world feedback and bi-directional sharing of threat intelligence via an access to millions of samples provided by the data scientists. This results in high accuracy rate for both existing and zero-day malware, and a lower false positive rate. Intercept X utilizes behavioral analysis to restrict new ransomware and boot-record attacks.  The Intercept X has been tested on several third parties such as NSS labs and received high-scores. It is also proven on VirusTotal since August of 2016. Maik Morgenstern, CTO, AV-TEST said, “One of the best performance scores we have ever seen in our tests.” Darktrace Antigena Darktrace Antigena is Darktrace’s active self-defense product. Antigena expands Darktrace’s core capabilities to detect and replicate the function of digital antibodies that identify and neutralize threats and viruses. Antigena makes use of Darktrace’s Enterprise Immune System to identify suspicious activity and responds to them in real-time, depending on the severity of the threat. With the help of underlying machine learning technology, Darktrace Antigena identifies and protects against unknown threats as they develop. It does this without the need for human intervention, prior knowledge of attacks, rules or signatures. With such automated response capability, organizations can respond to threats quickly, without disrupting the normal pattern of business activity. Darktrace Antigena modules help to regulate user and machine access to the internet, message protocols and machine and network connectivity via various products such as Antigena Internet, Antigena Communication, and Antigena network. IBM QRadar Advisor IBM’s QRadar Advisor uses the IBM Watson technology to fight against cyber attacks. It uses AI to auto-investigate indicators of any compromise or exploit. QRadar Advisor uses cognitive reasoning to give critical insights and further accelerates the response cycle. With the help of IBM’s QRadar Advisor, security analysts can assess threat incidents and reduce the risk of missing them. Features of the IBM QRadar Advisor Automatic investigations of incidents QRadar Advisor with Watson investigates threat incidents by mining local data using observables in the incident to gather broader local context. It later quickly assesses the threats regarding whether they have bypassed layered defenses or were blocked. Provides Intelligent reasoning QRadar identifies the likely threat by applying cognitive reasoning. It connects threat entities related to the original incident such as malicious files, suspicious IP addresses, and rogue entities to draw relationships among these entities. Identifies high priority risks With this tool, one can get critical insights on an incident, such as whether or not a malware has executed, with supporting evidence to focus your time on the higher risk threats. Then make a decision quickly on the best response method for your business. Key insights on users and critical assets IBM’s QRadar can detect suspicious behavior from insiders through integration with the User Behavior Analytics (UBA) App and understands how certain activities or profiles impact systems. Vectra’s Cognito Vectra’s Cognito platform uses AI to detect attackers in real-time. It automates threat detection and hunts for covert attackers. Cognito uses behavioral detection algorithms to collect network metadata, logs and cloud events. It further analyzes these events and stores them to reveal hidden attackers in workloads and user/IoT devices. Cognito platform consists of Cognito Detect and Cognito Recall. Cognito Detect reveals hidden attackers in real time using machine learning, data science, and behavioral analytics. It automatically triggers responses from existing security enforcement points by driving dynamic incident response rules. Cognito Recall determines exploits that exist in historical data. It further speeds up detection of incident investigations with actionable context about compromised devices and workloads over time. It’s a quick and easy fix to find all devices or workloads accessed by compromised accounts and identify files involved in exfiltration. Just as diamond cuts diamond, AI cuts AI. By using AI to attack and to prevent on either side, AI systems will learn different and newer patterns and also identify unique deviations to security analysts. This provides organizations to resolve an attack on the way much before it reaches to the core. Given the rate at which AI and machine learning are expanding, the days when AI will redefine the entire cybersecurity ecosystem are not that far. DeepMind AI can spot over 50 sight-threatening eye diseases with expert accuracy IBM’s DeepLocker: The Artificial Intelligence powered sneaky new breed of Malware 7 Black Hat USA 2018 conference cybersecurity training highlights Top 5 cybersecurity trends you should be aware of in 2018  
Read more
  • 0
  • 0
  • 16662
Unlock access to the largest independent learning library in Tech for FREE!
Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
Renews at $19.99/month. Cancel anytime
article-image-best-game-engines-for-ai-game-development
Natasha Mathur
24 Aug 2018
8 min read
Save for later

Best game engines for Artificial Intelligence game development

Natasha Mathur
24 Aug 2018
8 min read
"A computer would deserve to be called intelligent if it could deceive a human into believing that it was human" — Alan Turing It is quite common to find games which are initially exciting but take a boring turn eventually, making you want to quit the game. Then, there are games which are too difficult to hold your interest and you end up quitting in the beginning phase itself.  These are also two of the most common problems that game developers face when building games. This is where AI comes to your rescue, to spice things up. Why use Artificial Intelligence in games? The major reason for using AI in games is to provide a challenging opponent to make the game more fun to play. But, AI in the gaming industry is not a recent news. The gaming world has been leveraging the wonders of AI for a long time now. One of the first examples of AI is the computerized game, Nim, was created back in 1951. Other games such as Façade, Black & White, The Sims, Versu, and F.E.A.R. are all great AI games, that hit the market long time back. Even modern-day games like Need for Speed, Civilization, or Counter-Strike use AI. AI controls a lot of elements in games and is usually behind characters such as enemy creeps, neutral merchants, or even animals. AI in games is used to enable the non-human characters (NPCs) with responsive, adaptive, and intelligent behaviors similar to human-like intelligence. AI helps make NPCs seem intelligent as they are able to actively change their level of skills based on the person playing the game. This makes the game seem more personalized to the gamer. Playing video games is fun, and developing these games is equally fun. There are different game engines on the market to help with the development of games. A game engine is a software that provides game creators with the necessary set of features to build games quickly and efficiently. Let’s have a look at the top game engines for Artificial Intelligence game development. Unity3D Developer:  Unity Technologies Release Date: June 8, 2005 Unity is a cross-platform game engine which provides users with the ability to create games in both 2D and 3D. It is extremely popular and loved by game designers from large and small studios alike. Apart from 3D, and 2D games, it also helps with simulations for desktops, laptops, home consoles, smart TVs, and mobile devices. Key AI features: Unity offers a machine learning agents toolkit to the game developers, which help them include AI agents within games. As per the Unity team, “machine Learning Agents Toolkit (ML-Agents) is an open-source Unity plugin that enables games and simulations to serve as environments for training intelligent agents”. Unity AI - Unity 3D Artificial Intelligence  The ML-Agents SDK transforms games and simulations created using the Unity Editor into environments for training intelligent agents. These ML agents are trained using deep Reinforcement Learning, imitation learning, neuroevolution, or other machine learning methods via Python APIs. There’s also a TensorFlow based algorithm provided by Unity to allow game developers to easily train intelligent agents for 2D, 3D, and VR/AR games. These trained agents are then used for controlling the NPC behavior within games. The ML-Agents toolkit is beneficial for both game developers and AI researchers. Apart from this, Unity3D is easy to use and learn, compatible with every game platform and provides great community support. Learning Resources: Unity AI Programming Essentials Unity 2017 Game AI programming - Third Edition Unity 5.x Game AI Programming Cookbook Unreal Engine 4 Developer: Epic games Release Date: May 1998 Unreal Engine is widely used among developers all around the world. It is a collection of integrated tools for game developers which helps them build games, simulations, and visualization. It is also among the top game engines which are used to develop high-end AAA titles. Gears of War, Batman: Arkham Asylum and Mass Effect are some of the popular games developed using Unreal Engine. Key AI features: Unreal Engine uses a set of tools which helps add AI capabilities to a game. It uses tools such as behavior Tree, navigation Component, blackboard asset, enumeration, target point, AI Controller, and Navigation Volumes. Behavior tree creates different states and the logic behind AI. Navigation Component helps with handling movement for AI. Blackboard Asset stores information and acts as the local variable for AI. Enumeration creates states. It also allows alternating between these states. Target Point creates a basic Path node form. The AI Controller and Character tool is responsible for handling the communication between the world and the controlled pawn for AI. At last, the Navigation Volumes feature creates Navigation Mesh in the environment to allow easy Pathfinding for AI. There are also features such as Blueprint Visual Scripting which can be converted into performant C++ code, AIComponents, and the Environment Query System (EQS) which provides agents the ability to perceive their environment. Apart from its AI capabilities, the Unreal engine offers the largest community support with lifetime hours of video tutorials and assets. It is also compatible with a variety of operating platforms such as iOS, Android, Linux, Mac, Windows, and most game consoles. But there are certain inbuilt-tools in Unreal Engine which can be hard for beginners to learn. Learning resources: Unreal Engine 4 AI programming essentials CryEngine 3 Developer: Crytek Release Date: May 2, 2002 CryEngine is a powerful game development platform that comes packed with a set of tools and features to create world-class gaming experiences. It is the game engine behind games such as Sniper: Ghost Warrior 2, SNOW, etc. Key AI features: CryEngine comes with an AI system designed for easy creation of custom AI actors. This is flexible enough to handle a larger set of complex and different worlds. The core of CryEngine’s AI system is based on lots of scripting. There are different AI elements within this system that add the AI capabilities to the NPCs within the game. Some of these elements are AI Actions which allows the developers to script AI behaviors without creating new code. AI Actors Logger can log AI events and signals to files. AI Control Objects use AI object to control AI entities/actors. AI Debug Draw is the primary tool offered by CryEngine for information on the current state of the AI System and AI actors. AI Debugger registers the inputs that AI agents receive and the decisions that they make in real-time during a game session. AI Sequence system works in parallel to FG and AI systems. This helps to simplify and group AI control. CryEngine offers the easiest A.l. coding of any tech currently on the market. Since CryEngine is relatively new as compared to other game engines, it does not have a very flourishing community yet. Despite the easy AI coding, the overall learning curve of Unreal Engine is high. Panda3D Developer: Disney Interactive until 2010,  Walt Disney Imagineering, Carnegie Mellon University Release Date: 2002 Panda3D is a game engine, a framework for 3D rendering and game development for Python and C++ programs. It includes graphics, audio, I/O, collision detection, and other abilities for the creation of 3D games. Key AI features: Panda3D comes packed with an AI library named PandAI v1.0. PandAI is an AI library which provides 'Artificially Intelligent' behavior in NPC (Non-Playable Characters) in games. The PandAI library offers functionality for Steering Behaviors (Seek, Flee, Pursue, Evade, Wander, Flock, Obstacle Avoidance, Path Following) and path finding (helps the NPCs to intelligently avoiding obstacles via the shortest path ). This AI library is composed of several different entities. For instance, there’s a main AIWorld Class to update any AICharacters added to it. Each AICharacter has its own AIBehavior object for tracking all the position and rotation updates. Each AIBehavior object has the functionality to implement all the steering behaviors and pathfinding behaviors. These features within Panda3D gives you the ability to call the respective functions. Panda3D is a relatively simple game engine which lets you add AI capabilities within your games. The community is not as robust and has a low learning curve. AI is a fantastic tool which makes the entities in games seem more organic, alive, and real. The main goal here is not to copy the entire human thought process but to just sell the illusion of life. These game engines provide the developers with the entire framework needed to add AI capabilities to their games. The entire game development process is more fun as there is no need to create all systems including the physics, graphics, and AI, from scratch. Now, if you’re wondering about the best AI game engines out of the four mentioned in this article then there is no specific answer to that as selecting the best AI game engine depends on the requirements of your project. Game Engine Wars: Unity vs Unreal Engine Unity switches to WebAssembly as the output format for the Unity WebGL build target Developing Games Using AI  
Read more
  • 0
  • 1
  • 17965

article-image-tensorflow-always-tops-machine-learning-artificial-intelligence-tool-surveys
Sunith Shetty
23 Aug 2018
9 min read
Save for later

Why TensorFlow always tops machine learning and artificial intelligence tool surveys

Sunith Shetty
23 Aug 2018
9 min read
TensorFlow is an open source machine learning framework for carrying out high-performance numerical computations. It provides excellent architecture support which allows easy deployment of computations across a variety of platforms ranging from desktops to clusters of servers, mobiles, and edge devices. Have you ever thought, why TensorFlow has become so popular in such a short span of time? What made TensorFlow so special, that we seeing a huge surge of developers and researchers opting for the TensorFlow framework? Interestingly, when it comes to artificial intelligence frameworks showdown, you will find TensorFlow emerging as a clear winner most of the time. The major credit goes to the soaring popularity and contributions across various forums such as GitHub, Stack Overflow, and Quora. The fact is, TensorFlow is being used in over 6000 open source repositories showing their roots in many real-world research and applications. How TensorFlow came to be The library was developed by a group of researchers and engineers from the Google Brain team within Google AI organization. They wanted a library that provides strong support for machine learning and deep learning and advanced numerical computations across different scientific domains. Since the time Google open sourced its machine learning framework in 2015, TensorFlow has grown in popularity with more than 1500 projects mentions on GitHub. The constant updates made to the TensorFlow ecosystem is the real cherry on the cake. This has ensured all the new challenges developers and researchers face are addressed, thus easing the complex computations and providing newer features, promises, and performance improvements with the support of high-level APIs. By open sourcing the library, the Google research team have received all the benefits from a huge set of contributors outside their existing core team. Their idea was to make TensorFlow popular by open sourcing it, thus making sure all new research ideas are implemented in TensorFlow first allowing Google to productize those ideas. Read Also: 6 reasons why Google open sourced TensorFlow What makes TensorFlow different from the rest? With more and more research and real-life use cases going mainstream, we can see a big trend among programmers, and developers flocking towards the tool called TensorFlow. The popularity for TensorFlow is quite evident, with big names adopting TensorFlow for carrying out artificial intelligence tasks. Many popular companies such as NVIDIA, Twitter, Snapchat, Uber and more are using TensorFlow for all their major operations and research areas. On one hand, someone can make a case that TensorFlow’s popularity is based on its origins/legacy. TensorFlow being developed under the house of “Google” enjoys the reputation of the household name. There’s no doubt, TensorFlow has been better marketed than some of its competitors. Source: The Data Incubator However that’s not the full story. There are many other compelling reasons why small scale to large scale companies prefer using TensorFlow over other machine learning tools TensorFlow key functionalities TensorFlow provides an accessible and readable syntax which is essential for making these programming resources easier to use. The complex syntax is the last thing developers need to know given machine learning’s advanced nature. TensorFlow provides excellent functionalities and services when compared to other popular deep learning frameworks. These high-level operations are essential for carrying out complex parallel computations and for building advanced neural network models. TensorFlow is a low-level library which provides more flexibility. Thus you can define your own functionalities or services for your models. This is a very important parameter for researchers because it allows them to change the model based on changing user requirements. TensorFlow provides more network control. Thus allowing developers and researchers to understand how operations are implemented across the network. They can always keep track of new changes done over time. Distributed training The trend of distributed deep learning began in 2017, when Facebook released a paper showing a set of methods to reduce the training time of a convolutional neural network model. The test was done on RESNET-50 model on ImageNet dataset which took one hour to train instead of two weeks. 256 GPUs spread over 32 servers were used. This revolutionary test has open the gates for many research work which have massively reduced the experimentation time by running many tasks in parallel on multiple GPUs. Google’s distributed TensorFlow has allowed all the researchers and developers to scale out complex distributed training using in-built methods and operations that optimizes distributed deep learning among servers. . Google’s distributed TensorFlow engine which is part of the regular TensorFlow repo, works exceptionally well with the existing TensorFlow’s operations and functionalities. It has allowed exploring two of the most important distributed methods: Distribute the training time of a neural network model over many servers to reduce the training time. Searching for good hyperparameters by running parallel experiments over multiple servers. Google has given distributed TensorFlow engine the required power to steal the share of the market acquired by other distributed projects such as Microsoft’s CNTK, AMPLab's SparkNet, and CaffeOnSpark. Even though the competition is tough, Google has still managed to become more popular when compared to the other alternatives in the market. From research to production Google has, in some ways, democratized deep learning., The key reason is TensorFlow’s high-level APIs making deep learning accessible to everyone. TensorFlow provides pre-built functions and advanced operations to ease the task of building different neural network models. It provides the required infrastructure and hardware which makes them one of the leading libraries used extensively by researchers and students in the deep learning domain. In addition to research tools, TensorFlow extends the services by bringing the model in production using TensorFlow Serving. It is specifically designed for production environments, which provides a flexible, high-performance serving system for machine learning models. It provides all the functionalities and operations which makes it easy to deploy new algorithms and experiments as per changing requirements and preferences. It provides an excellent feature of out-of-the-box integration with TensorFlow models which can be easily extended to serve other types of models and data. TensorFlow’s API is a complete package which is easier to use and read, plus provides helpful operators, debugging and monitoring tools, and deployment features. This has led to growing use of TensorFlow library as a complete package within the ecosystem by the emerging body of students, researchers, developers, production engineers from various fields who are gravitating towards artificial intelligence. There is a TensorFlow for web, mobile, edge, embedded and more TensorFlow provides a range of services and modules within their existing ecosystem making them as one of the ground-breaking end-to-end tools to provide state-of-the-art deep learning. TensorFlow.js for machine learning on the web JavaScript library for training and deploying machine learning models in the browser. This library provides flexible and intuitive APIs to build and train new and pre-existing models from scratch right in the browser or under Node.js. TensorFlow Lite for mobile and embedded ML It is a TensorFlow lightweight solution used for mobile and embedded devices. It is fast since it enables on-device machine learning inference with low latency. It supports hardware acceleration with the Android Neural Networks API. The future releases of TensorFlow Lite will bring more built-in operators, performance improvements, and will support more models to simplify the developer’s experience of bringing machine learning services within mobile devices. TensorFlow Hub for reusable machine learning A library which is used extensively to reuse machine learning models. Thus you can transfer learning by reusing parts of machine learning models. TensorBoard for visual debugging While training a complex neural network model, the computations you use in TensorFlow can be very confusing. TensorBoard makes it very easy to understand and debug your TensorFlow programs in the form of visualizations. It allows you to easily inspect and understand your TensorFlow runs and graphs. Sonnet Sonnet is a DeepMind library which is built on top of TensorFlow extensively used to build complex neural network models. All of this factors have made the TensorFlow library immensely appealing for building a wide spectrum of machine learning and deep learning projects. This tool has become a preferred choice for everyone from space research giant NASA and other confidential government agencies, to an impressive roster of private sector giants. Road Ahead for TensorFlow TensorFlow no doubt is better marketed compared to the other deep learning frameworks. The community appears to be moving very fast. In any given hour, there are approximately 10 people around the world contributing or improving the TensorFlow project on GitHub. TensorFlow dominates the field with the largest active community. It will be interesting to see what new advances TensorFlow and other utilities make possible for the future of our digital world. Continuing the recent trend of rapid updates, the TensorFlow team is making sure they address all the current and active challenges faced by the contributors and the developers while building machine learning and deep learning models. TensorFlow 2.0 will be a major update, we can expect the release candidate by next year early March. The preview version of this major milestone is expected to hit later this year. The major focus will be on ease of use, additional support for more platforms and languages, and eager execution will be the central feature of TensorFlow 2.0. This breakthrough version will add more functionalities and operations to handle current research areas such as reinforcement learning, GANs, building advanced neural network models more efficiently. Google will continue to invest and upgrade their existing TensorFlow ecosystem. According to Google’s CEO, Sundar Pichai “artificial intelligence is more important than electricity or fire.” TensorFlow is the solution they have come up with to bring artificial intelligence into reality and provide a stepping stone to revolutionize humankind. Read more The 5 biggest announcements from TensorFlow Developer Summit 2018 The Deep Learning Framework Showdown: TensorFlow vs CNTK Tensor Processing Unit (TPU) 3.0: Google’s answer to cloud-ready Artificial Intelligence
Read more
  • 0
  • 0
  • 14578

article-image-ai-tools-data-scientists-might-not-know
Amey Varangaonkar
22 Aug 2018
8 min read
Save for later

5 artificial intelligence tools data scientists might not know

Amey Varangaonkar
22 Aug 2018
8 min read
With Artificial Intelligence going mainstream, it is not at all surprising to see the number of tools and platforms for AI development go up as well. Open source libraries such as Tensorflow, Keras and PyTorch are very popular today. Not just those - enterprise platforms such as Azure AI Platform, Google Cloud AI and Amazon Sagemaker are commonly used to build scalable production-grade AI applications. While you might be already familiar with these tools and frameworks, there are quite a few relatively unknown AI tools and services which can make your life as a data scientist much, much easier! In this article, we look at 5 such tools for AI development which you may or may not have heard of before. Wit.ai One of the most popular use-cases of Artificial Intelligence today is building bots that facilitate effective human-computer interaction. Wit.ai, a platform for building these conversational chatbots, finds applications across various platforms, including mobile apps, IoT as well as home automation. Used by over 150,000 developers across the world, this platform gives you the ability to build conversational UI that supports text categorization, classification, sentiment analysis and a whole host of other features. Why you should try this machine learning tool out There are a multitude of reasons why wit.ai is so popular among developers for creating conversational chatbots. Some of the major reasons are: Support for text as well as voice, which gives you more options and flexibility in the way you want to design your bots Support for multiple languages such as Python, Ruby and Node.js which facilitates better integration of your app with the website or the platform of your choice The documentation is very easy to follow Lots of built-in entities to ease the development of your chatbots Intel OpenVINO Toolkit Bringing together two of the most talked about technologies today, i.e. Artificial Intelligence and Edge Computing, we had to include Intel’s OpenVINO Toolkit in this list. Short for Open Visual Inference and Neural Network Optimization, this toolkit brings comprehensive computer vision and deep learning capabilities to the edge devices. It has proved to be an invaluable resource to industries looking to set up smart IoT systems for image recognition and processing using edge devices. The OpenVINO toolkit can be used with the commonly used popular frameworks such as OpenCV, Tensorflow as well as Caffe. It can be configured to leverage the power of the traditional CPUs as well as customized AI chips and FPGAs. Not just that, this toolkit also has support for the Vision Processing Unit, a processor developed specifically for machine vision. Why you should try this AI tool out Allows you to develop smart Computer Vision applications for IoT-specific use-cases Support for a large number of deep learning and image processing frameworks. Also, it can be used with the traditional CPUs as well as customized chips for AI/Computer Vision Its distributed capability allows you to develop scalable applications, which again is invaluable when deployed on edge devices You can know more about OpenVINO’s features and capabilities in our detailed coverage of the toolkit. Apache PredictionIO This one is for the machine learning engineers and data scientists looking to build large-scale machine learning solutions using the existing Big Data infrastructure. Apache PredictionIO is an open source, state-of-the-art Machine Learning server which can be easily integrated with the popular Big Data tools such as Apache Hadoop, Apache Spark and Elasticsearch to deploy smart applications. Source: PredictionIO System architecture As can be seen from the architecture diagram above, PredictionIO has modules that interact with the different components of the Big Data system and uses an App Server to communicate the results of the analysis to the outside devices. Why you should try this machine learning tool out Let’s you build production-ready models which can also be deployed as web services You can also leverage the machine learning capabilities of Apache Spark to build large-scale machine learning models Pre-built performance evaluation measures available to check the accuracy of your predictive models Most importantly, this tool helps you simplify your Big Data infrastructure without adding too many complexities IBM Snap ML A machine learning library that is 46 times faster than Tensorflow. If that’s not a reason to start using IBM’s Snap ML, what is? IBM have been taking some giant strides in the field of AI research in a bid to compete with the heavyweights in this space - mainly Google, Microsoft and Amazon. With Snap ML, they seem to have struck a goldmine. A library that can be used for high-speed machine learning models using the cutting edge CPU/GPU technology, Snap ML allows for agile development of models while scaling to process massive datasets. Why you should try this machine learning tool out It is insanely fast. Snap ML was used to train a logistic regression classifier on a terabyte-scale dataset in just under 100 seconds. It allows for GPU acceleration to avoid large data transfer overheads. With the enhanced GPU technology available today, Snap ML is one of the best tools you can have at your disposal to train models quickly and efficiently It allows for distributed model training and works on sparse data structures as well You should definitely check out our detailed coverage of Snap ML where we go into the depth of its features and understand why this is a very special tool. Crypto-ML It is common knowledge that cryptocurrency, especially Bitcoin, can be traded more efficiently and profitably by leveraging the power of machine learning. Large financial institutions and trading firms have been using the machine learning tools to great effect. However, it’s the individuals, on the other hand, who have relied on historical data and outdated techniques to forecast the trends. All that has now changed, thanks to Crypto-ML. Crypto-ML is a cryptocurrency trading platform designed specifically for individuals who want to get the most out of their investments in the most reliable, error-free ways. Using state-of-the-art deep learning techniques, Crypto-ML uses historical data to build models that predict future price movement. At the same time, it eliminates any human error or mistakes arising out of emotions. Why you should try this machine learning tool out No expertise in cryptocurrency trading is required if you want to use this tool Crypto-ML only makes use of historical data and builds data models to predict future prices without any human intervention Per the Crypto-ML website, the average gain on winning trades is close to 53%, whereas the average loss on losing trades is just close to 6%. If you are a data scientist or a machine learning developer with an interest in finance and cryptocurrency, this platform can also help you customize your own models for efficient trading. Here’s where you can read on how Crypto-ML works, in more detail. Other notable mentions Apart from the tools we mentioned above, there are a quite a few other tools that could not make it to the list, but deserve a special mention. Some of them are: ABBYY’s Real-time Recognition SDK for document recognition, language processing and data capturing is worth checking out. Vertex.ai’s PlaidML is an open source tool that allows you to build smart deep learning models across a variety of platforms. It leverages the power of Tile, a new machine learning language that facilitates tensor manipulation. Facebook recently open sourced MUSE, a Python library for efficient word embedding and other NLP tasks. This one’s worth keeping an eye on for sure! If you’re interested in browser-based machine learning, MachineLabs recently open sourced the entire code base of their machine learning platform. NVIDIA’s very own NVVL, their open source offering that provides GPU-accelerated video decoding for training deep learning models The vast ecosystem of tools and frameworks available for building smart, intelligent use-cases across various domains just points to the fact that AI is finding practical applications with every passing day. It is not an overstatement anymore to suggest that that AI is slowly becoming indispensable to businesses. This is not the end of it by any means either - expect to see more such tools spring to life in the near future, with some having game-changing, revolutionary consequences. So which tools are you planning to use for your machine learning / AI tasks? Is there any tool we missed out? Let us know! Read more Predictive Analytics with AWS: A quick look at Amazon ML Four interesting Amazon patents in 2018 that use machine learning, AR, and robotics How to earn $1m per year? Hint: Learn machine learning  
Read more
  • 0
  • 1
  • 4503

article-image-what-the-future-holds-for-privacy-its-got-artificial-intelligence
Guest Contributor
21 Aug 2018
8 min read
Save for later

Do you want to know what the future holds for privacy? It’s got Artificial Intelligence on both sides.

Guest Contributor
21 Aug 2018
8 min read
AI and machine learning are quickly becoming integral parts of modern society. They’ve become common personal and household objects in this era of the Internet of Things. No longer are they relegated to the inner workings of gigantic global corporations or military entities. AI is taking center stage in our very lives and there’s little we can do about it. Tech giants like Google and Amazon have made it very easy for anyone to get their hands on AI-based technology in the form of AI assistants and a plethora of MLaaS (machine-learning-as-a-service) offerings. These AI-powered devices can do anything like telling you the weather, finding you a recipe for your favorite pasta dish, and even letting you know your friend Brad is at the door- and opening that door for you. What’s more, democratized AI tools make it easy for anyone (even without coding experience) to try their hands on building machine learning based apps. Needless to say, a future filled with AI is a future filled with convenience. If Disney’s film “Wall-e” was any hint, we could spend our whole lives a chair while letting self-learning machines do everything we need to do for us (even raising our kids). However, the AI of today could paint an entirely different picture of the future for our privacy. The price of convenience Today’s AI is hungry for your personal information. Of course, this isn’t really surprising seeing as they were birthed by companies like Google that makes most of its yearly income from ad revenue. In one article written by Gizmodo, a privacy flaw was found in Google’s then newest AI creation. The AI assistant would be built into every Google Pixel phone and would run on their messenger app “Allo”. Users could simply ask the assistant questions like “what’s the weather like tomorrow” or “how do I get to Brad’s house”. Therein lies the problem. In order for an AI assistant to adjust according to your own personal preferences, it has to first learn and remember all of your personal information. Every intimate detail that makes you, you. It does this by raking in all the information stored in your device (like your contacts list, photos, messages, location). This poses a huge privacy issue since it means you’re sharing all your personal information with Google (or whichever company manufactures your AI-driven assistant). In the end, no one will know you better than yourself- except Google. Another problem with this AI is that it can only work if your message is unencrypted. You can either opt for more privacy by choosing to use the built-in end-to-end encrypted mode or opt for more convenience by turning off encrypted mode and letting the AI read/listen to your conversations. There is no middle ground yet. Why is this such a big problem? Two reasons: Companies, like Google, use or sell your private information to third parties to make their money; and Google isn’t exactly the most trustworthy with users’ secrets. If your AI manufacturer behaves like Google, that privacy policy that you’re relying on will mean nothing once the government starts knocking on their door. VPNs vs AI How AI learns from your personal information is just the tip of the iceberg. There’s a deeper privacy threat looming just behind the curtain: bad actors waiting to use AI for their own nefarious purposes. One study compared human hackers with artificial hackers to see who could get more Twitter users to click on malicious phishing links. The results showed that artificial hackers substantially outperformed their human counterparts. The artificial hacker pumped out more spear-phishing tweets that resulted in more conversions. This shows how powerful AI can be once it’s weaponized by hackers. Hackers may already be using AI right now- though it’s still hard to tell. Users are not without means to defend themselves, though. VPNs have long been used as a countermeasure against hackers. The VPN industry has even grown due to the recent problems regarding user data and personal information like the Facebook-Cambridge Analytica scandal and how the EU’s GDPR effectively drove many websites to block IPs from the EU. A VPN (Virtual Private Network) protects your privacy by masking your IP. It also routes your internet traffic through secure tunnels where it is encrypted. Most VPNs on the market currently use military-grade 256-bit AES to encrypt your data along with a multitude of various security features. The problem is that anyone with the time and resources can still break through your VPN’s defense- especially if you’re a high profile target. This can either be done by getting the key through some nefarious means or by exploiting known vulnerabilities to break into the VPN’s encryption. Breaking a VPN’s encryption is no easy task as it will take lots of computation and time- we’re talking years here. However, with the rise of AI, the process of breaking a VPN’s encryption may have become easier. Just 2 years ago, DARPA, the US government agency that commissions research for the US Department of Defense, funded the Cyber Grand Challenge. Here, computers were pitted against each other to find and fix bugs in their systems. The winner, a computer named “Mayhem” created by a team named “ForAllSecure”, took home the $2 million prize. It achieved its goal by not only patching any holes it found in its own system but also by finding and exploiting holes in its opponents’ software before they could be patched. Although the whole point of the challenge was to speed up the development of AI to defend against hackers, it also showed just how powerful an artificial hacker can be. A machine that could quickly process heaps and heaps of data while developing more ways to defend/attack from its own processes is a double-edged sword. This is why some VPN companies have started incorporating AI to defend against hackers- human or otherwise. The future of VPNs is AI augmented “If you can’t beat them, join them.” One VPN that has started using AI as part of their VPN service is Perfect Privacy. Their AI takes the form of Neuro routing (AI-based routing). With this, the AI makes a connection based on where the user is connecting to. The AI chooses the closest server to the destination server and does so separately for all connections. This means that if you’re in Romania but you’re connecting to a website hosted in New York, the VPN will choose a New York-based location as an exit server. This not only reduces latency but also ensures that all traffic remains in the VPN for as long as possible. This also makes the user appear to have different IPs on different sites which only bolsters privacy even more. Also, because the AI is dynamic in its approach, it frequently changes its route to be the shortest route possible. This makes its routes nigh impossible to predict. If you’d like a more detailed look at Perfect Privacy and its AI-based routing, check out this Perfect Privacy review. Some experts believe that someday in the future, we may just let AI handle our security in the Internet of Things for us. Just recently this year, a wireless VPN router called “Fortigis” was released and touted AI-based defenses. The router uses self-learning AI to keep your connection safe by learning from attack attempts made on any Fortigis router. All devices are then updated to defend against such attacks thereby ensuring up-to-date security. It also allows you to control who can connect to your home network, alarms you when someone is connecting and informs you of all the devices connected to your home network. These are just some of the ways the VPN industry is keeping up with the security needs of the times. Who knows what else the future could bring just around the corner. Whatever it is, one thing is for sure: Artificial intelligence will be a big part of it. About Author Dana Jackson, an U.S. expat living in Germany and the founder of PrivacyHub. She loves all things related to security and privacy. She holds a degree in Political Science, and loves to call herself a scientist. Dana also loves morning coffee and her dog Paw.   10 great tools to stay completely anonymous online Guide to safe cryptocurrency trading
Read more
  • 0
  • 0
  • 2142
article-image-multi-factor-authentication-system-good-idea-for-an-app
Mehul Rajput
20 Aug 2018
7 min read
Save for later

Multi-Factor Authentication System – Is it a Good Idea for an App?

Mehul Rajput
20 Aug 2018
7 min read
With cyber-attacks on the rise, strong passwords no longer guarantee enough protection to keep your online profiles safe from hackers. In fact, other security features such as antivirus software, encryption technology, firewall deployment, etc. are also susceptible to being bypassed by hackers when targeted explicitly and dedicatedly. A multi-factor authentication (MFA) system adds another layer of app security to ensure enhanced data safety. According to a survey, hackers use weak or stolen user credentials in a staggering 95% of all web application attacks. MFA implementation can prevent unauthorized access to your personal accounts, even if someone manages to steal your sign-in details. It has  low complexity, and the application does not require significant amount of time or resources. What is Multi-Factor Authentication? Multi-factor Authentication emerged as a reaction to the vulnerability and susceptibility of the existing security systems. It is a method that confirms the users’ identity multiple times, before granting them access. These pieces of evidence validating a user’s identity include: Knowledge factor: something you know (for e.g. a username, password, security question) Possession factor: something you have (for e.g. a registered phone number, hardware or software token that generate authentication code, smartcard) Inherence factor: something you are (biometric information such as a finger, face, or voice recognition, retina scans) When a system utilizes two or more verification mechanisms, it is known as a multi-factor authentication (MFA). The ultimate idea behind MFA is that the more number of steps a user has to take to access sensitive information, the harder it becomes for the hacker to breach the security. One of the most common methods of authentication is a password coupled with a verification code of unique string of numbers sent via SMS or email. This method is commonly used by Google, Twitter, and other popular services. iPhone X’s Face ID and Windows Hello use the latest innovations in advanced biometric scanners for fingerprints, retinas, or faces, that are built-in the devices. Moreover, you can also use a specialized app on your phone called an “authenticator”. The app is pre-set to work for a service and receives the codes that can be used whenever needed. Popular authentication apps include Google Authenticator, DuoMobile, and Twilio Authy. The authentication apps are more secure when compared to receiving codes via SMS. This is primarily because text messages can be intercepted and phone numbers can be hijacked. On the other hand, authentication apps do not rely on your service carriers. In fact, they function even in the absence of cell service. Importance of Multi-factor Authentication System Is MFA worth the hassle of additional verification? Yes, it absolutely is. The extra layer of security can save valuable and sensitive personal information from falling into the wrong hands. Password theft is constantly evolving. Hackers employ numerous methods including phishing, pharming, brute force, and keylogging to break into online accounts. Moreover, anti-virus systems and advanced firewalls are often incompetent and inefficient without user authentication. According to a Gemalto report, more than 2.5 billion data records were lost, stolen, or exposed worldwide in 2017, an 88% increase from 2016. Furthermore, cyber-attacks rake up huge financial losses to the compromised organization and even mere individuals; basically anyone connected to the internet. It is estimated that by 2021, cyber-crime will cause global financial damages of around $6 trillion annually. Despite the alarming statistics, only 38% of the global organizations are prepared to combat a cyber-attack. MFA implementation can mitigate cyber-attacks considerably. Organizations with multi-fold authentication in place can strengthen their access security. It not only will help them safeguard the personal assets of their employees and customers, but also protect the company’s integrity and reputation. Why Multi-factor Authentication System in Apps is good Numerous variables are taken into consideration during the app development process. You want the app to have a friendly user interface that provides a seamless experience. An appealing graphical design and innovative features are also top priorities. Furthermore, apps undergo rigorous testing to make them bug-free before releasing into the market. However, security breaches can taint the reputation of your app, especially if it holds sensitive information about the users. Here is why MFA is a good idea for your app: Intensified security As mentioned earlier, MFA can bolster the protection and reduce the risk associated with only password-protected apps. Additional means of authentication not only challenges the users to prove their identity, it can also provide the security team with broader visibility into a possible identity theft. Moreover, it is not necessary to prompt the user for MFA every time they log into the app. You can use data analytics to trigger MFA for a risk-based approach. Take into account the user’s geographical location, IP address, device in use, etc. before challenging the user’s identity and asking for additional authentication. High-risk scenarios that justify MFA include logging in from an unknown device or new location, accessing the app from a new IP address, or attempting to gain admission into a highly sensitive resource for the first time. Opt for risk-based approach only if your app holds valuable and intimate information about your client that can cause irrevocable personal damage to the user if divulged. Otherwise, such an approach requires complex data analytics, machine learning, and contextual recognition that can be difficult and time-consuming to program. Simplified login process You may consider MFA implementation as complicated and cumbersome. However, if you have multiple apps under your helm, you can offer more advanced login solutions like single sign-on. Once the user identity is validated, they can access multiple apps covered under the single sign-on. This practice provides practicality to the MFA process as the users are saved from the fatigue and stress of repeated logins. Increased customer satisfaction A customer’s satisfaction and trust is one of the biggest driving factors for any organization. When you offer MFA to your users, it builds a sense of trustworthiness amongst them and they are more at ease when sharing personal details. Compliance with standards In addition to the benefits to the users, there are certain compliance standards, mandated by state, federal or other authorities, which specify that companies should implement MFA in explicit situations. Moreover, there are fixed guidelines from the National Institute of Standards and Technology (NIST) that help you choose the right verification methods. Therefore, it is imperative that you do not only comply with the regulations but also implement the recommended MFA methods. The key is to deploy an MFA system that is not too laborious but offers optimal steps of authentication. Given the sheer number of methods available for MFA, choose the most appropriate options based on: Sensitivity of the data and assets being protected Convenience and ease of usability for the customers Compliance with the specific regulations Expediting implementation and management for IT department Summary MFA can strengthen the security of sensitive data and protect the user’s identity. It adds another layer of shield to safeguard the client’s online accounts, obstructing the efforts of dedicated hacking. Moreover, it allows you to comply with the standard guidelines proposed by the authorized officials. However, individual MFA implementation across different user environments and cloud services can be inconvenient to the users. Deploy single sign-on or adopt risk-based approach to eliminate security vulnerability while facilitating user access. Author Bio Mehul Rajput is a CEO and co-founder of Mindinventory which specializes in Android and iOS app development and provide web and mobile app solutions from startup to enterprise level businesses. He is an avid blogger and writes on mobile technologies, mobile app, app marketing, app development, startup and business. 5 application development tools that will matter in 2018 Implement an API Design-first approach for building APIs [Tutorial] Access application data with Entity Framework in .NET Core [Tutorial]
Read more
  • 0
  • 10
  • 8375

article-image-5-examples-of-artificial-intelligence-in-web-apps
Sugandha Lahoti
20 Aug 2018
7 min read
Save for later

5 examples of Artificial Intelligence in Web apps

Sugandha Lahoti
20 Aug 2018
7 min read
Modern day web app development is increasingly focused on building a customer-facing front-end presence with the use of Artificial Intelligence. Web apps, use Artificial Intelligence not just for intelligent automation, but also for building recommendation engines, website implementation, and image recognition, among other application areas. In this post, we look at five key areas, illustrated by real-world examples, where web apps are employing Artificial intelligence to automate some part of their system. Recommendation Engines of Amazon and Netflix Curating content based on the user’s context is one of the most widely used AI features in web apps. Amazon, for instance, uses item-based collaborative filtering for product classification. Amazon’s recommendation system uses a combination of goods-based recommendation (users are recommended for those similar to what they liked in the past) and buddy-based recommendation (users are recommended things which their Facebook friends like.) Not just for their recommendation system, Amazon has been using AI for multiple tasks. Their AI Management Strategy is called The Flywheel, where one part of Amazon acts as a catalyst for AI and machine learning growth in other areas. Read more: Four interesting Amazon patents in 2018 that use machine learning, AR, and robotics Another popular example is Netflix, who revamped their recommendation algorithm based on visual impressions. One of their research projects indicated that the artwork was not only the biggest influencer to a viewer's decision to watch content, but it also drew over 82% of their focus while browsing Netflix. This made them develop a new image recommendation algorithm which works in real time to project the image it thinks the user will respond to. They use implicit (user behavior) and Explicit data (user activity) and then feed this data to machine learning algorithms to figure out the relevant content for each user. For each title, users get the image with the highest rank based on their profile. Side by side, it continues collecting data from its 100 million other subscribers to improve its engine’s performance. Read more: What software stack does Netflix use? Google and Microsoft using Image recognition Image recognition can serve multiple uses for web apps including object and pattern recognition, locating duplicates (exact or partial), image search by fragments, and more. Two such unique applications of image recognition are Google’s Quickdraw and Microsoft’s Captionbot.ai. Quick Draw is Google’s AI-powered web app game, where users have to draw an everyday object that a neural network tries to recognize. Players are given 20 seconds to draw a random item, and Google’s neural network tries to match it with other 50 million hand-drawn sketches by other players to identify the correct one. Quickdraw aims to generate the world’s largest doodling data set, which is shared publicly to help further machine learning research. The data preserves user privacy by collecting only anonymous metadata, including timestamp, country code, whether or not the drawing was recognized, and which word the drawing corresponded to. This dataset was used in SketchRNN, a neural network that can draw words and interpolate between drawings. Another image recognition web app is Microsoft’s Captionbot.ai. The system can automatically generate a caption for an uploaded photograph. Users can rate how accurately it has detected what was on display. The algorithm learns from the rating, to make the captions more accurate. It uses three separate services to process the images. The Computer Vision API identifies the components of the photo, then mixes it with data from the Bing Image API, and runs any faces it spots through Emotion API. The Emotion API analyses facial expressions to detect anger, contempt, disgust, fear, and other traits. Based on the results from these APIs, the caption is generated. Google Docs powered by Natural Language Processing Modern Web apps can also be fueled with cognitive capabilities to make them stand apart from other apps. Instances of this include transforming human speech to text or conversing with people in natural language. One such example of a web app which includes natural language processing is Google Docs. Google Docs and Slides have an Explore feature to show text, images, and other features relevant to the document that a user is working on at any given point.  Docs can also use natural language to search through data and reports, and automatically generate formulas in Sheets. Google Docs recently incorporated an AI grammar checker, announced at Google Cloud Next. It uses a machine translation algorithm to recognize errors and suggest corrections as users type. Google Docs can also be integrated with Natural Language API to recognize the sentiment of selected text in a Google Doc and highlight it based on that sentiment. Web-based artificial intelligence Chatbots Web-based chatbots are just like app-based chatbots albeit they interact with users in the website browser. They use AI techniques such as natural language understanding and pattern recognition to store and distinguish between the context of the information provided and elicit a suitable response for future replies. An example of web-based chatbots are the Live Chat bots where the conversation with a visitor on a website is automated using a chatbot. Many live chat software companies are already experimenting with chatbots. Examples include the Operator bot used by Intercom, a company building customer messaging platform or Driftbot by Drift which gives your website a personal assistant. Read More: Top 4 chatbot development frameworks for developers Another example, are AI based chatbots which help in creating full websites. Right Click is a startup that introduced an A.I.-powered chatbot which uses Artificial Intelligence in a conversational interface to create websites. It asks general questions during the conversation like “What industry you belong to?” and “Why do you want to make a website?” and creates customized templates as per the given answers. Similarly, Wix’s Artificial Intelligence Design bot can tailor websites by learning about each person’s or business’ own needs. Web-based code helpers using AI Intelligent coding assistants are gaining popularity with their ability to understand the code and provide right suggestions at the right time. They can analyze code on the web and give fast and smart completions. Codota for Chrome is a smart web-based IDE which can build predictive models of code and suggest code completions and related content based on the current context present in the code. It combines program analysis, natural language processing, and machine learning to learn from the code. Users can look for Codota’s Icon on every code snippet on their browsers - in GitHub, StackOverflow and others. Another example is Deep Cognition’s Deep Learning Studio – Cloud. It is not exactly an IDE, but it features AI-powered drag & drop interface to help design deep learning models with ease. It features assisted modeling, for automated tensor size calculations and real-time validation. It also has AutoML feature to automatically build a neural network. [dropcap]E[/dropcap]ven though AI is a great choice to enhance your web apps, an important facet to keep in mind is ensuring fairness, accuracy, and transparency of your web apps. For instance, web apps powered by natural language should not discriminate people based on caste, color, or creed or hurt user sentiments. Similarly, those using neural networks for recognizing images should ensure the filtering of obscene images. Creating such types of artificial intelligence systems would require a hybrid of designers, programmers, ML engineers, and researchers. This collective group will have a good grasp of user experience, will be comfortable thinking in abstracts and algorithms, and equally well versed with the social impacts of artificial intelligence. Read More: 20 lessons on bias in machine learning systems by Kate Crawford at NIPS 2017 Uber introduces Fusion.js, a plugin-based web development framework for high-performance apps. Electron Fiddle: A ‘code playground’ for experimenting with cross-platform native apps. Warp: Rust’s new web framework for implementing WAI (Web Application Interface)
Read more
  • 0
  • 0
  • 27525

article-image-top-4-facebook-patents-to-battle-fake-news-and-improve-its-news-feed
Sugandha Lahoti
18 Aug 2018
7 min read
Save for later

Four 2018 Facebook patents to battle fake news and improve news feed

Sugandha Lahoti
18 Aug 2018
7 min read
The past few months saw Facebook struggling to maintain its integrity considering the number of fake news and data scandals linked to it - Alex Jones, accusations of discriminatory advertising and more. Not to mention, Facebook Stocks fell $120 billion in market value after Q2 2018 earnings call. Amidst these allegations of providing fake news and allowing discriminatory content on its news feed, Facebook patented its news feed filter tool last week to provide more relevant news to its users. In the past also, Facebook has made several interesting patents to enhance their news feed algorithm in order to curb fake news. This made us look into what other recent patents that Facebook have been granted around news feeds and fake news. Facebook’s News Feed has always been one of its signature features. The news feed is generated algorithmically (instead of chronologically), with a mix of status updates, page updates, and app updates that Facebook believes are interesting and relevant to you. Officially Facebook, successfully patented its News Feed in 2012, after filing for it in 2006. The patent gave the company a stronghold on the ability to let users see status messages, pictures, and links to videos of online friends, but also the actions those friends take. [box type="shadow" align="" class="" width=""]Note: According to United States Patent and Trademark Office (USPTO), Patent is an exclusive right to invention and “the right to exclude others from making, using, offering for sale, or selling the invention in the United States or “importing” the invention into the United States”.[/box] Here are four Facebook patents in 2018 pertaining to news feeds that we found interesting. Dynamically providing a feed of stories Date of Patent: April 10, 2018 Filed: December 10, 2015 Features: Facebook filed this patent to present their news feed in a more dynamic manner suiting to a particular person. Facebook’s News feed automatically generates a display that contains information relevant to a user about another user. This patent is titled Dynamically providing a feed of stories about a user of a social networking system. As per the patent application, recently, social networking websites have developed systems for tailoring connections between various users. Typically, however, these news items are disparate and disorganized. The proposed method generates news items regarding activities associated with a user. It attaches an informational link associated with at least one of the activities, to at least one of the news items. The method limits access to the news items to a predetermined set of viewers and assigns an order to the news items. Source: USPTO This patent is a viable solution to limit access to the news items which a particular section of users may find obscene. For instance, Facebook users below the age of 18, may be restricted from viewing graphic content. The patent received criticism with people ridiculing the patent for seeming to go against everything that the patent system is supposed to do. They say that such automatically generated news feeds are found in all sorts of systems and social networks these days. But now Facebook may have the right to prevent others from doing, what other social networks are inherently supposed to do. Generating a feed of content items from multiple sources Date of Patent: July 3, 2018 Filed: June 6, 2014 Features:  Facebook filed a patent allowing a feed of content items associated with a topic to be generated from multiple content sources. Per the Facebook patent, their newsfeed generation system receives content items from one or more content sources. It matches the content items to topics based on a measure of the affinity of each content item for one or more objects. These objects form a database that is associated with various topics. The feed associated with the topic is communicated to a user, allowing the user to readily identify content items associated with the topic. Source: USPTO Let us consider the example of sports. A sports database will contain an ontology defining relationships between objects such as teams, athletes, and coaches. The news feed system for a particular user interested in sports (an athlete or a coach or a player) will cover all content items associated with sports. Selecting organic content and advertisements based on user engagement Date of Patent: July 3, 2018 Filed: June 6, 2014 Features: Facebook wants to dynamically adjust its organic content items and advertisements, generated to a user by modifying a ranking. Partial engagement scores will be generated for organic content items based on an expected amount of user interaction with each organic content item. Advertisements scores will be generated based on expected user interaction and bid amounts associated with each organic content item. These advertisement and partial engagement scores are next used to determine two separator engagement scores measuring the user's estimated interaction with a content feed. One engagement score is of organic content items with advertisements and one without them. A difference between both these scores will modify a conversion factor used to combine expected user interaction and bid amounts to generate advertisement scores. This mechanism has been patented by Facebook as Selecting organic content and advertisements for presentation to social networking system users based on user engagement. For example, if a large number of advertisements are presented to a user, the user may become frustrated with the increased difficulty in viewing stories and interact less with the social networking system. However, advertisements also generate additional revenue for the social networking system. A balance is necessary. So, if the engagement score is greater than the additional engagement score by at least a threshold amount, the conversion factor is modified (e.g., decreased) to increase the number of organic content items included in the feed. If the engagement score is greater than the additional engagement score but less than the threshold amount, the conversion factor is modified (e.g., increased) to decrease the number of organic content items included in the feed. Source: USPTO Displaying news ticker content in a social networking system Date of Patent: January 9, 2018 Filed: February 10, 2016 Features: Facebook has also patented, Displaying news ticker content in a social networking system. This Facebook patent describes a system that displays stories about a user’s friends in a news ticker, as friends perform actions. The system monitors in real time for actions associated with users connected with the target user. The news ticker is updated such that stories including the identified actions and the associated connected users are displayed within a news ticker interface. The news ticker interface may be a dedicated portion of the website’s interface, for example in a column next to a newsfeed. Additional information related to the selected story may be displayed in a separate interface. Source: USPTO For example, a user may select a story displayed in the news ticker; let’s say movies. In response, additional information associated with movies (such as actors, director, songs etc) may be displayed, in an additional interface. The additional information can also depend on the movies liked by the friends of the target user. These patents talk lengths of how Facebook is trying to repair its image and make amendments to its news feed algorithms to curb fake and biased news. The dynamic algorithm may restrict content, the news ticket content and multiple source extractions will keep the feed relevant, and the balance between organic content and advertisements could lure users to stay on the site. As such there are no details currently on when or if these features will hit the Facebook feed, but once implemented could bring Zuckerberg’s vision of “bringing the world close together”, closer to reality. Read Next Four IBM facial recognition patents in 2018, we found intriguing Facebook patents its news feed filter tool to provide more relevant news to its users Four interesting Amazon patents in 2018 that use machine learning, AR, and robotics
Read more
  • 0
  • 0
  • 2769
article-image-how-everyone-at-netflix-uses-jupyter-notebooks-from-data-scientists-machine-learning-engineers-to-data-analysts
Bhagyashree R
18 Aug 2018
4 min read
Save for later

How everyone at Netflix uses Jupyter notebooks from data scientists, machine learning engineers, to data analysts

Bhagyashree R
18 Aug 2018
4 min read
Netflix uses a variety of tools to do data analysis. One of the big ways that data scientists and engineers at Netflix interact with their data is through Jupyter notebooks. In addition to providing execution environments to users, Netflix invests in various parts of the Jupyter ecosystem and tooling. They are “reimagining what a notebook can be, who can use it, and what they can do with it.” Netflix aims to provide personalized content to their 130 million viewers. For this every day more than 1 trillion events are written into a streaming ingestion pipeline. To support this, they’ve built an industry-leading data platform which is flexible, powerful, and complex. There are so many diverse users of this platform, such as analytics engineers, data engineers, and data scientists, requiring different sets of tools and languages. To help the platform scale, they wanted to minimize the number of tools and the solution to this was the open-source tool: Jupyter notebooks. Why Jupyter notebook is so compelling for Netflix? These are the functionalities provided by notebook that benefits Netflix’s data scientists and engineers: Standard messaging API: The Jupyter protocol provides a standard messaging API with the kernels that act as computational engines. It separates where the content is written and where the content is executed. This makes it language agnostic. Editable file format: It provides an editable file format that stores the code and results together. Web-based UI: It is web-based which helps interactively writing and running code as well as visualizing outputs. How Netflix uses Jupyter Notebooks? The following are some of the use cases they use Jupyter notebooks for: Data access: Notebooks were first introduced for workflows and their adoption grew among the data scientists. Seeing this, Netflix decided to leverage its versatility and architecture for general data access. Notebooks provide an user-friendly interface for interactively running code, exploring the outputs, and visualizing data all from a single cloud-based development environment. Notebook Templates: They introduced parameterized notebooks, which allow the use of parameters in the code and take values as input at runtime. These templates help: Data scientists to run an experiment with different coefficients and summarize the results Data engineers to execute data quality audits Data analysts to share prepared queries and visualizations Software engineers to email the results of a troubleshooting script Scheduling notebooks: Next they are using notebooks for creating a unifying layer for scheduling workflows. Notebooks are used for interactive work and allows smooth move to scheduling that work to run recurrently. Many users create an entire workflow in a notebook and just copy/paste it into separate files for scheduling when they’re ready to deploy it. Notebook infrastructure: The three fundamental components of the infrastructure are: storage, compute, and interface. Source: Netflix Tech Blog Storage: The Netflix Data Platform is made of Amazon S3 and EFS for cloud storage, which notebooks treat as virtual filesystems. Each user has a home directory on EFS containing a personal workspace for notebooks. This workspace is for storing any notebook created or uploaded by a user. When a user launches a notebook interactively, all the reading and writing happens at the workspace. Compute: All the jobs on the data platform run on containers including queries, pipelines and notebooks. A container with reasonable default resources is provisioned when a user launches a notebook. Users can request more resources if they find that the provided resources are not enough. A unified execution environment with a prepared container image is provided, which has common libraries and an array of default kernels preinstalled. The orchestration and environments are managed with Titus, their container management platform. Interface: They are using nteract, a React-based frontend for Jupyter notebooks, which emphasizes simplicity and composability as core design principles.They’re also introducing native support for parameterization, which makes it easier to schedule notebooks and create reusable templates. Netflix is planning to make investments in both the frontend and backend to improve the overall notebook experience. This year they are also sponsoring JupyterCon. To read more about how Jupyter is offering value to Netflix read Netflix’s original post at Medium. 10 reasons why data scientists love Jupyter notebooks What’s new in Jupyter Notebook 5.3.0 Netflix open sources Zuul 2 cloud gateway
Read more
  • 0
  • 0
  • 8654

article-image-jakarta-ee-past-present-and-future
David Heffelfinger
16 Aug 2018
10 min read
Save for later

Jakarta EE: Past, Present, and Future

David Heffelfinger
16 Aug 2018
10 min read
You may have heard some talk about a new Java framework called Jakarta EE, in this article we will cover what Jakarta EE actually is, how we got here, and what to expect when it’s actually released. History and Background In September of 2017, Oracle announced it was donating Java EE to the Eclipse Foundation. Isn’t Eclipse a Java IDE? Most Java developers are familiar with the hugely popular Eclipse IDE, therefore for many, when they hear the word “Eclipse”, the Eclipse IDE comes to mind. Not everybody knows that the Eclipse IDE is developed by the Eclipse Foundation, an open source foundation similar to the Apache Foundation and the Linux Foundation. In addition to the Eclipse IDE, the Eclipse Foundation develops several other Java tools and APIs such as Eclipse Vert.x, Eclipse Yasson, and EclipseLink. Java EE was the successor to J2EE; which was a wildly popular set of specifications for implementing enterprise software. In spite of its popularity, many J2EE APIs were cumbersome to use and required lots of boilerplate code. Sun Microsystems, together with the Java community as part of the Java Community Process (JCP), replaced J2EE with Java EE in 2006. Java EE introduced a much nicer, lightweight programming model, making enterprise Java development much more easier than what could be accomplished with J2EE. J2EE was so popular that, to this day, it is incorrectly used as a generic term for all server-side Java technologies. Many, to this day still refer to Java EE as J2EE, and incorrectly assume Java EE is a bloated, convoluted technology. In short, J2EE was so popular that even Java EE can’t shake its predecessor’s reputation for being a “heavyweight” technology. In 2010 Oracle purchased Sun Microsystems, and became the steward for Java technology, including Java EE. Java EE 7 was released in 2013, after the Sun Microsystems acquisition by Oracle, simplifying enterprise software development even further, and adding additional APIs to meet new demands of enterprise software systems. Work on Java EE 8, the latest version of the Java EE specification, began shortly after Java EE 7 was released. In the beginning everything seemed to be going well, however  in early 2016, the Java EE community started noticing a lack of progress in Java EE 8, particularly Java Specification Requests (JSRs) led by Oracle. The perceived lack of Java EE 8 progress became a big concern for many in the Java EE community. Since the specifications were owned by Oracle, there was no legal way for any other entity to continue making progress on Java EE 8. In response to the perceived lack of progress, several Java EE vendors, including big names such as IBM and Red Hat, got together and started the Microprofile initiative, which aimed to introduce new APIs to Java EE, with a focus on optimizing Java EE for developing systems based on a microservices architecture. The idea wasn’t to compete with Java EE per se, but to develop new specifications in the hopes that they would be eventually added to Java EE proper. In addition to big vendors reacting to the perceived Java EE progress, a grassroots organization called the Java EE Guardians was formed, led largely by prominent Java EE advocate Reza Rahman. The Java EE Guardians provided a way for Java EE developers and advocates to have a united, collective voice which could urge Oracle to either keep working on Java EE 8, or to allow the community to continue the work themselves. Nobody can say for sure how much influence the Microprofile initiative and Java EE Guardians had, but many speculate that Java EE would have never been donated to the Eclipse Foundation had it not been for these two initiatives. One Standard, Multiple Implementations It is worth mentioning that Java EE is not a framework per se, but a set of specifications for various APIs. Some examples of Java EE specifications include the Java API for RESTful Web Services (JAX-RS), Contexts and Dependency Injection (CDI), and the Java Persistence API (JPA). There are several implementations of Java EE, commonly known as application servers or runtimes, examples include Weblogic, JBoss, Websphere, Apache Tomee, GlassFish and Payara. Since all of these implement the Java EE specifications, code written against one of these servers can easily be migrated to another one, with minimal or no modifications. Coding against the Java EE standard provides protection against vendor lock-in. Once Jakarta EE is completely migrated to the Eclipse Foundation, it will continue being a specification with multiple implementations, keeping one of the biggest benefits of Java EE. To become Java EE certified, application server vendors had to pay Oracle a fee to obtain a Technology Compatibility Kit (TCK), which is a set of tests vendors can use to make sure their products comply 100% with the Java EE specification. The fact that the TCK is closed source and not publicly available has been a source of controversy among the Java EE community. It is expected that the TCK will be made publicly available once the transition to the Eclipse Foundation is complete. From Java EE to Jakarta EE Once the announcement of the donation was made, it became clear that for legal reasons Java EE would have to be renamed, as Oracle owns the “Java” trademark. The Eclipse Foundation requested input from the community, hundreds of suggestions were submitted. The Foundation made it clear that naming such a big project is no easy task, there are several constraints that may not be obvious to the casual observer, such as: the name must not be trademarked in any country, it must be catchy, and it must not spell profanity in any language. Out of hundreds of suggestions, the Eclipse Foundation narrowed them down to two choices, “Enterprise Profile” and “Jakarta EE”, and had the community vote for their favorite. “Jakarta EE” won by a fairly large margin. It is worth mentioning that the name “Jakarta” carries a bit of history in the Java world, as it used to be an umbrella project under the Apache Foundation. Several very popular Java tools and libraries used to fall under the Jakarta umbrella, such as the ANT build tool, the Struts MVC framework, and many others. Where we are in the transition Ever since the announcement, the Eclipse Foundation along with the Java EE community at large has been furiously working on transitioning Java EE to the Eclipse Foundation. Transitioning such a huge and far reaching project to an open source foundation is a huge undertaking, and as such it takes some time. Some of the progress so far includes relicensing all Oracle led Java EE technologies, including reference implementations (RI), Technology Compatibility Kits (TCK) and project documentation.  39 projects have been created under the Jakarta EE umbrella, corresponding to 39 Java EE specifications being donated to the Eclipse Foundation. Reference Implementations Each Java EE specification must include a reference implementation, which proves that the requirements on the specification can be met by actual code. For example, the reference implementation for JSF is called Mojarra, the CDI reference implementation is called Weld, and the JPA is called EclipseLink. Similarly, all other Java EE specifications have a corresponding reference implementation. These 39 projects are in different stages of completion, a small minority are still in the proposal stage; some have provisioned committers and other resources, but code and other artifacts hasn’t been transitioned yet; some have had the initial contribution (code and related content) transitioned already, the majority of the projects have had the initial contribution committed to the Eclipse Foundation’s Git repository, and a few have had their first Release Review, which is a formal announcement of the project’s release to the Eclipse Foundation, and a request for feedback. Current status for all 39 projects can be found at https://www.eclipse.org/ee4j/status.php. Additionally, the Jakarta EE working group was established, which includes Java EE implementation vendors, companies that either rely on Java EE or provide products or services complementary to Java EE, as well as individuals interested in advancing Jakarta EE. It is worth noting that Pivotal, the company behind the popular Spring Framework, has joined the Jakarta EE Working Group. This is worth pointing out as the Spring Framework and Java EE have traditionally been perceived as competing technologies. With Pivotal joining the Jakarta EE Working Group some are speculating that “the feud may soon be over”, with Jakarta EE and Spring cooperating with each other instead of competing. At the time of writing, it has been almost a year since the announcement that Java EE is moving to the Eclipse foundation, some may be wondering what is holding up the process. Transitioning a project of such a massive scale as Java EE involves several tasks that may not be obvious to the casual observer, both tasks related to legal compliance as well as technical tasks. For example, each individual source code file needs to be inspected to make sure it has the correct license header. Project dependencies for each API need to be analyzed. For legal reasons, some of the Java EE technologies need to be renamed, appropriate names need to be found. Additionally, build environments need to be created for each project under the Eclipse Foundation infrastructure. In short, there is more work than meets the eye. What to expect when the transition is complete The first release of Jakarta EE will be 100% compatible with Java EE. Existing Java EE applications, application servers and runtimes will also be Jakarta EE compliant. Sometime after the announcement, the Eclipse Foundation surveyed the Java EE community as to the direction Jakarta EE should take under the Foundation’s guidance. The community overwhelmingly stated that they want better support for cloud deployment, as well as better support for microservices. As such, expect Jakarta EE to evolve to better support these technologies. Representatives from the Eclipse Foundation have stated that release cadence for Jakarta EE will be more frequent than it was for Java EE when it was under Oracle. In summary, the first version of Jakarta EE will be an open version of Java EE 8, after that we can expect better support for cloud and microservices development, as well as a faster release cadence. Help Create the Future of Jakarta EE Anyone, from large corporations to individual contributors can contribute to Jakarta EE. I would like to invite interested readers to contribute! Here are a few ways to do so: Subscribe to Jakarta EE community mailing list: jakarta.ee-community@eclipse.org Contribute to EE4J projects: https://github.com/eclipse-ee4j You can also keep up to date with the latest Jakarta EE happenings by following Jakarta EE on Twitter at @JakartaEE or by visiting the Jakarta EE web site at https://jakarta.ee About the Author David R. Heffelfinger David R. Heffelfinger is an independent consultant based in the Washington D.C. area. He is a Java Champion, an Apache NetBeans committer, and a former member of the JavaOne content committee. He has written several books on Java EE, application servers, NetBeans, and JasperReports. David is a frequent speaker at software development conferences such as JavaOne, Oracle Code and NetBeans Day. You can follow him on Twitter at @ensode.  
Read more
  • 0
  • 0
  • 6143