How-To Tutorials - Front-End Web Development

In this article by Troy Miles, author of the book jQuery Essentials, we will learn that an event is the occurrence of anything that the system considers significant. It can originate in the browser, the form, the keyboard, or any other subsystem, and it can also be generated by the application via a trigger. An event can be as simple as a key press or as complex as the completion of an Ajax request. (For more resources related to this topic, see here.) While there are a myriad of potential events, events only matter when the application listens for them. This is also known as hooking an event. By hooking an event, you tell the browser that this occurrence is important to you and to let you know when it happens. When the event occurs, the browser calls your event handling code passing the event object to it. The event object holds important event data, including which page element triggered it. Let's take a look at the first learned and possibly most important event, the ready event. The ready event The first event that programmers new to jQuery usually learn about is the ready event, sometimes referred to as the document ready event. This event signifies that the DOM is fully loaded and that jQuery is open for business. The ready event is similar to the document load event, except that it doesn't wait for all of the page's images and other assets to load. It only waits for the DOM to be ready. Also, if the ready event fires before it is hooked, the handler code will be called at least once unlike most events. The .ready() event can only be attached to the document element. When you think about it, it makes sense because it fires when the DOM, also known as Document Object Model is fully loaded. The .ready() event has few different hooking styles. All of the styles do the same thing—hook the event. Which one you use is up to you. In its most basic form, the hooking code looks similar to the following: $(document).ready(handler); As it can only be attached to the document element, the selector can be omitted. In which case the event hook looks as follows: $().ready(handler); However, the jQuery documentation does not recommend using the preceding form. There is still a terser version of this event's hook. This version omits nearly everything, and it is only passing an event handler to the jQuery function. It looks similar to the following: $(handler); While all of the different styles work, I only recommend the first form because it is the most clear. While the other forms work and save a few bytes worth of characters, they do so at the expense of code clarity. If you are worried about the number of bytes an expression uses, you should use a JavaScript minimizer instead. It will do a much more thorough job of shrinking code than you could ever do by hand. The ready event can be hooked as many times as you'd like. When the event is triggered, the handlers are called in the order in which they were hooked. Let's take a look at an example in code. // ready event style no# 1 $(document).ready(function () { console.log("document ready event handler style no# 1"); // we're in the event handler so the event has already fired. // let's hook it again and see what happens $(document).ready(function () { console.log("We get this handler even though the ready event has already fired"); }); }); // ready event style no# 2 $().ready(function () { console.log("document ready event handler style no# 2"); }); // ready event style no# 3 $(function () { console.log("document ready event handler style no# 3"); }); In the preceding code, we hook the ready event three times, each one using a different hooking style. The handlers are called in the same order that they are hooked. In the first event handler, we hook the event again. As the event has been triggered already, we may expect that the handler will never be called, but we would be wrong. jQuery treats the ready event differently than other events. Its handler is always called, even if the event has already been triggered. This makes the ready event a great place for initialization and other code, which must be run. Hooking events The ready event is different as compared to all of the other events. Its handler will be called once, unlike other events. It is also hooked differently than other events. All of the other events are hooked by chaining the .on() method to the set of elements that you wish to use to trigger the event. The first parameter passed to the hook is the name of the event followed by the handling function, which can either be an anonymous function or the name of a function. This is the basic pattern for event hooking. It is as follows: $(selector).on('event name', handling function); The .on() method and its companion the .off() method were first added in version 1.7 of jQuery. For older versions of jQuery, the method that is used to hook the event is .bind(). Neither the .bind() method nor its companion the .unbind() method are deprecated, but .on() and .off() are preferred over them. If you are switching from .bind(), the call to .on() is identical at its simplest levels. The .on() method has capabilities beyond that of the .bind() method, which requires different sets of parameters to be passed to it. If you would like for more than one event to share the same handler, simply place the name of the next event after the previous one with a space separating them: $("#clickA").on("mouseenter mouseleave", eventHandler); Unhooking events The main method that is used to unhook an event handler is .off(). Calling it is simple; it is similar to the following: $(elements).off('event name', handling function); The handling function is optional and the event name is also optional. If the event name is omitted, then all events that are attached to the elements are removed. If the event name is included, then all handlers for the specified event are removed. This can create problems. Think about the following scenario. You write a click event handler for a button. A bit later in the app's life cycle, someone else also needs to know when the button is clicked. Not wanting to interfere with already working code, they add a second handler. When their code is complete, they remove the handler as follows: $('#myButton').off('click'); As the handler was called using only using the event name, it removed not only the handler that it added but also all of the handlers for the click event. This is not what was wanted. Don't despair however; there are two fixes for this problem: function clickBHandler(event){ console.log('Button B has been clicked, external'); } $('#clickB').on('click', clickBHandler); $('#clickB').on('click', function(event){ console.log('Button B has been clicked, anonymous'); // turn off the 1st handler without during off the 2nd $('#clickB').off('click', clickBHandler); }); The first fix is to pass the event handler to the .off() method. In the preceding code, we placed two click event handlers on the button named clickB. The first event handler is installed using a function declaration, and the second is installed using an anonymous function. When the button is clicked, both of the event handlers are called. The second one turns off the first one by calling the .off() method and passing its event handler as a parameter. By passing the event handler, the .off() method is able to match the signature of the handler that you'd like to turn off. If you are not using anonymous functions, this fix is works well. But, what if you want to pass an anonymous function as the event handler? Is there a way to turn off one handler without turning off the other? Yes there is, the second fix is to use event namespacing. Summary In this article, we learned a lot about one of the most important constructs in modern web programming—events. They are the things that make a site interactive. Resources for Article: Further resources on this subject: Preparing Your First jQuery Mobile Project[article] Building a Custom Version of jQuery[article] Learning jQuery[article]

Ember.js is a fantastic framework for developers and designers alike for building ambitious web applications. As touted by its website, Ember.js is built for productivity. Designed with the developer in mind, its friendly API’s help you get your job done fast. It also makes all the trivial choices for you. By taking care of certain architectural choices, it lets you concentrate on your application instead of reinventing the wheel or focusing on already solved problems. With Ember.js you will be empowered to rapidly prototype applications with more features for less code. Although Ember.js follows the MVC (Model-View-Controller) design pattern, it’s been slowly moving to a more component centric approach of building web applications. On this part 1 of 2 blog posts, I’ll be talking about how to quickly get started with Ember.js. I’ll go into detail on how to set up your development environment from beginning to end so you can immediately start building an Ember.js app with ember-cli – Ember’s build tool. Ember-cli provides an asset pipeline to handle your assets. It minifies and concatenates your JavaScript; it gives you a strong conventional project structure and a powerful addon system for extensions. In part two, I’ll guide you through setting up a very basic todo-like Ember.js application to get your feet wet with actual Ember.js development. Setup The first thing you need is node.js. Follow this guide on how to install node and npm from the npmjs.com website. Npm stands for Node Package Manager and it’s the most popular package manager out there for Node. Once you setup Node and Npm, you can install ember-cli with the following command on your terminal: npm install -g ember-cli You can verify whether you have correctly installed ember-cli by running the following command: ember –v If you see the output of the different versions you have for ember-cli, node, npm and your OS it means everything is correctly set up and you are ready to start building an Ember.js application. In order to get you more acquainted with ember-cli, you can run ember -h to see a list of useful ember-cli commands. Ember-cli gives you an easy way to install add-ons (packages created by the community to quickly set up functionality, so instead of creating a specific feature you need, someone may have already made a package for it. See http://emberobserver.com/). You can also generate a new project with ember init <app_name>, run the tests of your app with ember test and scaffold project files with ember generate. These are just one of the many useful commands ember-cli gives you by default. You can learn more specific subcommands for any given command by running ember <command_name> -help. Now that you know what ember-cli is useful for its time to move on to building a fun example application. Building an Ember.js app The application we will be building is an Ember.js application that will fetch a few posts from www.reddit.com/r/funny. It will display a list of these posts with some information about them such as title, author, and date. The purpose of this example application is to show you how easy it is to build an ember.js application that fetches data from a remote API and displays it. It will also show you have to leverage one of the most powerful features of Ember – its router. Now that you are more acquainted with ember-cli, let's create the skeleton of our application. There’s no need to worry and think about what packages and what features from ember we will need and we don’t even have to think about what local server to run in order to display our work in progress. First things first, run the following command on your terminal: ember new ember-r-funny We are running the ‘ember new’ command with the argument of ‘ember-r-funny’ which is what we are naming our app. Feel free to change the name to anything you’d like. From here, you’ll see a list of files being created. After it finishes, you’ll have the app directory with the app name being created in the current directory where you are working on. If you go into this directory and inspect the files, you’ll see that quite a few directories and files. For now don’t pay too much attention to these files except for the directory called ‘app’. This is where you’ll be mostly working on. On your terminal, if you got to the base path of your project (just inside of ember-r-funny/ ) and you run ember server, ember-cli will run a local server for you to see your app. If you now go on your browser to http://localhost:4200 you will see your newly created application, which is just a blank page with the wording Welcome to Ember. If you go into app/templates/application.js and change the <h1> tag and save the file, you’ll notice that your browser will automatically refresh the page. One thing to note before we continue, is that ember-cli projects allow you to use the ES6 JavaScript syntax. ES6 is a significant update to the language which although current browsers do not use it, ember-cli will compile your project to browser readable ES5. For a more in-depth explanation of ES6 visit: https://hacks.mozilla.org/2015/04/es6-in-depth-an-introduction/ Creating your first resource One of the strong points of Ember is the router. The router is responsible for displaying templates, loading data, and otherwise setting up application state. The next thing we need to do is to set up a route to display the /r/funny posts from reddit. Run the following command to create our base route: ember generate route index This will generate an index route. In Ember-speak, the index route is the base or lowest level route of the app. Now go to ‘app/routes/index.js’, and make sure the route looks like the following: import Ember from 'ember'; export default Ember.Route.extend({ beforeModel() { this.transitionTo('posts'); } }); This is telling the app that whenever a user lands on our base URL ‘/’, that it should transition to ‘posts’. Next, run the following command to generate our posts resource: ember generate resource posts If you open the ‘app/router.js file, you’ll see that that ‘this.route(‘posts’)’ was added. Change this to ‘this.resource(‘posts’)’ instead (since we want to deal with a resource and not a route). It should look like the following: import Ember from 'ember'; import config from './config/environment'; var Router = Ember.Router.extend({ location: config.locationType }); Router.map(function() { this.resource('posts'); }); export default Router;dfdf In this router.js file, we’ve created a ‘posts’ resource. So far, we’ve told the app to take the user to ‘/posts’ whenever the user goes to our app. Next, we’ll make sure to set up what data and templates to display when the user lands in ‘/posts’. Thanks to the generator, we now have a route file for posts under ‘app/routes/posts.js’. Open this file and make sure that it looks like the following: import Ember from 'ember'; export default Ember.Route.extend({ model() { return Ember.$.getJSON('https://www.reddit.com/r/funny.json?jsonp=?&limit=10').then(result => { return Ember.A(result.data.children).mapBy('data'); }); } }); Ember works by doing the asynchronous fetching of data on the model hook of our posts route. Routes are where we fetch the data so we can consume it and in this case, we are hitting reddit/r/funny and fetching the latest 10 posts. Once that data is returned, we filter out the unnecessary properties from the response so we can actually return an array with our 10 reddit post entries through the use of a handy function provided by ember called `mapBy`. One important thing to note is that you need to always return something on the model hook, be it either an array, object, or a Promise (which is what we are doing in this case; for a more in depth explanation of promises you can read more here). Now that we have our route wired up to fetch the information, let’s open the ‘app/templates/posts.hbs’ file, remove the current contents, and add the following: <ul> {{#each model as |post|}} <li>{{post.title}}</li> {{/each}} </ul> This is HTML mixed with the Handlebars syntax. Handlebars is the templating engine Ember.js uses to display your dynamic content. What this .hbs template is doing here is that it is looping through our model and displaying the title property for each object inside the model array. If you haven’t noticed yet, Ember.js is smart enough to know when the data has returned from the server and then displays it, so we don’t need to handle any callback functions as far as the model hook is concerned. At this point, it may be normal to see some deprecation warnings on the console, but if you see an error with the words ‘refused to load the script https://www.reddit.com/r/funny.json..’ you need to add the following key and value to the ‘config/environment.js’ file inside the ENV object: contentSecurityPolicy: { 'script-src': "'self' https://www.reddit.com" }, By default, ember-cli will prevent you from doing external requests and fetching external resources from different domain names. This is a security feature so we need to whitelist the reddit domain name so we can make requests against it. At this point, if you go to localhost:4200, you should be redirected to the /posts route and you should see something like the following: Congratulations, you’ve just created a simple ember.js app that displays the title of some reddit posts inside an html list element. So far we’ve added a few lines of code here and there and we already have most of what we need. In Part 2 of this blog, we will set up a more detailed view for each of our reddit posts. About the Author: Daniel Ochoa is a senior software engineer at Frog with a passion for crafting beautiful web and mobile experiences. His current interests are Node.js, Ember.js, Ruby on Rails, iOS development with Swift, and the Haskell language. He can be found on Twitter @DanyOchoaOzz.

In this two-part series, we're programing littleBits circuits using the Johnny-Five JavaScript Robotics Framework. Be sure to read over Part 1 before continuing here. Let's create a circuit to play with, using all of the modules from the littleBits Arduino Coding Kit. Attach a button to the Arduino connector labelled d0. Attach a dimmer to the connector marked a0 and a second dimmer to a1. Turn the dimmers all the way to the right (max) to start with. Attach a power module to the single connector on the left-hand side of the fork module, and the three output connectors of the fork module to all of the input modules. The bargraph should be connected to d5, and the servo to d9, and both set to PWM output mode using the switches on board of the Arduino. The servo module has two modes: turn and swing. Swing mode makes the servo sweep betwen maximum and minimum. Set it to swing mode using the onboard switch. Reading input values We'll create an instance of the Johnny-Five Button class to respond to button press events. Our button is connected to the connector labelled d0 (i.e. digital "pin" 0) on our Arduino, so we'll need to specify the pin as an argument when we create the button. var five = require("johnny-five"); var board = new five.Board(); board.on("ready", function() { var button = new five.Button(0); }); Our dimmers are connected to analog pins (A0 and A1), so we'll specify these as strings when we create Sensor objects to read their values. We can also provide options for reading the values; for example, we'll set the frequency to 250 milliseconds, so we'll be receiving 4 readings per second for both dimmers. var dimmer1 = new five.Sensor({ pin: "A0", freq: 250 }); var dimmer2 = new five.Sensor({ pin: "A1", freq: 250 }); We can attach a function that will be run any time the value changes (on "change") or anytime we get a reading (on "data"): dimmer1.on("change", function() { // raw value (between 0 and 1023) console.log("dimmer 1 is " + this.raw); }); Run the code and try turning dimmer 1. You should see the value printed to the console whenever the dimmer value changes. Triggering behavior Now we can use code to hook our input components up to our output components. To use, for example, the dimmer to control the brightness of the bargraph, change the code in the event handler: var led = new five.Led(5); dimmer1.on("change", function() { // set bargraph brightness to one quarter // of the raw value from dimmer led.brightness(Math.floor(this.raw / 4)); }); You'll see the bargraph brightness fade as you turn the dimmer. We can use the JavaScript Math library and operators to manipulate the brightness value before we send it to the bargraph. Writing code gives us more control over the mapping between input values and output behaviors than if we'd snapped our littleBits modules directly together without going via the Arduino. We set our d5 output to PWM mode, so all of the LEDs should fade in and out at the same time. If we set the output to analog mode instead, we'd see the behavior change to light up more or fewer LEDs depending on value of the brightness. Let's use the button to trigger the servo stop and start functions. Add a button press handler to your code, and a variable to keep track of whether the servo is running or not. We'll toggle this variable between true and false using JavaScript's boolean not operator (!). We can determine whether to stop or start the servo each time the button is pressed via a conditional statement based on the value of this variable. var servo = new five.Motor(9); servo.start(); var button = new five.Button(0); var toggle = false; var speed = 255; button.on("press", function(value){ toggle = !toggle; if (toggle) { servo.start(speed); } else { servo.stop(); } }); The other dimmer can be used to change the servo speed: dimmer2.on("change", function(){ speed = Math.floor(this.raw / 4); if (toggle) { servo.start(speed); } }); There are many input and output modules available within the littleBits system for you to experiment with. You can use the Sensor class with input modules, and check out the Johnny-Five API docs to see examples of types of outputs supported by the API. You can always fall back to using the Pin class to program any littleBits module. Using the REPL Johnny-Five includes a Read-Eval-Print-Loop (REPL) so you can interactively write code to control components instantly - no waiting for code to compile and upload! Any of the JavaScript objects from your program that you want to access from the REPL need to be "injected". The following code, for example, injects our servo and led objects. this.repl.inject({ led: led, servo: servo }); After running the program using Node.js, you'll see a >> prompt in your terminal. Try some of the following functions (hit Enter after each to see it take effect): servo.stop(): stop the servo servo.start(50): start servo moving at slow speed servo.start(255): start servo moving at max speed led.on(): turn LED on led.off(): turn LED off led.pulse(): slowly fade LED in and out led.stop(): stop pulsing LED led.brightness(100): set brightness of LED - the parameter should be between 0 and 255 LittleBits are fantastic for prototyping, and pairing the littleBits Arduino with JavaScript makes prototyping interactive electronic projects even faster and easier. About the author Anna Gerber is a full-stack developer with 15 years of experience in the university sector. Specializing in Digital Humanities, she was a Technical Project Manager at the University of Queensland’s eResearch centre, and she has worked at Brisbane’s Distributed System Technology Centre as a Research Scientist. Anna is a JavaScript robotics enthusiast who enjoys tinkering with soft circuits and 3D printers.

I spent the last six months working on data analytics and machine learning to feed my curiosity and prepare for my new job. It is a challenging mission and I chose to give up for a while on my current web projects to stay focused. Back then, I was coding a dashboard for an automated trading system, powered by an exciting new framework from Facebook : React. In my opinion, Web Components was the way to go and React seemed gentler with my brain than, say, Polymer. One just needed to carefully design components boundaries, properties and states and bam, you got a reusable piece of web to plug anywhere. Beautiful. This is quite a naive way to put it of course but, for an MVP, it actually kind of worked. Fast forward to last week, I was needing a new dashboard to monitor various metrics from my shiny new infrastructure. Specialized requirements kept me away from a full-fledged solution like InfluxDB and Grafana combo, so I naturally starred at my old code. Well, it turned out I did not reuse a single line of code. Since the last time I spent in web development, new tools, frameworks and methodologies had taken over the world : es6 (and transpilers), isomorphic applications, one-way data flow, hot reloading, module bundler, ... Even starter kits are remarkably complex (at least for me) and I got overwhelmed. But those new toys are also truly empowering and I persevered. In this post, we will learn to leverage them, build the simplest dashboard possible and pave the way toward modern, real-time metrics monitoring. Tooling & Motivations I think the points of so much tooling are productivity and complexity management. New single page applications usually involve a significant number of moving parts : front and backend development, data management, scaling, appealing UX, ... Isomorphic webapps with nodejs and es6 try to harmonize this workflow sharing one readable language across the stack. Node already sells the "javascript everywhere" argument but here, it goes even further, with code that can be executed both on the server and in the browser, indifferently. Team work and reusability are improved, as well as SEO (Search Engine optimization) when rendering HTML on server-side. Yet, applications' codebase can turn into a massive mess and that's where Web Components come handy. Providing clear contracts between modules, a developer is able to focus on subpart of the UI with an explicit definition of its parameters and states. This level of abstraction makes the application much more easy to navigate, maintain and reuse. Working with React gives a sense of clarity with components as Javascript objects. Lifecycle and behavior are explicitly detailed by pre-defined hooks, while properties and states are distinct attributes. We still need to glue all of those components and their dependencies together. That's where npm, Webpack and Gulp join the party. Npm is the de facto package manager for nodejs, and more and more for frontend development. What's more, it can run for you scripts and spare you from using a task runner like Gulp. Webpack, meanwhile, bundles pretty much anything thanks to its loaders. Feed it an entrypoint which require your js, jsx, css, whatever ... and it will transform and package them for the browser. Given the steep learning curve of modern full-stack development, I hope you can see the mean of those tools. Last pieces I would like to introduce for our little project are metrics-graphics and react-sparklines (that I won't actually describe but worth noting for our purpose). Both are neat frameworks to visualize data and play nicely with React, as we are going to see now. Graph Component When building components-based interfaces, first things to define are what subpart of the UI those components are. Since we start a spartiate implementation, we are only going to define a Graph. // Graph.jsx // new es6 import syntax import React from 'react'; // graph renderer import MG from 'metrics-graphics'; export default class Graph extends React.Component { // called after the `render` method below componentDidMount () { // use d3 to load data from metrics-graphics samples d3.json('node_modules/metrics-graphics/examples/data/confidence_band.json', function(data) { data = MG.convert.date(data, 'date'); MG.data_graphic({ title: {this.props.title}, data: data, format: 'percentage', width: 600, height: 200, right: 40, target: '#confidence', show_secondary_x_label: false, show_confidence_band: ['l', 'u'], x_extended_ticks: true }); }); } render () { // render the element targeted by the graph return <div id="confidence"></div>; } } This code, a trendy combination of es6 and jsx, defines in the DOM a standalone graph from the json data in confidence_band.json I stole on Mozilla official examples. Now let's actually mount and render the DOM in the main entrypoint of the application (I mentioned above with Webpack). // main.jsx // tell webpack to bundle style along with the javascript import 'metrics-graphics/dist/metricsgraphics.css'; import 'metrics-graphics/examples/css/metricsgraphics-demo.css'; import 'metrics-graphics/examples/css/highlightjs-default.css'; import React from 'react'; import Graph from './components/Graph'; function main() { // it is recommended to not directly render on body var app = document.createElement('div'); document.body.appendChild(app); // key/value pairs are available under `this.props` hash within the component React.render(<Graph title={Keep calm and build a dashboard}/>, app); } main(); Now that we defined in plain javascript the web page, it's time for our tools to take over and actually build it. Build workflow This is mostly a matter of configuration. First, create the following structure. $ tree . ├── app │ ├── components │ │ ├── Graph.jsx │ ├── main.jsx ├── build └── package.json Where package.json is defined like below. { "name": "react-dashboard", "scripts": { "build": "TARGET=build webpack", "dev": "TARGET=dev webpack-dev-server --host 0.0.0.0 --devtool eval-source --progress --colors --hot --inline --history-api-fallback" }, "devDependencies": { "babel-core": "^5.6.18", "babel-loader": "^5.3.2", "css-loader": "^0.15.1", "html-webpack-plugin": "^1.5.2", "node-libs-browser": "^0.5.2", "react-hot-loader": "^1.2.7", "style-loader": "^0.12.3", "webpack": "^1.10.1", "webpack-dev-server": "^1.10.1", "webpack-merge": "^0.1.2" }, "dependencies": { "metrics-graphics": "^2.6.0", "react": "^0.13.3" } } A quick npm install will download every package we need for development and production. Two scripts are even defined to build a static version of the site, or serve a dynamic one that will be updated on file changes detection. This formidable feature becomes essential once tasted. But we have yet to configure Webpack to enjoy it. var path = require('path'); var HtmlWebpackPlugin = require('html-webpack-plugin'); var webpack = require('webpack'); var merge = require('webpack-merge'); // discern development server from static build var TARGET = process.env.TARGET; // webpack prefers abolute path var ROOT_PATH = path.resolve(__dirname); // common environments configuration var common = { // input main.js we wrote earlier entry: [path.resolve(ROOT_PATH, 'app/main')], // import requirements with following extensions resolve: { extensions: ['', '.js', '.jsx'] }, // define the single bundle file output by the build output: { path: path.resolve(ROOT_PATH, 'build'), filename: 'bundle.js' }, module: { // also support css loading from main.js loaders: [ { test: /.css$/, loaders: ['style', 'css'] } ] }, plugins: [ // automatically generate a standard index.html to attach on the React app new HtmlWebpackPlugin({ title: 'React Dashboard' }) ] }; // production specific configuration if(TARGET === 'build') { module.exports = merge(common, { module: { // compile es6 jsx to standard es5 loaders: [ { test: /.jsx?$/, loader: 'babel?stage=1', include: path.resolve(ROOT_PATH, 'app') } ] }, // optimize output size plugins: [ new webpack.DefinePlugin({ 'process.env': { // This has effect on the react lib size 'NODE_ENV': JSON.stringify('production') } }), new webpack.optimize.UglifyJsPlugin({ compress: { warnings: false } }) ] }); } // development specific configuration if(TARGET === 'dev') { module.exports = merge(common, { module: { // also transpile javascript, but also use react-hot-loader, to automagically update web page on changes loaders: [ { test: /.jsx?$/, loaders: ['react-hot', 'babel?stage=1'], include: path.resolve(ROOT_PATH, 'app'), }, ], }, }); } Webpack configuration can be hard to swallow at first but, given the huge amount of transformations to operate, this style scales very well. Plus, once setup, the development environment becomes remarkably productive. To convince yourself, run webpack-dev-server and reach localhost:8080/assets/bundle.js in your browser. Tweak the title argument in main.jsx, save the file and watch the browser update itself. We are ready to build new components and extend our modular dashboard. Conclusion We condensed in a few paragraphs a lot of what makes the current web ecosystem effervescent. I strongly encourage the reader to deepen its knowledge on those matters and consider this post as it is : an introduction. Web components, like micro-services, are fun, powerful and bleeding edges. But also complex, fast-moving and unstable. The tooling, especially, is impressive. Spend a hard time to master them and craft something cool ! About the Author Xavier Bruhiere is a Lead Developer at AppTurbo in Paris, where he develops innovative prototypes to support company growth. He is addicted to learning, hacking on intriguing hot techs (both soft and hard), and practicing high intensity sports.

Who wouldn't want an army of robots to help out around the home and garden? It's not science fiction: Robots are devices that sense and respond to the world around us, so with some off-the-shelf hardware, and the power of the Johnny-Five JavaScript Robotics framework, we can build and program simple "robots" to automate every day tasks. In this two part article series, we'll build an internet-connected device for monitoring plants. Bill of materials You'll need these parts to build this project Part Source Particle Core (or Photon) Particle 3xAA Battery holder e.g. with micro USB connector from DF Robot Jumper wires Any electronics supplier e.g. Sparkfun Solderless breadboard Any electronics supplier e.g. Sparkfun Photo resistor Any electronics supplier e.g. Sparkfun 1K resistor Any electronics supplier e.g. Sparkfun Soil moisture sensor e.g. Sparkfun Plants   Particle (formerly known as Spark) is a platform for developing devices for the Internet of Things. The Particle Core was their first generation Wifi development board, and has since been supeceded by the Photon. Johnny-Five supports both of these boards, as well as Arduino, BeagleBone Black, Raspberry Pi, Edison, Galileo, Electric Imp, Tessel and many other device platforms, so you can use the framework with your device of choice. The Platform Support page lists the features currently supported on each device. Any device with Analog Read support is suitable for this project. Setting up the Particle board Make sure you have a recent version of Node.js installed. We're using npm (Node Package Manager) to install the tools and libraries required for this project. Install the Particle command line tools with npm (via the Terminal on Mac, or Command Prompt on Windows): npm install -g particle-cli Particle boards need to be registered with the Particle Cloud service, and you must also configure your device to connect to your wireless network. So the first thing you'll need to do is connect it to your computer via USB and run the setup program. See the Particle Setup docs. The LED on the Particle Core should be blinking blue when you plug it in for the first time (if not, press and hold the mode button). Sign up for a Particle Account and then follow the prompts to setup your device via the Particle website, or if you prefer you can run the setup program from the command line. You'll be prompted to sign in and then to enter your Wifi SSID and password: particle setup After setup is complete, the Particle Core can be disconnected from your computer and powered by batteries or a separate USB power supply - we will connect to the board wirelessly from now on. Flashing the board We also need to flash the board with the Voodoospark firmware. Use the CLI tool to sign in to the Particle Cloud and list your devices to find out the ID of your board: particle cloud login particle list Download the firmware.cpp file and use the flash command to write the Voodoospark firmware to your device: particle cloud flash <Your Device ID> voodoospark.cpp See the Voodoospark Getting Started page for more details. You should see the following message: Flash device OK: Update started The LED on the board will flash magenta. This will take about a minute, and will change back to green when the board is ready to use. Creating a Johnny-Five project We'll be installing a few dependencies from npm, so to help manage these, we'll set up our project as an npm package. Run the init command, filling in the project details at the prompts. npm init After init has completed, you'll have a package.json file with the metadata that you entered about your project. Dependencies for the project can also be saved to this file. We'll use the --save command line argument to npm when installing packages to persist dependencies to our package.json file. We'll need the Johnny-Five npm module as well as the Particle-IO IO Plugin for Johnny-Five. npm install johnny-five --save npm install particle-io --save Johnny-Five uses the Firmata protocol to communicate with Arduino-based devices. IO Plugins provide Firmata compatible interfaces to allow Johnny-Five to communicate with non-Arduino-based devices. The Particle-IO Plugin allows you to run Node.js applications on your computer that communicate with the Particle board over Wifi, so that you can read from sensors or control components that are connected to the board. When you connect to your board, you'll need to specify your Device ID and your Particle API Access Token. You can look up your access token under Settings in the Particle IDE. It's a good idea to copy these to environment variables rather than hardcoding them into your programs. If you are on Mac or Linux, you can create a file called .particlerc then run source .particlerc: export PARTICLE_TOKEN=<Your Token Here> export PARTICLE_DEVICE_ID=<Your Device ID Here> Reading from a sensor Now we're ready to get our hands dirty! Let's confirm that we can communicate with our Particle board using Johnny-Five, by taking a reading from our soil moisture sensor. Using jumper wires, connect one pin on the soil sensor to pin A0 (analog pin 0) and the other to GND (ground). The probes go into the soil in your plant pot. Create a JavaScript file named sensor.js using your preferred text editor or IDE. We use require statements to include the Johnny-Five module and the Particle-IO plugin. We're creating an instance of the Particle IO plugin (with our token and deviceId read from our environment variables) and providing this as the io config option when creating our Board object. var five = require("johnny-five"); var Particle = require("particle-io"); var board = new five.Board({ io: new Particle({ token: process.env.PARTICLE_TOKEN, deviceId: process.env.PARTICLE_DEVICE_ID }) }); board.on("ready", function() { console.log("CONNECTED"); var soilSensor = new five.Sensor("A0"); soilSensor.on("change", function() { console.log(this.value); }); }); After the board is ready, we create a Sensor object to monitor changes on pin A0, and then print the value from the sensor to the Node.js console whenever it changes. Run the program using Node.js: node sensor.js Try pulling the sensor out of the soil or watering your plant to make the sensor reading change. See the Sensor API for more methods that you can use with Sensors. You can hit control-C to end the program. In the next installment we'll connect our light sensor and extend our Node.js application to monitor our plant's environment. Continue reading now! About the author Anna Gerber is a full-stack developer with 15 years’ experience in the university sector, formerly a Technical Project Manager at The University of Queensland ITEE eResearchspecializing in Digital Humanities and Research Scientist at the Distributed System Technology Centre (DSTC). Anna is a JavaScript robotics enthusiast and maker who enjoys tinkering with soft circuits and 3D printers.

In this two-part article series, we're building an internet-connected garden bot using JavaScript. In part one, we set up a Particle Core board, created a Johnny-Five project, and ran a Node.js program to read raw values from a soil moisture sensor. Adding a light sensor Let's connect another sensor. We'll extend our circuit to add a photo resistor to measure the ambient light levels around our plants. Connect one lead of the photo resistor to ground, and the other to analog pin 4, with a 1K pull-down resistor from A4 to the 3.3V pin. The value of the pull-down resistor determines the raw readings from the sensor. We're using a 1K resistor so that the sensor values don't saturate under tropical sun conditions. For plants kept inside a dark room, or in a less sunny climate, a 10K resistor might be a better choice. Read more about how pull-down resistors work with photo resistors at AdaFruit. Now, in our board's ready callback function, we add another sensor instance, this time on pin A4: var lightSensor = new five.Sensor({ pin: "A4", freq: 1000 }); lightSensor.on("data", function() { console.log("Light reading " + this.value); }); For this sensor we are logging the sensor value every second, not just when it changes. We can control how often sensor events are emitted by specifying the number of milliseconds in the freq option when creating the sensor. We can use the threshold config option can be used to control when the change callback occurs. Calibrating the soil sensor The soil sensor uses the electrical resistance between two probes to provide a measure of the moisture content of the soil. We're using a commercial sensor, but you could make your own simply using two pieces of wire spaced about an inch apart (using galvinized wire to avoid rust). Water is a good conductor of electricity, so a low reading means that the soil is moist, while a high amount of resistance indicates that the soil is dry. Because these aren't very sophisticated sensors, the readings will vary from sensor to sensor. In order to do anything meaningful with the readings within our application, we'll need to calibrate our sensor. Calibrate by making a note of the sensor values for very dry soil, wet soil, and in between to get a sense of what the optimal range of values should be. For an imprecise sensor like this, it also helps to map the raw readings onto ranges that can be used to display different messages (e.g. very dry, dry, damp, wet) or trigger different actions. The scale method on the Sensor class can come in handy for this. For example, we could convert the raw readings from 0 - 1023 to a 0 - 5 scale: soilSensor.scale(0, 5).on("change", function() { console.log(this.value); }); However, the raw readings for this sensor range between about 50 (wet) to 500 (fairly dry soil). If we're only interested in when the soil is dry, i.e. when readings are above 300, we could use a conditional statement within our callback function, or use the within method so that the function is only triggered when the values are inside a range of values we care about. soilSensor.within([ 300, 500 ], function() { console.log("Water me!"); }); Our raw soil sensor values will vary depending on the temperature of the soil, so this type of sensor is best for indoor plants that aren't exposed to weather extremes. If you are installing a soil moisture sensor outdoors, consider adding a temperature sensor and then calibrate for values at different temperature ranges. Connecting more sensors We have seven analog and seven digital IO pins on the Particle Core, so we could attach more sensors, perhaps more of the same type to monitor additional planters, or different types of sensors to monitor additional conditions. There are many kinds of environmental sensors available through online marketplaces like AliExpress and ebay. These include sensors for temperature, humidity, dust, gas, water depth, particulate detection etc. Some of these sensors are straightforward analog or digital devices that can be used directly with the Johnny-Five Sensor class, as we have with our soil and light sensors. The Johnny-Five API also includes subclasses like Temperature, with controllers for some widely used sensor components. However, some sensors use protocols like SPI, I2C or OneWire, which are not as well supported by Johnny-Five across all platforms. This is always improving, for example, I2C was added to the Particle-IO plugin in October 2015. Keep an eye on I2C component backpacks which are providing support for additional sensors via secondary microcontrollers. Automation If you are gardening at scale, or going away on extended vacation, you might want more than just monitoring. You might want to automate some basic garden maintenance tasks, like turning on grow lights on overcast days, or controlling a pump to water the plants when the soil moisture level gets low. This can be acheived with relays. For example, we can connect a relay with a daylight bulb to a digital pin, and use it to turn lights on in response to the light readings, but only between certain hours: var five = require("johnny-five"); var Particle = require("particle-io"); var moment = require("moment"); var board = new five.Board({ io: new Particle({ token: process.env.PARTICLE_TOKEN, deviceId: process.env.PARTICLE_DEVICE_ID }) }); board.on("ready", function() { var lightSensor = new five.Sensor("A4"); var lampRelay = new five.Relay(2); lightSensor.scale(0, 5).on("change", function() { console.log("light reading is " + this.value) var now = moment(); var nightCurfew = now.endOf('day').subtract(4,'h'); var morningCurfew = now.startOf('day').add(6,'h'); if (this.value > 4) { if (!lampRelay.isOn && now.isAfter(morningCurfew) && now.isBefore(nightCurfew)) { lampRelay.on(); } } else { lampRelay.off(); } }); }); And beyond... One of the great things about using Node.js with hardware is that we can extend our apps with modules from npm. We could publish an Atom feed of sensor readings over time, push the data to a web UI using socket-io, build an alert system or create a data visualization layer, or we might build an API to control lights or pumps attached via relays to our board. It's never been easier to program your own internet-connected robot helpers and smart devices using JavaScript. Build more exciting robotics projects with servos and motors – click here to find out how. About the author Anna Gerber is a full-stack developer with 15 years’ experience in the university sector, formerly a Technical Project Manager at The University of Queensland ITEE eResearchspecializing in Digital Humanities and Research Scientist at the Distributed System Technology Centre (DSTC). Anna is a JavaScript robotics enthusiast and maker who enjoys tinkering with soft circuits and 3D printers.

In this article by Abiee Echamea, author of the book Mastering Backbone.js, you will see that one of the best things about Backbone is the freedom of building applications with the libraries of your choice, no batteries included. Backbone is not a framework but a library. Building applications with it can be challenging as no structure is provided. The developer is responsible for code organization and how to wire the pieces of code across the application; it's a big responsibility. Bad decisions about code organization can lead to buggy and unmaintainable applications that nobody wants to see. In this article, you will learn the following topics: Delegating the right responsibilities to Backbone objects Splitting the application into small and maintainable scripts (For more resources related to this topic, see here.) The big picture We can split application into two big logical parts. The first is an infrastructure part or root application, which is responsible for providing common components and utilities to the whole system. It has handlers to show error messages, activate menu items, manage breadcrumbs, and so on. It also owns common views such as dialog layouts or loading the progress bar. A root application is responsible for providing common components and utilities to the whole system. A root application is the main entry point to the system. It bootstraps the common objects, sets the global configuration, instantiates routers, attaches general services to a global application, renders the main application layout at the body element, sets up third-party plugins, starts a Backbone history, and instantiates, renders, and initializes components such as a header or breadcrumb. However, the root application itself does nothing; it is just the infrastructure to provide services to the other parts that we can call subapplications or modules. Subapplications are small applications that run business value code. It's where the real work happens. Subapplications are focused on a specific domain area, for example, invoices, mailboxes, or chats, and should be decoupled from the other applications. Each subapplication has its own router, entities, and views. To decouple subapplications from the root application, communication is made through a message bus implemented with the Backbone.Events or Backbone.Radio plugin such that services are requested to the application by triggering events instead of call methods on an object. Subapplications are focused on a specific domain area and should be decoupled from the root application and other subapplications. Figure 1.1 shows a component diagram of the application. As you can see, the root application depends on the routers of the subapplications due to the Backbone.history requirement to instantiate all the routers before calling the start method and the root application does this. Once Backbone.history is started, the browser's URL is processed and a route handler in a subapplication is triggered; this is the entry point for subapplications. Additionally, a default route can be defined in the root application for any route that is not handled on the subapplications. Figure 1.1: Logical organization of a Backbone application When you build Backbone applications in this way, you know exactly which object has the responsibility, so debugging and improving the application is easier. Remember, divide and conquer. Also by doing this, you make your code more testable, improving its robustness. Responsibilities of the Backbone objects One of the biggest issues with the Backbone documentation is no clues about how to use its objects. Developers should figure out the responsibilities for each object across the application but at least you have some experience working with Backbone already and this is not an easy task. The next sections will describe the best uses for each Backbone object. In this way, you will have a clearer idea about the scope of responsibilities of Backbone, and this will be the starting point of designing our application architecture. Keep in mind, Backbone is a library with foundation objects, so you will need to bring your own objects and structure to make an awesome Backbone application. Models This is the place where the general business logic lives. A specific business logic should be placed on other sites. A general business logic is all the rules that are so general that they can be used on multiple use cases, while specific business logic is a use case itself. Let's imagine a shopping cart. A model can be an item in the cart. The logic behind this model can include calculating the total by multiplying the unit price by the quantity or setting a new quantity. In this scenario, assume that the shop has a business rule that a customer can buy the same product only three times. This is a specific business rule because it is specific for this business, or how many stores do you know with this rule? These business rules take place on other sites and should be avoided on models. Also, it's a good idea to validate the model data before sending requests to the server. Backbone helps us with the validate method for this, so it's reasonable to put validation logic here too. Models often synchronize the data with the server, so direct calls to servers such as AJAX calls should be encapsulated at the model level. Models are the most basic pieces of information and logic; keep this in mind. Collections Consider collections as data repositories similar to a database. Collections are often used to fetch the data from the server and render its contents as lists or tables. It's not usual to see business logic here. Resource servers have different ways to deal with lists of resources. For instance, while some servers accept a skip parameter for pagination, others have a page parameter for the same purpose. Another case is responses; a server can respond with a plain array while other prefer sending an object with a data, list, or some other key, where an array of objects is placed. There is no standard way. Collections can deal with these issues, making server requests transparent for the rest of the application. Views Views have the responsibility of handling Document Object Model (DOM). Views work closely with the template engines rendering the templates and putting the results in DOM. Listen for low-level events using a jQuery API and transform them into domain ones. Views abstract the user interactions transforming his/her actions into data structures for the application, for example, clicking on a save button in a form view will create a plain object with the information in the input and trigger a domain event such as save:contact with this object attached. Then a domain-specific object can apply domain logic to the data and show a result. Business logic on views should be avoided, but basic form validations are allowed, such as accepting only numbers, but complex validations should be done on the model. Routers Routers have a simple responsibility: listening for URL changes on the browser and transforming them into a call to a handler. A router knows which handler to call for a given URL and also decodes the URL parameters and passes them to the handlers. The root application bootstraps the infrastructure, but routers decide which subapplication will be executed. In this way, routers are a kind of entry point. Domain objects It is possible to develop Backbone applications using only the Backbone objects described in the previous section, but for a medium-to-large application, it's not sufficient. We need to introduce a new kind of object with well-delimited responsibilities that use and coordinate the Backbone foundation objects. Subapplication facade This object is the public interface of a subapplication. Any interaction with the subapplication should be done through its methods. Direct calls to internal objects of the subapplication are discouraged. Typically, methods on this controller are called from the router but can be called from anywhere. The main responsibility of this object is simplifying the subapplication internals, so its work is to fetch the data from the server through models or collections and in case an error occurs during the process, it has to show an error message to the user. Once the data is loaded in a model or collection, it creates a subapplication controller that knows the views that should be rendered and has the handlers to deal with its events. The subapplication facade will transform the URL request into a Backbone data object. It shows the right error message; creates a subapplication controller, and delegates the control to it. The subapplication controller or mediator This object acts as an air traffic controller for the views, models, and collections. With a Backbone data object, it will instantiate and render the appropriate views and then coordinate them. However, the coordination task is not easy in complex layouts. Due to loose coupling reasons, a view cannot call the methods or events of the other views directly. Instead of this, a view triggers an event and the controller handles the event and orchestrates the view's behavior, if necessary. Note how the views are isolated, handling just their owned portion of DOM and triggering events when they need to communicate something. Business logic for simple use cases can be implemented here, but for more complex interactions, another strategy is needed. This object implements the mediator pattern allowing other basic objects such as views and models to keep it simple and allow loose coupling. The logic workflow The application starts bootstrapping common components and then initializes all the routers available for the subapplications and starts Backbone.history. See Figure 1.2, After initialization, the URL on the browser will trigger a route for a subapplication, then a route handler instantiates a subapplication facade object and calls the method that knows how to handle the request. The facade will create a Backbone data object, such as a collection, and fetch the data from the server calling its fetch method. If an error is issued while fetching the data, the subapplication facade will ask the root application to show the error, for example, a 500 Internal Server Error. Figure 1.2: Abstract architecture for subapplications Once the data is in a model or collection, the subapplication facade will instantiate the subapplication object that knows the business rules for the use case and pass the model or collection to it. Then, it renders one or more view with the information of the model or collection and places the results in the DOM. The views will listen for DOM events, for example, click, and transform them into a higher-level event to be consumed by the application object. The subapplication object listens for events on models and views and coordinates them when an event is triggered. When the business rules are not too complex, they can be implemented on this application object, such as deleting a model. Models and views can be in sync with the Backbone events or use a library for bindings such as Backbone.Stickit. In the next section, we will describe this process step by step with code examples for a better understanding of the concepts explained. Route handling The entry point for a subapplication is given by its routes, which ideally share the same namespace. For instance, a contacts subapplication can have these routes: contacts: Lists all the available contacts Contacts/page/:page: Paginates the contacts collection contacts/new: Shows a form to create a new contact contacts/view/:id: Shows an invoice given its ID contacts/edit/:id: Shows a form to edit a contact Note how all the routes start with the /contacts prefix. It's a good practice to use the same prefix for all the subapplication routes. In this way, the user will know where he/she is in the application, and you will have a clean separation of responsibilities. Use the same prefix for all URLs in one subapplication; avoid mixing routes with the other subapplications. When the user points the browser to one of these routes, a route handler is triggered. The function handler parses the URL request and delegates the request to the subapplication object, as follows: var ContactsRouter = Backbone.Router.extend({ routes: { "contacts": "showContactList", "contacts/page/:page": "showContactList", "contacts/new": "createContact", "contacts/view/:id": "showContact", "contacts/edit/:id": "editContact" }, showContactList: function(page) { page = page || 1; page = page > 0 ? page : 1; var region = new Region({el: '#main'}); var app = new ContactsApp({region: region}); app.showContactList(page); }, createContact: function() { var region = new Region({el: '#main'}); var app = new ContactsApp({region: region}); app.showNewContactForm(); }, showContact: function(contactId) { var region = new Region({el: '#main'}); var app = new ContactsApp({region: region}); app.showContactById(contactId); }, editContact: function(contactId) { var region = new Region({el: '#main'}); var app = new ContactsApp({region: region}); app.showContactEditorById(contactId); } }); The validation of the URL parameters should be done on the router as shown in the showContactList method. Once the validation is done, ContactsRouter instantiates an application object, ContactsApp, which is a facade for the Contacts subapplication; finally, ContactsRouter calls an API method to handle the user request. The router doesn't know anything about business logic; it just knows how to decode the URL requests and which object to call in order to handle the request. Here, the region object points to an existing DOM node by passing the application and tells us where the application should be rendered. The subapplication facade A subapplication is composed of smaller pieces that handle specific use cases. In the case of the contacts app, a use case can be see a contact, create a new contact, or edit a contact. The implementation of these use cases is separated on different objects that handle views, events, and business logic for a specific use case. The facade basically fetches the data from the server, handles the connection errors, and creates the objects needed for the use case, as shown here: function ContactsApp(options) { this.region = options.region; this.showContactList = function(page) { App.trigger("loading:start"); new ContactCollection().fetch({ success: _.bind(function(collection, response, options) { this._showList(collection); App.trigger("loading:stop"); }, this), fail: function(collection, response, options) { App.trigger("loading:stop"); App.trigger("server:error", response); } }); }; this._showList = function(contacts) { var contactList = new ContactList({region: this.region}); contactList.showList(contacts); } this.showNewContactForm = function() { this._showEditor(new Contact()); }; this.showContactEditorById = function(contactId) { new Contact({id: contactId}).fetch({ success: _.bind(function(model, response, options) { this._showEditor(model); App.trigger("loading:stop"); }, this), fail: function(collection, response, options) { App.trigger("loading:stop"); App.trigger("server:error", response); } }); }; this._showEditor = function(contact) { var contactEditor = new ContactEditor({region: this.region}); contactEditor.showEditor(contact); } this.showContactById = function(contactId) { new Contact({id: contactId}).fetch({ success: _.bind(function(model, response, options) { this._showViewer(model); App.trigger("loading:stop"); }, this), fail: function(collection, response, options) { App.trigger("loading:stop"); App.trigger("server:error", response); } }); }; this._showViewer = function(contact) { var contactViewer = new ContactViewer({region: this.region}); contactViewer.showContact(contact); } } The simplest handler is showNewContactForm, which is called when the user wants to create a new contact. This creates a new Contact object and passes to the _showEditor method, which will render an editor for a blank Contact. The handler doesn't need to know how to do this because the ContactEditor application will do the job. Other handlers follow the same pattern, triggering an event for the root application to show a loading widget to the user while fetching the data from the server. Once the server responds successfully, it calls another method to handle the result. If an error occurs during the operation, it triggers an event to the root application to show a friendly error to the user. Handlers receive an object and create an application object that renders a set of views and handles the user interactions. The object created will respond to the action of the users, that is, let's imagine the object handling a form to save a contact. When users click on the save button, it will handle the save process and maybe show a message such as Are you sure want to save the changes and take the right action? The subapplication mediator The responsibility of the subapplication mediator object is to render the required layout and views to be showed to the user. It knows which views need to be rendered and in which order, so instantiate the views with the models if needed and put the results on the DOM. After rendering the necessary views, it will listen for user interactions as Backbone events triggered from the views; methods on the object will handle the interaction as described in the use cases. The mediator pattern is applied to this object to coordinate efforts between the views. For example, imagine that we have a form with contact data. As the user made some input in the edition form, other views will render a preview business card for the contact; in this case, the form view will trigger changes to the application object and the application object will tell the business card view to use a new set of data each time. As you can see, the views are decoupled and this is the objective of the application object. The following snippet shows the application that shows a list of contacts. It creates a ContactListView view, which knows how to render a collection of contacts and pass the contacts collection to be rendered: var ContactList = function(options) { _.extend(this, Backbone.Events); this.region = options.region; this.showList = function(contacts) { var contactList = new ContactListView({ collection: contacts }); this.region.show(contactList); this.listenTo(contactList, "item:contact:delete", this._deleteContact); } this._deleteContact = function(contact) { if (confirm('Are you sure?')) { contact.collection.remove(contact); } } this.close = function() { this.stopListening(); } } The ContactListView view will be responsible for transforming this into the DOM nodes and responding to collection events such as adding a new contact or removing one. Once the view is initialized, it is rendered on a specific region previously specified. When the view is finally on DOM, the application listens for the "item:contact:delete" event, which will be triggered if the user clicks on a delete button rendered for each contact. To see a contact, a ContactViewer application is responsible for managing the use case, which is as follows: var ContactViewer = function(options) { _.extend(this, Backbone.Events); this.region = options.region; this.showContact = function(contact) { var contactView = new ContactView({model: contact}); this.region.show(contactView); this.listenTo(contactView, "contact:delete", this._deleteContact); }, this._deleteContact = function(contact) { if (confirm("Are you sure?")) { contact.destroy({ success: function() { App.router.navigate("/contacts", true); }, error: function() { alert("Something goes wrong"); } }); } } } It's the same situation, that is, the contact list creates a view that manages the DOM interactions, renders on the specified region, and listens for events. From the details view of a contact, users can delete them. Similar to a list, a _deleteContact method handles the event, but the difference is when a contact is deleted, the application is redirected to the list of contacts, which is the expected behavior. You can see how the handler uses the root application infrastructure by calling the navigate method of the global App.router. The handler forms to create or edit contacts are very similar, so the same ContactEditor can be used for both the cases. This object will show a form to the user and will wait for the save action, as shown in the following code: var ContactEditor = function(options) { _.extend(this, Backbone.Events) this.region = options.region; this.showEditor = function(contact) { var contactForm = new ContactForm({model: contact}); this.region.show(contactForm); this.listenTo(contactForm, "contact:save", this._saveContact); }, this._saveContact = function(contact) { contact.save({ success: function() { alert("Successfully saved"); App.router.navigate("/contacts"); }, error: function() { alert("Something goes wrong"); } }); } } In this case, the model can have modifications in its data. In simple layouts, the views and model can work nicely with the model-view data bindings, so no extra code is needed. In this case, we will assume that the model is updated as the user puts in information in the form, for example, Backbone.Stickit. When the save button is clicked, a "contact:save" event is triggered and the application responds with the _saveContact method. See how the method issues a save call to the standard Backbone model and waits for the result. In successful requests, a message will be displayed and the user is redirected to the contact list. In errors, a message will tell the user that the application found a problem while saving the contact. The implementation details about the views are outside of the scope of this article, but you can abstract the work made by this object by seeing the snippets in this section. Summary In this article, we started by describing in a general way how a Backbone application works. It describes two main parts, a root application and subapplications. A root application provides common infrastructure to the other smaller and focused applications that we call subapplications. Subapplications are loose-coupled with the other subapplications and should own resources such as views, controllers, routers, and so on. A subapplication manages a small part of the system and no more. Communication between the subapplications and root application is made through an event-driven bus, such as Backbone.Events or Backbone.Radio. The user interacts with the application using views that a subapplication renders. A subapplication mediator orchestrates interaction between the views, models, and collections. It also handles the business logic such as saving or deleting a resource. Resources for Article: Further resources on this subject: Object-Oriented JavaScript with Backbone Classes [article] Building a Simple Blog [article] Marionette View Types and Their Use [article]

In this article by Andy Bailey and Sudheer Jonna, the authors of the book, PrimeFaces Theme Development, we'll cover icons, which add a lot of value to an application based on the principle that a picture is worth a thousand words. Equally important is the fact that they can, when well designed, please the eye and serve as memory joggers for your user. We humans strongly associate symbols with actions. For example, a save button with a disc icon is more evocative. The association becomes even stronger when we use the same icon for the same action in menus and button bars. It is also possible to use icons in place of text labels. It is an important thing to keep in mind when designing the user interface of your application that the navigational and action elements (such as buttons) should not be so intrusive that the application becomes too cluttered with the things that can be done. The user wants to be able to see the information that they want to see and use input dialogs to add more. What they don't want is to be distracted with links, lots of link and button text, and glaring visuals. In this article, we will cover the following topics: The standard theme icon set Creating a set of icons of our own Adding new icons to a theme Using custom icons in a commandButton component Using custom icons in a menu component The FontAwesome icons as an alternative to the ThemeRoller icons (For more resources related to this topic, see here.) Introducing the standard theme icon set jQuery UI provides a big set of standard icons that can be applied by just adding icon class names to HTML elements. The full list of icons is available at its official site, which can be viewed by visiting http://api.jqueryui.com/theming/icons/. Also, available in some of the published icon cheat sheets at http://www.petefreitag.com/cheatsheets/jqueryui-icons/. The icon class names follow the following syntax in order to add them for HTML elements: .ui-icon-{icon type}-{icon sub description}-{direction} .ui-icon-{icon type}-{icon sub description}-{direction} For example, the following span element will display an icon of a triangle pointing to the south: <span class="ui-icon ui-icon-triangle-1-s"></span> Other icons such as ui-icon-triangle-1-n, ui-icon-triangle-1-e, and ui-icon-triangle-1-w represent icons of triangles pointing to the north, east, and west respectively. The direction element is optional, and it is available only for a few icons such as a triangle, an arrow, and so on. These theme icons will be integrated in a number of jQuery UI-based widgets such as buttons, menus, dialogs, date picker components, and so on. The aforementioned standard set of icons is available in the ThemeRoller as one image sprite instead of a separate image for each icon. That is, ThemeRoller is designed to use the image sprites technology for icons. The different image sprites that vary in color (based on the widget state) are available in the images folder of each downloaded theme. An image sprite is a collection of images put into a single image. A webpage with many images may take a long time to load and generate multiple server requests. For a high-performance application, this idea will reduce the number of server requests and bandwidth. Also, it centralizes the image locations so that all the icons can be found at one location. The basic image sprite for the PrimeFaces Aristo theme looks like this: The image sprite's look and feel will vary based on the screen area of the widget and its components such as the header and content and widget states such as hover, active, highlight, and error styles. Let us now consider a JSF/PF-based example, where we can add a standard set of icons for UI components such as the commandButton and menu bar. First, we will create a new folder in web pages called chapter6. Then, we will create a new JSF template client called standardThemeIcons.xhtml and add a link to it in the chaptersTemplate.xhtml template file. When adding a submenu, use Chapter 6 for the label name and for the menu item, use Standard Icon Set as its value. In the title section, replace the text title with the respective topic of this article, which is Standard Icons: <ui:define name="title">   Standard Icons </ui:define> In the content section, replace the text content with the code for commandButton and menu components. Let's start with the commandButton components. The set of commandButton components uses the standard theme icon set with the help of the icon attribute, as follows: <h:panelGroup style="margin-left:830px">   <h3 style="margin-top: 0">Buttons</h3>   <p:commandButton value="Edit" icon="ui-icon-pencil"     type="button" />   <p:commandButton value="Bookmark" icon="ui-icon-bookmark"     type="button" />   <p:commandButton value="Next" icon="ui-icon-circle-arrow-e"     type="button" />   <p:commandButton value="Previous" icon="ui-icon-circle-arrow-w"     type="button" /> </h:panelGroup> The generated HTML for the first commandButton that is used to display the standard icon will be as follows: <button id="mainForm:j_idt15" name="mainForm:j_idt15" class="ui-   button ui-widget ui-state-default ui-corner-all ui-button-text-   icon-left" type="button" role="button" aria-disabled="false">   <span class="ui-button-icon-left ui-icon ui-c   ui-icon-     pencil"></span>   <span class="ui-button-text ui-c">Edit</span> </button> The PrimeFaces commandButton renderer appends the icon position CSS class based on the icon position (left or right) to the HTML button element, apart from the icon CSS class in one child span element and text CSS class in another child span element. This way, it displays the icon on commandButton based on the icon position property. By default, the position of the icon is left. Now, we will move on to the menu components. A menu component uses the standard theme icon set with the help of the menu item icon attribute. Add the following code snippets of the menu component to your page: <h3>Menu</h3> <p:menu style="margin-left:500px">   <p:submenu label="File">     <p:menuitem value="New" url="#" icon="ui-icon-plus" />     <p:menuitem value="Delete" url="#" icon="ui-icon-close" />     <p:menuitem value="Refresh" url="#" icon="ui-icon-refresh" />     <p:menuitem value="Print" url="#" icon="ui-icon-print" />   </p:submenu>   <p:submenu label="Navigations">     <p:menuitem value="Home" url="http://www.primefaces.org"       icon="ui-icon home" />     <p:menuitem value="Admin" url="#" icon="ui-icon-person" />     <p:menuitem value="Contact Us" url="#" icon="ui-icon-       contact" />   </p:submenu> </p:menu> You may have observed from the preceding code snippets that each icon from ThemeRoller starts with ui-icon for consistency. Now, run the application and navigate your way to the newly created page, and you should see the standard ThemeRoller icons applied to buttons and menu items, as shown in the following screenshot: For further information, you can use PrimeFaces showcase (http://www.primefaces.org/showcase/), where you can see the default icons used for components, applying standard theme icons with the help of the icon attribute, and so on. Creating a set of icons of our own In this section, we are going to discuss how to create our own icons for the PrimeFaces web application. Instead of using images, you need to use image sprites by considering the impact of application performance. Most of the time, we might be interested in adding custom icons to UI components apart from the regular standard icon set. Generally, in order to create our own custom icons, we need to provide CSS classes with the background-image property, which is referred to the image in the theme images folder. For example, the following commandButton components will use a custom icon: <p:commandButton value="With Icon" icon="disk"/> <p:commandButton icon="disk"/> The disk icon is created by adding the .disk CSS class with the background image property. In order to display the image, you need to provide the correct relative path of the image from the web application, as follows: .disk {   background-image: url('disk.png') !important; } However, as discussed earlier, we are going to use the image sprite technology instead of a separate image for each icon to optimize web performance. Before creating an image sprite, you need to select all the required images and convert those images (PNG, JPG, and so on) to the icon format with a size almost equal to to that of the ThemeRoller icons. In this article, we used the Paint.NET tool to convert images to the ICO format with a size of 16 by 16 pixels. Paint.NET is a free raster graphics editor for Microsoft Windows, and it is developed on the .NET framework. It is a good replacement for the Microsoft Paint program in an editor with support for layers blending, transparency, and plugins. If the ICO format is not available, then you have to add the file type plug-in for the Paint.NET installation directory. So, this is just a two-step process for the conversion: The image (PNG, JPG, and so on) need to be saved as the Icons (*.ico) option from the Save as type dropdown. Then, select 16 by 16 dimensions with the supported bit system (8-bit, 32-bit, and so on). All the PrimeFaces theme icons are designed to have the same dimensions. There are many online and offline tools available that can be used to create an image sprite. I used Instant Sprite, an open source CSS sprite generator tool, to create an image sprite in this article. You can have a look at the official site for this CSS generator tool by visiting http://instantsprite.com/. Let's go through the following step-by-step process to create an image sprite using the Instant Sprite tool: First, either select multiple icons from your computer, or drag and drop icons on to the tool page. In the Thumbnails section, just drag and drop the images to change their order in the sprite. Change the offset (in pixels), direction (horizontal, vertical, and diagonal), and the type (.png or .gif) values in the Options section. In the Sprite section, right-click on the image to save it on your computer. You can also save the image in a new window or as a base64 type. In the Usage section, you will find the generated sprite CSS classes and HTML. Once the image is created, you will be able to see the image in the preview section before finalizing the image. Now, let's start creating the image sprite for button bar and menu components, which are going to be used in later sections. First, download or copy the required individual icons on the computer. Then, select all those files and drag and drop them in a particular order, as follows: We can also configure a few options, such as an offset of 10 px for icon padding, direction as horizontal to display them horizontally, and then finally selecting the image as the PNG type: The image sprite is generated in the sprite section, as follows: Right-click on the image to save it on your computer. Now, we have created a custom image sprite from the set of icons. Once the image sprite has been created, change the sprite name to ui-custom-icons and copy the generated CSS styles for later. In the generated HTML, note that each div class is appended with the ui-icon class to display the icon with a width of 16 px and height of 16 px. Adding the new icons to your theme In order to apply the custom icons to your web page, we first need to copy the generated image sprite file and then add the generated CSS classes from the previous section. The following generated sprite file has to be added to the images folder of the primefaces-moodyBlue2 custom theme. Let's name the file ui-custom-icons: After this, copy the generated CSS rules from the previous section. The first CSS class (ui-icon) contains the image sprite to display custom icons using the background URL property and dimensions such as the width and height properties for each icon. But since we are going to add the image reference in widget state style classes, you need to remove the background image URL property from the ui-icon class. Hence, the ui-icon class contains only the width and height dimensions: .ui-icon {   width: 16px;   height: 16px; } Later, modify the icon-specific CSS class names as shown in the following format. Each icon has its own icon name: .ui-icon-{icon name} The following CSS classes are used to refer individual icons with the help of the background-position property. Now after modification, the positioning CSS classes will look like this: .ui-icon-edit { background-position: 0 0; } .ui-icon-bookmark { background-position: -26px 0; } .ui-icon-next { background-position: -52px 0; } .ui-icon-previous { background-position: -78px 0; } .ui-icon-new { background-position: -104px 0; } .ui-icon-delete { background-position: -130px 0; } .ui-icon-refresh { background-position: -156px 0; } .ui-icon-print { background-position: -182px 0; } .ui-icon-home { background-position: -208px 0; } .ui-icon-admin { background-position: -234px 0; } .ui-icon-contactus { background-position: -260px 0; } Apart from the preceding CSS classes, we have to add the component state CSS classes. Widget states such as hover, focus, highlight, active, and error need to refer to different image sprites in order to display the component state behavior for user interactions. For demonstration purposes, we created only one image sprite and used it for all the CSS classes. But in real-time development, the image will vary based on the widget state. The following widget states refer to image sprites for different widget states: .ui-icon, .ui-widget-content .ui-icon {   background-image: url("#{resource['primefaces-     moodyblue2:images/ui-custom-icons.png']}"); } .ui-widget-header .ui-icon {   background-image: url("#{resource['primefaces-     moodyblue2:images/ui-custom-icons.png']}"); } .ui-state-default .ui-icon {   background-image: url("#{resource['primefaces-     moodyblue2:images/ui-custom-icons.png']}"); } .ui-state-hover .ui-icon, .ui-state-focus .ui-icon {   background-image: url("#{resource['primefaces-     moodyblue2:images/ui-custom-icons.png']}"); } .ui-state-active .ui-icon {   background-image: url("#{resource['primefaces-     moodyblue2:images/ui-custom-icons.png']}"); } .ui-state-highlight .ui-icon {   background-image: url("#{resource['primefaces-     moodyblue2:images/ui-custom-icons.png']}"); } .ui-state-error .ui-icon, .ui-state-error-text .ui-icon {   background-image: url("#{resource['primefaces-     moodyblue2:images/ui-custom-icons.png']}"); } In the JSF ecosystem, image references in the theme.css file must be converted to an expression that JSF resource loading can understand. So at first, in the preceding CSS classes, all the image URLs are appeared in the following expression: background-image: url("images/ui-custom-icons.png"); The preceding expression, when modified, looks like this: background-image: url("#{resource['primefaces-   moodyblue2:images/ui-custom-icons.png']}");  We need to make sure that the default state classes are commented out in the theme.css (the moodyblue2 theme) file to display the custom icons. By default, custom theme classes (such as the state classes and icon classes available under custom states and images and custom icons positioning) are commented out in the source code of the GitHub project. So, we need to uncomment these sections and comment out the default theme classes (such as the state classes and icon classes available under states and images and positioning). This means that the default or custom style classes only need to be available in the theme.css file. (OR) You can see all these changes in moodyblue3 theme as well. The custom icons appeared in Custom Icons screen by just changing the current theme to moodyblue3. Using custom icons in the commandButton components After applying the new icons to the theme, you are ready to use them on the PrimeFaces components. In this section, we will add custom icons to command buttons. Let's add a link named Custom Icons to the chaptersTemplate.xhtml file. The title of this page is also named Custom Icons. The following code snippets show how custom icons are added to command buttons using the icon attribute: <h3 style="margin-top: 0">Buttons</h3> <p:commandButton value="Edit" icon="ui-icon-edit" type="button" /> <p:commandButton value="Bookmark" icon="ui-icon-bookmark"   type="button" /> <p:commandButton value="Next" icon="ui-icon-next" type="button" /> <p:commandButton value="Previous" icon="ui-icon-previous"   type="button" /> Now, run the application and navigate to the newly created page. You should see the custom icons applied to the command buttons, as shown in the following screenshot: The commandButton component also supports the iconpos attribute if you wish to display the icon either to the left or right side. The default value is left. Using custom icons in a menu component In this section, we are going to add custom icons to a menu component. The menuitem tag supports the icon attribute to attach a custom icon. The following code snippets show how custom icons are added to the menu component: <h3>Menu</h3> <p:menu style="margin-left:500px">   <p:submenu label="File">     <p:menuitem value="New" url="#" icon="ui-icon-new" />     <p:menuitem value="Delete" url="#" icon="ui-icon-delete" />     <p:menuitem value="Refresh" url="#" icon="ui-icon-refresh" />     <p:menuitem value="Print" url="#" icon="ui-icon-print" />   </p:submenu>   <p:submenu label="Navigations">     <p:menuitem value="Home" url="http://www.primefaces.org"       icon="ui-icon-home" />     <p:menuitem value="Admin" url="#" icon="ui-icon-admin" />     <p:menuitem value="Contact Us" url="#" icon="ui-icon-       contactus" />   </p:submenu> </p:menu> Now, run the application and navigate to the newly created page. You will see the custom icons applied to the menu component, as shown in the following screenshot: Thus, you can apply custom icons on a PrimeFaces component that supports the icon attribute. The FontAwesome icons as an alternative to the ThemeRoller icons In addition to the default ThemeRoller icon set, the PrimeFaces team provided and supported a set of alternative icons named the FontAwesome iconic font and CSS framework. Originally, it was designed for the Twitter Bootstrap frontend framework. Currently, it works well with all frameworks. The official site for the FontAwesome toolkit is http://fortawesome.github.io/Font-Awesome/. The features of FontAwesome that make it a powerful iconic font and CSS toolkit are as follows: One font, 519 icons: In a single collection, FontAwesome is a pictographic language of web-related actions No JavaScript required: It has minimum compatibility issues because FontAwesome doesn't required JavaScript Infinite scalability: SVG (short for Scalable Vector Graphics) icons look awesome in any size Free to use: It is completely free and can be used for commercial usage CSS control: It's easy to style the icon color, size, shadow, and so on Perfect on retina displays: It looks gorgeous on high resolution displays It can be easily integrated with all frameworks Desktop-friendly Compatible with screen readers FontAwesome is an extension to Bootstrap by providing various icons based on scalable vector graphics. This FontAwesome feature is available from the PrimeFaces 5.2 release onwards. These icons can be customized in terms of size, color, drop and shadow and so on with the power of CSS. The full list of icons is available at both the official site of FontAwesome (http://fortawesome.github.io/Font-Awesome/icons/) as well as the PrimeFaces showcase (http://www.primefaces.org/showcase/ui/misc/fa.xhtml). In order to enable this feature, we have to set primefaces.FONT_AWESOME context param in web.xml to true, as follows: <context-param>   <param-name>primefaces.FONT_AWESOME</param-name>   <param-value>true</param-value> </context-param> The usage is as simple as using the standard ThemeRoller icons. PrimeFaces components such as buttons or menu items provide an icon attribute, which accepts an icon from the FontAwesome icon set. Remember that the icons should be prefixed by fa in a component. The general syntax of the FontAwesome icons will be as follows: fa fa-[name]-[shape]-[o]-[direction] Here, [name] is the name of the icon, [shape] is the optional shape of the icon's background (either circle or square), [o] is the optional outlined version of the icon, and [direction] is the direction in which certain icons point. Now, we first create a new navigation link named FontAwesome under chapter6 inside the chapterTemplate.xhtml template file. Then, we create a JSF template client called fontawesome.xhtml, where it explains the FontAwesome feature with the help of buttons and menu. This page has been added as a menu item for the top-level menu bar. In the content section, replace the text content with the following code snippets. The following set of buttons displays the FontAwesome icons with the help of the icon attribute. You may have observed that the fa-fw style class used to set icons at a fixed width. This is useful when variable widths throw off alignment: <h3 style="margin-top: 0">Buttons</h3> <p:commandButton value="Edit" icon="fa fa-fw fa-edit"   type="button" /> <p:commandButton value="Bookmark" icon="fa fa-fw fa-bookmark"   type="button" /> <p:commandButton value="Next" icon="fa fa-fw fa-arrow-right"   type="button" /> <p:commandButton value="Previous" icon="fa fa-fw fa-arrow-  left"   type="button" /> After this, apply the FontAwesome icons to navigation lists, such as the menu component, to display the icons just to the left of the component text content, as follows: <h3>Menu</h3> <p:menu style="margin-left:500px">   <p:submenu label="File">     <p:menuitem value="New" url="#" icon="fa fa-plus" />     <p:menuitem value="Delete" url="#" icon="fa fa-close" />     <p:menuitem value="Refresh" url="#" icon="fa fa-refresh" />     <p:menuitem value="Print" url="#" icon="fa fa-print" />   </p:submenu>   <p:submenu label="Navigations">     <p:menuitem value="Home" url="http://www.primefaces.org"       icon="fa fa-home" />     <p:menuitem value="Admin" url="#" icon="fa fa-user" />     <p:menuitem value="Contact Us" url="#" icon="fa fa-       picture-o" />   </p:submenu> </p:menu> Now, run the application and navigate to the newly created page. You should see the FontAwesome icons applied to buttons and menu items, as shown in the following screenshot: Note that the 40 shiny new icons of FontAwesome are available only in the PrimeFaces Elite 5.2.2 release and the community PrimeFaces 5.3 release because PrimeFaces was upgraded to FontAwesome 4.3 version since its 5.2.2 release. Summary In this article, we explored the standard theme icon set and how to use it on various PrimeFaces components. We also learned how to create our own set of icons in the form of the image sprite technology. We saw how to create image sprites using open source online tools and add them on a PrimeFaces theme. Finally, we had a look at the FontAwesome CSS framework, which was introduced as an alternative to the standard ThemeRoller icons. To ensure best practice, we learned how to use icons on commandButton and menu components. Now that you've come to the end of this article, you should be comfortable using web icons for PrimeFaces components in different ways. Resources for Article: Further resources on this subject: Introducing Primefaces [article] Setting Up Primefaces [article] Components Of Primefaces Extensions [article]

In this article by Michael David, author of the book Wordpress Search Engine Optimization, you will learn about Google Analytics and Google Webmaster/Search Console. Once you have built your website and started promoting it, you'll want to monitor your progress to ensure that your hard work is yielding both high rankings, search engine visibility, and web traffic. In this article, we'll cover a range of tools with which you will monitor the quality of your website, learn how search spiders interact with your site, measure your rankings in search engines for various keywords, and analyze how your visitors behave, when they are on your site. With this information, you can gauge your progress and make adjustments to your strategy. (For more resources related to this topic, see here.) Obviously, you'll want to know where your web traffic is coming from, what search terms are being used to find your website, and where you are ranking in the search engines for each of these terms. This information will allow you to see what you still need to work on, in terms of building links to the pages on your website. There are five main tools you will use to analyze your site and evaluate your traffic and rankings, and in this article, we will cover each in turn. They are Google Analytics, Google Search Console (formerly Webmaster Tools), HTML Validator, Bing Webmaster, and Link Assistant's Rank Tracker. As an alternative to Bing Webmaster, you may also want to employ Majestic SEO to check your backlinks. We'll cover each of these tools in turn. Google Analytics Google Analytics monitors and analyzes your website's traffic. With this tool, you can see how many website visitors you have had, whether they found your site through the search engines or clicked through from another website, how these visitors behaved once they were on your site, and much more. You can even connect your Google AdSense account to one or more of the domains you are monitoring in Google Analytics, to get information about which pages and keywords generate the most income. While there are other analytics services available, none match the scope and scale of what Google Analytics offers. Setting up Google Analytics for your website To set up Google Analytics for your website, perform the following steps: To sign up for Google Analytics, visit http://www.google.com/analytics/. On the top-right corner of the page, you'll see a button that says, Sign In to Google Analytics. You'll need to have a Google account before signing in. You will have to click Sign up on the next page if you haven't already, and on the next page you'll enter your website's URL and time zone. If you have more than one website, just start with one. You can add the other sites later. The account name you use is not important, but you can add all of your sites to one account, so you might want to use something generic like your name or your business name. Select the time zone country and time zone, and then click on Continue. Enter your name and country, and click on Continue again. Then you will need to accept the Google Analytics terms of service. After you have accepted the terms, you will be given a snippet of HTML code, that you'll need to insert into the pages of your website. The snippet will look like the following: <script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-xxxxxx-x', 'auto'); ga('send', 'pageview');</script> The code must be placed just before the closing head tag in each page on your website. There are several ways to install the code. Check first to see, if your WordPress template offers a field to insert the code in the template admin area—most modern templates offer this feature. If not, you can insert the code manually just before the closing the </head> tag, which you will find in the file called header.php within your WordPress template files. There is yet a third way to do this, if you don't want to tinker with your WordPress code. You can download and install one of the many WordPress analytics plugins. Two sound choices for an analytics plugin would be Google Analyticator or Google Analytics by Yoast; both the plugins are available at Wordpress.org. If you have more than one website, you can add more websites within your analytics account. Personally, I like to keep one website per analytics account just to keep things neat, but some like to have all their websites under one analytics account. To add a second or third website to an analytics account, navigate to Admin on the top menu bar, and then pull down the menu under Account in the left column, and then click Create a New Account. When you are done adding websites, click Home on the top menu bar to navigate to the main analytics page. From here, you can select a web property. This is the screen that greets you when you log in to Google Analytics, the Audience Overview report. It offers an effective quick look at how your website is performing. Using Google Analytics Once you have installed Google Analytics on your websites, you'll need to wait a few weeks, or perhaps longer, for Analytics to collect enough data from your websites to be useful. Remember that with website analytics, like any type of statistical analysis, larger sets of data reveal more accurate and useful information. It may not take that long if you have a high-traffic site, but if you have a fairly new site that doesn't get a lot of traffic, it will take some time, so come back in a few weeks to a month to check to see how your sites are doing. The Audience Overview report is your default report, displayed to you, when you first log into analytics and gives you a quick overview of each site's traffic. You can see at a glance, whether the tracking code is receiving data, how many visits (sessions) your website has gotten in the past 30 days, the average time visitors stay on your site (session duration), your bounce rate (the percentage of users that come to visit your site and leave without visiting another page), and the percentage of new sessions (users that haven't visited before) in the past 30 days. The data displayed on the dashboard is just a small taste of what you can learn from Google Analytics. To drill down to more detailed information, you'll navigate to other sections of analytics using the left menu. Your main report areas in analytics are Audience (who your visitors are), Acquisition (how your visitors found your site), Behavior (what your visitors did on your site), and Conversions (did they make a purchase or complete a contact form). Within any of the main report areas, are a dozens of available reports. Google Analytics can offer you tremendous detail on almost any imaginable metric related to your website. Most of your time, however, is best spent with a few key reports. Understanding key analytics reports Your key analytics reports will be the Audience Overview, Acquisition Overview, and the Behavior Overview. You access each of these reports by navigating to the left menu area and each corresponding overview report is the first link in each list. The Audience Overview report is your default report, discussed earlier. The Acquisition Overview report drills down into how your visitors are finding your site, whether it is through organic search, pay per click programs, social media, or referrals from other websites. These different pathways by which customers find your site are referred to as channels or mediums on other reports, but mean essentially the same thing. The Acquisition Overview report also shows some very basic conversion data, although the reports in the Conversions section offer much more meaningful conversion data. The following screenshot is an Acquisition Overview report: Why is the Acquisition Overview report important? It shows us the relative strength of our inbound channels. In the case above, organic search is delivering the highest number of users. This tells us that we are getting most of our users from the organic search results—our organic campaign is running strong. Referrals generated 384 visits, which means we've got good links delivering traffic. Our 369 direct visitors don't come from other websites, it's a fresh browser page where users type our URL directly or follow a bookmark they've saved. That is a welcome figure, because it means we've got strong brand and name recognition and in the conversions column on the right side of the table, we can see that our direct traffic generated a measurable conversion rate. That's a positive sign that our brand recognition and reputation is strong. The Behavior Overview report tells us how users behave once they visit our site. Are they bouncing immediately or viewing several pages? What content are they viewing the most? Here's a sample Behavior Overview report: Some information is repeated on the Behavior Overview report, such as Pageviews and Bounce Rate. What is important here is the table under the graph. This table shows you the relative popularity of your pages. This data is important because it shows you what content is generating the most user interest. As you can see from the table, the page titled /add-sidebar-wordpress generated over 1,000 pageviews, more than the home page, which is indicated in analytics by a single slash (/). This table shows you your most popular content—these pages are your greatest success. And remember, you can click on any individual page and then see individual metrics for that page. One way to maximize your site earnings is to focus on improving the performance of the pages that are already earning money. Chances are, you earn at least 80 percent of your income from the top 20 percent of the pages on your site. Focus on improving the rankings for those pages in order to get the best return for your efforts. With any analytics report, by default the statistics shown will all be for the past month, but you can adjust the time period, by clicking on the down arrow next to the dates in the upper right hand corner. Setting up automated analytics reports You can also use Google Analytics to e-mail you daily, weekly, or monthly reports. To enable this feature, simply navigate to the report that you'd like to be sent to you. Then, click the Email link just under the report title. The following pop-up will appear: The Frequency field lets you determine how often the report is sent, or you can simply send the report one time. Google Webmasters/Search Console Now, we are going to go into a bit more detail, and show you how to use Google Webmaster Tools to obtain information that you can use to improve your website. Understanding your website's search queries The Search Console now shows you data on what search queries users entered in Google search to find their way to your website. This is valuable: it teaches you what query terms are effectively delivering customers. To see the report, expand Search Traffic on the left navigation menu and select Search Analytics. The following report will display: Examine the top queries that Search Console shows you are getting traffic for, and add them to your list of keywords to work on, if they are not already there. You will probably find that it is most beneficial to focus on those keywords, that are currently ranked between #4 and #10 in Google, to try to get them moved up to one of the top three spots. You'll also want to check for crawl errors on each of your sites while you work in the Search Console. A crawl error is an error that Google's spider encounters, when trying to find pages on your site. A dead link, a link to a page on your site that no longer exists, is a common and perfect example of a crawl error. Crawl errors are detrimental to rankings. First, crawl errors send a poor quality signal to search engines. Remember that, Google wants to deliver a great experience to users of its search engine and properties. Dead links and lost pages do not deliver a great experience to users. To see the crawl error data, expand the Crawl link on the left navigation bar and then click Crawl Errors. The crawl error report will show you any pages that Google attempted to read but could not reach. Not all entries on this report are bad news. If you remove a piece of content that Google crawled in the past, Google will attempt for months to try to find that piece of content again. So, a removed page will generate a crawl error. Another way crawl errors get generated is if you have a sitemap file with page entries that no longer exist. Even inbound links from third party websites to pages that don't exist, will generate crawl errors. Other errors you find might be the result of a typo or the other error that involves going into a specific page on your website to fix. For example, perhaps you made a mistake typing in the URL when linking from one page to another, resulting in a 404 error. To fix that, you need to edit the page that the URL with the error was linked from and correct the URL. Other errors might require editing the files for your template to fix multiple errors simultaneously. For example, if you see that the Googlebot is getting a 404 (page not found) error every time it attempts to crawl the comments feed for a post, then the template file probably doesn't have the right format for creating those URLs. Once you correct the template file, all of the crawl errors related to that problem will be fixed. There are other things you can do in Google Webmaster Tools, for example, you can check to see how many links to your site Google is detecting. Checking your website's code with a HTML Validator HyperText Markup Language (HTML) is a coding standard with a reasonable degree of complexity. HTML standards develop over time, and a valid HTML code displays more websites more accurately in a wider range of browsers. Sites that have substantial amounts of HTML coding errors can potentially be punished by search engines. For this reason, you should periodically check your website for HTML coding errors. There are two principal tools, that web professionals use to check the quality of their websites' code: the W3C HTML Validator and the CSE HTML Validator. The W3C HTML Validator (http://validator.w3.org) is the less robust of the two validators, but it is free. Both validators work in the same way; they examine the code on your website and issue a report advising you of any errors. The CSE HTML Validator (http://htmlvalidator.com) is not a free tool, but you can get a free trial of the software that is good for 30 days or 200 validations, whichever comes first. This software catches errors in HTML, XHTML, CSS, PHP, and JavaScript. It also checks your links, to ensure that they are all valid. It includes an accessibility checker, spell checker, and a SEO checker. With all of these functions, there is a good chance that if your website has any problems, the CSE HTML Validator will be able to find them. After downloading and installing the demo version of CSE HTML Validator, you will be given the option to go to a page that contains two video demos. It is a good idea to watch these videos, or at least the HTML validation video, before trying to use the program. They are not very long, and watching them will reduce the amount of time it takes you to learn to use the program. To validate a file that is on your hard drive, first open the file in the editor, then click the down arrow next to the Validate button on the task bar. You will see several options. Selecting Full will give you not only errors, but messages containing suggestions as well. Errors only will only show you actual errors, and Errors and warnings only will tell you if there are things that could be errors, but might not be. You can experiment with the different options to see which one you like best. After you select a validation option, a box will appear at the bottom of the screen listing all of the errors, as well as warnings and messages depending on the option you chose. You might be surprised at how many errors there are, especially if you are using WordPress to create the code for your site. The code is often not as clean as you might expect it to be. However, not all of the errors you see, will be things that you need to worry about or correct. Yes, it is better to have a website with perfect coding, but one of the advantages of using WordPress is that, you don't have to know how to code to build a website. If you do not know anything about HTML coding, you may do more harm than good by trying to fix minor errors, such as omission of a slash at the end of a tag. Most of these errors will not cause problems anyhow. You should look through the errors and fix the ones you know how to fix. If you are completely mystified by what you see here, don't worry about it too much unless you are having a problem with the way your website loads or displays. If the errors are causing problems, you'll either have to learn a bit about coding or hire someone who knows what they're doing to fix your website. If you want to be able to check the code of an entire website at once, you'll need to buy the Pro version of CSE HTML Validator. You can then use the batch wizard to check your website. This feature is not available in the Standard or Lite versions of the software. To use the batch wizard, click on Tools, then Batch Wizard. A new window will pop up, allowing you to choose the files you want to check. Click on the button with the large green plus sign, and select the appropriate option to add files or URLs to your list. You can add files individually, add an entire folder, or even add a URL. To check an entire site, you can add the root file for your domain from your hard drive, or you can add the URL for the root domain. Once you have added your target, click on it. Now, click on Target in the main menu bar, then on Properties. Click on the Follow Links tab in the box that pops up, then check the box in front of Follow and validate links. Click on the OK button and then click on the Process button to start validating. Checking your inbound link count with Bing Webmaster Bing Webmaster allows you to get information about backlinks, crawl errors, and search traffic to your websites. In order to get the maximum value from this tool, you'll need to authenticate each of your websites to prove that you own them. To get started, go to https://www.bing.com/webmaster/ and sign up with your Microsoft account. As a part of the sign up process, you'll get a HTML file, that you'll install in the root directory of your WordPress installation. This file validates that you are the owner of the website and entitled to see the data that Bing collects. One core use for Bing Webmaster is that it presents a highly accurate picture of one's inbound link counts. If you recall, Google does not present accurate inbound link counts to users. Thus, Bing Webmaster is the most authoritative picture from a search engine, that you'll have of how many backlinks your site enjoys. To see your inbound links, simply log in and navigate to the Dashboard. At the lower right, you'll see the Inbound Links table shown here: The table shows you the inbound links to your website for each page of content. This helpful feature lets you determine, which articles of content are garnering the most interest from the other webmasters. High link counts are always good, but you also want to make sure you are getting high quality links from websites in the same general category as your site. Bing offers an additional feature: link keyword research. Expand the Diagnostics & Tools entry on the navigation bar on the left, and click Keyword Research. The search traffic section will give you valuable information about the search terms you should be targeting for your site, as well as allowing you to see which of the terms you are already targeting are getting traffic. Just as you did with the keywords shown in Google Analytics and Google Webmaster Tools, you want to find keywords that are getting traffic, but are not currently ranked in the top one to three positions in the search engine results pages. Send more links to these pages with the keyword phrase you want to target as anchor text, in order to move these pages up in the rankings. Monitoring ranking positions and movement with Rank Tracker Rank Tracker is a paid tool and we've included it because it is tremendously valuable and noteworthy. Rank Tracker is a software tool that can monitor your website's rankings for thousands of terms in hundreds of different search engines. Even more, it maintains a historical ranking data that helps you gauge your progress as you work. It is a valuable tool that is used by many SEO professionals. There is a free version, although the free version does not allow you to save your work (and thus does not let you save historical information). To really harness the power of this software, you'll need the paid version. You can download either version from http://www.link-assistant.com/rank-tracker/. After you install the program, you will be prompted to enter the URL of your website. On the next screen, the program will ask you to input the keywords you wish to track. If you have them all listed in a spreadsheet somewhere, you can just copy the column that has your keywords in it and paste them all into the tool. Then Rank Tracker will ask you which search engines you are targeting and will check the rank of each keyword in all of the search engines you select. It only takes a few minutes for Rank Tracker to update hundreds of keywords, so you can find out where you are ranking in very little time. This screenshot shows the main interface of the Rank Tracker software. For monitoring progress on large numbers of keywords on several different search engines, Rank Tracker can be a real time-saver: Once your rankings have been updated, you can sort them by rank to see which ones will benefit the most from additional link building. Remember that more than half of the people who search for something in Google, click on the first result. If you are not in the top three to five results, you will get very little traffic from people searching for your targeted keyword. For this reason, you will usually get the best results by working on the keywords that are ranking between #4 and #10 in the search engine results pages. If you want more data to play with, you can click on Update KEI, to find out the keyword effectiveness index for each keyword. This tool gathers the data from Google, to tell you how many searches per month and how much competition there is for each keyword, and then calculates the KEI score based on that data. As a general rule, the higher the KEI number is, the easier it will be to rank for a given keyword. Keep in mind, however, that if you are already ranking for the keyword, it will usually not be that difficult to move up in the rankings, even if the KEI is low. To make it easier to see which keywords will be easy to rank, they are color-coded in the Rank Tracker tool. The easiest ones are green, and the hardest are red. Yellow and orange fall in between. In addition to checking your rankings on keywords you are targeting, you can use the Rank Tracker tool to find more keywords to target. If you navigate to Tools and then Get Keyword Suggestions, a window will pop up that will let you choose from a range of different methods of finding keywords. It is recommended that you start with the Google AdWords Keyword Tool. After you choose the method, you'll be asked to enter keywords that are relevant to the content of your website. You will only get about 100 results no matter how many keywords you enter, so it's best to work on just one at a time. On the next page, you will be presented with a list of keywords. Select the ones you want to add and click on Next. When the tool is done updating the data for the selected keywords, click on the Finish button to add them to your project. You can repeat this process as many times as you want to find new keywords for your website, and you can experiment with the other fifteen tools as well, which should give you more variety. When you find keywords that look promising, put them on a list, and plan on writing posts to target those keywords in the future. Look for keywords that have at least 100 searches per month, with a relatively low amount of competition. Rank Tracker not only allows you to check your current rankings, but it also keeps track of your historical data as well. Every time you update, a new point will be added to the progress graph so you can see your progress over time. This allows you to see, whether what you are doing is working or not. If you see that your rank is dropping for a certain keyword, you can use that information to figure out whether something you changed had the opposite effect that you intended. If you are doing SEO for clients, you'll find the reports in Rank Tracker to be extremely useful (if not mandatory). You can create a monthly report for each client that shows how many keywords are ranked #1, as well as how many are in the top 10, 20, or 100. The report also shows the number of keywords that moved up and the number that moved down. You can use these reports to show your clients how much progress you are making and demonstrate your value to the client. If you want to take advantage of the historical data, you'll have to purchase the paid version of Rank Tracker. The free version does not support saving your projects, so you won't have data from your past rankings to compare and see whether you are moving up or down in the search engine results. You also won't be able to generate reports, that tell you what has changed from the last time you updated your rankings. Monitoring backlinks with majestic SEO If you want to see how many backlinks you have pointing to your site along with a range of additional data, the king of free backlink tools is the powerful Majestic SEO backlink checker tool. To use this tool, go to https://majestic.com/ and enter your domain in the box at the top of the page. The tool will generate a report that shows how many URLs are indexed for your domain, how many total backlinks you have, and how many unique domains link to your website. For heavy-duty link reconnaissance, you'll want the paid upgrade. Underneath the site info, you can see the stats for each page on your site. The tool shows the number of backlinks for each page, as well as the number of domains linking to each page. You can only see ten results per page, so you'll have to click through numerous pages to see all of the results if you have a large site. Majestic SEO offers a few details that you won't get from Google or Bing Webmaster. Majestic SEO calculates and reports the number of separate C class subnets upon which your backlinks appear. As we have learned, links from sites on separate C class subnets are more valuable, because they are perceived by search engines as being truly non-duplicate links. Majestic SEO also reports the number of .edu and .gov upon which your links appear. This extra information gives you a clear picture of how effective your link building efforts are progressing. Majestic offers another feature of note: Majestic crawls the web more deeply, so you'll see higher link counts. Majestic is particularly useful when doing link cleanup, because it scrapes low-value sites that Google and Bing don't bother indexing. This screenshot highlights some of Majestic SEO's more robust features: it shows you the number of backlinks from .edu and .gov domains as well as the number of separate Class C subnets upon which your inbound links appear: Majestic SEO offers one more special feature; it records and graphs your backlink acquisition over time. The graph in the screenshot just above, shows this feature in action. Majestic SEO is not a search engine, so it will show you a count of inbound links without any regards to the quality of the pages on which your links appear. Put another way, Bing Webmaster will only show you links that appear on indexed pages. Lower value pages, such as pages with duplicate content or pages in low-value link directories, tend not to appear in search engine indexes. As such, Majestic SEO reports higher link counts than Bing Webmaster or Google. Summary In this article, we learned how to monitor your progress through the use of free and paid tools. We learned how to set up and employ Google Analytics, to measure and monitor where your website visitors are coming from and how they behave on your site. We learned how to set up and use Google Webmaster Tools to detect crawling errors on your site and learned how Googlebot interacts with your site. We discovered two HTML validation tools that you can use to ensure that your website's code meets current HTML standards. Finally, we learned how to measure and monitor your backlink efforts with Bing Webmaster and Majestic SEO. With the tools and techniques in this article, you can ensure that your optimization efforts are effective and remain on track. Resources for Article: Further resources on this subject: Creating Blog Content in WordPress [Article] Responsive Web Design with WordPress [Article] Introduction to a WordPress application's frontend [Article]

 In this article, by Hernán G. Resnizky, author of the book Learning Shiny, the main objective will be to learn how to install all the needed components to build an application in R with Shiny. Additionally, some general ideas about what R is will be covered in order to be able to dive deeper into programming using R. The following topics will be covered: A brief introduction to R, RStudio, and Shiny Installation of R and Shiny General tips and tricks (For more resources related to this topic, see here.) About R As stated on the R-project main website: "R is a language and environment for statistical computing and graphics." R is a successor of S and is a GNU project. This means, briefly, that anyone can have access to its source codes and can modify or adapt it to their needs. Nowadays, it is gaining territory over classic commercial software, and it is, along with Python, the most used language for statistics and data science. Regarding R's main characteristics, the following can be considered: Object oriented: R is a language that is composed mainly of objects and functions. Can be easily contributed to: Similar to GNU projects, R is constantly being enriched by user's contributions either by making their codes accessible via "packages" or libraries, or by editing/improving its source code. There are actually almost 7000 packages in the common R repository, Comprehensive R Archive Network (CRAN). Additionally, there are R repositories of public access, such as bioconductor project that contains packages for bioinformatics. Runtime execution: Unlike C or Java, R does not need compilation. This means that you can, for instance, write 2 + 2 in the console and it will return the value. Extensibility: The R functionalities can be extended through the installation of packages and libraries. Standard proven libraries can be found in CRAN repositories and are accessible directly from R by typing install.packages(). Installing R R can be installed in every operating system. It is highly recommended to download the program directly from http://cran.rstudio.com/ when working on Windows or Mac OS. On Ubuntu, R can be easily installed from the terminal as follows: sudo apt-get update sudo apt-get install r-base sudo apt-get install r-base-dev The installation of r-base-dev is highly recommended as it is a package that enables users to compile the R packages from source, that is, maintain the packages or install additional R packages directly from the R console using the install.packages() command. To install R on other UNIX-based operating systems, visit the following links: http://cran.rstudio.com/ http://cran.r-project.org/doc/manuals/r-release/R-admin.html#Obtaining-R A quick guide to R When working on Windows, R can be launched via its application. After the installation, it is available as any other program on Windows. When opening the program, a window like this will appear: When working on Linux, you can access the R console directly by typing R on the command line: In both the cases, R executes in runtime. This means that you can type in code, press Enter, and the result will be given immediately as follows: > 2+2 [1] 4 The R application in any operating system does not provide an easy environment to develop code. For this reason, it is highly recommended (not only to write web applications in R with Shiny, but for any task you want to perform in R) to use an Integrated Development Environment (IDE). About RStudio As with other programming languages, there is a huge variety of IDEs available for R. IDEs are applications that make code development easier and clearer for the programmer. RStudio is one of the most important ones for R, and it is especially recommended to write web applications in R with Shiny because this contains features specially designed for R. Additionally, RStudio provides facilities to write C++, Latex, or HTML documents and also integrates them to the R code. RStudio also provides version control, project management, and debugging features among many others. Installing RStudio RStudio for desktop computers can be downloaded from its official website at http://www.rstudio.com/products/rstudio/download/ where you can get versions of the software for Windows, MAC OS X, Ubuntu, Debian, and Fedora. Quick guide to RStudio Before installing and running RStudio, it is important to have R installed. As it is an IDE and not the programming language, it will not work at all. The following screenshot shows RStudio's starting view: At the first glance, the following four main windows are available: Text editor: This provides facilities to write the R scripts such as highlighting and a code completer (when hitting Tab, you can see the available options to complete the code written). It is also possible to include the R code in an HTML, Latex, or C++ piece of code. Environment and history: They are defined as follows: In the Environment section, you can see the active objects in each environment. By clicking on Global Environment (which is the environment shown by default), you can change the environment and see the active objects. In the History tab, the pieces of codes executed are stored line by line. You can select one or more lines and send them either to the editor or to the console. In addition, you can look up for a certain specific piece of code by typing it in the textbox in the top right part of this window. Console: This is an exact equivalent of R console, as described in Quick guide of R. Tabs: The different tabs are defined as follows: Files: This consists of a file browser with several additional features (renaming, deleting, and copying). Clicking on a file will open it in editor or the Environment tab depending on the type of the file. If it is a .rda or .RData file, it will open in both. If it is a text file, it will open in one of them. Plots: Whenever a plot is executed, it will be displayed in that tab. Packages: This shows a list of available and active packages. When the package is active, it will appear as clicked. Packages can also be installed interactively by clicking on Install Packages. Help: This is a window to seek and read active packages' documentation. Viewer: This enables us to see the HTML-generated content within RStudio. Along with numerous features, RStudio also provides keyboard shortcuts. A few of them are listed as follows: Description Windows/Linux OSX Complete the code. Tab Tab Run the selected piece of code. If no piece of code is selected, the active line is run. Ctrl + Enter ⌘ + Enter Comment the selected block of code. Ctrl + Shift + C ⌘ + / Create a section of code, which can be expanded or compressed by clicking on the arrow to the left. Additionally, it can be accessed by clicking on it in the bottom left menu. ##### ##### Find and replace. Ctrl + F ⌘ + F The following screenshots show how a block of code can be collapsed by clicking on the arrow and how it can be accessed quickly by clicking on its name in the bottom-left part of the window: Clicking on the circled arrow will collapse the Section 1 block, as follows: The full list of shortcuts can be found at https://support.rstudio.com/hc/en-us/articles/200711853-Keyboard-Shortcuts. For further information about other RStudio features, the full documentation is available at https://support.rstudio.com/hc/en-us/categories/200035113-Documentation. About Shiny Shiny is a package created by RStudio, which enables to easily interface R with a web browser. As stated in its official documentation, Shiny is a web application framework for R that makes it incredibly easy to build interactive web applications with R. One of its main advantages is that there is no need to combine R code with HTML/JavaScript code as the framework already contains prebuilt features that cover the most commonly used functionalities in a web interactive application. There is a wide range of software that has web application functionalities, especially oriented to interactive data visualization. What are the advantages of using R/Shiny then, you ask? They are as follows: It is free not only in terms of money, but as all GNU projects, in terms of freedom. As stated in the GNU main page: To understand the concept (GNU), you should think of free as in free speech, not as in free beer. Free software is a matter of the users' freedom to run, copy, distribute, study, change, and improve the software. All the possibilities of a powerful language such as R is available. Thanks to its contributive essence, you can develop a web application that can display any R-generated output. This means that you can, for instance, run complex statistical models and return the output in a friendly way in the browser, obtain and integrate data from the various sources and formats (for instance, SQL, XML, JSON, and so on) the way you need, and subset, process, and dynamically aggregate the data the way you want. These options are not available (or are much more difficult to accomplish) under most of the commercial BI tools. Installing and loading Shiny As with any other package available in the CRAN repositories, the easiest way to install Shiny is by executing install.packages("shiny"). The following output should appear on the console: Due to R's extensibility, many of its packages use elements (mostly functions) from other packages. For this reason, these packages are loaded or installed when the package that is dependent on them is loaded or installed. This is called dependency. Shiny (on its 0.10.2.1 version) depends on Rcpp, httpuv, mime, htmltools, and R6. An R session is started only with the minimal packages loaded. So if functions from other packages are used, they need to be loaded before using them. The corresponding command for this is as follows: library(shiny) When installing a package, the package name must be quoted but when loading the package, it must be unquoted. Summary After these instructions, the reader should be able to install all the fundamental elements to create a web application with Shiny. Additionally, he or she must have acquired at least a general idea of what R and the R project is. Resources for Article: Further resources on this subject: R ─ Classification and Regression Trees[article] An overview of common machine learning tasks[article] Taking Control of Reactivity, Inputs, and Outputs [article]

React is an open-sourced JavaScript library made by Facebook for building UI applications. The project has a strong emphasis on the component-based approach and utilizes the full power of JavaScript for constructing all elements. The React Native project was introduced during the first React conference in January 2015. It allows you to build native mobile applications using the same concepts from React. In this post I am going to explain the main building blocks of React Native through the example of an iOS demo application. I assume that you have previous experience in writing web applications with React. Setup Please go through getting started section on the React Native website if you would like to build an application on your machine. Quick start When all of the necessary tools are installed, let's initialize the new React application with the following command: react-native init LastFmTopArtists After the command fetches the code and the dependencies, you can open the new project (LastFmTopArtists/LastFmTopArtists.xcodeproj) in Xcode. Then you can build and run the app with cmd+R. You will see a similar screen on the iOS simulator: You can make changes in index.ios.js, then press cmd+R and see instant changes in the simulator. Demo app In this post I will show you how to build a list of popular artists using the Last.fm api. We will display them with help of ListView component and redirect on the artist page using WebView. First screen Let's start with adding a new screen into our application. For now it will contain dump text. Create file ArtistListScreen with the following code: var React = require('react-native'); var { ListView, StyleSheet, Text, View, } = React; class ArtistListScreen extendsReact.Component { render() { return ( <View style={styles.container}> <Text>Artist list would be here</Text> </View> ); } } var styles = StyleSheet.create({ container: { flex: 1, backgroundColor: 'white', marginTop: 64 } }) module.exports = ArtistListScreen; Here are some things to note: I declare react components with ES6 Classes syntax. ES6 Destructuring assignment syntax is used for React objects declaration. FlexBox is a default layout system in React Native. Flex values can be either integers or doubles, indicating the relative size of the box. So, when you have multiple elements they will fill the relative proportion of the view based on their flex value. ListView is declared but will be used later. From index.ios.js we call ArtistListScreen using NavigatorIOS component: var React = require('react-native'); var ArtistListScreen = require('./ArtistListScreen'); var { AppRegistry, NavigatorIOS, StyleSheet } = React; var LastFmArtists = React.createClass({ render: function() { return ( <NavigatorIOS style={styles.container} initialRoute={{ title: "last.fm Top Artists", component: ArtistListScreen }} /> ); } }); var styles = StyleSheet.create({ container: { flex: 1, backgroundColor: 'white', }, }); Switch to iOS Simulator, refresh with cmd+R and you will see: ListView After we have got the empty screen, let's render some mock data in a ListView component. This component has a number of performance improvements such as rendering of only visible elements and removing which are off screen. The new version of ArtistListScreen looks like the following: class ArtistListScreen extendsReact.Component { constructor(props) { super(props) this.state = { isLoading: false, dataSource: newListView.DataSource({ rowHasChanged: (row1, row2) => row1 !== row2 }) } } componentDidMount() { this.loadArtists(); } loadArtists() { this.setState({ dataSource: this.getDataSource([{name: 'Muse'}, {name: 'Radiohead'}]) }) } getDataSource(artists: Array<any>): ListView.DataSource { returnthis.state.dataSource.cloneWithRows(artists); } renderRow(artist) { return ( <Text>{artist.name}</Text> ); } render() { return ( <View style={styles.container}> <ListView dataSource={this.state.dataSource} renderRow={this.renderRow.bind(this)} automaticallyAdjustContentInsets={false} /> </View> ); } } Side notes: The DataSource is an interface that ListView is using to determine which rows have changed over the course of updates. ES6 constructor is an analog of getInitialState. The end result of the changes: Api token The Last.fm web api is free to use but you will need a personal api token in order to access it. At first it is necessary to join Last.fm and then get an API account. Fetching real data I assume you have successfully set up the API account. Let's call a real web service using fetch API: const API_KEY='put token here'; const API_URL = 'http://ws.audioscrobbler.com/2.0/?method=geo.gettopartists&country=ukraine&format=json&limit=40'; const REQUEST_URL = API_URL + '&api_key=' + API_KEY; loadArtists() { this.setState({ isLoading: true }); fetch(REQUEST_URL) .then((response) => response.json()) .catch((error) => { console.error(error); }) .then((responseData) => { this.setState({ isLoading: false, dataSource: this.getDataSource(responseData.topartists.artist) }) }) .done(); } After a refresh, the iOS simulator should display: ArtistCell Since we have real data, it is time to add artist's images and rank them on the display. Let's move artist cell display logic into separate component ArtistCell: 'use strict'; var React = require('react-native'); var { Image, View, Text, TouchableHighlight, StyleSheet } = React; class ArtistCell extendsReact.Component { render() { return ( <View> <View style={styles.container}> <Image source={{uri: this.props.artist.image[2]["#text"]}} style={styles.artistImage} /> <View style={styles.rightContainer}> <Text style={styles.rank}>## {this.props.artist["@attr"].rank}</Text> <Text style={styles.name}>{this.props.artist.name}</Text> </View> </View> <View style={styles.separator}/> </View> ); } } var styles = StyleSheet.create({ container: { flex: 1, flexDirection: 'row', justifyContent: 'center', alignItems: 'center', padding: 5 }, artistImage: { height: 84, width: 126, marginRight: 10 }, rightContainer: { flex: 1 }, name: { textAlign: 'center', fontSize: 14, color: '#999999' }, rank: { textAlign: 'center', marginBottom: 2, fontWeight: '500', fontSize: 16 }, separator: { height: 1, backgroundColor: '#E3E3E3', flex: 1 } }) module.exports = ArtistCell; Changes in ArtistListScreen: // declare new component var ArtistCell = require('./ArtistCell'); // use it in renderRow method: renderRow(artist) { return ( <ArtistCell artist={artist} /> ); } Press cmd+R in iOS Simulator: WebView The last piece of the application would be to open a web page by clicking in ListView. Declare new component WebView: 'use strict'; var React = require('react-native'); var { View, WebView, StyleSheet } = React; class Web extendsReact.Component { render() { return ( <View style={styles.container}> <WebView url={this.props.url}/> </View> ); } } var styles = StyleSheet.create({ container: { flex: 1, backgroundColor: '#F6F6EF', flexDirection: 'column', }, }); Web.propTypes = { url: React.PropTypes.string.isRequired }; module.exports = Web; Then by using TouchableHighlight we will call onOpenPage from ArtistCell: class ArtistCell extendsReact.Component { render() { return ( <View> <TouchableHighlight onPress={this.props.onOpenPage} underlayColor='transparent'> <View style={styles.container}> <Image source={{uri: this.props.artist.image[2]["#text"]}} style={styles.artistImage} /> <View style={styles.rightContainer}> <Text style={styles.rank}>## {this.props.artist["@attr"].rank}</Text> <Text style={styles.name}>{this.props.artist.name}</Text> </View> </View> </TouchableHighlight> <View style={styles.separator}/> </View> ); } } Finally open web page from ArtistListScreen component: // declare new component var WebView = require('WebView'); class ArtistListScreen extendsReact.Component { // will be called on touch from ArtistCell openPage(url) { this.props.navigator.push({ title: 'Web View', component: WebView, passProps: {url} }); } renderRow(artist) { return ( <ArtistCell artist={artist} // specify artist's url on render onOpenPage={this.openPage.bind(this, artist.url)} /> ); } } Now a touch on any cell in ListView will load a web page for selected artist: Conclusion You can explore source code of the app on Github repo. For me it was a real fun to play with React Native. I found debugging in Chrome and error stack messages extremely easy to work with. By using React's component-based approach you can build complex UI without much effort. I highly recommend to explore this technology for rapid prototyping and maybe for your next awesome project. Useful links Building a flashcard app with React Native Examples of React Native apps React Native Videos Video course on React Native Want more JavaScript? Visit our dedicated page here. About the author Eugene Safronov is a software engineer with a proven record of delivering high quality software. He has an extensive experience building successful teams and adjusting development processes to the project’s needs. His primary focuses are Web (.NET, node.js stacks) and cross-platform mobile development (native and hybrid). He can be found on Twitter @sejoker.

This article by Adnan Jaswal, the author of the book, KnockoutJS by Example, will render a map of the application and allow the users to place markers on it. The users will also be able to get directions between two addresses, both as description and route on the map. (For more resources related to this topic, see here.) Placing marker on the map This feature is about placing markers on the map for the selected addresses. To implement this feature, we will: Update the address model to hold the marker Create a method to place a marker on the map Create a method to remove an existing marker Register subscribers to trigger the removal of the existing markers when an address changes Update the module to add a marker to the map Let's get started by updating the address model. Open the MapsApplication module and locate the AddressModel variable. Add an observable to this model to hold the marker like this: /* generic model for address */ var AddressModel = function() { this.marker = ko.observable(); this.location = ko.observable(); this.streetNumber = ko.observable(); this.streetName = ko.observable(); this.city = ko.observable(); this.state = ko.observable(); this.postCode = ko.observable(); this.country = ko.observable(); }; Next, we create a method that will create and place the marker on the map. This method should take location and address model as parameters. The method will also store the marker in the address model. Use the google.maps.Marker class to create and place the marker. Our implementation of this method looks similar to this: /* method to place a marker on the map */ var placeMarker = function (location, value) { // create and place marker on the map var marker = new google.maps.Marker({ position: location, map: map }); //store the newly created marker in the address model value().marker(marker); }; Now, create a method that checks for an existing marker in the address model and removes it from the map. Name this method removeMarker. It should look similar to this: /* method to remove old marker from the map */ var removeMarker = function(address) { if(address != null) { address.marker().setMap(null); } }; The next step is to register subscribers that will trigger when an address changes. We will use these subscribers to trigger the removal of the existing markers. We will use the beforeChange event of the subscribers so that we have access to the existing markers in the model. Add subscribers to the fromAddress and toAddress observables to trigger on the beforeChange event. Remove the existing markers on the trigger. To achieve this, I created a method called registerSubscribers. This method is called from the init method of the module. The method registers the two subscribers that triggers calls to removeMarker. Our implementation looks similar to this: /* method to register subscriber */ var registerSubscribers = function () { //fire before from address is changed mapsModel.fromAddress.subscribe(function(oldValue) { removeMarker(oldValue); }, null, "beforeChange"); //fire before to address is changed mapsModel.toAddress.subscribe(function(oldValue) { removeMarker(oldValue); }, null, "beforeChange"); }; We are now ready to bring the methods we created together and place a marker on the map. Create a map called updateAddress. This method should take two parameters: the place object and the value binding. The method should call populateAddress to extract and populate the address model, and placeMarker to place a new marker on the map. Our implementation looks similar to this: /* method to update the address model */ var updateAddress = function(place, value) { populateAddress(place, value); placeMarker(place.geometry.location, value); }; Call the updateAddress method from the event listener in the addressAutoComplete custom binding: google.maps.event.addListener(autocomplete, 'place_changed', function() { var place = autocomplete.getPlace(); console.log(place); updateAddress(place, value); }); Open the application in your browser. Select from and to addresses. You should now see markers appear for the two selected addresses. In our browser, the application looks similar to the following screenshot: Displaying a route between the markers The last feature of the application is to draw a route between the two address markers. To implement this feature, we will: Create and initialize the direction service Request routing information from the direction service and draw the route Update the view to add a button to get directions Let's get started by creating and initializing the direction service. We will use the google.maps.DirectionsService class to get the routing information and the google.maps.DirectionsRenderer to draw the route on the map. Create two attributes in the MapsApplication module: one for directions service and the other for directions renderer: /* the directions service */ var directionsService; /* the directions renderer */ var directionsRenderer; Next, create a method to create and initialize the preceding attributes: /* initialise the direction service and display */ var initDirectionService = function () { directionsService = new google.maps.DirectionsService(); directionsRenderer = new google.maps.DirectionsRenderer({suppressMarkers: true}); directionsRenderer.setMap(map); }; Call this method from the mapPanel custom binding handler after the map has been created and cantered. The updated mapPanel custom binding should look similar to this: /* custom binding handler for maps panel */ ko.bindingHandlers.mapPanel = { init: function(element, valueAccessor){ map = new google.maps.Map(element, { zoom: 10 }); centerMap(localLocation); initDirectionService(); } }; The next step is to create a method that will build and fire a request to the direction service to fetch the direction information. The direction information will then be used by the direction renderer to draw the route on the map. Our implementation of this method looks similar to this: /* method to get directions and display route */ var getDirections = function () { //create request for directions var routeRequest = { origin: mapsModel.fromAddress().location(), destination: mapsModel.toAddress().location(), travelMode: google.maps.TravelMode.DRIVING }; //fire request to route based on request directionsService.route(routeRequest, function(response, status) { if (status == google.maps.DirectionsStatus.OK) { directionsRenderer.setDirections(response); } else { console.log("No directions returned ..."); } }); }; We create a routing request in the first part of the method. The request object consists of origin, destination, and travelMode. The origin and destination values are set to the locations for from and to addresses. The travelMode is set to google.maps.TravelMode.DRIVING, which, as the name suggests, specifies that we require driving route. Add the getDirections method to the return statement of the module as we will bind it to a button in the view. One last step before we can work on the view is to clear the route on the map when the user selects a new address. This can be achieved by adding an instruction to clear the route information in the subscribers we registerd earlier. Update the subscribers in the registerSubscribers method to clear the routes on the map: /* method to register subscriber */ var registerSubscribers = function () { //fire before from address is changed mapsModel.fromAddress.subscribe(function(oldValue) { removeMarker(oldValue); directionsRenderer.set('directions', null); }, null, "beforeChange"); //fire before to address is changed mapsModel.toAddress.subscribe(function(oldValue) { removeMarker(oldValue); directionsRenderer.set('directions', null); }, null, "beforeChange"); }; The last step is to update the view. Open the view and add a button under the address input components. Add click binding to the button and bind it to the getDirections method of the module. Add enable binding to make the button clickable only after the user has selected the two addresses. The button should look similar to this: <button type="button" class="btn btn-default" data-bind="enable: MapsApplication.mapsModel.fromAddress && MapsApplication.mapsModel.toAddress, click: MapsApplication.getDirections"> Get Directions </button> Open the application in your browser and select the From address and To address option. The address details and markers should appear for the two selected addresses. Click on the Get Directions button. You should see the route drawn on the map between the two markers. In our browser, the application looks similar to the following screenshot: Summary In this article, we walked through placing markers on the map and displaying the route between the markers. Resources for Article: Further resources on this subject: KnockoutJS Templates[article] Components [article] Web Application Testing [article]

 In this article by Alex Antonov, author of the book Spring Boot Cookbook, we will cover the following topics: Understanding Spring Boot autoconfiguration Creating a custom Spring Boot autoconfiguration starter (For more resources related to this topic, see here.) Introduction Its time to take a look behind the scenes and find out the magic behind the Spring Boot autoconfiguration and write some starters of our own as well. This is a very useful capability to possess, especially for large software enterprises where the presence of a proprietary code is inevitable and it is very helpful to be able to create internal custom starters that would automatically add some of the configuration or functionalities to the applications. Some likely candidates can be custom configuration systems, libraries, and configurations that deal with connecting to databases, using custom connection pools, http clients, servers, and so on. We will go through the internals of Spring Boot autoconfiguration, take a look at how new starters are created, explore conditional initialization and wiring of beans based on various rules, and see that annotations can be a powerful tool, which provides the consumers of the starters more control over dictating what configurations should be used and where. Understanding Spring Boot autoconfiguration Spring Boot has a lot of power when it comes to bootstrapping an application and configuring it with exactly the things that are needed, all without much of the glue code that is required of us, the developers. The secret behind this power actually comes from Spring itself or rather from the Java Configuration functionality that it provides. As we add more starters as dependencies, more and more classes will appear in our classpath. Spring Boot detects the presence or absence of specific classes and based on this information, makes some decisions, which are fairly complicated at times, and automatically creates and wires the necessary beans to the application context. Sounds simple, right? How to do it… Conveniently, Spring Boot provides us with an ability to get the AUTO-CONFIGURATION REPORT by simply starting the application with the debug flag. This can be passed to the application either as an environment variable, DEBUG, as a system property, -Ddebug, or as an application property, --debug. Start the application by running DEBUG=true ./gradlew clean bootRun. Now, if you look at the console logs, you will see a lot more information printed there that is marked with the DEBUG level log. At the end of the startup log sequence, we will see the AUTO-CONFIGURATION REPORT as follows: ========================= AUTO-CONFIGURATION REPORT ========================= Positive matches: ----------------- … DataSourceAutoConfiguration - @ConditionalOnClass classes found: javax.sql.DataSource,org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType (OnClassCondition) … Negative matches: ----------------- … GsonAutoConfiguration - required @ConditionalOnClass classes not found: com.google.gson.Gson (OnClassCondition) … How it works… As you can see, the amount of information that is printed in the debug mode can be somewhat overwhelming; so I've selected only one example of positive and negative matches each. For each line of the report, Spring Boot tells us why certain configurations have been selected to be included, what they have been positively matched on, or, for the negative matches, what was missing that prevented a particular configuration to be included in the mix. Let's look at the positive match for DataSourceAutoConfiguration: The @ConditionalOnClass classes found tells us that Spring Boot has detected the presence of a particular class, specifically two classes in our case: javax.sql.DataSource and org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType. The OnClassCondition indicates the kind of matching that was used. This is supported by the @ConditionalOnClass and @ConditionalOnMissingClass annotations. While OnClassCondition is the most common kind of detection, Spring Boot also uses many other conditions. For example, OnBeanCondition is used to check the presence or absence of specific bean instances, OnPropertyCondition is used to check the presence, absence, or a specific value of a property as well as any number of the custom conditions that can be defined using the @Conditional annotation and Condition interface implementations. The negative matches show us a list of configurations that Spring Boot has evaluated, which means that they do exist in the classpath and were scanned by Spring Boot but didn't pass the conditions required for their inclusion. GsonAutoConfiguration, while available in the classpath as it is a part of the imported spring-boot-autoconfigure artifact, was not included because the required com.google.gson.Gson class was not detected as present in the classpath, thus failing the OnClassCondition. The implementation of the GsonAutoConfiguration file looks as follows: @Configuration @ConditionalOnClass(Gson.class) public class GsonAutoConfiguration { @Bean @ConditionalOnMissingBean public Gson gson() { return new Gson(); } } After looking at the code, it is very easy to make the connection between the conditional annotations and report information that is provided by Spring Boot at the start time. Creating a custom Spring Boot autoconfiguration starter We have a high-level idea of the process by which Spring Boot decides which configurations to include in the formation of the application context. Now, let's take a stab at creating our own Spring Boot starter artifact, which we can include as an autoconfigurable dependency in our build. Let's build a simple starter that will create another CommandLineRunner that will take the collection of all the Repository instances and print out the count of the total entries for each. We will start by adding a child Gradle project to our existing project that will house the codebase for the starter artifact. We will call it db-count-starter. How to do it… We will start by creating a new directory named db-count-starter in the root of our project. As our project has now become what is known as a multiproject build, we will need to create a settings.gradle configuration file in the root of our project with the following content: include 'db-count-starter' We should also create a separate build.gradle configuration file for our subproject in the db-count-starter directory in the root of our project with the following content: apply plugin: 'java' repositories { mavenCentral() maven { url "https://repo.spring.io/snapshot" } maven { url "https://repo.spring.io/milestone" } } dependencies { compile("org.springframework.boot:spring- boot:1.2.3.RELEASE") compile("org.springframework.data:spring-data- commons:1.9.2.RELEASE") } Now we are ready to start coding. So, the first thing is to create the directory structure, src/main/java/org/test/bookpubstarter/dbcount, in the db-count-starter directory in the root of our project. In the newly created directory, let's add our implementation of the CommandLineRunner file named DbCountRunner.java with the following content: public class DbCountRunner implements CommandLineRunner { protected final Log logger = LogFactory.getLog(getClass()); private Collection<CrudRepository> repositories; public DbCountRunner(Collection<CrudRepository> repositories) { this.repositories = repositories; } @Override public void run(String... args) throws Exception { repositories.forEach(crudRepository -> logger.info(String.format("%s has %s entries", getRepositoryName(crudRepository.getClass()), crudRepository.count()))); } private static String getRepositoryName(Class crudRepositoryClass) { for(Class repositoryInterface : crudRepositoryClass.getInterfaces()) { if (repositoryInterface.getName(). startsWith("org.test.bookpub.repository")) { return repositoryInterface.getSimpleName(); } } return "UnknownRepository"; } } With the actual implementation of DbCountRunner in place, we will now need to create the configuration object that will declaratively create an instance during the configuration phase. So, let's create a new class file called DbCountAutoConfiguration.java with the following content: @Configuration public class DbCountAutoConfiguration { @Bean public DbCountRunner dbCountRunner(Collection<CrudRepository> repositories) { return new DbCountRunner(repositories); } } We will also need to tell Spring Boot that our newly created JAR artifact contains the autoconfiguration classes. For this, we will need to create a resources/META-INF directory in the db-count-starter/src/main directory in the root of our project. In this newly created directory, we will place the file named spring.factories with the following content: org.springframework.boot.autoconfigure.EnableAutoConfiguration=org.test.bookpubstarter.dbcount.DbCountAutoConfiguration For the purpose of our demo, we will add the dependency to our starter artifact in the main project's build.gradle by adding the following entry in the dependencies section: compile project(':db-count-starter') Start the application by running ./gradlew clean bootRun. Once the application is complied and has started, we should see the following in the console logs: 2015-04-05 INFO org.test.bookpub.StartupRunner : Welcome to the Book Catalog System! 2015-04-05 INFO o.t.b.dbcount.DbCountRunner : AuthorRepository has 1 entries 2015-04-05 INFO o.t.b.dbcount.DbCountRunner : PublisherRepository has 1 entries 2015-04-05 INFO o.t.b.dbcount.DbCountRunner : BookRepository has 1 entries 2015-04-05 INFO o.t.b.dbcount.DbCountRunner : ReviewerRepository has 0 entries 2015-04-05 INFO org.test.bookpub.BookPubApplication : Started BookPubApplication in 8.528 seconds (JVM running for 9.002) 2015-04-05 INFO org.test.bookpub.StartupRunner           : Number of books: 1 How it works… Congratulations! You have now built your very own Spring Boot autoconfiguration starter. First, let's quickly walk through the changes that we made to our Gradle build configuration and then we will examine the starter setup in detail. As the Spring Boot starter is a separate, independent artifact, just adding more classes to our existing project source tree would not really demonstrate much. To make this separate artifact, we had a few choices: making a separate Gradle configuration in our existing project or creating a completely separate project altogether. The most ideal solution, however, was to just convert our build to Gradle Multi-Project Build by adding a nested project directory and subproject dependency to build.gradle of the root project. By doing this, Gradle actually creates a separate artifact JAR for us but we don't have to publish it anywhere, only include it as a compile project(':db-count-starter') dependency. For more information about Gradle multi-project builds, you can check out the manual at http://gradle.org/docs/current/userguide/multi_project_builds.html. Spring Boot Auto-Configuration Starter is nothing more than a regular Spring Java Configuration class annotated with the @Configuration annotation and the presence of spring.factories in the classpath in the META-INF directory with the appropriate configuration entries. During the application startup, Spring Boot uses SpringFactoriesLoader, which is a part of Spring Core, in order to get a list of the Spring Java Configurations that are configured for the org.springframework.boot.autoconfigure.EnableAutoConfiguration property key. Under the hood, this call collects all the spring.factories files located in the META-INF directory from all the jars or other entries in the classpath and builds a composite list to be added as application context configurations. In addition to the EnableAutoConfiguration key, we can declare the following automatically initializable other startup implementations in a similar fashion: org.springframework.context.ApplicationContextInitializer org.springframework.context.ApplicationListener org.springframework.boot.SpringApplicationRunListener org.springframework.boot.env.PropertySourceLoader org.springframework.boot.autoconfigure.template.TemplateAvailabilityProvider org.springframework.test.contex.TestExecutionListener Ironically enough, a Spring Boot Starter does not need to depend on the Spring Boot library as its compile time dependency. If we look at the list of class imports in the DbCountAutoConfiguration class, we will not see anything from the org.springframework.boot package. The only reason that we have a dependency declared on Spring Boot is because our implementation of DbCountRunner implements the org.springframework.boot.CommandLineRunner interface. Summary Resources for Article: Further resources on this subject: Welcome to the Spring Framework[article] Time Travelling with Spring[article] Aggregators, File exchange Over FTP/FTPS, Social Integration, and Enterprise Messaging[article]

 In this article by Mohsin Hijazee, the author of the book Mastering Google App Engine, we will go through learning, but unlearning something is even harder. The main reason why learning something is hard is not because it is hard in and of itself, but for the fact that most of the times, you have to unlearn a lot in order to learn a little. This is quite true for a datastore. Basically, it is built to scale the so-called Google scale. That's why, in order to be proficient with it, you will have to unlearn some of the things that you know. Your learning as a computer science student or a programmer has been deeply enriched by the relational model so much so that it is natural to you. Anything else may seem quite hard to grasp, and this is the reason why learning Google datastore is quite hard. However, if this were the only glitch in all that, things would have been way simpler because you could ask yourself to forget the relational world and consider the new paradigm afresh. Things have been complicated due to Google's own official documentation, where it presents a datastore in a manner where it seems closer to something such as Django's ORM, Rails ActiveRecord, or SQLAlchemy. However, all of a sudden, it starts to enlist its limitations with a very brief mention or, at times, no mention of why the limitations exist. Since you only know the limitations but not why the limitations are there in the first place, a lack of reason may result to you being unable to work around those limitations or mold your problem space into the new solution space, which is Google datastore. We will try to fix this. Hence, the following will be our goals in this article: To understand BigTable and its data model To have a look at the physical data storage in BigTable and the operations that are available in it To understand how BigTable scales To understand datastore and the way it models data on top of BigTable So, there's a lot more to learn. Let's get started on our journey of exploring datastore. The BigTable If you decided to fetch every web page hosted on the planet, download and store a copy of it, and later process every page to extract data from it, you'll find out that your own laptop or desktop is not good enough to accomplish this task. It has barely enough storage to store every page. Usually, laptops come with 1 TB hard disk drives, and this seems to be quite enough for a person who is not much into video content such as movies. Assuming that there are 2 billion websites, each with an average of 50 pages and each page weighing around 250 KB, it sums up to around 23,000+ TB (or roughly 22 petabytes), which would need 23,000 such laptops to store all the web pages with a 1 TB hard drive in each. Assuming the same statistics, if you are able to download at a whopping speed of 100 MBps, it would take you about seven years to download the whole content to one such gigantic hard drive if you had one in your laptop. Let's suppose that you downloaded the content in whatever time it took and stored it. Now, you need to analyze and process it too. If processing takes about 50 milliseconds per page, it would take about two months to process the entire data that you downloaded. The world would have changed a lot by then already, leaving your data and processed results obsolete. This is the Kind of scale for which BigTable is built. Every Google product that you see—Search Analytics, Finance, Gmail, Docs, Drive, and Google Maps—is built on top of BigTable. If you want to read more about BigTable, you can go through the academic paper from Google Research, which is available at http://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf. The data model Let's examine the data model of BigTable at a logical level. BigTable is basically a key-value store. So, everything that you store falls under a unique key, just like PHP' arrays, Ruby's hash, or Python's dict: # PHP $person['name'] = 'Mohsin'; # Ruby or Python person['name'] = 'Mohsin' However, this is a partial picture. We will learn the details gradually in a while. So, let's understand this step by step. A BigTable installation can have multiple tables, just like a MySQL database can have multiple tables. The difference here is that a MySQL installation might have multiple databases, which in turn might have multiple tables. However, in the case of BigTable, the first major storage unit is a table. Each table can have hundreds of columns, which can be divided into groups called column families. You can define column families at the time of creating a table. They cannot be altered later, but each column family might have hundreds of columns that you can define even after the creation of the table. The notation that is used to address a column and its column families is like job:title, where job is a column family and title is the column. So here, you have a job column family that stores all the information about the job of the user, and title is supposed to store the job title. However, one of the important facts about these columns is that there's no concept of datatypes in BigTable as you'd encounter in other relational database systems. Everything is just an uninterpreted sequence of bytes, which means nothing to BigTable. What they really mean is just up to you. It might be a very long integer, a string, or a JSON-encoded data. Now, let's turn our attention to the rows. There are two major characteristics of the rows that we are concerned about. First, each row has a key, which must be unique. The contents of the key again consist of an uninterpreted string of bytes that is up to 64 KB in length. A key can be anything that you want it to be. All that's required is that it must be unique within the table, and in case it is not, you will have to overwrite the contents of the row with the same content. Which key should you use for a row in your table? That's the question that requires some consideration. To answer this, you need to understand how the data is actually stored. Till then, you can assume that each key has to be a unique string of bytes within the scope of a table and should be up to 64 KB in length. Now that we know about tables, column families, columns, rows, and row keys, let's look at an example of BigTable that stores 'employees' information. Let's pretend that we are creating something similar to LinkedIn here. So, here's the table: Personal Professional Key(name) personal:lastname personal:age professinal:company professional:designation Mohsin Hijazee 29 Sony Senior Designer Peter Smith 34 Panasonic General Manager Kim Yong 32 Sony Director Ricky Martin 45 Panasonic CTO Paul Jefferson 39 LG Sales Head So, 'this is a sample BigTable. The first column is the name, and we have chosen it as a key. It is of course not a good key, because the first name cannot necessarily be unique, even in small groups, let alone in millions of records. However, for the sake of this example, we will assume that the name is unique. Another reason behind assuming the name's uniqueness is that we want to increase our understanding gradually. So, the key point here is that we picked the first name as the row's key for now, but we will improve on this as we learn more. Next, we have two column groups. The personal column group holds all the personal attributes of the employees, and the other column family named professional has all the other attributes pertaining to the professional aspects. When referring to a column within a family, the notation is family:column. So, personal:age contains the age of the employees. If you look at professinal:designation and personal:age, it seems that the first one's contents are strings, while the second one stores integers. That's false. No column stores anything but just plain bytes without any distinction of what they mean. The meaning and interpretation of these bytes is up to the user of the data. From the point of view of BigTable', each column just contains plain old bytes. Another thing that is drastically different from RDBMS is such as MySQL is that each row need not have the same number of columns. Each row can adopt the layout that they want. So, the second row's personal column family can have two more columns that store gender and nationality. For this particular example, the data is in no particular order, and I wrote it down as it came to my mind. Hence, there's no order of any sort in the data at all. To summarize, BigTable is a key-value storage where keys should be unique and have a length that is less than or equal to 64 KB. The columns are divided into column families, which can be created at the time of defining the table, but each column family might have hundreds of columns created as and when needed. Also, contents have no data type and comprise just plain old bytes. There's one minor detail left, which is not important as regards our purpose. However, for the sake of the completeness of the BigTable's data model, I will mention it now. Each value of the column is stored with a timestamp that is accurate to the microseconds, and in this way, multiple versions of a column value are available. The number of last versions that should be kept is something that is configurable at the table level, but since we are not going to deal with BigTable directly, this detail is not important to us. How data is stored? Now that we know about row keys, column families, and columns, we will gradually move towards examining this data model in detail and understand how the data is actually stored. We will examine the logical storage and then dive into the actual structure, as it ends up on the disk. The data that we presented in the earlier table had no order and were listed as they came to my mind. However, while storing, the data is always sorted by the row key. So now, the data will actually be stored like this: personal professional Key(name) personal:lastname personal:age professinal:company professional:designation Kim Yong 32 Sony Director Mohsin Hijazee 29 Sony Senior Designer Paul Jefferson 39 LG Sales Head Peter Smith 34 Panasonic General Manager Ricky Martin 45 Panasonic CTO OK, so what happened here? The name column indicates the key of the table and now, the whole table is sorted by the key. That's exactly how it is stored on the disk as well. 'An important thing about sorting is lexicographic sorting and not semantic sorting. By lexicographic, we mean that they are sorted by the byte value and not by the textness or the semantic sort. This matters because even within the Latin character set, different languages have different sort orders for letters, such as letters in English versus German and French. However, all of this and the Unicode collation order isn't valid here. It is just sorted by byte values. In our instance, since K has a smaller byte value (because K has a lower ASCII/Unicode value) than letter M, it comes first. Now, suppose that some European language considers and sorts M before K. That's not how the data would be laid out here, because it is a plain, blind, and simple sort. The data is sorted by the byte value, with no regard for the semantic value. In fact, for BigTable, this is not even text. It's just a plain string of bytes. Just a hint. This order of keys is something that we will exploit when modeling data. How? We'll see later. The Physical storage Now that we understand the logical data model and how it is organized, it's time to take a closer look at how this data is actually stored on the disk. On a physical disk, the stored data is sorted by the key. So, key 1 is followed by its respective value, key 2 is followed by its respective value, and so on. At the end of the file, there's a sorted list of just the keys and their offset in the file from the start, which is something like the block to the right: Ignore the block on your left that is labeled Index. We will come back to it in a while. This particular format actually has a name SSTable (String Storage Table) because it has strings (the keys), and they are sorted. It is of course tabular data, and hence the name. Whenever your data is sorted, you have certain advantages, with the first and foremost advantage being that when you look up for an item or a range of items, 'your dataset is sorted. We will discuss this in detail later in this article. Now, if we start from the beginning of the file and read sequentially, noting down every key and then its offset in a format such as key:offset, we effectively create an index of the whole file in a single scan. That's where the first block to your left in the preceding diagram comes from. Since the keys are sorted in the file, we simply read it sequentially till the end of the file, hence effectively creating an index of the data. Furthermore, since this index only contains keys and their offsets in the file, it is much smaller in terms of the space it occupies. Now, assuming that SSTable has a table that is, say, 500 MB in size, we only need to load the index from the end of the file into the memory, and whenever we are asked for a key or a range of keys, we just search within a memory index (thus not touching the disk at all). If we find the data, only then do we seek the disk at the given offset because we know the offset of that particular key from the index that we loaded in the memory. Some limitations Pretty smart, neat, and elegant, you would say! Yes it is. However, there's a catch. If you want to create a new row, key must come in a sorted order, and even if you are sure about where exactly this key should be placed in the file to avoid the need to sort the data, you still need to rewrite the whole file in a new, sorted order along with the index. Hence, large amounts of I/O are required for just a single row insertion. The same goes for deleting a row because now, the file should be sorted and rewritten again. Updates are OK as long as the key itself is not altered because, in that case, it is sort of having a new key altogether. This is so because a modified key would have a different place in the sorted order, depending on what the key actually is. Hence, the whole file would be rewritten. Just for an example, say you have a row with the key as all-boys, and then you change the key of that row to x-rays-of-zebra. Now, you will see that after the new modification, the row will end up at nearly the end of the file, whereas previously, it was probably at the beginning of the file because all-boys comes before x-rays-of-zebra when sorted. This seems pretty limiting, and it looks like inserting or removing a key is quite expensive. However, this is not the case, as we will see later. Random writes and deletion There's one last thing that's worth a mention before we examine the operations that are available on a BigTable. We'd like to examine how random writes and the deletion of rows are handled because that seems quite expensive, as we just examined in the preceding section. The idea is very simple. All the read, writes, and removals don't go straight to the disk. Instead, an in-memory SSTable is created along with its index, both of which are empty when created. We'll call it MemTable from this point onwards for the sake of simplicity. Every read checks the index of this table, and if a record is found from here, it's well and good. If it is not, then the index of the SSTable on the disk is checked and the desired row is returned. When a new row has to be read, we don't look at anything and simply enter the row in the MemTable along with its record in the index of this MemTable. To delete a key, we simply mark it deleted in the memory, regardless of whether it is in MemTable or in the on disk table. As shown here the allocation of block into Mem Table: Now, when the size of the MemTable grows up to a certain size, it is written to the disk as a new SSTable. Since this only depends on the size of the MemTable and of course happens much infrequently, it is much faster. Each time the MemTable grows beyond a configured size, it is flushed to the disk as a new SSTable. However, the index of each flushed SSTable is still kept in the memory so that we can quickly check the incoming read requests and locate it in any table without touching the disk. Finally, when the number of SSTables reaches a certain count, the SSTables are merged and collapsed into a single SSTable. Since each SSTable is just a sorted set of keys, a merge sort is applied. This merging process is quite fast. Congratulations! You've just learned the most atomic storage unit in BigData solutions such as BigTable, Hbase, Hypertable, Cassandara, and LevelDB. That's how they actually store and process the data. Now that we know how a big table is actually stored on the disk and how the read and writes are handled, it's time to take a closer look at the available operations. Operations on BigTable Until this point, we know that a BigTable table is a collection of rows that have unique keys up to 64 KB in length and the data is stored according to the lexicographic sort order of the keys. We also examined how it is laid out on the disk and how read, writes, and removals are handled. Now, the question is, which operations are available on this data? The following are the operations that are available to us: Fetching a row by using its key Inserting a new key Deleting a row Updating a row Reading a range of rows from the starting row key to the ending row key Reading Now, the first operation is pretty simple. You have a key, and you want the associated row. Since the whole data set is sorted by the key, all we need to do is perform a binary search on it, and you'll be able to locate your desired row within a few lookups, even within a set of a million rows. In practice, the index at the end of the SSTable is loaded in the memory, and the binary search is actually performed on it. If we take a closer look at this operation in light of what we know from the previous section, the index is already in the memory of the MemTable that we saw in the previous section. In case there are multiple SSTables because MemTable was flushed many times to the disk as it grew too large, all the indexes of all the SSTables are present in the memory, and a quick binary search is performed on them. Writing The second operation that is available to us is the ability to insert a new row. So, we have a key and the values that we want to insert in the table. According to our new knowledge about physical storage and SSTables, we can understand this very well. The write directly happens on the in-memory MemTable and its index is updated, which is also in the memory. Since no disk access is required to write the row as we are writing in memory, the whole file doesn't have to be rewritten on disk, because yet again, all of it is in the memory. This operation is very fast and almost instantaneous. However, if the MemTable grows in size, it will be flushed to the disk as a new SSTable along with the index while retaining a copy of its index in the memory. Finally, we also saw that when the number of SSTables reaches a certain number, they are merged and collapsed to form a new, bigger table. Deleting It seems that since all the keys are in a sorted order on the disk and deleting a key would mean disrupting the sort order, a rewrite of the whole file would be a big I/O overhead. However, it is not, as it can be handled smartly. Since all the indexes, including the MemTable and the tables that were the result of flushing a larger MemTable to the disk, are already in the memory, deleting a row only requires us to find the required key in the in-memory indexes and mark it as deleted. Now, whenever someone tries to read the row, the in-memory indexes will be checked, and although an entry will be there, it will be marked as deleted and won't be returned. When MemTable is being flushed to the disk or multiple tables are being collapsed, this key and the associated row will be excluded in the write process. Hence, they are totally gone from the storage. Updating Updating a row is no different, but it has two cases. The first case is in which not only the values, but also the key is modified. In this case, it is like removing the row with an old key and inserting a row with a new key. We already have seen both of these cases in detail. So, the operation should be obvious. However, the case where only the values are modified is even simpler. We only have to locate the row from the indexes, load it in the memory if it is not already there, and modify. That's all. Scanning a range This last operation is quite interesting. You can scan a range of keys from a starting key to an ending key. For instance, you can return all the rows that have a key greater than or equal to key1 and less than or equal to key2, effectively forming a range. Since the looking up of a single key is a fast operation, we only have to locate the first key of the range. Then, we start reading the consecutive keys one after the other till we encounter a key that is greater than key2, at which point, we will stop the scanning, and the keys that we scanned so far are our query's result. This is how it looks like: Name Department Company Chris Harris Research & Development Google Christopher Graham Research & Development LG Debra Lee Accounting Sony Ernest Morrison Accounting Apple Fred Black Research & Development Sony Janice Young Research & Development Google Jennifer Sims Research & Development Panasonic Joyce Garrett Human Resources Apple Joyce Robinson Research & Development Apple Judy Bishop Human Resources Google Kathryn Crawford Human Resources Google Kelly Bailey Research & Development LG Lori Tucker Human Resources Sony Nancy Campbell Accounting Sony Nicole Martinez Research & Development LG Norma Miller Human Resources Sony Patrick Ward Research & Development Sony Paula Harvey Research & Development LG Stephanie Chavez Accounting Sony Stephanie Mccoy Human Resources Panasonic In the preceding table, we said that the starting key will be greater than or equal to Ernest and ending key will be less than or equal to Kathryn. So, we locate the first key that is greater than or equal to Ernest, which happens to be Ernest Morrison. Then, we start scanning further, picking and returning each key as long as it is less than or equal to Kathryn. When we reach Judy, it is less than or equal to Kathryn, but Kathryn isn't. So, this row is not returned. However, the rows before this are returned. This is the last operation that is available to us on BigTable. Selecting a key Now that we have examined the data model and the storage layout, we are in a better position to talk about the key selection for a table. As we know that the stored data is sorted by the key, it does not impact the writing, deleting, and updating to fetch a single row. However, the operation that is impacted by the key is that of scanning a range. Let's think about the previous table again and assume that this table is a part of some system that processes payrolls for companies, and the companies pay us for the task of processing their payroll. Now, let's suppose that Sony asks us to process their data and generate a payroll for them. Right now, we cannot do anything of this kind. We can just make our program scan the whole table, and hence all the records (which might be in millions), and only pick the records where job:company has the value of Sony. This would be inefficient. Instead, what we can do is put this sorted nature of row keys to our service. Select the company name as the key and concatenate the designation and name along with it. So, the new table will look like this: Key Name Department Company Apple-Accounting-Ernest Morrison Ernest Morrison Accounting Apple Apple-Human Resources-Joyce Garrett Joyce Garrett Human Resources Apple Apple-Research & Development-Joyce Robinson Joyce Robinson Research & Development Apple Google-Human Resources-Judy Bishop Chris Harris Research & Development Google Google-Human Resources-Kathryn Crawford Janice Young Research & Development Google Google-Research & Development-Chris Harris Judy Bishop Human Resources Google Google-Research & Development-Janice Young Kathryn Crawford Human Resources Google LG-Research & Development-Christopher Graham Christopher Graham Research & Development LG LG-Research & Development-Kelly Bailey Kelly Bailey Research & Development LG LG-Research & Development-Nicole Martinez Nicole Martinez Research & Development LG LG-Research & Development-Paula Harvey Paula Harvey Research & Development LG Panasonic-Human Resources-Stephanie Mccoy Jennifer Sims Research & Development Panasonic Panasonic-Research & Development-Jennifer Sims Stephanie Mccoy Human Resources Panasonic Sony-Accounting-Debra Lee Debra Lee Accounting Sony Sony-Accounting-Nancy Campbell Fred Black Research & Development Sony Sony-Accounting-Stephanie Chavez Lori Tucker Human Resources Sony Sony-Human Resources-Lori Tucker Nancy Campbell Accounting Sony Sony-Human Resources-Norma Miller Norma Miller Human Resources Sony Sony-Research & Development-Fred Black Patrick Ward Research & Development Sony Sony-Research & Development-Patrick Ward Stephanie Chavez Accounting Sony So, this is a new format. We just welded the company, department, and name as the key and as the table will always be sorted by the key, that's what it looks like, as shown in the preceding table. Now, suppose that we receive a request from Google to process their data. All we have to do is perform a scan, starting from the key greater than or equal to Google and less then L because that's the next letter. This scan is highlighted in the previous table. Now, the next request is more specific. Sony asks us to process their data, but only for their accounting department. How do we do that? Quite simple! In this case, our starting key will be greater than or equal to Sony-Accounting, and the ending key can be Sony-Accountinga, where a is appended to indicate the end key in the range. The scanned range and the returned rows are highlighted in the previous table. BigTable – a hands-on approach Okay, enough of the theory. It is now time to take a break and perform some hands-on experimentation. By now, we know that about 80 percent of the BigTable and the other 20 percent of the complexity is scaling it to more than one machine. Our current discussion only assumed and focused on a single machine environment, and we assumed that the BigTable table is on our laptop and that's about it. You might really want to experiment with what you learned. Fortunately, given that you have the latest version of Google Chrome or Mozilla Firefox, that's easy. You have BigTable right there! How? Let me explain. Basically, from the ideas that we looked at pertaining to the stored key value, the sorted layout, the indexes of the sorted files, and all the operations that were performed on them, including scanning, we extracted a separate component called LevelDB. Meanwhile, as HTML was evolving towards HTML5, a need was felt to store data locally. Initially, SQLite3 was embedded in browsers, and there was a querying interface for you to play with. So all in all, you had an SQL database in the browser, which yielded a lot of possibilities. However, in recent years, W3C deprecated this specification and urged browser vendors to not implement it. Instead of web databases that were based on SQLite3, they now have databases based on LevelDB that are actually key-value stores, where storage is always sorted by key. Hence, besides looking up for a key, you can scan across a range of keys. Covering the IndexedDB API here would be beyond the scope of this book, but if you want to understand it and find out what the theory that we talked about looks like in practice, you can try using IndexedDB in your browser by visiting http://code.tutsplus.com/tutorials/working-with-indexeddb--net-34673. The concepts of keys and the scanning of key ranges are exactly like those that we examined here as regards BigTable, and those about indexes are mainly from the concepts that we will examine in a later section about datastores. Scaling BigTable to BigData By now, you have probably understood the data model of BigTable, how it is laid out on the disk, and the advantages it offers. To recap once again, the BigTable installation may have many tables, each table may have many column families that are defined at the time of creating the table, and each column family may have many columns, as required. Rows are identified by keys, which have a maximum length of 64 KB, and the stored data is sorted by the key. We can receive, update, and delete a single row. We can also scan a range of rows from a starting key to an ending key. So now, the question comes, how does this scale? We will provide a very high-level overview, neglecting the micro details to keep things simple and build a mental model that is useful to us as the consumers of BigTable, as we're not supposed to clone BigTable's implementation after all. As we saw earlier, the basic storage unit in BigTable is a file format called SSTable that stores key-value pairs, which are sorted by the key, and has an index at its end. We also examined how the read, write, and delete work on an in-memory copy of the table and merged periodically with the table that is present on the disk. Lastly, we also mentioned that when the in memory is flushed as SSTables on the disk when reach a certain configurable count, they are merged into a bigger table. The view so far presents the data model, its physical layout, and how operations work on it in cases where the data resides on a single machine, such as a situation where your laptop has a telephone directory of the entire Europe. However, how does that work at larger scales? Neglecting the minor implementation details and complexities that arise in distributed systems, the overall architecture and working principles are simple. In case of a single machine, there's only one SSTable (or a few in case they are not merged into one) file that has to be taken care of, and all the operations have to be performed on it. However, in case this file does not fit on a single machine, we will of course have to add another machine, and half of the SSTable will reside on one machine, while the other half will be on the another machine. This split would of course mean that each machine would have a range of keys. For instance, if we have 1 million keys (that look like key1, key2, key3, and so on), then the keys from key1 to key500000 might be on one machine, while the keys from key500001 to key1000000 will be on the second machine. So, we can say that each machine has a different key range for the same table. Now, although the data resides on two different machines, it is of course a single table that sprawls over two machines. These partitions or separate parts are called tablets. Let's see the Key allocation on two machines: We will keep this system to only two machines and 1 million rows for the sake of discussion, but there may be cases where there are about 20 billion keys sprawling over some 12,000 machines, with each machine having a different range of keys. However, let's continue with this small cluster consisting of only two nodes. Now, the problem is that as an external user who has no knowledge of which machine has which portion of the SSTable (and eventually, the key ranges on each machine), how can a key, say, key489087 be located? For this, we will have to add something like a telephone directory, where I look up the table name and my desired key and I get to know the machine that I should contact to get the data associated with the key. So, we are going to add another node, which will be called the master. This master will again contain simple, plain SSTable, which is familiar to us. However, the key-value pair would be a very interesting one. Since this table would contain data about the other BigTable tables, let's call it the METADATA table. In the METADATA table, we will adopt the following format for the keys: tablename_ending-row-key Since we have only two machines and each machine has two tablets, the METADATA table will look like this: Key Value employees_key500000 192.168.0.2 employees_key1000000 192.168.0.3 The master stores the location of each tablet server with the row key that is the encoding of the table name and the ending row of the tablet. So, the tablet has to be scanned. The master assigns tablets to different machines when required. Each tablet is about 100 MB to 200 MB in size. So, if we want to fetch a key, all we need to know is the following: Location of the master server Table in which we are looking for the key The key itself Now, we will concatenate the table name with the key and perform a scan on the METADATA table on the master node. Let's suppose that we are looking for key600000 in employees table. So, we would first be actually looking for the employees_key600000 key in the table on master machine. As you are familiar with the scan operation on SSTable (and METADATA is just an SSTable), we are looking for a key that is greater than or equal to employees_key600000, which happens to be employees_key1000000. From this lookup, the key that we get is employees_key1000000 against which, IP address 192.168.0.3 is listed. This means that this is the machine that we should connect to fetch our data. We used the word keys and not the key because it is a range scan operation. This will be clearer with another example. Let's suppose that we want to process rows with keys starting from key400000 to key800000. Now, if you look at the distribution of data across the machine, you'll know that half of the required range is on one machine, while the other half is on the other. Now, in this case, when we consult the METADATA table, two rows will be returned to us because key400000 is less then key500000 (which is the ending row key for data on the first machine) and key800000 is less then key1000000, which is the ending row for the data on the second machine. So, with these two rows returned, we have two locations to fetch our data from. This leads to an interesting side-effect. As the data resides on two different machines, this can be read or processed in parallel, which leads to an improved system performance. This is one reason why even with larger datasets, the performance of BigTable won't deteriorate as badly as it would have if it were a single, large machine with all the data on it. The datastore thyself So until now, everything that we talked about was about BigTable, and we did not mention datastore at all. Now is the time to look at datastore in detail because we understand BigTable quite well now. Datastore is an effectively solution that was built on top of BigTable as a persistent NoSQL layer for Google App Engine. As we know that BigTable might have different tables, data for all the applications is stored in six separate tables, where each table stores a different aspect or information about the data. Don't worry about memorizing things about data modeling and how to use it for now, as this is something that we are going to look into in greater detail later. The fundamental unit of storage in datastore is called a property. You can think of a property as a column. So, a property has a name and type. You can group multiple properties into a Kind, which effectively is a Python class and analogous to a table in the RDBMS world. Here's a pseudo code sample: # 1. Define our Kind and how it looks like. class Person(object): name = StringProperty() age = IntegerProperty() # 2. Create an entity of kind person ali = Person(name='Ali', age='24) bob = Person(name='Bob', age='34) david = Person(name='David', age='44) zain = Person(name='Zain', age='54) # 3. Save it ali.put() bob.put() david.put() zain.put() This looks a lot like an ORM such as Django's ORM, SQLAlchemy, or Rails ActiveRecord. So, Person class is called a Kind in App Engine's terminology. The StringProperty and IntegerProperty property classes are used to indicate the type of the data that is supposed to be stored. We created an instance of the Person class as mohsin. This instance is called an entity in App Engine's terminology. Each entity, when stored, has a key that is not only unique throughout your application, but also combined with your application ID. It becomes unique throughout all the applications that are hosted over Google App Engine. All entities of all kinds for all apps are stored in a single BigTable, and they are stored in a way where all the property values are serialized and stored in a single BigTable column. Hence, no separate columns are defined for each property. This is interesting and required as well because if we are Google App Engine's architects, we do not know the Kind of data that people are going to store or the number and types of properties that they would define so that it makes sense to serialize the whole thing as one and store them in one column. So, this is how it looks like: Key Kind Data agtkZXZ-bWdhZS0wMXIQTXIGUGVyc29uIgNBbGkM Person {name: 'Ali', age: 24} agtkZXZ-bWdhZS0wMXIPCxNTVVyc29uIgNBbGkM Person {name: 'Bob', age: 34} agtkZXZ-bWdhZS0wMXIPCxIGUGVyc29uIgNBbBQM Person {name: 'David', age: 44} agtkZXZ-bWdhZS0wMXIPCxIGUGVyc29uIRJ3bGkM Person {name: 'Zain', age: 54} The key appears to be random, but it is not. A key is formed by concatenating your application ID, your Kind name (Person here), and either a unique identifier that is auto generated by Google App Engine, or a string that is supplied by you. The key seems cryptic, but it is not safe to pass it around in public, as someone might decode it and take advantage of it. Basically, it is just base 64 encoded and can easily be decoded to know the entity's Kind name and ID. A better way would be to encrypt it using a secret key and then pass it around in public. On the other hand, to receive it, you will have to decrypt it using the same key. A gist of this is available on GitHub that can serve the purpose. To view this, visit https://gist.github.com/mohsinhijazee/07cdfc2826a565b50a68. However, for it to work, you need to edit your app.yaml file so that it includes the following: libraries: - name: pycrypto version: latest Then, you can call the encrypt() method on the key while passing around and decrypt it back using the decrypt() method, as follows: person = Person(name='peter', age=10) key = person.put() url_safe_key = key.urlsafe() safe_to_pass_around = encrypt(SECRET_KEY, url_safe_key) Now, when you have a key from the outside, you should first decrypt it and then use it, as follows: key_from_outside = request.params.get('key') url_safe_key = decrypt(SECRET_KEY, key_from_outside) key = ndb.Key(urlsafe=url_safe_key) person = key.get() The key object is now good for use. To summarize, just get the URL safe key by calling the ndb.Key.urlsafe() method and encrypt it so that it can be passed around. On return, just do the reverse. If you really want to see how the encrypt and decrypt operations are implemented, they are reproduced as follows without any documentation/comments, as cryptography is not our main subject: import os import base64 from Crypto.Cipher import AES BLOCK_SIZE = 32 PADDING='#' def _pad(data, pad_with=PADDING): return data + (BLOCK_SIZE - len(data) % BLOCK_SIZE) * PADDING def encrypt(secret_key, data): cipher = AES.new(_pad(secret_key, '@')[:32]) return base64.b64encode(cipher.encrypt(_pad(data))) def decrypt(secret_key, encrypted_data): cipher = AES.new(_pad(secret_key, '@')[:32]) return cipher.decrypt(base64.b64decode (encrypted_data)).rstrip(PADDING) KEY='your-key-super-duper-secret-key-here-only-first-32-characters-are-used' decrypted = encrypt(KEY, 'Hello, world!') print decrypted print decrypt(KEY, decrypted) More explanation on how this works is given at https://gist.github.com/mohsinhijazee/07cdfc2826a565b50a68. Now, let's come back to our main subject, datastore. As you can see, all the data is stored in a single column, and if we want to query something, for instance, people who are older than 25, we have no way to do this. So, how will this work? Let's examine this next. Supporting queries Now, what if we want to get information pertaining to all the people who are older than, say, 30? In the current scheme of things, this does not seem to be something that is doable, because the data is serialized and dumped, as shown in the previous table. Datastore solves this problem by putting the sorted values to be queried upon as keys. So here, we want to query by age. Datastore will create a record in another table called the Index table. This index table is nothing but just a plain BigTable, where the row keys are actually the property value that you want to query. Hence, a scan and a quick lookup is possible. Here's how it would look like: Key Entity key Myapp-person-age-24 agtkZXZ-bWdhZS0wMXIQTXIGUGVyc29uIgNBbGkM Myapp-person-age-34 agtkZXZ-bWdhZS0wMXIPCxNTVVyc29uIgNBbGkM Myapp-person-age-44 agtkZXZ-bWdhZS0wMXIPCxIGUGVyc29uIgNBbBQM Myapp-person-age-54 agtkZXZ-bWdhZS0wMXIPCxIGUGVyc29uIRJ3bGkM Implementation details So, all in all, Datastore actually builds a NoSQL solution on top of BigTable by using the following six tables: A table to store entities A table to store entities by kind A table to store indexes for the property values in the ascending order A table to store indexes for the property values in the descending order A table to store indexes for multiple properties together A table to keep a track of the next unique ID for Kind Let us look at each table in turn. The first table is used to store entities for all the applications. We have examined this in an example. The second table just stores the Kind names. Nothing fancy here. It's just some metadata that datastore maintains for itself. Think of this—you want to get all the entities that are of the Person Kind. How will you do this? If you look at the entities table alone and the operations that are available to us on a BigTable table, you will know that there's no such way for us to fetch all the entities of a certain Kind. This table does exactly this. It looks like this: Key Entity key Myapp-Person-agtkZXZ-bWdhZS0wMXIQTXIGUGVyc29uIgNBbGkM AgtkZXZ-bWdhZS0wMXIQTXIGUGVyc29uIgNBbGkM Myapp-Person-agtkZXZ-bWdhZS0wMXIQTXIGUGVyc29uIgNBb854 agtkZXZ-bWdhZS0wMXIQTXIGUGVyc29uIgNBb854 Myapp-Person-agtkZXZ-bWdhZS0wMXIQTXIGUGVy748IgNBbGkM agtkZXZ-agtkZXZ-bWdhZS0wMXIQTXIGUGVy748IgNBbGkM So, as you can see, this is just a simple BigTable table where the keys are of the [app ID]-[Kind name]-[entity key] pattern. The tables 3, 4, and 5 from the six tables that were mentioned in the preceding list are similar to the table that we examined in the Supporting queries section labeled Data as stored in BigTable. This leaves us with the last table. As you know that while storing entities, it is important to have a unique key for each row. Since all the entities from all the apps are stored in a single table, they should be unique across the whole table. When datastore generates a key for an entity that has to be stored, it combines your application ID and the Kind name of the entity. Now, this much part of the key only makes it unique across all the other entities in the table, but not within the set of your own entities. To do this, you need a number that should be appended to this. This is exactly similar to how AUTO INCREMENT works in the RDBMS world, where the value of a column is automatically incremented to ensure that it is unique. So, that's exactly what the last table is for. It keeps a track of the last ID that was used by each Kind of each application, and it looks like this: Key Next ID Myapp-Person 65 So, in this table, the key is of the [application ID]-[Kind name] format, and the value is the next value, which is 65 in this particular case. When a new entity of kind Person is created, it will be assigned 65 as the ID, and the row will have a new value of 66. Our application has only one Kind defined, which is Person. Therefore, there's only one row in this table because we are only keeping track for the next ID for this Kind. If we had another Kind, say, Group, it will have its own row in this table. Summary We started this article with the problem of storing huge amounts of data, processing it in bulk, and randomly accessing it. This arose from the fact that we were ambitious to store every single web page on earth and process it to extract some results from it. We introduced a solution called BigTable and examined its data model. We saw that in BigTable, we can define multiple tables, with each table having multiple column families, which are defined at the time of creating the table. We learned that column families are logical groupings of columns, and new columns can be defined in a column family, as needed. We also learned that the data store in BigTable has no meaning on its own, and it stores them just as plain bytes; its interpretation and meanings depend on the user of data. We also learned that each row in BigTable has a unique row key, which has a length of 64 KB. Lastly, we turned our attention to datastore, a NoSQL storage solution built on top of BigTable for Google App Engine. We briefly mentioned some datastore terminology such as properties (columns), entities (rows), and kinds (tables). We learned that all data is stored across six different BigTable tables. This captured a different aspect of data. Most importantly, we learned that all the entities of all the apps hosted on Google App Engine are stored in a single BigTable and all properties go to a single BigTable column. We also learned how querying is supported by additional tables that are keyed by the property values that list the corresponding keys. This concludes our discussion on Google App Engine's datastore and its underlying technology, workings, and related concepts. Next, we will learn how to model our data on top of datastore. What we learned in this chapter will help us enormously in understanding how to better model our data to take full advantage of the underlying mechanisms. Resources for Article: Further resources on this subject: Google Guice[article] The EventBus Class[article] Integrating Google Play Services [article]

 In this article by Wojciech Bancer, author of the book, Symfony2 Essentials, we will learn the basics of Symfony, its installation, configuration, and use. The Symfony framework is currently one of the most popular PHP frameworks existing within the PHP developer's environment. Version 2, which was released a few years ago, has been a great improvement, and in my opinion was one of the key elements for making the PHP ecosystem suitable for larger enterprise projects. The framework version 2.0 not only required the modern PHP version (minimal version required for Symfony is PHP 5.3.8), but also uses state-of-the-art technology — namespaces and anonymous functions. Authors also put a lot of efforts to provide long term support and to minimize changes, which break the compatibility between versions. Also, Symfony forced developers to use a few useful design concepts. The key one, introduced in Symfony, was DependencyInjection. (For more resources related to this topic, see here.) In most cases, the article will refer to the framework as Symfony2. If you want to look over the Internet or Google about this framework, apart from using Symfony keyword you may also try to use the Symfony2 keyword. This was the way recommended some time ago by one of the creators to make searching or referencing to the specific framework version easier in future. Key reasons to choose Symfony2 Symfony2 is recognized in the PHP ecosystem as a very well-written and well-maintained framework. Design patterns that are recommended and forced within the framework allow work to be more efficient in the group, this allows better tests and the creation of reusable code. Symfony's knowledge can also be verified through a certificate system, and this allows its developers to be easily found and be more recognized on the market. Last but not least, the Symfony2 components are used as parts of other projects, for example, look at the following: Drupal phpBB Laravel eZ Publish and more Over time, there is a good chance that you will find the parts of the Symfony2 components within other open source solutions. Bundles and extendable architecture are also some of the key Symfony2 features. They not only allow you to make your work easier through the easy development of reusable code, but also allows you to find smaller or larger pieces of code that you can embed and use within your project to speed up and make your work faster. The standards of Symfony2 also make it easier to catch errors and to write high-quality code; its community is growing every year. The history of Symfony There are many Symfony versions around, and it's good to know the differences between them to learn how the framework was evolving during these years. The first stable Symfony version — 1.0 — was released in the beginning of 2007 and was supported for three years. In mid-2008, version 1.1 was presented, which wasn't compatible with the previous release, and it was difficult to upgrade any old project to this. Symfony 1.2 version was released shortly after this, at the end of 2008. Migrating between these versions was much easier, and there were no dramatic changes in the structure. The final versions of Symfony 1's legacy family was released nearly one year later. Simultaneously, there were two version releases, 1.3 and 1.4. Both were identical, but Symfony 1.4 did not have deprecated features, and it was recommended to start new projects with it. Version 1.4 had 3 years of support. If you look into the code, version 1.x was very different from version 2. The company that was behind Symfony (the French company, SensioLabs) made a bold move and decided to rewrite the whole framework from scratch. The first release of Symfony2 wasn't perfect, but it was very promising. It relied on Git submodules (the composer did not exist back then). The 2.1 and 2.2 versions were closer to the one we use now, although it required a lot of effort to migrate to the upper level. Finally, the Symfony 2.3 was released — the first long-term support version within the 2.x branch. After this version, the changes provided within the next major versions (2.4, 2.5, and 2.6) are not so drastic and usually they do not break compatibility. This article was written based on the latest stable Symfony 2.7.4 version and was tested with PHP 5.5). This Symfony version is marked as the so called long-term support version, and updates for it will be released for 3 years since the first 2.7 version release. Installation Prior to installing Symfony2, you don't need to have a configured web server. If you have at least PHP version 5.4, you can use the standalone server provided by Symfony2. This server is suitable for development purposes and should not be used for production. It is strongly recommend to work with a Linux/UNIX system for both development and production deployment of Symfony2 framework applications. While it is possible to install and operate on a Windows box, due to its different nature, working with Windows can sometimes force you to maintain a separate fragment of code for this system. Even if your primary OS is Windows, it is strongly recommended to configure Linux system in a virtual environment. Also, there are solutions that will help you in automating the whole process. As an example, see more on https://www.vagrantup.com/ website. To install Symfony2, you can use a few methods as follows: Use a new Symfony2 installer script (currently, the only officially recommended). Please note that installer requires at least PHP 5.4. Use a composer dependency manager to install a Symfony project. Download a zip or tgz package and unpack it. It does not really matter which method you choose, as they all give you similar results. Installing Symfony2 by using an installer To install Symfony2 through an installer, go to the Symfony website at http://symfony.com/download, and install the Symfony2 installer by issuing the following commands: $ sudo curl -LsS http://symfony.com/installer -o /usr/local/bin/symfony $ sudo chmod +x /usr/local/bin/symfony After this, you can install Symfony by just typing the following command: $ symfony new <new_project_folder> To install the Symfony2 framework for a to-do application, execute the following command: $ symfony new <new_project_folder> This command installs the latest Symfony2 stable version on the newly created todoapp folder, creates the Symfony2 application, and prepares some basic structure for you to work with. After the app creation, you can verify that your local PHP is properly configured for Symfony2 by typing the following command: $ php app/check.php If everything goes fine, the script should complete with the following message: [OK] Your system is ready to run Symfony projects Symfony2 is equipped with a standalone server. It makes development easier. If you want to run this, type the following command: $ php app/console server:run If everything went alright, you will see a message that your server is working on the IP 127.0.0.1 and port 8000. If there is an error, make sure you are not running anything else that is listening on port 8000. It is also possible to run the server on a different port or IP, if you have such a requirement, by adding the address and port as a parameter, that is: $ php app/console server:run 127.0.0.1:8080 If everything works, you can now type the following: http://127.0.0.1:8000/ Now, you will visit Symfony's welcome page. This page presents you with a nice welcome information and useful documentation link. The Symfony2 directory structure Let's dive in to the initial directory structure within the typical Symfony application. Here it is: app bin src vendor web While Symfony2 is very flexible in terms of directory structure, it is recommended to keep the basic structure mentioned earlier. The following table describes their purpose: Directory Used for app This holds information about general configuration, routing, security configuration, database parameters, and many others. It is also the recommended place for putting new view files. This directory is a starting point. bin It holds some helper executables. It is not really important during the development process, and rarely modified. src This directory holds the project PHP code (usually your bundles). vendor These are third-party libraries used within the project. Usually, this directory contains all the open source third-party bundles, libraries, and other resources. It's worth to mention that it's recommended to keep the files within this directory outside the versioning system. It means that you should not modify them under any circumstances. Fortunately, there are ways to modify the code, if it suits your needs more. This will be demonstrated when we implement user management within our to-do application. web This is the directory that is accessible through the web server. It holds the main entry point to the application (usually the app.php and app_dev.php files), CSS files, JavaScript files, and all the files that need to be available through the web server (user uploadable files). So, in most cases, you will be usually modifying and creating the PHP files within the src/ directory, the view and configuration files within the app/ directory, and the JS/CSS files within the web/ directory. The main directory also holds a few files as follows: .gitignore README.md composer.json composer.lock The .gitignore file's purpose is to provide some preconfigured settings for the Git repository, while the composer.json and composer.lock files are the files used by the composer dependency manager. What is a bundle? Within the Symfony2 application, you will be using the "bundle" term quite often. Bundle is something similar to plugins. So it can literally hold any code controllers, views, models, and services. A bundle can integrate other non-Symfony2 libraries and hold some JavaScript/CSS code as well. We can say that almost everything is a bundle in Symfony2; even some of the core framework features together form a bundle. A bundle usually implements a single feature or functionality. The code you are writing when you write a Symfony2 application is also a bundle. There are two types of bundles. The first kind of bundle is the one you write within the application, which is project-specific and not reusable. For this purpose, there is a special bundle called AppBundle created for you when you install the Symfony2 project. Also, there are reusable bundles that are shared across the various projects either written by you, your team, or provided by a third-party vendors. Your own bundles are usually stored within the src/ directory, while the third-party bundles sit within the vendor/ directory. The vendor directory is used to store third-party libraries and is managed by the composer. As such, it should never be modified by you. There are many reusable open source bundles, which help you to implement various features within the application. You can find many of them to help you with User Management, writing RESTful APIs, making better documentation, connecting to Facebook and AWS, and even generating a whole admin panel. There are tons of bundles, and everyday brings new ones. If you want to explore open source bundles, and want to look around what's available, I recommend you to start with the http://knpbundles.com/ website. The bundle name is correlated with the PHP namespace. As such, it needs to follow some technical rules, and it needs to end with the Bundle suffix. A few examples of correct names are AppBundle and AcmeDemoBundle, CompanyBlogBundle or CompanySocialForumBundle, and so on. Composer Symfony2 is built based on components, and it would be very difficult to manage the dependencies between them and the framework without a dependency manager. To make installing and managing these components easier, Symfony2 uses a manager called composer. You can get it from the https://getcomposer.org/ website. The composer makes it easy to install and check all dependencies, download them, and integrate them to your work. If you want to find additional packages that can be installed with the composer, you should visit https://packagist.org/. This site is the main composer repository, and contains information about most of the packages that are installable with the composer. To install the composer, go to https://getcomposer.org/download/ and see the download instruction. The download instruction should be similar to the following: $ curl -sS https://getcomposer.org/installer | php If the download was successful, you should see the composer.phar file in your directory. Move this to the project location in the same place where you have the composer.json and composer.lock files. You can also install it globally, if you prefer to, with these two commands: $ curl -sS https://getcomposer.org/installer | php $ sudo mv composer.phar /usr/local/bin/composer You will usually need to use only three composer commands: require, install, and update. The require command is executed when you need to add a new dependency. The install command is used to install the package. The update command is used when you need to fetch the latest version of your dependencies as specified within the JSON file. The difference between install and update is subtle, but very important. If you are executing the update command, your composer.lock file gets updated with the version of the code you just fetched and downloaded. The install command uses the information stored in the composer.lock file and the fetch version stored in this file. When to use install? For example, if you deploy the code to the server, you should use install rather than update, as it will deploy the version of the code stored in composer.lock, rather than download the latest version (which may be untested by you). Also, if you work in a team and you just got an update through Git, you should use install to fetch the vendor code updated by other developers. You should use the update command if you want to check whether there is an updated version of the package you have installed, that is, whether a new minor version of Symfony2 will be released, then the update command will fetch everything. As an example, let's install one extra package for user management called FOSUserBundle (FOS is a shortcut of Friends of Symfony). We will only install it here; we will not configure it. To install FOSUserBundle, we need to know the correct package name and version. The easiest way is to look in the packagist site at https://packagist.org/ and search for the package there. If you type fosuserbundle, the search should return a package called friendsofsymfony/user-bundle as one of the top results. The download counts visible on the right-hand side might be also helpful in determining how popular the bundle is. If you click on this, you will end up on the page with the detailed information about that bundle, such as homepage, versions, and requirements of the package. Type the following command: $ php composer.phar require friendsofsymfony/user-bundle ^1.3 Using version ^1.3 for friendsofsymfony/user-bundle ./composer.json has been updated Loading composer repositories with package information Updating dependencies (including require-dev) - Installing friendsofsymfony/user-bundle (v1.3.6) Loading from cache friendsofsymfony/user-bundle suggests installing willdurand/propel-typehintable-behavior (Needed when using the propel implementation) Writing lock file Generating autoload files ... Which version of the package you choose is up to you. If you are interested in package versioning standards, see the composer website at https://getcomposer.org/doc/01-basic-usage.md#package-versions to get more information on it. The composer holds all the configurable information about dependencies and where to install them in a special JSON file called composer.json. Let's take a look at this: { "name": "wbancer/todoapp", "license": "proprietary", "type": "project", "autoload": { "psr-0": { "": "src/", "SymfonyStandard": "app/SymfonyStandard/" } }, "require": { "php": ">=5.3.9", "symfony/symfony": "2.7.*", "doctrine/orm": "~2.2,>=2.2.3,<2.5", // [...] "incenteev/composer-parameter-handler": "~2.0", "friendsofsymfony/user-bundle": "^1.3" }, "require-dev": { "sensio/generator-bundle": "~2.3" }, "scripts": { "post-root-package-install": [ "SymfonyStandard\\Composer::hookRootPackageInstall" ], "post-install-cmd": [ // post installation steps ], "post-update-cmd": [ // post update steps ] }, "config": { "bin-dir": "bin" }, "extra": { // [...] } } The most important section is the one with the require key. It holds all the information about the packages we want to use within the project. The key scripts contain a set of instructions to run post-install and post-update. The extra key in this case contains some settings specific to the Symfony2 framework. Note that one of the values in here points out to the parameter.yml file. This file is the main file holding the custom machine-specific parameters. The meaning of the other keys is rather obvious. If you look into the vendor/ directory, you will notice that our package has been installed in the vendor/friendsofsymfony/user-bundle directory. The configuration files Each application has a need to hold some global and machine-specific parameters and configurations. Symfony2 holds configuration within the app/config directory and it is split into a few files as follows: config.yml config_dev.yml config_prod.yml config_test.yml parameters.yml parameters.yml.dist routing.yml routing_dev.yml security.yml services.yml All the files except the parameters.yml* files contain global configuration, while the parameters.yml file holds machine-specific information such as database host, database name, user, password, and SMTP configuration. The default configuration file generated by the new Symfony command will be similar to the following one. This file is auto-generated during the composer install: parameters: database_driver: pdo_mysql database_host: 127.0.0.1 database_port: null database_name: symfony database_user: root database_password: null mailer_transport: smtp mailer_host: 127.0.0.1 mailer_user: null mailer_password: null secret: 93b0eebeffd9e229701f74597e10f8ecf4d94d7f As you can see, it mostly holds the parameters related to database, SMTP, locale settings, and secret key that are used internally by Symfony2. Here, you can add your custom parameters using the same syntax. It is a good practice to keep machine-specific data such as passwords, tokens, api-keys, and access keys within this file only. Putting passwords in the general config.yml file is considered as a security risk bug. The global configuration file (config.yml) is split into a few other files called routing*.yml that contain information about routing on the development and production configuration. The file called as security.yml holds information related to authentication and securing the application access. Note that some files contains information for development, production, or test mode. You can define your mode when you run Symfony through the command-line console and when you run it through the web server. In most cases, while developing you will be using the dev mode. The Symfony2 console To finish, let's take a look at the Symfony console script. We used it before to fire up the development server, but it offers more. Execute the following: $ php app/console You will see a list of supported commands. Each command has a short description. Each of the standard commands come with help, so I will not be describing each of them here, but it is worth to mention a few commonly used ones: Command Description app/console: cache:clear Symfony in production uses a lot of caching. Therefore, if you need to change values within a template (twig) or within configuration files while in production mode, you will need to clear the cache. Cache is also one of the reasons why it's worth to work in the development mode. app/console container:debug Displays all configured public services app/console router:debug Displays all routing configuration along with method, scheme, host, and path. app/console security:check Checks your composer and packages version against known security vulnerabilities. You should run this command regularly. Summary In this article, we have demonstrated how to use the Symfony2 installer, test the configuration, run the deployment server, and play around with the Symfony2 command line. We have also installed the composer and learned how to install a package using it. To demonstrate how Symfony2 enables you to make web applications faster, we will try to learn through examples that can be found in real life. To make this task easier, we will try to produce a real to-do web application with modern look and a few working features. In case you are interested in knowing other Symfony books that Packt has in store for you, here is the link: Symfony 1.3 Web Application Development, Tim Bowler, Wojciech Bancer Extending Symfony2 Web Application Framework, Sébastien Armand Resources for Article: Further resources on this subject: A Command-line Companion Called Artisan[article] Creating and Using Composer Packages[article] Services [article]