What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Learning Search-driven Application Development with SharePoint 2013

Chapter 1. Getting Started with SharePoint 2013 Search

SharePoint 2013 feels like a breeze of fresh air, offering many new features and changes over older versions. In addition to a whole new social experience, a new development model called Apps, and native HTML5 support, SharePoint 2013 introduces a new and improved search engine. As the title of the book implies, this book is all about the new search engine. In this introductory chapter we will get a taste of the new features SharePoint 2013 Search brings to the table and then deep-dive into the architecture that holds this system together.

In this chapter, we will cover the following topics:

New features of SharePoint 2013 Search
The new search architecture

New features of SharePoint 2013 Search

The SharePoint 2013 Search engine is the most powerful enterprise search engine Microsoft has created to date. With this new release, Microsoft combined all of the best features of the legacy SharePoint Enterprise search engine with the best features of the FAST search engine, which Microsoft acquired back in 2008.

The new features of SharePoint 2013 Search can be divided into four main categories as follows:

Search administration
UI changes and customization
Relevance and ranking features
New development methods

Search administration

One drawback of search in previous versions of SharePoint was that almost everything had to be managed from the central administration page, which meant that search was managed at the farm level.

SharePoint 2013 changed that by adding most of the search settings from the farm level to site collections and sites (SPWebs). As SharePoint 2013 is offered as a cloud service (through Office 365), and cloud users have no access to settings in the farm-administration level, this was a welcome change that both cloud and on-premise site administrations can take advantage of.

Let's have a look at what settings are available for us to administrate; these are shown in the following screenshot:

We will discuss these settings in detail in Chapter 2, Using the Out of the Box Search Components, but for now just keep in mind that a site administrator can configure the search experience on his/her site in ways that were reserved exclusively to farm administrators in previous versions.

In addition, Microsoft introduces a new crawling mode, continuous crawl. Continuous crawl helps to keep the search index as fresh as possible by crawling SharePoint sites (and only SharePoint sites) every 15 minutes, by default; we can change this value using PowerShell, as shown in the following snippet:

$ssa = Get-SPEnterpriseSearchServiceApplication
$ssa.SetProperty("ContinuousCrawlInterval",<minutes>)

The value we use for <minutes> is the number of minutes between crawling.

When running, the crawler gets changes from the crawled SharePoint sites and pushes them to the content processing component, which will process the new content on the fly.

By enabling the continuous crawler, items appear in the search almost immediately after being crawled.

UI changes and customization

If there is one change in SharePoint 2013 Search that just pops to the eyes, it is the new and fresh user interface (UI). If you worked with SharePoint 2010 search, you'll remember the following screenshot, showing a search-results page:

By looking at the preceding screenshot, we can see that it sports a pretty simple UI. We have textual refinements on the left side; predefined search scopes for websites and people (All Sites and People) on top, and a main, simply styled results area without grouping or categorization of results.

To customize the way the results are shown, we had to use XSL/XSLT, which is quite a messy and unattractive way to design.

Fast forward to the present day. The following screenshot displays how the results page looks like in SharePoint 2013:

Now that's quite different, isn't it? The UI is modern thanks to the use of HTML and JavaScript templates. Instead of messing around with XSLT, we now have display templates to design our results using languages we already know and love: HTML, JavaScript, and CSS.

Take a look at the refinement panel on the left. While we still have textual refiners, we also have graphical ones, such as a scroller for dates.

We have new out-of-the-box search scopes—Videos and Reports—and results are grouped by their types; for example, the first two results are discussion items from a discussion board.

The biggest and most notable change, however, is the new hover panel. Whenever we hover over a search result, we are presented with a floating panel containing additional information about the hovered item. As SharePoint 2013 seamlessly integrates with Office web apps, any Office document we hover over will show a preview of its content in the new hover panel. The most important thing about the hover panel, however, is that we, as developers, have complete control over the content of this hover panel. Just like search results, the hover panel is also controlled by HTML, CSS, and JavaScript.

We will discuss all of these new and exciting customizations features in detail in Chapter 4, Customizing the Look.

Relevance and ranking features

As mentioned earlier, SharePoint 2013 Search took the best features of SharePoint Search and FAST and improved them. As such, SharePoint 2013 uses new and improved ranking models to determine which items are to be displayed and what would be their rank (the order in which they are displayed).

The key to successfully determine the relevancy of search results is to satisfy the intent of the person who issues the query. Let's explain this statement with an example; say I'm performing a search for Apple. Now, did I search for apple the fruit or Apple the technology company?

SharePoint 2013 Search continuously tracks and analyzes search usage to determine how content is connected, how often an item appears in search results, and which search results people click in order to continuously improve the relevance of items to the search query. So, if I clicked on a lot of fruit-related results, the search engine will assume I was looking for apple the fruit, and not the technology company.

We will discuss these new features in Chapter 2, Using the Out of the Box Search Components, and Chapter 3, Using the New CSOM and RESTful APIs.

New development methods

With this new release of SharePoint, Microsoft made changes to the search-development model. The old SOAP web service (ASMX) has been deprecated alongside the SQL query syntax that we could use to query against SharePoint data.

But, just like the the old saying goes, "out with the old and in with the new", we get some new features to play with to replace the ones that are gone.

A new C lient Side Object Model (CSOM) object which enable us to access the search service using JavaScript and C#. With the help of the search CSOM, we can create search-driven applications even for servers that don't have SharePoint 2013 installed on them.
A dedicated Representational State Transfer (REST) service that enables us to execute queries against the search service from client applications using libraries such as jQuery or RestSharp. The REST service supports all of the properties available in the CSOM object, but instead of working against objects, we use the URL's query string to send parameters to it.
An enhanced keyword query language with new and improved operators such as ONEAR and XRANK.
Enhancements to the Business Connectivity Services Connector Framework, which improves capturing and logging of exceptions to help us troubleshoot errors during the crawl process.

We will discuss all of these topics in detail in Chapter 3, Using the New CSOM and RESTful APIs, and Chapter 5, Extending Beyond SharePoint.

Now that we have a general idea about what's new in SharePoint 2013 Search, let's go ahead and discuss the architecture that makes all of this happen.

The search architecture

SharePoint 2013 Search introduces a new search architecture that includes significant changes and new additions compared to previous versions. Since Microsoft consolidated FAST and SharePoint Search, the new search architecture has inherited components from both products while maintaining high scalability and performance.

Let's have a look at the new search architecture and discuss its components; refer to the following screenshot:

As we can see from the diagram, the search architecture can be divided into four components groups as follows:

Content components
Query components
The index component
The analytics-processing component

Content components

The content components are in charge of getting content ready for indexing. Each component has a well-defined role, which we will discuss next.

Crawl component

The crawl component is responsible for crawling content sources. It is the first stop for data that is about to be indexed by the search engine. The crawl component invokes connectors (both out-of-the-box and custom ones) that interact with the content source in order to crawl it.

While indexing, the crawl component uses one (or more) crawl database to temporarily store detailed tracking and historical information about the crawled item, such as the last time the item was crawled and the type of update during the last crawl.

Once an item is crawled, meaning both its data and its associated metadata is crawled, the crawl component delivers it to the content-processing component.

Content-processing component

The content-processing component's job is to analyze content it receives from the crawl component and feed it to the index component for indexing.

Content analysis is done by following a flow known as the Content Processing Flow, which is depicted in the following diagram:

The rectangular blocks in the diagram represent stages that we cannot interact with. We won't be discussing them as they are quite self-explanatory. The curved rectangular blocks, however, represent stages that we can interact with during the processing flow.

The Web service callout stage is similar to the pipeline extensibility stage of FAST for SharePoint 2010, and allows you to add a callout from the content-processing component to a web service of your own so you can manipulate the crawled content before it gets indexed by the index component.

Unlike FAST's pipeline-extensibility stage, where code had to be executed in a sandbox, the web service callout accepts a web service endpoint, which is much easier and reduces the overhead involved in writing a console application to accompany the content-flow process.

Calling a web service during the processing stage can be useful for two scenarios.

Creating new refiners by extracting data from unstructured text using our own logic
Calculating new refiners based on the data of managed properties

You can find a great example on using the web service callout in Kathrine Hammervold's post, Customize the SharePoint 2013 search experience with a Content Enrichment web service, located at http://blogs.msdn.com/b/sharepointdev/archive/2012/11/13/customize-the-sharepoint-2013-search-experience-with-a-content-enrichment-web-service.aspx.

The next point of interaction is the word-breaking stage, which allows you to write your own custom word-breaking logic for the content processor. Please refer to the MSDN documentation on custom word breakers, located at http://msdn.microsoft.com/en-us/library/jj163981.aspx.

Query components

The query components are in charge of analyzing the search query and processing the results.

Web frontend

The web frontend is where the search process actually begins. A user can interact with the search service by either writing a search query in the search center (or a search box) or developing against the new public APIs: REST/OData services and the CSOM. Both the search center and public APIs are hosted on the frontend.

Once the user creates a query, the query is sent to the query-processing component for analysis. The query-processing component analyzes the query and forwards it to the index component. The index component returns the matching results to the query-processing component for another analysis and from there the results are forwarded to the web frontend to be displayed.

Query processing component

As mentioned previously, the query-processing component's job is to analyze and process both search queries and results.

When the query-processing component receives a search query from the frontend, it analyzes it in an attempt to optimize its precision and relevance. A site administrator can interact with a query using different techniques such as query rules or result source. We will discuss these techniques in detail in the next chapter, but for now it is important to understand that these manipulations are handled within the query-processing components. As part of its query handling, the query-processing component performs linguistic processes on the query, such as word-breaking and stemming.

Once the query is optimized, it is sent to the index component, which will process the optimized query and return a result set back to the query-processing component and from there to the search frontend.

The index component

The index component is the heart of the search service, and without proper planning it can easily become the bottleneck of the service as well.

The index component has the following two roles:

Input: The index component is in charge of writing the optimized content it gets from the content-processing component to the index file
Output: The index component is in charge of returning results from the index file to the query-processing component, by request

How the index component saves and manages this index file is out of the scope of this book, but you can read more about this in the TechNet article Manage the index component in SharePoint Server 2013, located at http://technet.microsoft.com/en-us/library/jj862355.aspx.

Analytics processing component

The analytics-processing component is a new addition to SharePoint Search. Its role is to analyze both content and user actions with the content in order to improve the search relevance for the user.

The analytics architecture consists of three main parts, as follows:

The analytics-processing component, which runs the analytics jobs.
The analytics-reporting database, which stores statistical information such as usage data.
The link database, which stores information about searches and crawled documents. In addition, the link database is shared with the Content Processing Component, which in turn stores links and anchors in it. The information, the content-processing component stores is later used by the analytics-processing component.

The analytics-processing component runs two types of analytics: search analytics and usage analytics. The search analytics analyzes content from the content-processing component for information such as links, information related to people, and recommendations. The usage analytics analyzes user actions on an item, such as the number of views it had or how many users clicked on it.

An important output of usage analytics are the recommendations. The recommendations analysis creates recommendations on items based on how users have interacted with this specific item in the past. The analysis calculates an item-to-item relationship graph and updates it continuously based on search usage.

Keep in mind that the analytics-processing component is a "learning" component, which means it learns by usage. The more usage the search system will have, the better analytics it will provide.

Key benefits

Create search-driven applications using the new SharePoint 2013 enterprise search engine

Learn how to respond intelligently to user's search queries using Query Rules

Filled with helpful tips, diagrams, and practical examples to make your organization's search experience smarter

Description

SharePoint 2013 feels like a of fresh air, offering many new features and changes over older versions. Among these new features is a completely revamped search engine. "Learning Search-driven Application Development with SharePoint 2013" is a quick-start guide to creating search-driven applications using the new and exciting features that have revolutionized the SharePoint enterprise search experience. "Learning Search-driven Application Development with SharePoint 2013" is a fast-paced, practical, hands-on guide to the world of enterprise search in SharePoint 2013. With step-by-step tutorials and real-world examples, this book will give you a head start creating fresh and exciting search-driven applications using SharePoint 2013's new search engine. "Learning Search-driven Application Development with SharePoint 2013" is an accelerated way to create search-driven applications for SharePoint 2013. By covering the basics first and gradually covering all search related topics, this book will be your guide through the world of SharePoint's enterprise search. Furthermore, you will learn how to use the powerful feature of Query Rules to create smart conditions that help respond to user's search queries intelligently. We will also discuss how to style search results and make them stand out, how to index external content so it will be searchable using SharePoint's powerful search engine, and how to use the new client side search APIs, which will allow us to take advantage of search in Apps, the new development model for SharePoint 2013. After reading Learning Search-driven Application Development with SharePoint 2013, you will understand what it takes to create applications that use search as a content provider. Using applications that are based on real world examples and step-by-step tutorials, you'll get hands-on experience in developing search driven applications.

What you will learn

Getting acquainted with the new features of SharePoint 2013 search and how its architecture is designed

Using query rules to define smart search conditions

Building search verticals

Using the new Content Search Web Part

Using SharePoint 2013 s client facing APIs and learn how to use them in a SharePoint hosted app

Styling results using Result Types and Display Templates

Learning how to use the business connectivity services application

Creating a BCS search indexer connector to index content that is external to SharePoint

What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

$60.99

Microsoft SharePoint 2013 Disaster Recovery Guide

$48.99

Learning Search-driven Application Development with SharePoint 2013

$43.99

Total $ 153.97

JuanBerenguer Mar 18, 2014

At nearly $40 one expects a thorough examination of a given topic within a technology book of this nature. At a measly 87 pages of content, this "book" whisks over topics and gives you a very high level idea of the components that make up search in SharePoint 2013. The content within this book is more akin to what one would expect from a free E-book or a whitepaper. Spend 15 minutes searching the internet on this topic and you not only have a better grasp of Search Driven Application Development, but you'll have an extra $40 sitting in your pocket.

Amazon Verified review

Learning Search-driven Application Development with SharePoint 2013: The search engine in SharePoint 2013 is a refreshed version and this book will show you how to make the most of it with a range of methodologies for developing search-driven applications. JavaScript experience required.

What do you get with Print?

Learning Search-driven Application Development with SharePoint 2013

Chapter 1. Getting Started with SharePoint 2013 Search

New features of SharePoint 2013 Search

Search administration

UI changes and customization

Relevance and ranking features

New development methods

The search architecture

Content components

Crawl component

Content-processing component

Query components

Web frontend

Query processing component

The index component

Analytics processing component

Summary

Page 1 of 4

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs

Learning Search-driven Application Development with SharePoint 2013: The search engine in SharePoint 2013 is a refreshed version and this book will show you how to make the most of it with a range of methodologies for developing search-driven applications. JavaScript experience required.

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs