Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Scaling Big Data with Hadoop and Solr, Second Edition

You're reading from   Scaling Big Data with Hadoop and Solr, Second Edition Understand, design, build, and optimize your big data search engine with Hadoop and Apache Solr

Arrow left icon
Product type Paperback
Published in Apr 2015
Publisher
ISBN-13 9781783553396
Length 166 pages
Edition 1st Edition
Tools
Concepts
Arrow right icon
Author (1):
Arrow left icon
Hrishikesh Vijay Karambelkar Hrishikesh Vijay Karambelkar
Author Profile Icon Hrishikesh Vijay Karambelkar
Hrishikesh Vijay Karambelkar
Arrow right icon
View More author details
Toc

Sharding algorithm and fault tolerance

We have already seen the sharding, collection and replicas. In this section we will look at some of the important aspects of sharding, and how it plays a role in scalability and high availability. The strategy for creating new shards is highly dependent upon the hardware and the shard size. Let's say, you have two machines M1 & M2, of, the same configuration, each with one shard. Shard A is loaded with 1 million index documents, and shard B is loaded with 100 documents. When a query is fired, the query response to any Solr queries is determined by the query response of slowest node (in this case shard A). Hence having a shard with near to equal shard sizes can perform better in this case.

Document Routing and Sharding

Typically, when any enterprise search is deployed, the size of documents to be indexed keeps growing over time. Since SolrCloud provides a way to create a cluster of Solr nodes running on index shards, it becomes feasible to scale...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image