Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Hands-On Big Data Modeling
Hands-On Big Data Modeling

Hands-On Big Data Modeling: Effective database design techniques for data architects and business intelligence professionals

eBook
€17.99 €26.99
Paperback
€32.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Table of content icon View table of contents Preview book icon Preview Book

Hands-On Big Data Modeling

Data Modeling and Management Platforms

In the previous chapter, we discussed some theoretical and managerial aspects of big data, as well as the importance of big data in general. In this chapter, we will list some must-ask questions with regard to big data management. Also, you will learn about companies and governmental agencies that deal with big data, in terms of its storage, management, and processing. Big data presents new challenges, requiring framework-based developers to come up with solutions. They have introduced several types of scalable, portable frameworks for the storage, management, and processing of big data.

In this chapter, we will discuss various big data management approaches, including the steps involved in using the different options, as well as the vendors that provide them. Additionally, we will explore various programming platforms that are available...

Big data management

Big data management guarantees a certain level of data quality and accessibility for business intelligence, and also for big data analytics applications. Big data management incorporates the policies, techniques, and processes used for data collection, storage, administration, and the delivery of large repositories. The step can involve cleaning, integration, migration, and reporting.

Data ingestion

Data ingestion refers to the process of procuring data into the system. This can be done via manual, semi-automatic, or automatic methods.

Data ingestion means the process of getting the data into the data system that we are building or using.

In a smaller system, users prefer to have some kind of web form...

Big data management services

There are different varieties of big data management solution that organizations can choose from. Different vendors support different technological stacks, and have different pricing models. There are vendors that offer a variety of standalone or multi-featured big data management tools. In this section, we will present some services from data management vendors.

Data cleansing

Data cleansing is the process of identifying and fixing corrupt or fallacious records in a record set, table, or database. It also deals with identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data, and then replacing, modifying, or deleting the infected data. Data entry and acquisition is inherently...

Big data management vendors

Due to the high prevalence of big data and its use in business intelligence, there are thousands of big data management vendors available in the market, ready to provide all data management services. Going through each of them would be beyond the scope of this book. In this section, we will explore ten big data management vendors. The list contained in this book does not guarantee the popularity or performance of each vendor. The list was created based on the use of state-of-the-art data analysis:

  • Alation: Alation is a newer organization, providing cloud-based data services that were created to centralize knowledge about a company's data and how to apply it. You can find out more about the company from the link provided in the Further study section.
  • AtScale: AtScale provides technology to facilitate familiar BI tools, in order to query data stored...

Big data storage and data models

We have now explored what big data is, as well as its importance. A demand for storing and processing massive-scale datasets has encouraged the development of data storage and database systems over the past few years. With this advancement, storage systems and data models for big data have been improved and enhanced over time, from local storage to clustered, distributed to cloud-based storage. In this section, you will learn about some most popular storage models and data models:

Figure 2.1

The preceding diagram shows the taxonomy of data stores and data platforms, and the corresponding systems that are currently available in the market. Let's dig into each category in the next section.

Storage models

...

Big data programming models

Big data programming models portray the style of and show the interface paradigm for developers to write big data applications and programs. Programming models are normally the core feature of big data frameworks, as they implicitly affect the execution model of big data processing engines, and also drive revealing and constructing big data applications and programs.

In this section, we will discuss and compare the major programming models for writing big data applications, based on their taxonomy.

MapReduce

MapReduce refers to a programming model that is suitable for processing big amounts of data. For example, Hadoop is capable of executing a MapReduce program written in several programming languages...

Getting started with Python and R

In this section, we will get started with the most common open source programming languages in the big data world. We will show you how to get started with Python and R on a Windows machine and a macOS machine.

Python on macOS

macOS X, High Sierra has a preloaded version of Python 2.7 out-of-the-box. If you have macOS X, you will not have to install or configure anything else in order to use Python 2. If you want to use Python3, then follow these instructions.

Before you install Python, you’ll be required to install GCC:

Summary

In this chapter, we discussed various theoretical aspects of big data management. Additionally, we enlightened you about different data management services, including data cleansing and data integration. There are also different storage models and data models, which we covered briefly in this chapter. Finally, in this chapter, we explained the different programming models, and how to get started with the Python and R programming languages. In the next chapter, you will learn more about the different types of data models, and the structured, unstructured, and operations performed on them.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Create effective models that get the most out of big data
  • Apply your knowledge to datasets from Twitter and weather data to learn big data
  • Tackle different data modeling challenges with expert techniques presented in this book

Description

Modeling and managing data is a central focus of all big data projects. In fact, a database is considered to be effective only if you have a logical and sophisticated data model. This book will help you develop practical skills in modeling your own big data projects and improve the performance of analytical queries for your specific business requirements. To start with, you’ll get a quick introduction to big data and understand the different data modeling and data management platforms for big data. Then you’ll work with structured and semi-structured data with the help of real-life examples. Once you’ve got to grips with the basics, you’ll use the SQL Developer Data Modeler to create your own data models containing different file types such as CSV, XML, and JSON. You’ll also learn to create graph data models and explore data modeling with streaming data using real-world datasets. By the end of this book, you’ll be able to design and develop efficient data models for varying data sizes easily and efficiently.

Who is this book for?

This book is great for programmers, geologists, biologists, and every professional who deals with spatial data. If you want to learn how to handle GIS, GPS, and remote sensing data, then this book is for you. Basic knowledge of R and QGIS would be helpful.

What you will learn

  • Get insights into big data and discover various data models
  • Explore conceptual, logical, and big data models
  • Understand how to model data containing different file types
  • Run through data modeling with examples of Twitter, Bitcoin, IMDB and weather data modeling
  • Create data models such as Graph Data and Vector Space
  • Model structured and unstructured data using Python and R
Estimated delivery fee Deliver to Switzerland

Standard delivery 10 - 13 business days

€11.95

Premium delivery 3 - 6 business days

€16.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Nov 30, 2018
Length: 306 pages
Edition : 1st
Language : English
ISBN-13 : 9781788620901
Category :
Languages :
Concepts :
Tools :

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Estimated delivery fee Deliver to Switzerland

Standard delivery 10 - 13 business days

€11.95

Premium delivery 3 - 6 business days

€16.95
(Includes tracking information)

Product Details

Publication date : Nov 30, 2018
Length: 306 pages
Edition : 1st
Language : English
ISBN-13 : 9781788620901
Category :
Languages :
Concepts :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 111.97
Hands-On Big Data Modeling
€32.99
Hands-On Data Science with R
€32.99
Big Data Architect???s Handbook
€45.99
Total 111.97 Stars icon

Table of Contents

16 Chapters
Introduction to Big Data and Data Management Chevron down icon Chevron up icon
Data Modeling and Management Platforms Chevron down icon Chevron up icon
Defining Data Models Chevron down icon Chevron up icon
Categorizing Data Models Chevron down icon Chevron up icon
Structures of Data Models Chevron down icon Chevron up icon
Modeling Structured Data Chevron down icon Chevron up icon
Modeling with Unstructured Data Chevron down icon Chevron up icon
Modeling with Streaming Data Chevron down icon Chevron up icon
Streaming Sensor Data Chevron down icon Chevron up icon
Concept and Approaches of Big-Data Management Chevron down icon Chevron up icon
DBMS to BDMS Chevron down icon Chevron up icon
Modeling Bitcoin Data Points with Python Chevron down icon Chevron up icon
Modeling Twitter Feeds Using Python Chevron down icon Chevron up icon
Modeling Weather Data Points with Python Chevron down icon Chevron up icon
Modeling IMDb Data Points with Python Chevron down icon Chevron up icon
Other Books You May Enjoy Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Half star icon Empty star icon Empty star icon 2.3
(4 Ratings)
5 star 25%
4 star 0%
3 star 0%
2 star 25%
1 star 50%
Pooja Dec 31, 2019
Full star icon Empty star icon Empty star icon Empty star icon Empty star icon 1
This book is just a bunch of buzz words thrown together with no real in depth discussion of any topic.
Amazon Verified review Amazon
Jimmy Boam Sep 22, 2019
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Just finished reading the book. I'd regard the book as required reading for anyone involved in big data/ dataware housing. I recently asked my team lead to get 10 copies for 15 peers in my department. Buying this book is a no-brainer with respect to personal ROI.
Amazon Verified review Amazon
Amazon Customer Jun 07, 2019
Full star icon Empty star icon Empty star icon Empty star icon Empty star icon 1
I don't normally write reviews, but decided I should for this to warn others. This is one of the worst technical books I've read in some time. It seems it was hastily thrown together as a cash grab.
Amazon Verified review Amazon
Jack Mok Mar 03, 2019
Full star icon Full star icon Empty star icon Empty star icon Empty star icon 2
The content is so simple
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela