Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
PostgreSQL Replication, Second Edition
PostgreSQL Replication, Second Edition

PostgreSQL Replication, Second Edition: Leverage the power of PostgreSQL replication to make your databases more robust, secure, scalable, and fast

Arrow left icon
Profile Icon Hans-Jürgen Schönig
Arrow right icon
$27.98 $39.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.5 (4 Ratings)
eBook Jul 2015 322 pages 1st Edition
eBook
$27.98 $39.99
Paperback
$48.99
Subscription
Free Trial
Renews at $19.99p/m
Arrow left icon
Profile Icon Hans-Jürgen Schönig
Arrow right icon
$27.98 $39.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.5 (4 Ratings)
eBook Jul 2015 322 pages 1st Edition
eBook
$27.98 $39.99
Paperback
$48.99
Subscription
Free Trial
Renews at $19.99p/m
eBook
$27.98 $39.99
Paperback
$48.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

PostgreSQL Replication, Second Edition

Chapter 2. Understanding the PostgreSQL Transaction Log

In the previous chapter, we dealt with various replication concepts. That was meant to be more of a theoretical overview to sharpen your senses for what is to come, and was supposed to introduce the topic in general.

In this chapter, we will move closer to practical solutions, and you will learn how PostgreSQL works internally and what it means for replication. We will see what the so-called transaction log (XLOG) does and how it operates. The XLOG is the very backbone of the PostgreSQL onboard replication machinery. It is essential to understand how this part works in order to fully understand replication. You will learn the following topics in this chapter:

  • How PostgreSQL writes data
  • Which memory and storage parameters are involved
  • How writes can be optimized
  • How writes are replicated
  • How data consistency can be ensured

In this chapter, you will also learn about replication slots and logical decoding. Once you have completed reading...

How PostgreSQL writes data

PostgreSQL replication is all about writing data. Therefore, the way PostgreSQL writes a chunk of data internally is highly relevant and directly connected to replication and replication concepts. In this section, we will dig into writes.

The PostgreSQL disk layout

One of the first things we want to take a look at in this chapter is the PostgreSQL disk layout. Knowing about the disk layout can be very helpful when inspecting an existing setup, and it can be helpful when designing an efficient, high-performance installation.

In contrast to other database systems, such as Oracle, PostgreSQL will always rely on a filesystem to store data. PostgreSQL does not use raw devices. The idea behind this is that if a filesystem developer has done their job well, there is no need to reimplement the filesystem functionality over and over again.

Looking into the data directory

To understand the filesystem layout used by PostgreSQL, we can take a look at what we can find inside the...

The XLOG and replication

In this chapter, you have already learned that the transaction log of PostgreSQL has all the changes made to the database. The transaction log itself is packed into nice and easy-to-use 16 MB segments.

The idea of using this set of changes to replicate data is not farfetched. In fact, it is a logical step in the development of every relational (or maybe even a nonrelational) database system. For the rest of this book, you will see in many ways how the PostgreSQL transaction log can be used, fetched, stored, replicated, and analyzed.

In most replicated systems, the PostgreSQL transaction log is the backbone of the entire architecture (for synchronous as well as for asynchronous replication).

Understanding consistency and data loss

Digging into the PostgreSQL transaction log without thinking about consistency is impossible. In the first part of this chapter, we tried hard to explain the basic idea of the transaction log in general. You learned that it is hard, or even impossible, to keep data files in good shape without the ability to log changes beforehand.

So far, we have mostly talked about corruption. It is definitely not good to lose data files because of corrupted entries in them, but corruption is not the only issue you have to be concerned about. Two other important topics are:

  • Performance
  • Data loss

While these might be an obvious choice for important topics, we have the feeling that they are not well understood and honored. Therefore, they have been taken into consideration.

In our daily business as PostgreSQL consultants and trainers, we usually tend to see people who are only focused on performance.

Performance is everything. We want to be fast; tell us how to be fast&...

Tuning checkpoints and the XLOG

So far, this chapter has hopefully provided some insights into how PostgreSQL writes data and what the XLOG is used for in general. Given this knowledge, we can now move on and see what we can do to make our databases work even more efficiently, both in the case of replication and in the case of running just a single server.

Understanding the checkpoints

In this chapter, we have seen that data has to be written to the XLOG before it can go anywhere. The thing is that if the XLOG was never deleted, clearly, we would not write to it forever without filling up the disk at some point in time.

To solve this problem, the XLOG has to be deleted at some point. This process is called checkpointing.

The main question arising from this issue is, "When can the XLOG be truncated up to a certain point?" The answer is, "When PostgreSQL has put everything that is already in the XLOG into the storage files." If all the changes made to the XLOG are also made...

Experiencing the XLOG in action

We will use the transaction log throughout this book, and to give you a deeper insight into how things work on a technical level, we have added this section dealing exclusively with the internal workings of the XLOG machinery. We will avoid going down to the C level as this would be way beyond the scope of this book, but we will provide you with insights that are hopefully deep enough.

Understanding the XLOG records

Changes made to the XLOG are record-based. What does that mean? Let's assume you are adding a row to a table:

test=# INSERT INTO t_test VALUES (1, 'hans');
INSERT 0 1

In this example, we are inserting values into a table containing two columns. For the sake of this example, we want to assume that both columns are indexed.

Remember what you learned before: the purpose of the XLOG is to keep those data files safe. So this operation will trigger a series of XLOG entries. First, the data file (or files) related to the table will be written...

How PostgreSQL writes data


PostgreSQL replication is all about writing data. Therefore, the way PostgreSQL writes a chunk of data internally is highly relevant and directly connected to replication and replication concepts. In this section, we will dig into writes.

The PostgreSQL disk layout

One of the first things we want to take a look at in this chapter is the PostgreSQL disk layout. Knowing about the disk layout can be very helpful when inspecting an existing setup, and it can be helpful when designing an efficient, high-performance installation.

In contrast to other database systems, such as Oracle, PostgreSQL will always rely on a filesystem to store data. PostgreSQL does not use raw devices. The idea behind this is that if a filesystem developer has done their job well, there is no need to reimplement the filesystem functionality over and over again.

Looking into the data directory

To understand the filesystem layout used by PostgreSQL, we can take a look at what we can find inside the...

The XLOG and replication


In this chapter, you have already learned that the transaction log of PostgreSQL has all the changes made to the database. The transaction log itself is packed into nice and easy-to-use 16 MB segments.

The idea of using this set of changes to replicate data is not farfetched. In fact, it is a logical step in the development of every relational (or maybe even a nonrelational) database system. For the rest of this book, you will see in many ways how the PostgreSQL transaction log can be used, fetched, stored, replicated, and analyzed.

In most replicated systems, the PostgreSQL transaction log is the backbone of the entire architecture (for synchronous as well as for asynchronous replication).

Left arrow icon Right arrow icon

Description

This book is ideal for PostgreSQL administrators who want to set up and understand replication. By the end of the book, you will be able to make your databases more robust and secure by getting to grips with PostgreSQL replication.

Who is this book for?

This book is ideal for PostgreSQL administrators who want to set up and understand replication. By the end of the book, you will be able to make your databases more robust and secure by getting to grips with PostgreSQL replication.

What you will learn

  • Use Pointintime Recovery to perform data recovery as well as replication
  • Set up synchronous as well as asynchronous streaming replication
  • Get familiarized with the transaction log, the core component of most replication setups and its purpose
  • Improve speed and reliability with an understanding of pgpool and PgBouncer
  • Increase your data security and geographically distribute data
  • Make your systems more available and secure with Linux High Availability
  • Scale out with PL/Proxy and PostgresXC
  • Detect, investigate, and solve replicationrelated problems

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jul 28, 2015
Length: 322 pages
Edition : 1st
Language : English
ISBN-13 : 9781783988549
Category :
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Jul 28, 2015
Length: 322 pages
Edition : 1st
Language : English
ISBN-13 : 9781783988549
Category :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 142.97
Troubleshooting PostgreSQL
$32.99
PostgreSQL Replication, Second Edition
$48.99
Learning PostgreSQL
$60.99
Total $ 142.97 Stars icon

Table of Contents

16 Chapters
1. Understanding the Concepts of Replication Chevron down icon Chevron up icon
2. Understanding the PostgreSQL Transaction Log Chevron down icon Chevron up icon
3. Understanding Point-in-time Recovery Chevron down icon Chevron up icon
4. Setting Up Asynchronous Replication Chevron down icon Chevron up icon
5. Setting Up Synchronous Replication Chevron down icon Chevron up icon
6. Monitoring Your Setup Chevron down icon Chevron up icon
7. Understanding Linux High Availability Chevron down icon Chevron up icon
8. Working with PgBouncer Chevron down icon Chevron up icon
9. Working with pgpool Chevron down icon Chevron up icon
10. Configuring Slony Chevron down icon Chevron up icon
11. Using SkyTools Chevron down icon Chevron up icon
12. Working with Postgres-XC Chevron down icon Chevron up icon
13. Scaling with PL/Proxy Chevron down icon Chevron up icon
14. Scaling with BDR Chevron down icon Chevron up icon
15. Working with Walbouncer Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.5
(4 Ratings)
5 star 50%
4 star 50%
3 star 0%
2 star 0%
1 star 0%
Jeff L Sep 21, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book is a great introductory text to anyone needing to get started with data replication on PostgreSQL and is unsure about how to get started. It covers all of the major replication technologies currently available and describes some of the benefits and limitations of each. General optimization strategies to improve replication performance are also covered. Since the scope of coverage is pretty wide across all of the different replication technologies, this book doesn't quiet have enough space to devote more than single chapter to each, so you may still need to consult other manuals and reference materials once you've narrowed down your replication strategy.
Amazon Verified review Amazon
Lance Alligood Nov 12, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Normally when I am googling for assistance with PostgreSQL, it's all but guaranteed that at least one of the first 3 hits is for the postgresql.org. However, when it came to configuring master-slave replication, that wasn't the case. Most of the hits were instead for various blog posts or forums. Any details were frequently poorly formatted, glossed over--or omitted(!)--important steps, and never supplied adequate background information for the values assigned to parameters in postgresql.conf. To make matters worse, there was little mention of what version of PostgreSQL the instructions were for in those posts. My frustration level was off the charts as I tried to make sense of this poor & inconsistent "documentation"!I was in (desperate!) need of some accurate, explanatory guidance to get replication & point-in-time-recovery (PITR) up & going on my PostgreSQL 9.4 servers. Normally the PostgreSQL project is fantastic in supplying clear & precise documentation for commands & components of the database, but that just isn't the case at all when it came to making sense of replication and/or PITR. However, I must fully acknowledge that the PostgreSQL project has been exerting considerable effort into improving the ease of configuration & use when it comes to replication & PITR. (And it's only a matter of time the website documentation is there to go along with it.) Based on my searches, there have been massive changes in replication & PITR for (at least) 9.1, 9.2, & 9.3.PostgreSQL Replication Second Edition just came out in July 2015 so it is thankfully relevant to 9.4, although the changes from 9.3 -> 9.4 (command-wise at least) appear to be less dramatic than the previous 9.x releases. This was very much the documentation I was looking for! I have no issues with an author nor a publisher putting out a well-written book, but why PostgreSQL doesn't have webpages on its site comparable to this excellent tome is a real head-scratcher. That said, this book definitely lives up to the hype of showing you what it takes to get either/both replication or PITR running on PostgreSQL. I was particularly thankful for the explanation of the main files/directories that make up the database & all of the necessary parameters in postgresql.conf & pg_hba.conf!The project admits that there will certainly be more changes in the future--and I'm quite certain they'll all be for the better!--when it comes to replication & PITR, but for the hear & now in late 2015, this book will get you up to speed on configuring those with the current tools right now, quickly, & with little effort.
Amazon Verified review Amazon
Ángel Mar 28, 2018
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
Tiene base teórica, ejemplos prácticos, bastante completo en sus explicaciones, además de que referencia a documentos en internet (las URLs han cambiado desde la edición del libro y es el único fallo que le encuentro).
Amazon Verified review Amazon
Ajay Thakur Aug 12, 2015
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
This is an amazing book on Replication.Concepts are well explained and things are well described too.RegardsChitij Chauhan
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.