9. Tools to Move Data in and out of PostgreSQL | PostgreSQL for Data Architects

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

PostgreSQL for Data Architects

By : Jayadevan M

5 (2)

PostgreSQL for Data Architects

5 (2)

By: Jayadevan M

Overview of this book

This book is for developers and data architects who have some exposure to databases. It is assumed that you understand the basic concepts of tables and common database objects, including privileges and security.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. Installing PostgreSQL

1. Installing PostgreSQL

Installation options

Dependencies to compile the source

Configuring and creating the makefile

Building and creating the executables

Installing and moving the files to where they belong

Initializing a cluster

Working with extensions

Summary

2. Server Architecture

2. Server Architecture

Starting with the daemon process

Understanding the shared buffer

Checkpoint

WAL and the WAL writer process

The background writer

The autovacuum launcher process

The logging process

The stats collector process

The WAL sender and WAL receiver

Sorting in memory with work_mem

Maintenance with maintenance_work_mem

Summary

3. PostgreSQL – Object Hierarchy and Roles

3. PostgreSQL – Object Hierarchy and Roles

The PostgreSQL cluster

Understanding tablespaces

Databases, schemas, and search_path

Roles and privileges

Summary

4. Working with Transactions

4. Working with Transactions

Understanding transactions

PostgreSQL and MVCC

Summary

5. Data Modeling with SQL Power Architect

5. Data Modeling with SQL Power Architect

Tools for databases and their uses

Database design tools

Generating SQL

Summary

6. Client Tools

6. Client Tools

GUI tools and command-line tools

pgAdmin – downloading and installation

psql – working from the command line

Summary

7. SQL Tuning

7. SQL Tuning

Understanding basic facts about databases

Query execution components

Finding the execution plan

Optimization guidelines and catches

Summary

8. Server Tuning

8. Server Tuning

Server-wide memory settings

Managing writes, connections, and maintenance

Seek/scan cost and statistics parameters

Materialized views

Partitioned tables

Summary

9. Tools to Move Data in and out of PostgreSQL

9. Tools to Move Data in and out of PostgreSQL

Setting up the production database – considerations

COPY

Fast loading with pg_bulkload

pg_dump

Filtering options

Summary

10. Scaling, Replication, and Backup and Recovery

10. Scaling, Replication, and Backup and Recovery

Scalability

Point-in-time recovery

Summary

11. PostgreSQL – Troubleshooting

11. PostgreSQL – Troubleshooting

Connection issues

Authentication and permission issues

Parameter changes not effective

Query not responding

Summary

12. PostgreSQL – Extras

12. PostgreSQL – Extras

Interesting data types

Features over time

Summary

Index

Index

Fast loading with pg_bulkload

The pg_bulkload utility is the one that provides a lot more flexibility than the COPY utility. The biggest advantage of pg_bulkload is speed. This utility lets us skip the shared buffers and WAL logging. This also implies a separate recovery process if something goes wrong. This is facilitated by postgresql, a script that comes with the pg_bulkload installation.

We will start with the installation of the utility. To do this, follow these steps:

Download the source with the following command:

wget \
http://pgfoundry.org/frs/download.php/3653/pg_bulkload-\
3.1.6.tar.gz

Extract the contents as follows:
```
tar xvf ./pg_bulkload-3.1.6.tar.gz
```
Change the directory with the following command:
```
cd pg_bulkload-3.1.6
```
Make and install the binaries:
```
make USE_PGXS=1
```
Then, use su with the following command:
```
make USE_PGXS=1 install
```
Install the extension. Log in to the database where we will be using the utility and at the psql prompt:
```
CREATE EXTENSION  pg_bulkload;
CREATE EXTENSION
```

We will see...

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

PostgreSQL for Data Architects

Search

Your notes and bookmarks