Tantor PipelineDB installation

PostgreSQL installation procedure

Since Tantor PipelineDB works as an extension for PostgreSQL, start by installing PostgreSQL.

Note

Currently Tantor PipelineDB supports PostgreSQL 15, 16 on 64-bit architectures including x86_64 and ARM.

After installing PostgreSQL on your system, you need to download and run the db_extension_installer.sh script from our Nexus repository to install the Tantor PipelineDB binaries. After installing the binary files, the Tantor PipelineDB extension shall be created in the PostgreSQL database.

Let’s look at the installation process using db_extension_installer.

Downloading the installer

Start by downloading the installer script from the provided URL:

wget https://public.tantorlabs.ru/db_extension_installer.sh

After the download is complete, grant execution rights to the script:

chmod +x db_extension_installer.sh

Tantor PipelineDB installation

Set the required environment variables before running the installer. These variables provide the installer with the credentials and URL to access the Nexus repository:

export NEXUS_USER="<username>"
export NEXUS_USER_PASSWORD="<password>"
export NEXUS_URL="nexus.tantorlabs.ru"

You can then run the installer with the desired parameters. For example, to run the database after installation, specify the major version and build with the following command:

db_extension_installer.sh \
  --database-type=<PostgreSQL release> \
  --database-major-version=<database version> \
  --edition=<edition, TantorDB only> \
  --extension=<extension name>

To customize the installation process to your needs, the installer script comes with several parameters. You can look through them using the --help option:

db_extension_installer.sh --help

Parameters:

--database-type — specifies PostgreSQL release (pgdg, tantor). The default is tantor.
--edition — if you are using a TantorDB release (tantor), you must specify its edition: be, se, se-1c or certified.
--database-major-version — defines the major version of the database: 15, 16. The default is 15.
--extension — sets the name of the extension. The default is pipelinedb.
--extension-version — sets the version of the extension if a specific version needs to be installed. The default is the latest version.

Example of use:

Here is a use case that demonstrates how to set environment variables and run the installer with certain parameters:

export NEXUS_USER="user_name"
export NEXUS_USER_PASSWORD="user_password"
export NEXUS_URL="nexus.tantorlabs.ru"

db_extension_installer.sh \
  --database-type=tantor \
  --database-major-version=15 \
  --edition=se \
  --extension=pipelinedb

Creating Tantor PipelineDB extension

For Tantor PipelineDB to work, the shared_preload_libraries configuration parameter must be configured in postgresql.conf, which is located in your data directory. It is also recommended to set max_worker_processes to a high enough value to ensure that the Tantor PipelineDB workflows have enough capacity:

# At the bottom of <data directory>/postgresql.conf
shared_preload_libraries = 'pipelinedb'
max_worker_processes = 128

Starting PostgreSQL

To run the PostgreSQL server in the background, use the pg_ctl driver and specify a newly initialized data directory:

pg_ctl -D <data directory> -l postgresql.log start

To connect to a running server, use the standard PostgreSQL client psql, which can be used to create the Tantor PipelineDB extension:

psql -c "CREATE EXTENSION pipelinedb"

Once the extension is created, you can start using Tantor PipelineDB!

You can read the Quick start, and immediately start transferring data to Tantor PipelineDB.

Configuration

By default, PostgreSQL does not accept incoming connections from remote hosts. To allow incoming connections, first configure the listen_addresses line in postgresql.conf:

listen_addresses = '*'

And in pg_hba.conf, add a line similar to the following to allow incoming connections:

host    all             all             <ip address>/<subnet>            md5

For example, to allow incoming connections from any host:

host    all             all             0.0.0.0/0            md5

Removing Tantor PipelineDB

To remove Tantor PipelineDB, use your distribution’s package management system. On Debian-based systems, you can remove Tantor PipelineDB using the following command:

apt-get remove <extension name>-<postgreSQL release>-<edition, TantorDB only>-<database version>

For example, for the above example:

apt-get remove pipelinedb-tantor-se-15