PostgreSQL is a free and open-source relational database management system (RDBMS) emphasizing extensibility and SQL compliance.
In a YAML file, the config
section contains the following properties:
Connectorname: PostgreSQL
host: Hostname or IP address of the server
port: Server running port
username: Username
password: Password
database: Database
drivername: postgresql+pg8000
In the select
section, specify the table name list to load tables from the PostgreSQL server.
In the metadata
section, define the mode of data refresh. There are two modes: INCREMENTAL
and FULL_TABLE
. It only supports Date/DateTime datatype columns.
INCREMENTAL |
|
This mode fetches data from the date column mentioned in the replication key from the start date as mentioned in the replication value. Once it is scheduled, the replication value is updated automatically from the imported data. |
FULL_TABLE |
|
This mode fetches data from the date column specified in the replication key starting from the date specified in the replication value. Once scheduled, the replication value is updated according to the interval_type and interval_value from the imported data. For example, if the interval_type is set to 'year' and the interval_value is set to '1', the first schedule will fetch records from January 1, 2000 to December 31, 2000. In the next schedule, it will fetch records from January 1, 2001 to December 31, 2001, and so on. |
version: 1
encrypt_credentials: false
plugins:
extractors:
- name: tap_postgres
connectorname: PostgreSQL
config:
host: <HOSTNAME>
port: <PORT>
username: <USERNAME>
database: <DATABASENAME>
password: <PASSWORD>
drivername: postgresql+pg8000
select:
- TABLE1
- TABLE2
metadata:
TABLE1:
replication_method: FULL_TABLE
replication_key: last_modified_on
replication_value: 2023-07-19 00:00:00
interval_type: days
interval_value: 6
TABLE2:
replication_method: FULL_TABLE
replication_key: last_modified_on
replication_value: 2023-07-19 00:00:00
interval_type: days
interval_value: 6
version: 1
encrypt_credentials: false
plugins:
extractors:
- name: tap_postgres
connectorname: PostgreSQL
config:
host: <HOSTNAME>
port: <PORT>
username: <USERNAME>
database: <DATABASENAME>
password: <PASSWORD>
drivername: postgresql+pg8000
select:
- TABLE1
- TABLE2
metadata:
TABLE1:
replication_method: INCREMENTAL
replication_key: last_modified_on
replication_value: 2023-07-19 00:00:00
TABLE2:
replication_method: INCREMENTAL
replication_key: last_modified_on
replication_value: 2023-07-19 00:00:00