> For the complete documentation index, see [llms.txt](https://upsolver.gitbook.io/content/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://upsolver.gitbook.io/content/quickstarts-1/data-ingestion-wizard/using-the-wizard/source-set-up/postgresql.md). # PostgreSQL ### Prerequisite - Configure PostgreSQL for CDC Upsolver supports ingesting CDC data from relational databases including PostgreSQL. Upsolver provides CDC capabilities by running a Debezium Engine under the hood. Connectors detect and ingest changes automatically. Before you begin, [please follow the linked guide to prepare PostgreSQL for CDC](/content/how-to-guides-1/connectors/enable-cdc/postgresql.md). ## Step 1 - Connect to PostgreSQL ### Create a new connection Click **Create a new connection**, if it is not already selected. Enter your connection string in the following format: ```markup jdbc:postgresql://HOST:PORT/DB_NAME ``` where: * `HOST` - hostname (e.g. database instance endpoint in AWS RDS) * `PORT` - port number * `DB_NAME` - database name Provide the **Username** and **Password** to authenticate to the database. In the **Name your connection** field, type in the name for this connection. Please note this connection will be available to other users in your organization.

Create a new connection to PostgreSQL to ingest your CDC data.

### Use an existing connection By default, if you have already created a connection, Upsolver selects **Use an existing connection**, and your PostgreSQL connection is populated in the list. For organizations with multiple connections, select the source connection you want to use.

Select your existing PostgreSQL connection as the source for your ingestion job.

## Step 2 - Select schemas, tables, and columns to ingest Upsolver requires the publication of replicated tables in order to subscribe to change events. Select the **Publication Name** created as part of the [PostgreSQL prerequisite setup](/content/how-to-guides-1/connectors/enable-cdc/postgresql.md). Upsolver uses a **Heartbeat Table** to protect your database server from high disk usage. Although this is an optional setting, it is best practice to include a Heartbeat table, especially when the source database does not contain at least one frequently changing table. Select an option from: * **Start without Heartbeat table** * **Use an existing Heartbeat table**: choose the Heartbeat table from the select list * **Create a new table**: select a schema, enter a table name, and click **Create** To learn more, and get the commands to create the Heartbeat table, see the [PostgreSQL prerequisite setup](/content/how-to-guides-1/connectors/enable-cdc/postgresql.md#setting-up-a-heartbeat-table). There are two ways to replicate objects to the target: 1. Manually select the schemas, tables, and columns. 2. Use regular expressions to specify which tables are included and which columns are excluded. The following examples show how to use a regular expression to include specific tables:
ExpressionResults
db_name.*Select all tables in the db_name database
db_name.users, db_name.itemsSelect users and items tables in the db_name database
db1.items_.*Select all tables in the db1 database that start with items_
By default, Upsolver ingests all columns in the tables that match the regular expression. Below are examples of regular expressions to exclude columns from included tables:
ExpressionResults
db.users.address_.*Select all columns starting with address_ in the users table in the db database.
db.*.(.*_pii)Select all columns ending in _pii across all tables in the db database.
{% hint style="info" %} **Note** If a PostgreSQL table does not have a primary key, the ingestion will be insert only. Auto-generated jobs assume that the source table has a primary key that can be used for the `ON` clause, `e.g. source.PK = target.PK`. No error is displayed if a primary key does not exist, so please be aware of this prior to creating your ingestion job. {% endhint %} **Schema evolution** By default, new schemas, tables, and columns added after the job is launched, will be replicated to the target. You can overwrite the default behavior and set specific behavior for newly created schemas, tables in a specific schema, or columns in a specific table. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://upsolver.gitbook.io/content/quickstarts-1/data-ingestion-wizard/using-the-wizard/source-set-up/postgresql.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.