> For the complete documentation index, see [llms.txt](https://upsolver.gitbook.io/content/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://upsolver.gitbook.io/content/how-to-guides-1/connectors/configure-access/apache-kafka.md).

# Apache Kafka

This article explains how to configure SSL for your Apache Kafka connection in Upsolver, detailing the necessary steps to upload the certificates to the Upsolver servers and configure the connection.

{% hint style="success" %}
The following guide also applies to configuring your **Confluent Cloud** connection.
{% endhint %}

## Configure SSL for Apache Kafka

To enable Upsolver to successfully connect to your Apache Kafka cluster using SSL, the key and certificate files generated when SSL was deployed should be provided as part of your Upsolver cluster configuration. This involves uploading the required certificates to the Upsolver servers by running a patch HTTP request for each cluster you wish to use with your connection.

<details>

<summary>Prerequisites</summary>

Before starting, ensure you have SSL authentication configured within your Kafka cluster.

See [Encryption and authentication with SSL](https://docs.confluent.io/platform/current/kafka/authentication_ssl.html) for more information.

</details>

{% hint style="warning" %}
Note that the instructions below were written for a Linux-based system.

If you are working with Windows, you can use a Linux mimicking program or file a ticket via the [Upsolver support portal](https://support.upsolver.com/hc/en-us) and send the certificates for us to update it for you.
{% endhint %}

### Configure your Upsolver cluster

#### Step 1: Upload your files

There are two request types you can use to upload the files:

1. **`ModifyServerFiles`** - Can be used to upload multiple files at once. Replaces existing files, only the files in the latest upload will be saved, and all previously uploaded files will be removed.
2. **`ModifyServerFile`** - Can be used to upload a single file with each request. Does not remove the previously uploaded files.

The name must be unique: uploading a new file with an existing name will replace the previously uploaded file. This applies to both scripts.

**For Multiple Files**

First, run the following request for the API server:

{% code fullWidth="false" %}

```shell
echo {} | jq '{ clazz: "ModifyServerFiles", serverFiles: [ { name: "kafka.client.keystore.jks", "path": "/opt/kafka.client.keystore.jks", "content": $file1 }, { name: "kafka.client.truststore.jks", "path": "/opt/kafka.client.truststore.jks", "content": $file2 } ]  }' --arg file1 $(cat /<FILE_PATH>/kafka.client.keystore.jks | base64) --arg file2 $(cat /<FILE_PATH>/kafka.client.truststore.jks | base64) |
http PATCH "https://api.upsolver.com/environments/<API_SERVER_ID>/" "Authorization: <API_TOKEN>" "x-user-organization: <ORG_ID>"
```

{% endcode %}

Then run this request for the cluster you wish to upload the files to:

```shell
echo {} | jq '{ clazz: "ModifyServerFiles", serverFiles: [ { name: "kafka.client.keystore.jks", "path": "/opt/kafka.client.keystore.jks", "content": $file1 }, { name: "kafka.client.truststore.jks", "path": "/opt/kafka.client.truststore.jks", "content": $file2 } ]  }' --arg file1 $(cat /<FILE_PATH>/kafka.client.keystore.jks | base64) --arg file2 $(cat /<FILE_PATH>/kafka.client.truststore.jks | base64) |
http PATCH "https://api.upsolver.com/environments/<CLUSTER_ID>/" "Authorization: <API_TOKEN>" "x-user-organization: <ORG_ID>"
```

Note that the two requests only differ in the **IDs** provided within the URLs for each request.

The first line of the request creates the JSON array `serverFiles`, which contains the `path` and `content` of the file you are uploading.

The `path` referenced within the array itself is the path the file is written to within the server; it is also the path that should be provided when using this file to establish a connection.

The `content` of the file is passed through as an argument with `--arg`. Here, `<FILE_PATH>` represents the path to the file you are uploading on your local computer.

{% hint style="info" %}
This example uploads two files to the server, but the `serverFiles` array elements can be adjusted to upload either one or more files. Note that running this request overrides any files that may have been uploaded previously.
{% endhint %}

**For a Single File**

To upload a single file without overriding any existing files, run these requests instead:

```shell
echo {} | jq '{ clazz: "ModifyServerFile", serverFile: { name: "cert.pem", "path": "/opt/cert.pem", "content": $file1 }  }' --arg file1 $(cat ~/Downloads/cert.pem | base64) |
http PATCH "https://api.upsolver.com/environments/<API_SERVER_ID>/" "Authorization: $(cat ~/.upsolver/token)" "X-Api-Impersonate-Organization: <ORG_ID>"
```

```shell
echo {} | jq '{ clazz: "ModifyServerFile", serverFile: { name: "cert.pem", "path": "/opt/cert.pem", "content: $file1 }  }' --arg file1 $(cat ~/Downloads/cert.pem | base64) |
http PATCH "https://api.upsolver.com/environments/<CLUSTER_ID>/" "Authorization: $(cat ~/.upsolver/token)" "X-Api-Impersonate-Organization: <ORG_ID>"
```

{% hint style="info" %}
The name parameter must be unique across different files: using the same name will replace the existing file with that name.
{% endhint %}

Make sure to replace the placeholder values `<API_SERVER_ID>` and `<CLUSTER_ID>`, as well as your `<API_TOKEN>` and `<ORG_ID>`.

To learn how to generate an API token, see [Enable API Integration](/content/how-to-guides-1/setup/enable-api-integration.md).

<details>

<summary>How to find your <code>&#x3C;API_SERVER_ID></code> and <code>&#x3C;CLUSTER_ID></code></summary>

1. Open a new worksheet.
2. Run the query `select * from system.information_schema.clusters;`
3. Lookup the relevant clusters and cluster ids in the results pane.

</details>

<details>

<summary>How to find your <code>&#x3C;ORG_ID></code></summary>

1. Navigate to the **Security Information** page by clicking **Settings > Security Information**.
2. Your org id can be found at the **Trusted entity** as the `sts:ExternalId` value.

</details>

#### Step 2: Roll your cluster

Once the certificates have been uploaded, roll the modified cluster to apply the changes.

<details>

<summary>How to roll a cluster</summary>

1. Open a worksheet.
2. Run the following statement: `roll cluster "cluster_name"`

</details>

### Create your connection

To use your key and certificate files to connect Upsolver to your Kafka cluster, you should provide the paths to your uploaded files as part of the properties in the [`CREATE KAFKA CONNECTION`](/content/reference-1/sql-commands/connections/create-connection/apache-kafka.md) command.

To allow the connection to be used for reading data, the key store and trust store locations should be configured as [`CONSUMER_PROPERTIES`](/content/reference-1/sql-commands/connections/create-connection/apache-kafka.md#consumer_properties-editable).

**Example**

```sql
CREATE KAFKA CONNECTION my_kafka_connection
  HOSTS = ('<bootstrap_server_1>:<port_number>','<bootstrap_server_2>:<port_number>')
  CONSUMER_PROPERTIES = 'security.protocol=SSL
      ssl.truststore.location=/opt/kafka.client.truststore.jks
      ssl.keystore.location=/opt/kafka.client.keystore.jks
      ssl.keystore.password=<PASSWORD>
      ssl.key.password=<PASSWORD>';
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://upsolver.gitbook.io/content/how-to-guides-1/connectors/configure-access/apache-kafka.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.