> For the complete documentation index, see [llms.txt](https://upsolver.gitbook.io/content/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://upsolver.gitbook.io/content/reference-1/sql-commands/jobs/create-job/transformation/job-options/snowflake.md).

# Snowflake

## Job options

```sql
[ ADD_MISSING_COLUMNS = { TRUE | FALSE } ]
[ AGGREGATION_PARALLELISM = <integer> ]
[ COMMENT = '<comment>' ]
[ COMMIT_INTERVAL = <integer> { MINUTE[S] | HOUR[S] | DAY[S] } ]
[ COMPUTE_CLUSTER = <cluster_identifier> ]
[ CREATE_TABLE_IF_MISSING = { TRUE | FALSE } ]
[ CUSTOM_INSERT_EXPRESSIONS = (<column_name> = '<expression>', ...) ]
[ CUSTOM_UPDATE_EXPRESSIONS = (<column_name> = '<expression>', ...) ]
[ END_AT = { NOW | timestamp } ]
[ KEEP_EXISTING_VALUES_WHEN_NULL = { TRUE | FALSE } ]
[ RUN_INTERVAL = <integer> { MINUTE[S] | HOUR[S] | DAY[S] } ]
[ RUN_PARALLELISM = <integer> ]
[ START_FROM = { NOW | BEGINNING | timestamp } ]
```

#### Jump to

Snowflake job options:

* [`ADD_MISSING_COLUMNS`](#add_missing_columns)
* [`ON_COLUMN_TYPE_MISMATCH`](#on_column_type_mismatch)
* [`AGGREGATION_PARALLELISM`](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options.md#aggregation_parallelism-editable)
* [`COMMIT_INTERVAL`](#commit_interval)
* [`CREATE_TABLE_IF_MISSING`](#create_table_if_missing)
* [`CUSTOM_INSERT_EXPRESSIONS`](#custom_insert_expressions-editable)
* [`CUSTOM_UPDATE_EXPRESSIONS`](#custom_update_expressions)
* [`KEEP_EXISTING_VALUES_WHEN_NULL`](#keep_existing_values_when_null-editable)
* [`RUN_PARALLELISM`](#run_parallelism-editable)

General job options:

* [`COMMENT`](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options.md#comment-editable)
* [`COMPUTE_CLUSTER`](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options.md#compute_cluster-editable)
* [`END_AT`](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options.md#end_at-editable)
* [`RUN_INTERVAL`](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options.md#run_interval)
* [`START_FROM`](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options.md#start_from)

#### `ADD_MISSING_COLUMNS`

Type: `Boolean`

Default: `false`

(Optional) When `true`, columns that don't exist in the target table are added automatically when encountered.

When `false`, you cannot do `SELECT *` within the `SELECT` statement of your transformation job.

#### `ON_COLUMN_TYPE_MISMATCH`

Type: `String`

Default: `None`

Possible values:  `ADD_COLUMN`,`NONE`

This option is applicable only if `ADD_MISSING_COLUMNS=TRUE`.\
It determines how to handle cases where the datatype of the source data does not match the datatype of the corresponding column in the target table.

If set to `ADD_COLUMN`, Upsolver will attempt to cast the incoming data to the original column's datatype. If the cast fails, a new column with the format \<originalColumnName>\_newDataType will be created, and the mismatched data will be written to this new column. For example, if the CLIENT\_ID column is a number in the target table and a VARCHAR arrives in that column, a new column called CLIENT\_ID\_VARCHAR will be added for the string data. The original column will continue to be populated if the data can be cast successfully.

If set to `NONE`, no new columns will be added, and only valid casts will be written to the original column.

#### `AGGREGATION_PARALLELISM` — editable

Type: `integer`

Default: `1`

(Optional) Only supported when the query contains aggregations. Formally known as "output sharding."

#### `COMMIT_INTERVAL`

Type: `<integer> { MINUTE[S] | HOUR[S] | DAY[S] }`

Default: [`RUN_INTERVAL`](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options.md#run_interval)

(Optional) Defines how often the job will commit to Snowflake. If empty, the `RUN_INTERVAL` value will be used.&#x20;

The `COMMIT_INTERVAL` value must be bigger and divisible by `RUN_INTERVAL`.

#### `CREATE_TABLE_IF_MISSING`&#x20;

Type: Boolean&#x20;

Default: `false`&#x20;

(Optional) When true, the Snowflake table will be created automatically, otherwise the table should already exist.

#### `CUSTOM_INSERT_EXPRESSIONS`

Type: `array[(column, expression)]`

Default: `()`

(Optional) Configure a list of custom expression transformations to apply to the value of each column when **inserting unmatched (new) rows**. Note this is only used in [MERGE](/content/reference-1/sql-commands/jobs/create-job/transformation/merge.md) jobs.

Note:&#x20;

1. You can use `{}` as a placeholder for the `SELECT` statement's mapped value.
2. The expressions will not create the target columns in the table. The columns should either exist in the target table or be mapped in the `SELECT` clause, for example `SELECT timestamp '1970-01-01 00:00:00' as INSERT_TIME`.

**Example:**

The following example will set the **INSERT\_TIME** column of the target table to the value of `CURRENT_TIMESTAMP()` and will add `1` to the value of `some_value` when inserting new rows:

```sql
CREATE SYNC JOB my_merge_job
    START_FROM = BEGINNING
    RUN_INTERVAL = 1 MINUTE
    CUSTOM_INSERT_EXPRESSIONS = (INSERT_TIME = 'CURRENT_TIMESTAMP()', 
                                MY_VALUE = '{} + 1')
 AS MERGE INTO SNOWFLAKE snow.PUBLIC.MY_TABLE AS target       
    USING (SELECT id AS ID, some_value AS MY_VALUE
	  FROM glue.staging.my_staging_table
	  WHERE $event_time BETWEEN run_start_time() AND run_end_time()) AS source
     ON (target.ID = source.ID)
     WHEN MATCHED THEN REPLACE
     WHEN NOT MATCHED THEN INSERT MAP_COLUMNS_BY_NAME;
```

#### `CUSTOM_UPDATE_EXPRESSIONS`

Type: `array[(column, expression)]`

Default: `()`

(Optional) Configure a list of custom expression transformations to apply to the value of each column when **updating matched rows**. Note this is only used in [MERGE](/content/reference-1/sql-commands/jobs/create-job/transformation/merge.md) jobs.

Notes:&#x20;

1. You can use `{}` as a placeholder for the mapped value from the `SELECT` statement or `{current_value}` as a placeholder for the existing value in the table.
2. The expressions will not create the target columns in the table. The columns should either exist in the target table or be mapped in the `SELECT` clause, for example `SELECT timestamp '1970-01-01 00:00:00' as UPDATE_TIME`.

**Example:**

The following example will set the **UPDATE\_TIME** column of the target table to the value of `CURRENT_TIMESTAMP()` and will add `1` to the value of `some_value` when updating existing rows:

```sql
CREATE SYNC JOB my_merge_job
    START_FROM = BEGINNING
    RUN_INTERVAL = 1 MINUTE
    CUSTOM_UPDATED_EXPRESSIONS = (UPDATE_TIME = 'CURRENT_TIMESTAMP()', 
                                  MY_VALUE = '{} + 1',
                                  INSERT_TIME = '{current_value}')
 AS MERGE INTO SNOWFLAKE snow.PUBLIC.MY_TABLE AS target       
    USING (SELECT id AS ID, some_value AS MY_VALUE
	  FROM glue.staging.my_staging_table
	  WHERE $event_time BETWEEN run_start_time() AND run_end_time()) AS source
     ON (target.ID = source.ID)
     WHEN MATCHED THEN REPLACE
     WHEN NOT MATCHED THEN INSERT MAP_COLUMNS_BY_NAME;
```

#### `KEEP_EXISTING_VALUES_WHEN_NULL`

Type: `Boolean`

Default: `false`

(Optional) If enabled, updates to the table preserve the previous non-null value. This option is useful if your update events only contain values for modified columns. This works by coalescing the new value with the existing value. If the new value is null the previous value will be preserved. This means that updating values to `null` is not supported.&#x20;

#### `RUN_PARALLELISM` — editable

Type: `integer`

Default: `1`

(Optional) Controls how many jobs run in parallel to process a single minute of data from the source table.&#x20;

Increasing this can lower the end-to-end latency if you have lots of data per minute.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://upsolver.gitbook.io/content/reference-1/sql-commands/jobs/create-job/transformation/job-options/snowflake.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.