> For the complete documentation index, see [llms.txt](https://upsolver.gitbook.io/content/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://upsolver.gitbook.io/content/release-notes/earlier-releases/2024/june-2024.md). # June 2024 {% hint style="info" %} **Release Notes Blog** For more detailed information on these updates, check out the [Upsolver July 2024 Feature Summary](https://www.upsolver.com/blog/upsolver-july-2024-feature-summary) blog. {% endhint %} ### 2024.06.30-08.33 ⬆️ **Enhancements** * Apache Iceberg: * Added support for creating Iceberg tables with hidden partitions using partition transforms * Added support for setting retention on partitions with type Long/Integer, enabling retention on hidden partitions that use time truncate transformations such as year/month/day/hour * Amazon S3: Reduced the number of upload file part requests by using larger parts * [Amazon Redshift](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options/amazon-redshift.md) Targets: Added `TRUNCATE_COLUMNS` property to truncate VARCHAR or CHARACTER VARYING fields by the target column size when inserting into an existing table 🔧 **Bug Fixes** * \[BREAKING CHANGE]: Removed `$commit_time` column from all system tables * Amazon S3: Fixed cases where incomplete multipart upload files were not cleaned up * Apache Iceberg: Schema evolution was assumed for new fields with a suffix and type that match other fields * Minor bug fixes ### 2024.06.20-15.35 ⬆️ **Enhancements** * Apache Iceberg: * [`CREATE EXTERNAL ICEBERG TABLE`](/content/reference-1/sql-commands/iceberg-tables/external-iceberg-tables/create-external-iceberg-table.md) statement: * Changed the default value for the `OPTIMIZE` property to `FALSE` when creating external Iceberg tables * [`CREATE ICEBERG TABLE`](/content/reference-1/sql-commands/iceberg-tables/upsolver-managed-tables/create-iceberg-table.md) statement: * Allow the setting Iceberg properties using `iceberg.property.name` syntax * Removed the `COLUMN_DATA_RETENTION` property when creating Iceberg tables, as this property is only supported in Upsolver Managed Hive tables * Implemented deletion of temporary intermediate files in [ingestion jobs](/content/reference-1/sql-commands/jobs/create-job/ingestion.md) after their usage. This feature will be enabled gradually * [Ingestion wizard](/content/quickstarts-1/data-ingestion-wizard/using-the-wizard.md): * When creating [replication (CDC) jobs](/content/quickstarts-1/data-ingestion-wizard/using-the-wizard/job-configuration/job-configuration-for-cdc.md) via the wizard, we have now added the ability to map selected source schemas to target schemas automatically. If target schemas do not already exist, they will be created automatically. The source and target schemas will be identical. This option was previously available only via SQL. In addition, we have improved the experience when configuring source to schema mapping 🔧 **Bug Fixes** * Minor bug fixes ### 2024.06.10-17.20 ⬆️ **Enhancements** * Query Iceberg Tables - Customers can now query Iceberg tables using Upsolver's Query Engine for preview * Auto Sharding: Output files can now be split across shards within a partition * Improved the compaction speed for sorted Iceberg tables 🔧 **Bug Fixes** * Fixed a bug that caused jobs writing to Iceberg tables not to produce any rows when the job's source was a system table * Fixed Iceberg creating dangling files during failed compactions * CDC Inputs: Implemented deletion of intermediate files that store the metadata of the source database * Fixed the shard number in [system.monitoring.recent\_compactions](/content/reference-1/monitoring/system-catalog/monitoring/recent-compactions.md) * External Iceberg Tables: Added support for data retention on external Iceberg tables *** ### 2024.06.04-11.16 ⬆️ **Enhancements** * [Amazon Redshift](/content/reference-1/sql-commands/jobs/create-job/replication.md#commit_interval) outputs now support custom commit interval 🔧 **Bug Fixes** * Non-Sync jobs reading from Apache Iceberg with an interval that is larger than 1 minute did not read all data from the source table * Jobs writing to Hive tables with an interval equal to or larger than 7 days could get stuck unless extra configuration was done by Upsolver support * Fixed a bug that prevented a job writing to a table with an end time from being marked as **Completed** when the end time was reached *** ### 2024.06.02-15.50 #### :sparkles: New Features * [Iceberg Table Mirrors](/content/how-to-guides-1/data/query-upsolver-iceberg-tables-from-snowflake.md) * Create mirrors of your Iceberg tables in Snowflake. Maintains an external Iceberg table in Snowflake that Upsolver keeps up-to-date by pointing to the latest snapshot of the table. * [Data Ingestion Wizard](/content/quickstarts-1/data-ingestion-wizard/using-the-wizard.md) * Support direct ingestion to [Amazon Redshift](/content/quickstarts-1/data-ingestion-wizard/using-the-wizard/target-set-up/amazon-redshift.md) 🔧 **Bug Fixes** * Minor bug fixes --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://upsolver.gitbook.io/content/release-notes/earlier-releases/2024/june-2024.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.