> For the complete documentation index, see [llms.txt](https://upsolver.gitbook.io/content/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://upsolver.gitbook.io/content/release-notes/january-2025.md). # January 2025 ### 2025.01.13-09.11 ⬆️ **Enhancements** * API: Excluded statistics of deleted jobs from table ingested data statistics. * Kafka: Added support for using regex in the `TOPIC` property of Kafka jobs, enclosed in `/regex/` syntax, alongside existing string and glob patterns. * Iceberg: New cleanup task to get rid of unused deleted files * Monitoring - introduced a new system table, `system.monitoring.retention_deletions`, that shows the retention executions of Iceberg tables. * Replication Job: Added support for configuring the primary key column name for each replication group using the new `PRIMARY_KEY_COLUMN` property 🔧 **Bug Fixes** * Iceberg: * Fixed high memory usage in compaction of tables with a large number of position delete files. * Improved performance and reduced memory usage when optimizing and collecting statistics on tables with a large number of files. * Fixed a rare issue where compaction could duplicate data * Fixed a bug where deleted jobs failed to remove intermediate files that were no longer needed. ### 2025.01.08-07.52 ⬆️ **Enhancements** * [Replication Jobs](/content/reference-1/sql-commands/jobs/create-job/replication.md): Newly discovered tables in the source database will be created immediately in the target. Additionally, empty tables and columns within the target table will be created. Currently, this feature is limited to [PostgreSQL](/content/reference-1/sql-commands/jobs/create-job/ingestion/postgresql.md) source tables, and [Snowflake](/content/reference-1/sql-commands/jobs/create-job/transformation/job-options/snowflake.md) and [Iceberg](broken://spaces/WKMq8oT1OPM3KjP8vlg2/pages/c8LdMf4jwBdPmLh3EZoe) target tables. * You can now decide what should be the schema strategy on column type mismatch, using the `ON_COLUMN_TYPE_MISMATCH` key - ignore or add a new column for the new type * Iceberg: Now you can configure Iceberg table properties in replication jobs: * We added support for converting equality deletes to position deletes files, instead of fully re-writing data files during compactions * When the partition field is missing or null, the row will be written into a null partition 🔧 **Bug Fixes** * Hive Tables: Fixed the issue when setting an end time for a sync job to a point earlier than the last processed time and then dropping the job, could cause delays in other jobs reading from the same table. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://upsolver.gitbook.io/content/release-notes/january-2025.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.