> For the complete documentation index, see [llms.txt](https://upsolver.gitbook.io/content/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://upsolver.gitbook.io/content/reference-1/monitoring/job-status/stream-and-file-sources/monitoring-v1.md). # Monitoring (V1) ## Metric groups Some metrics may not apply to the job type you created or only be relevant to particular data sources or target destinations. * **Summary**: Displays the most relevant metrics to your job. * **All**: All metrics applicable to your job. Job metrics are categorized into three groups: 1. [Job Execution Status](#job-execution-status) 2. [Data Scanned](#data-scanned) 3. [Cluster](#cluster) ## Job Execution Status The Job Execution Status metrics include all jobs running on the cluster:

Metric	Description
Job executions currently running	The number of currently running job executions.
Job executions currently in queue	The number of queued job executions pending.
Job executions completed - today	The number of job executions completed today.
Job executions completed - lifetime	The number of job executions completed over the lifetime of the job.
Job executions currently waiting for dependencies	The number of job executions that are waiting for a dependency to complete.
Job executions currently retrying after failure	The number of job executions that encountered an error and are currently retrying.
Execution Failure Reason	The error message detailing why the job failed.

## Data Scanned The following metrics provide monitoring information regarding the data processed by your job:

Metric	Description
Rows read (completed executions)	The total number of rows scanned by completed executions today. This is a measure of rows that were processed successfully.
Rows filtered by WHERE clause	The number of rows that were filtered out because they didn’t pass the WHERE clause predicate defined in the job.
Average rows scanned per execution	The average number of rows scanned per job execution.
Maximum rows scanned in an execution	The maximum number of rows scanned in a single job execution today.
Rows Pending Processing	The number of rows in the source table that have not been processed yet.
Discovered Files	The number of files to load discovered by the job.
Discovered Bytes	The number of bytes to load discovered in the source stream.
Parse Errors (for ingestion jobs)	The number of items that failed to parse. This value represents a lower bound as malformed items may corrupt subsequent items in the same file as well.
Rows written (completed executions)	The number of rows written to the target by the job.
Rows filtered by HAVING clause	The number of rows that were filtered out because they did not pass the HAVING clause predicate defined in the job.
Rows filtered due to missing partition	The number of rows that were filtered out because some or all of the partition columns were NULL or empty string.
Rows filtered due to missing Primary Key	The number of rows that were filtered out because some or all of the primary key columns were NULL.
Bytes written (completed executions)	The size of the data written by the job.
Columns written	The number of columns written to by the job. This value can change over time if the query uses * in the SELECT clause.
Columns written - sparse	The number of sparse columns written to today. A sparse column is a column that appears in less than 0.01% of all rows.

## Cluster The following metrics provide information about the cluster running your job:

Metric	Description
Utilization Percent	Represents how much of the server's processing capacity is in use.
Tasks in Queue	The number of job tasks pending execution in the cluster queue.
Memory Load Index (%GC)	The percentage of time that the server is doing garbage collection rather than working.
Reload from disk percent	The percent of bytes re-loaded into memory from disk.
Cluster Crashes	How many server crashes happened in the job’s cluster today.

--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://upsolver.gitbook.io/content/reference-1/monitoring/job-status/stream-and-file-sources/monitoring-v1.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.