Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: insert and mutation progress #17014

Merged
merged 7 commits into from
Dec 10, 2024
Merged

Conversation

zhyass
Copy link
Member

@zhyass zhyass commented Dec 8, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  1. Fixing Write and Scan Progress in Mutation/Compact

  2. Fixing Empty result in Fast Mutation
    Before:

root@localhost:8000/default> delete from t where 1=0;
0 row written in 0.099 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)
  1. Inject Row Count Result After Insert Operations
root@localhost:8000/default> create table t(a int);

CREATE TABLE t (a Int32)

0 row written in 0.071 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

root@localhost:8000/default> insert into t select number from numbers(10);

┌─────────────────────────┐
│ number of rows inserted │
│          UInt64         │
├─────────────────────────┤
│                      10 │
└─────────────────────────┘
10 rows written in 0.085 sec. Processed 10 rows, 42 B (117.65 rows/s, 494 B/s)

root@localhost:8000/default> update t set a=a+1 where a % 2 = 1;

┌────────────────────────┐
│ number of rows updated │
│         UInt64         │
├────────────────────────┤
│                      5 │
└────────────────────────┘
10 rows written in 0.249 sec. Processed 10 rows, 42 B (40.16 rows/s, 168 B/s)

root@localhost:8000/default> delete from t where 1=0;

┌────────────────────────┐
│ number of rows deleted │
│         UInt64         │
├────────────────────────┤
│                      0 │
└────────────────────────┘
0 row written in 0.099 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

root@localhost:8000/default> delete from t where 1;

┌────────────────────────┐
│ number of rows deleted │
│         UInt64         │
├────────────────────────┤
│                     10 │
└────────────────────────┘
0 row written in 0.114 sec. Processed 0 row, 0 B (0 row/s, 0 B/s)

logs:

{"log_type":1,"log_type_name":"Start","query_kind":"Insert","query_text":"INSERT INTO t SELECT number FROM numbers(10)","query_duration_ms":0,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":0,"written_bytes":0,"written_io_bytes":0,"written_io_bytes_cost_ms":0,"scan_rows":0,"scan_bytes":0,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":0,"total_partitions":0,"result_rows":0,"result_bytes":0""}
{"log_type":2,"log_type_name":"Finish","query_kind":"Insert","query_text":"INSERT INTO t SELECT number FROM numbers(10)","query_duration_ms":85,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":10,"written_bytes":42,"written_io_bytes":1125,"written_io_bytes_cost_ms":0,"scan_rows":10,"scan_bytes":80,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":0,"total_partitions":0,"result_rows":0,"result_bytes":0""}
{"log_type":1,"log_type_name":"Start","query_kind":"Update","query_text":"UPDATE t SET a = a + 1 WHERE a % 2 = 1","query_duration_ms":0,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":0,"written_bytes":0,"written_io_bytes":0,"written_io_bytes_cost_ms":0,"scan_rows":0,"scan_bytes":0,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":0,"total_partitions":0,"result_rows":0,"result_bytes":0""}
{"log_type":2,"log_type_name":"Finish","query_kind":"Update","query_text":"UPDATE t SET a = a + 1 WHERE a % 2 = 1","query_duration_ms":249,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":10,"written_bytes":42,"written_io_bytes":0,"written_io_bytes_cost_ms":0,"scan_rows":10,"scan_bytes":42,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":1,"total_partitions":1,"result_rows":0,"result_bytes":0""}
{"log_type":1,"log_type_name":"Start","query_kind":"Update","query_text":"DELETE FROM t WHERE 1 = 0","query_duration_ms":0,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":0,"written_bytes":0,"written_io_bytes":0,"written_io_bytes_cost_ms":0,"scan_rows":0,"scan_bytes":0,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":0,"total_partitions":0,"result_rows":0,"result_bytes":0""}
{"log_type":2,"log_type_name":"Finish","query_kind":"Update","query_text":"DELETE FROM t WHERE 1 = 0","query_duration_ms":99,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":0,"written_bytes":0,"written_io_bytes":0,"written_io_bytes_cost_ms":0,"scan_rows":0,"scan_bytes":0,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":0,"total_partitions":0,"result_rows":0,"result_bytes":0""}
{"log_type":1,"log_type_name":"Start","query_kind":"Update","query_text":"DELETE FROM t WHERE 1","query_duration_ms":0,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":0,"written_bytes":0,"written_io_bytes":0,"written_io_bytes_cost_ms":0,"scan_rows":0,"scan_bytes":0,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":0,"total_partitions":0,"result_rows":0,"result_bytes":0""}
{"log_type":2,"log_type_name":"Finish","query_kind":"Update","query_text":"DELETE FROM t WHERE 1","query_duration_ms":114,"query_queued_duration_ms":0,"current_database":"default","databases":"","tables":"","columns":"","projections":"","written_rows":0,"written_bytes":0,"written_io_bytes":514,"written_io_bytes_cost_ms":0,"scan_rows":0,"scan_bytes":0,"scan_io_bytes":0,"scan_io_bytes_cost_ms":0,"scan_partitions":0,"total_partitions":0,"result_rows":0,"result_bytes":0""}

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@zhyass zhyass marked this pull request as draft December 8, 2024 05:41
@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Dec 8, 2024
@zhyass zhyass changed the title fix: insert and mutation write progress fix: insert and mutation progress Dec 9, 2024
@zhyass zhyass added the A-cloud Area: cloud infra label Dec 9, 2024
@zhyass zhyass marked this pull request as ready for review December 9, 2024 12:45
@zhyass zhyass requested a review from youngsofun December 9, 2024 13:19
@dantengsky
Copy link
Member

LGTM.

The INSERT statement now always returns a result. I'm not sure if this could potentially break anything. @hantmac @youngsofun @sundy-li please take a look

Copy link
Member

@hantmac hantmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dantengsky dantengsky added this pull request to the merge queue Dec 10, 2024
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Dec 10, 2024
@BohuTANG BohuTANG merged commit ece1166 into databendlabs:main Dec 10, 2024
76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cloud Area: cloud infra pr-bugfix this PR patches a bug in codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: update sql did not have read bytes and read rows progress
4 participants