This worklog has been replaced with mariadb.org/jira

This site is here for historical purposes only. Do not add or edit tasks here!

 
 
 

WorkLog Frontpage Log in / Register
High-Level Description | Task Dependencies | High-Level Specification | Low-Level Design | File Attachments | User Comments | Time Estimates | Funding and Votes | Progress Reports

 Make mysqlbinlog not to output unneeded statements when using --database
Title
Task ID38
Queue
Version N/A
Status
PriorityN/A
Copies to
Created byPsergey08 Aug 2009Done
Supervisor N/A  
Lead Architect    
Architecture Review  
Implementor  
Code Review  
QA  
Documentation  
 High-Level Description
This comes from MySQL BUG#23890 and BUG#23894: when one runs 

 mysqlbinlog --database=bar binlog_file 

the produced SQL may contain useless sequences of commands like:

COMMIT; 
COMMIT;
COMMIT;
...

or 

SET INSERT_ID=val1;
SET INSERT_ID=val2;
SET INSERT_ID=val3;
...

This happens because the statements between COMMIT (or SET) statements had no
effect on the specified database and so were filtered out.  COMMIT and SET
statements themselves are not associated with any database and were left in.

Presence of redundant COMMIT or SET statements makes binlog SQL script
unnecessarily big and it will take more client<->server roundtrips to apply it.
 Task Dependencies
Others waiting for Task 38Task 38 is waiting forGraph
 
 High-Level Specification
Monty's suggestions for fix:

A way to fix this for 'most' cases are:

If we do filtering (like mysqlbinlog --database='xxx') then:

- In mysql_bin_log(), do a flush of the Log_event::cache() between
  each statement.
- Log on statement.
- If the statement was ignored (we need a flag for this) and
  there is something in the cache and the file position didn't change
  (the cache didn't overflow), then reset the cache.

Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single
database is used
- Could be fixed by having a flag to mark if something was printed
  to the log since last commit.
 Low-Level Design
 File Attachments
 NameTypeSizeByDate
 User Comments
 Time Estimates
NameHours WorkedLast Updated
Total0 
 Hrs WorkedProgressCurrentOriginal
Total000
 
 Funding and Votes
Votes: 0: 0%
 Make vote: Useless    Nice to have    Important    Very important    

Funding: 0 offers, total 0 Euro
 Progress Reports
(Sergei - Thu, 01 Jul 2010, 06:17
    
Version updated.
--- /tmp/wklog.38.old.29367	2010-07-01 06:17:34.000000000 +0000
+++ /tmp/wklog.38.new.29367	2010-07-01 06:17:34.000000000 +0000
@@ -1 +1 @@
-Benchmarks-3.0
+9.x

(Sergei - Thu, 01 Jul 2010, 06:17
    
Category updated.
--- /tmp/wklog.38.old.29367	2010-07-01 06:17:34.000000000 +0000
+++ /tmp/wklog.38.new.29367	2010-07-01 06:17:34.000000000 +0000
@@ -1 +1 @@
-Server-RawIdeaBin
+Client-RawIdeaBin

(Guest - Thu, 17 Jun 2010, 00:36
    
Dependency deleted: WL#39 no longer depends on WL#38

(Psergey - Sun, 09 Aug 2009, 12:24
    
Dependency created: WL#39 now depends on WL#38

(Guest - Sun, 09 Aug 2009, 12:22
    
High-Level Specification modified.
--- /tmp/wklog.38.old.20756	2009-08-09 12:22:52.000000000 +0300
+++ /tmp/wklog.38.new.20756	2009-08-09 12:22:52.000000000 +0300
@@ -1 +1,18 @@
+Monty's suggestions for fix:
+
+A way to fix this for 'most' cases are:
+
+If we do filtering (like mysqlbinlog --database='xxx') then:
+
+- In mysql_bin_log(), do a flush of the Log_event::cache() between
+  each statement.
+- Log on statement.
+- If the statement was ignored (we need a flag for this) and
+  there is something in the cache and the file position didn't change
+  (the cache didn't overflow), then reset the cache.
+
+Bug #23890 mysqlbinlog outputs COMMIT unnecessarily when single
+database is used
+- Could be fixed by having a flag to mark if something was printed
+  to the log since last commit.
 

(Guest - Sun, 09 Aug 2009, 12:20
    
High Level Description modified.
--- /tmp/wklog.38.old.20618	2009-08-09 12:20:16.000000000 +0300
+++ /tmp/wklog.38.new.20618	2009-08-09 12:20:16.000000000 +0300
@@ -1,10 +1,24 @@
-This comes from MySQL BUG#23890: 
+This comes from MySQL BUG#23890 and BUG#23894: when one runs 
 
-  mysqlbinlog --database=bar N-bin.000003 
+ mysqlbinlog --database=bar binlog_file 
 
-will output all the COMMIT statements in the binary log even if it didn't print
-any statements between the COMMITs (because all the statements that were there
-were for the other databases).
+the produced SQL may contain useless sequences of commands like:
 
-The fix is trivial: in mysqlbinlog, check if we've printed anything after we've
-printed the previous commit statement.
+COMMIT; 
+COMMIT;
+COMMIT;
+...
+
+or 
+
+SET INSERT_ID=val1;
+SET INSERT_ID=val2;
+SET INSERT_ID=val3;
+...
+
+This happens because the statements between COMMIT (or SET) statements had no
+effect on the specified database and so were filtered out.  COMMIT and SET
+statements themselves are not associated with any database and were left in.
+
+Presence of redundant COMMIT or SET statements makes binlog SQL script
+unnecessarily big and it will take more client<->server roundtrips to apply it.

(Guest - Sun, 09 Aug 2009, 12:19
    
Title modified.
--- /tmp/wklog.38.old.20544	2009-08-09 12:19:22.000000000 +0300
+++ /tmp/wklog.38.new.20544	2009-08-09 12:19:22.000000000 +0300
@@ -1 +1 @@
-Make mysqlbinlog not to output unneeded COMMIT statements
+Make mysqlbinlog not to output unneeded statements when using --database


Report Generator:
 
Saved Reports:

WorkLog v4.0.0
  © 2010  Sergei Golubchik and Monty Program AB
  © 2004  Andrew Sweger <yDNA@perlocity.org> and Addnorya
  © 2003  Matt Wagner <matt@mysql.com> and MySQL AB