This worklog has been replaced with mariadb.org/jira

This site is here for historical purposes only. Do not add or edit tasks here!

 
 
 

WorkLog Frontpage Log in / Register
High-Level Description | Task Dependencies | High-Level Specification | Low-Level Design | File Attachments | User Comments | Time Estimates | Funding and Votes | Progress Reports

 optional nonblocking client API
Title
Task ID192
Queue
Version
Status
Priority
Copies toSergei

Created byMdcallag30 Mar 2011Done
Supervisor   
Lead Architect    
Architecture Review  
Implementor 22 Sep 2011
Code Review 22 Sep 2011
QA  
Documentation  
 High-Level Description
We want some of the calls in libmysqlclient to optionally be non-blocking so it 
can be used with libevent as described by http://jan.kneschke.de/2008/9/9/async-
mysql-queries-with-c-api. We prefer to extend libmysqlclient rather than replace 
it.

Is this possible?

=======================================================================

Below is user documentation for this feature.


Non-blocking client library
---------------------------

MariaDB from version 5.5.21 support _non-bloking_ operation in the
client-library. This allow an application to start a query or other operation
against the database, and then continue to do other work (in the same thread)
while the request is sent over the network, query is processed in the server,
and result travels back. As parts of the result become ready, the application
can - at its leisure - call back into the library to continue processing,
repeating this until the operation is completed.

Non-blocking operation is entirely implemented in the client library. This
means that no special server support is necessary; non-blocking operation
works with any version of the MariaDB or MySQL server, same as the normal
blocking API. It also means that it is not possible to have two queries
running at the same time on the same connection (this is a protocol
limitation). But a single thread can have any number of non-blocking queries
running at the same time, each using its own MYSQL connection object.

Non-blocking operation is useful when an application needs to run a number of
independent queries in parallel at the same time, to speed up operation
compared to running then sequentially one after the other. This could be
multiple queries against a single server (to better utilise multiple CPU cores
and/or high-capacity I/O system on the server), or it could be queries against
multiple servers (eg. SHOW STATUS against all running servers for monitoring,
or map/reduce-like operation against a big sharded database).

Non-blocking operation is also very useful in applications that are already
written in a non-blocking style, for example using a framework like libevent
(http://libevent.org/), or eg. a GUI-application using an event loop. Using
the non-blocking client library allows to integrate database queries into such
applications, without risking that long-running queries will "hang" the user
interface or stall the event loop, and without having to manually spawn
separate threads to run the queries and re-synchronise with the threads to get
the results back.

In this context, "blocking" means the situation where communication on the
network socket to the server has to wait while processing the query. Waiting
can be necessary because the server has not yet had time to process the query,
or because we must wait for the data to travel over the network from the
server, or even because we need to wait for the first part of a large request
to be sent out on the network before local socket buffers can accept the last
part. Whenever such wait is necessary, control returns to the application. The
application will then run select() or poll() or similar to detect when any
wait condition is satisfied, and then call back into the library to continue
processing.

An example program is available in the MariaDB source tree:

    tests/async_queries.c

It uses libevent to run a set of queries in parallel from within a single
thread / event loop. This is a good example for how to integrate non-blocking
query processing into an event-based framework.

The non-blocking API in the client library is entirely optional. The new
library is completely ABI- and source-compatible with existing
applications. Applications not using non-blocking operations are not affected,
nor is there any significant performance penalty for having support for
non-blocking operations in the library for applications that do not use them.


Using the non-blocking library
------------------------------

The MariaDB non-blocking client API is modelled after the normal blocking
library calls. This makes it easy to learn/remember, makes it easier
to translate code using the blocking API to do non-blocking operation (or vice
versa), and also makes it simple to mix blocking and non-blocking calls in the
same code path.

For every library call that may block on socket I/O, such as
int mysql_real_query(MYSQL, query, query_length), two additional non-blocking calls
are introduced:

    int mysql_real_query_start(&status, MYSQL, query, query_length)
    int mysql_real_query_start(&status, MYSQL, wait_status)

To do non-blocking operation, an application first calls
mysql_real_query_start() instead of mysql_real_query(), passing the same
parameters.

If mysql_real_query_start() returns zero, then the operation completed without
blocking, and `status' is set to the value that would normally be returned
from mysql_real_query().

Else, the return value from mysql_real_query_start() is a bitmask of events
that the library is waiting on. This can be MYSQL_WAIT_READ / MYSQL_WAIT_WRITE
/ MYSQL_WAIT_EXCEPT, corresponding to the similar flags for select() or
poll(); and it can include MYSQL_WAIT_TIMEOUT when waiting for a timeout to
occur (eg. connection timeout).

In this case, the application continues other processing and eventually checks
for the appropriate condition(s) to occur on the socket (or for timeout). When
this occurs, the application can resume the operation by calling
mysql_real_query_cont(), passing in `wait_status' a bitmask of the events that
actually occured.

Just like mysql_real_query_start(), mysql_real_query_cont() returns zero when
done, or bitmask of events it needs to wait on. Thus the application continues
to repeatedly call mysql_real_query_cont(), intermixed with other processing
of its choice; until zero is returned, after which the result of the operation
is stored in `status'.

Some calls, like mysql_option(), do not do any socket I/O, and so can never
block. For these, there are no separate _start() or _cont() calls. See the
section "Non-blocking API reference" below for a full list of what functions
can and can not block.

The checking for events on the socket / timeout can be done with select() or
poll() or similar mechanism. Though often it will be done using a higher-level
framework (such as libevent), which supplies facilities for registering and
acting on such conditions.

The descriptor of the socket on which to check for events can be obtained by
calling mysql_get_socket(). The duration of any timeout can be obtained from
mysql_get_timeout_value().

Here is a trivial but full example of running a query with the non-blocking
API. The example is found in the MariaDB source tree as
client/async_example.c. (A larger, more realistic example using libevent is
found as tests/async_queries.c in the source):

  static void run_query(const char *host, const char *user, const char *password) {
    int err, status;
    MYSQL mysql, *ret;
    MYSQL_RES *res;
    MYSQL_ROW row;

    mysql_init(&mysql);
    mysql_options(&mysql, MYSQL_OPT_NONBLOCK, 0);

    status = mysql_real_connect_start(&ret, &mysql, host, user, password, NULL,
0, NULL, 0);
    while (status) {
      status = wait_for_mysql(&mysql, status);
      status = mysql_real_connect_cont(&ret, &mysql, status);
    }

    if (!ret)
      fatal(&mysql, "Failed to mysql_real_connect()");

    status = mysql_real_query_start(&err, &mysql, SL("SHOW STATUS"));
    while (status) {
      status = wait_for_mysql(&mysql, status);
      status = mysql_real_query_cont(&err, &mysql, status);
    }
    if (err)
      fatal(&mysql, "mysql_real_query() returns error");

    /* This method cannot block. */
    res= mysql_use_result(&mysql);
    if (!res)
      fatal(&mysql, "mysql_use_result() returns error");

    for (;;) {
      status= mysql_fetch_row_start(&row, res);
      while (status) {
        status= wait_for_mysql(&mysql, status);
        status= mysql_fetch_row_cont(&row, res, status);
      }
      if (!row)
        break;
      printf("%s: %s\n", row[0], row[1]);
    }
    if (mysql_errno(&mysql))
      fatal(&mysql, "Got error while retrieving rows");
    mysql_free_result(res);
    mysql_close(&mysql);
  }

  /* Helper function to do the waiting for events on the socket. */
  static int wait_for_mysql(MYSQL *mysql, int status) {
    struct pollfd pfd;
    int timeout, res;

    pfd.fd = mysql_get_socket(mysql);
    pfd.events =
      (status & MYSQL_WAIT_READ ? POLLIN : 0) |
      (status & MYSQL_WAIT_WRITE ? POLLOUT : 0) |
      (status & MYSQL_WAIT_EXCEPT ? POLLPRI : 0);
    if (status & MYSQL_WAIT_TIMEOUT)
      timeout = 1000*mysql_get_timeout_value(mysql);
    else
      timeout = -1;
    res = poll(&pfd, 1, timeout);
    if (res == 0)
      return MYSQL_WAIT_TIMEOUT;
    else if (res < 0)
      return MYSQL_WAIT_TIMEOUT;
    else {
      int status = 0;
      if (pfd.revents & POLLIN) status |= MYSQL_WAIT_READ;
      if (pfd.revents & POLLOUT) status |= MYSQL_WAIT_WRITE;
      if (pfd.revents & POLLPRI) status |= MYSQL_WAIT_EXCEPT;
      return status;
    }
  }


Setting MYSQL_OPT_NONBLOCK
--------------------------

Before using any non-blocking operation, it is necessary to enable it first
by setting the MYSQL_OPT_NONBLOCK option:

    mysql_options(&mysql, MYSQL_OPT_NONBLOCK, 0);

This call can be made at any time - typically it will be done at the start,
before mysql_real_connect(), but it can be done at any time to start using
non-blocking operations.

If a non-blocking operation is attempted without setting the
MYSQL_OPT_NONBLOCK option, the program will typically crash with a NULL
pointer exception.

The argument for MYSQL_OPT_NONBLOCK is the size of the stack used to save the
state of a non-blocking operation while it is waiting for I/O and the
application is doing other processing. Normally, applications will not have to
change this, and it can be passed as zero to use the default value.


Mixing blocking and non-blocking operation
------------------------------------------

It is possible to freely mix blocking and non-blocking calls on the same MYSQL
connection.

Thus, an application can do a normal blocking mysql_real_connect() and
subsequently do a non-blocking mysql_real_query_start(). Or vice versa, do a
non-blocking mysql_real_connect_start(), and later do a blocking
mysql_real_query() on the resulting connection.

Mixing can be useful to allow code to use the simpler blocking API in parts of
the program where waiting is not a problem. For example establishing the
connection(s) at program startup, or doing small quick queries between large,
long-running ones.

The only restriction is that any previous non-blocking operation must have
finished before starting a new blocking (or non-blocking) operation, see
"Terminating a non-blocking operation early" below.


Terminating a non-blocking operation early
------------------------------------------

When a non-blocking operation is started with mysql_real_query_start() or
other _start() function, it must be allowed to finish before starting a new
operation. Thus, the application must continue calling mysql_real_query_cont()
until zero is returned, indicating that the operation is completed. It is not
allowed to leave one operation "hanging" in the middle of processing and then
start a new one on top of it.

It is, however, permissible to terminate the connection completely with
mysql_close() in the middle of processing a non-blocking call. A new
connection must then be initiated with mysql_real_connect before new queries
can be run, either with a new MYSQL object or re-using the old one.

In the future, we may implement an abort facility to force an on-going
operation to terminate as quickly as possible (but it will still be necessary
to call mysql_real_query_cont() one last time after abort, allowing it to
clean up the operation and return immediately with an appropriate error code).


Restrictions
------------

When mysql_real_connect_start() is passed a hostname (as opposed to a local
unix socket or an IP address, it may need to look up the hostname in DNS,
depending on local host configuration (ie. if the name is not in /etc/hosts or
cached). Such DNS lookup does _not_ happen in a non-blocking way. This means
that mysql_real_connect_start() will not return control to the application
while waiting for the DNS response. Thus the application may "hang" for some
time if DNS is slow or non-functional.

If this is a problem, the application can pass an IP address to
mysql_real_connect_start() instead of a hostname, which avoids the
problem. The IP address can be obtained by the application with whatever
non-blocking DNS loopup operation is available to it from the operating system
or event framework used. Alternatively, a simple solution may be to just add
the hostname to the local host lookup file (/etc/hosts on Posix/Unix/Linux
machines).

There is no support in the non-blocking API for connections using Windows
named pipes.

Named pipes can still be used, using either the blocking or the non-blocking
API. However, operations that need to wait on I/O on the named pipe will not
return control to the application; instead they will "hang" waiting for the
operation to complete, just like the normal blocking API calls.


Non-blocking API reference
--------------------------

Here is a list of all functions in the non-blocking client API and their
parameters. Apart from operating in the non-blocking way, they all work
exactly the same as their non-blocking counterpart, so their exact semantics
can be obtained from the documentation of the normal client API.

The API contains additionally the following two functions used to get the
socket fd and timeout value when waiting for events to occur:

  my_socket mysql_get_socket(const MYSQL *mysql)

  Return the descriptor of the socket used for the connection.


  unsigned int STDCALL mysql_get_timeout_value(const MYSQL *mysql)

  This should only be called when a _start() or _cont() function returns a
  value with the MYSQL_WAIT_TIMEOUT flag set. In this case, it returns the
  value, in seconds, after which a timeout has occured and the application
  should call the appropriate _cont() function passing MYSQL_WAIT_TIMEOUT as
  the event that occured.

  This is used to handle connection and read timeouts.


At the end is a list of all functions from the normal API that can be used
safely in a non-blocking program, since they never need to block.


int mysql_real_connect_start(MYSQL **ret, MYSQL *mysql, const char *host,
                         const char *user, const char *passwd, const char *db,
                         unsigned int port, const char *unix_socket,
                         unsigned long client_flags)
int mysql_real_connect_cont(MYSQL **ret, MYSQL *mysql, int ready_status)

mysql_real_connect_start() initiates a non-blocking connection request to a
server.

When mysql_real_connect_start() or mysql_real_connect_cont() returns zero, a
copy of the passed `mysql' argument is stored in *ret.


int mysql_real_query_start(int *ret, MYSQL *mysql, const char *stmt_str,
unsigned long length)
int mysql_real_query_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_fetch_row_start(MYSQL_ROW *ret, MYSQL_RES *result)
int mysql_fetch_row_cont(MYSQL_ROW *ret, MYSQL_RES *result, int ready_status)

Initiate fetch of another row from a SELECT query.

If the MYSQL_RES was obtained from mysql_use_result(), then this function
allows stream processing, where initial rows are returned to the application
while the server is still sending subsequent rows. When no more data is
available on the socket, mysql_fetch_row_start() or mysql_fetch_row_cont()
will return MYSQL_WAIT_READ (or possibly MYSQL_WAIT_WRITE if using SSL and SSL
re-negotiation is needed; also MYSQL_WAIT_TIMEOUT may be set if read timeout
is enabled). When data becomes available, more rows can be fetched with
mysql_fetch_row_cont().

If the MYSQL_RES was obtained from mysql_store_result() /
mysql_store_result_start() / mysql_store_result_cont(), then this function
cannot block - mysql_fetch_row_start() will always return 0 (and if desired,
plain mysql_fetch_row() may be used instead with equivalent effect).


int mysql_set_character_set_start(int *ret, MYSQL *mysql, const char *csname)
int mysql_set_character_set_cont(int *ret, MYSQL *mysql, int ready_status)


mysql_select_db_start(int *ret, MYSQL *mysql, const char *db)
int mysql_select_db_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_send_query_start(int *ret, MYSQL *mysql, const char *q, unsigned long
length)
int mysql_send_query_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_store_result_start(MYSQL_RES **ret, MYSQL *mysql)
int mysql_store_result_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)


int mysql_free_result_start(MYSQL_RES *result)
int mysql_free_result_cont(MYSQL_RES *result, int ready_status)

This function can need to wait if not all rows were fetched before it was
called (then it needs to consume any pending rows sent from the server so they
do not interfere with any subsequent queries sent).

If all rows were already fetched, then this function will not need to
wait. mysql_free_result_start() will return zero (or if so desired, plain
mysql_free_result() may be used instead).

Note that mysql_free_result() returns no value, so there is no extra `ret'
parameter for mysql_free_result_start() or mysql_free_result_cont().


int mysql_close_start(MYSQL *sock)
int mysql_close_cont(MYSQL *sock, int ready_status)

mysql_close() sends a COM_QUIT request to the server, though it does not wait
for any reply.

Thus teoretically it can block (if the socket buffer is full), though in
practise it is probably unlikely to occur frequently.

The non-blocking version of mysql_close() is provided for completeness; for
many applications using the normal mysql_close() is probably sufficient (and
may be simpler).

Note that mysql_close() returns no value, so there is no extra `ret' parameter
for mysql_close_start() or mysql_close_cont().


int mysql_change_user_start(my_bool *ret, MYSQL *mysql, const char *user, const
char *passwd, const char *db)
int mysql_change_user_cont(my_bool *ret, MYSQL *mysql, int ready_status)


int mysql_query_start(int *ret, MYSQL *mysql, const char *q)
int mysql_query_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_shutdown_start(int *ret, MYSQL *mysql, enum mysql_enum_shutdown_level
shutdown_level)
int mysql_shutdown_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_dump_debug_info_start(int *ret, MYSQL *mysql)
int mysql_dump_debug_info_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_refresh_start(int *ret, MYSQL *mysql, unsigned int refresh_options)
int mysql_refresh_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_kill_start(int *ret, MYSQL *mysql, unsigned long pid)
int mysql_kill_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_set_server_option_start(int *ret, MYSQL *mysql,
                              enum enum_mysql_set_option option)
int mysql_set_server_option_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_ping_start(int *ret, MYSQL *mysql)
int mysql_ping_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_stat_start(const char **ret, MYSQL *mysql)
int mysql_stat_cont(const char **ret, MYSQL *mysql, int ready_status)


int mysql_list_dbs_start(MYSQL_RES **ret, MYSQL *mysql, const char *wild)
int mysql_list_dbs_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)


int mysql_list_tables_start(MYSQL_RES **ret, MYSQL *mysql, const char *wild)
int mysql_list_tables_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)


int mysql_list_processes_start(MYSQL_RES **ret, MYSQL *mysql)
int mysql_list_processes_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)


int mysql_list_fields_start(MYSQL_RES **ret, MYSQL *mysql, const char *table,
                        const char *wild)
int mysql_list_fields_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)


int mysql_read_query_result_start(my_bool *ret, MYSQL *mysql)
int mysql_read_query_result_cont(my_bool *ret, MYSQL *mysql, int ready_status)


int mysql_stmt_prepare_start(int *ret, MYSQL_STMT *stmt, const char *query,
                         unsigned long length)
int mysql_stmt_prepare_cont(int *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_stmt_execute_start(int *ret, MYSQL_STMT *stmt)
int mysql_stmt_execute_cont(int *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_stmt_fetch_start(int *ret, MYSQL_STMT *stmt)
int mysql_stmt_fetch_cont(int *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_stmt_store_result_start(int *ret, MYSQL_STMT *stmt)
int mysql_stmt_store_result_cont(int *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_stmt_close_start(my_bool *ret, MYSQL_STMT *stmt)
int mysql_stmt_close_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_stmt_reset_start(my_bool *ret, MYSQL_STMT *stmt)
int mysql_stmt_reset_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_stmt_free_result_start(my_bool *ret, MYSQL_STMT *stmt)
int mysql_stmt_free_result_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_stmt_send_long_data_start(my_bool *ret, MYSQL_STMT *stmt,
                                unsigned int param_number,
                                const char *data, unsigned long length)
int mysql_stmt_send_long_data_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)


int mysql_commit_start(my_bool *ret, MYSQL *mysql)
int mysql_commit_cont(my_bool *ret, MYSQL *mysql, int ready_status)


int mysql_rollback_start(my_bool *ret, MYSQL *mysql)
int mysql_rollback_cont(my_bool *ret, MYSQL *mysql, int ready_status)


int mysql_autocommit_start(my_bool *ret, MYSQL *mysql, my_bool auto_mode)
int mysql_autocommit_cont(my_bool *ret, MYSQL *mysql, int ready_status)


int mysql_next_result_start(int *ret, MYSQL *mysql)
int mysql_next_result_cont(int *ret, MYSQL *mysql, int ready_status)


int mysql_stmt_next_result_start(int *ret, MYSQL_STMT *stmt)
int mysql_stmt_next_result_cont(int *ret, MYSQL_STMT *stmt, int ready_status)


The following client API functions never need to do I/O and thus can never
block. Therefore, they can be used as normal in programs using non-blocking
operations; no need to call any special _start() variant. (Even if a _start()
variant was available, it would always return zero, so no _cont() call would
ever be needed).

    mysql_num_rows()
    mysql_num_fields()
    mysql_eof()
    mysql_fetch_field_direct()
    mysql_fetch_fields()
    mysql_row_tell()
    mysql_field_tell()
    mysql_field_count()
    mysql_affected_rows()
    mysql_insert_id()
    mysql_errno()
    mysql_error()
    mysql_sqlstate()
    mysql_warning_count()
    mysql_info()
    mysql_thread_id()
    mysql_character_set_name()
    mysql_init()
    mysql_ssl_set()
    mysql_get_ssl_cipher()
    mysql_use_result()
    mysql_get_character_set_info()
    mysql_set_local_infile_handler()
    mysql_set_local_infile_default()
    mysql_get_server_info()
    mysql_get_server_name()
    mysql_get_client_info()
    mysql_get_client_version()
    mysql_get_host_info()
    mysql_get_server_version()
    mysql_get_proto_info()
    mysql_options()
    mysql_data_seek()
    mysql_row_seek()
    mysql_field_seek()
    mysql_fetch_lengths()
    mysql_fetch_field()
    mysql_escape_string()
    mysql_hex_string()
    mysql_real_escape_string()
    mysql_debug()
    myodbc_remove_escape()
    mysql_thread_safe()
    mysql_embedded()
    mariadb_connection()
    mysql_stmt_init()
    mysql_stmt_fetch_column()
    mysql_stmt_param_count()
    mysql_stmt_attr_set()
    mysql_stmt_attr_get()
    mysql_stmt_bind_param()
    mysql_stmt_bind_result()
    mysql_stmt_result_metadata()
    mysql_stmt_param_metadata()
    mysql_stmt_errno()
    mysql_stmt_error()
    mysql_stmt_sqlstate()
    mysql_stmt_row_seek()
    mysql_stmt_row_tell()
    mysql_stmt_data_seek()
    mysql_stmt_num_rows()
    mysql_stmt_affected_rows()
    mysql_stmt_insert_id()
    mysql_stmt_field_count()
    mysql_more_results()
    mysql_get_socket()
    mysql_get_timeout_value()
 Task Dependencies
Others waiting for Task 192Task 192 is waiting forGraph
 
 High-Level Specification
I suggest to extend libmysqlclient with an extra set of functions. These
new functions never block. Instead, when the library needs to wait for data
from the server, the new functions return control to the application. The
application must then arrange to wait for data to be available eg. with
select() or poll() or similar, and meanwhile can do any other processing it
wants. When data becomes available, the application then calls into the
library again to continue the suspended operation.

Here is a simple example of using the suggested API:


    int err;
    MYSQL mysql, *ret;
    MYSQL_RES *res;
    MYSQL_ROW row;
    int status;

    mysql_init(&mysql);
    /* Returns 0 when done, or flag for what to wait for when need to block. */
    status= mysql_real_connect_start(&ret, &mysql, "localhost", "test",
                                     "testpass", "test", 0, NULL, 0);
    while (status)
    {
      status= wait_for_mysql(&mysql, status);
      status= mysql_real_connect_cont(&ret, &mysql, status);
    }
    if (!ret)
      fatal(&mysql, "Failed to mysql_real_connect()");

    status= mysql_real_query_start(&err, &mysql, SL("SHOW STATUS"));
    while (status)
    {
      status= wait_for_mysql(&mysql, status);
      status= mysql_real_query_cont(&err, &mysql, status);
    }
    if (err)
      fatal(&mysql, "mysql_real_query() returns error");

    /* This method cannot block. */
    res= mysql_use_result(&mysql);
    if (!res)
      fatal(&mysql, "mysql_use_result() returns error");

    for (;;)
    {
      status= mysql_fetch_row_start(&row, res);
      while (status)
      {
        status= wait_for_mysql(&mysql, status);
        status= mysql_fetch_row_cont(&row, res, status);
      }
      if (!row)
        break;
      printf("%s: %s\n", row[0], row[1]);
    }
    if (mysql_errno(&mysql))
      fatal(&mysql, "Got error while retrieving rows");

    /* I suppose these must be non-blocking too. */
    mysql_free_result(res);
    mysql_close(&mysql);


The wait_for_mysql() could be implemented like this:

    static void
    wait_for_mysql(MYSQL *mysql, int status)
    {
      struct pollfd pfd;
      struct timeval tv;
      p.fd= mysql_get_socket_fd(&mysql);
      p.events=
        (status & MYSQL_WAIT_READ ? POLLIN : 0) |
        (status & MYSQL_WAIT_WRITE ? POLLOUT : 0) |
        (status & MYSQL_WAIT_EXCEPT ? POLLPRI : 0);
      tv.tv_sec= mysql_get_timeout_value(mysql);
      tv.tv_usec= 0;
      if (0 == poll(&pfd, 1, (status & MYSQL_WAIT_TIMEOUT ? &tv : 0)))
        return MYSQL_WAIT_TIMEOUT;
      return (pfd.revents & POLLIN ? MYSQL_WAIT_READ : 0) |
             (pfd.revents & POLLOUT ? MYSQL_WAIT_WRITE : 0) |
             (pfd.revents & POLLPRI ? MYSQL_WAIT_EXCEPT : 0);
    }

Of course in this simple example the non-blocking behaviour is not needed, but
it demonstrates how the waiting/blocking can be done under control of the
application, and it is hopefully clear how to do the same example in an
event-driven style hooking into an event loop such as provided with libevent
or similar.

The basic idea is that for each libmysqlclient function that can potentially
block, like mysql_real_query(), we introduce two new functions,
mysql_real_query_start(), and mysql_real_query_cont() that can be used for
non-blocking operations.

The mysql_real_query_start() function takes the same parameters as the
original mysql_real_query(), and in addition one extra parameter "result". The
mysql_real_query_cont() takes three parameters: the MYSQL object, the extra
"result" parameter, and a MYSQL_ASYNC_STATUS flag of events that
occured. They both return a status value of MYSQL_ASYNC_STATUS flags:


int
mysql_real_query(MYSQL *mysql, const char *stmt_str, unsigned long length)

int
mysql_real_query_start(int *result,
                       MYSQL *mysql, const char *stmt_str, unsigned long length)

int
mysql_real_query_cont(int *result,
                       MYSQL *mysql, int ready_status)


The application first calls mysql_real_query_start(). When libmysqlclient
needs to block, it returns a non-zero status flag which is a combination of
these values:

  MYSQL_WAIT_READ= 1,   // Application should select socket for read ready
  MYSQL_WAIT_WRITE= 2,  // Application should select socket for write ready
  MYSQL_WAIT_EXCEPT= 4, // Wait for select() to mark exception on socket
  MYSQL_WAIT_TIMEOUT= 8 // Limit for how long to wait (eg. for connect timeout)

These values should be used to hook the server socket into the appropriate
event loop mechanism or used in select()/poll() as appropriate. When the wait
is done, the application calls back into the library with
mysql_real_query_cont(), passing back a status value indicating which event(s)
of those requested actually occured.

When the operation is finally done, the mysql_real_query_start() or
mysql_real_query_cont() functions set the "result" parameter to the value that
the normal mysql_real_query() would return, and returns a status of zero to
indicate to the application that the operation is done.

So given this normal blocking operation:

    result= mysql_real_query(mysql, "SELECT * FROM t1", 16)

The non-block version first does

    status= mysql_real_query_start(&result, mysql, "SELECT * FROM t1", 16)

Then as long as status is non-zero, it waits for the condition indicated by
the bits in status and repeatedly calls

    status= mysql_real_query_cont(&result, mysql, events_occured)

When status is returned as zero, the operation is complete with the same
effect as in the blocking operation.

The application can obtain the file descriptor for the socket and any timeout
for the wait with these two functions:

    socket_fd= mysql_get_socket(&mysql)
    wait_microseconds= mysql_get_wait_timeout(&mysql)

The third parameter to mysql_real_query_cont() passes back information about
which event actually occured, of the several events requested to be waited
for. This is mostly for mysql_real_connect(); it will wait for the connection
attempt to either succeed or timeout. Without this information, the library
cannot determine which of the two occured. Note that when automatic reconnect
is enabled, most commands can do a reconnect, so the information about which
event occured is needed for all commands.

Note also that when SSL connections are used, SSL renegotiation can cause both
extra reads and writes on the socket. Thus in particular,
mysql_fetch_row_start() and mysql_fetch_row_cont() can both return requesting
MYSQL_WAIT_WRITE, even though they only read data from the socket (though in
practise it is perhaps unlikely that SSL renegotiation will be able to fill
the socket buffer by itself and need to block).

In general, for any function in libmysqlclient that can potentially block, we
introduce two new functions. A _start() function, which takes an extra
"result" parameter that points to a location that will eventually be set to
the return value of the original blocking function. The _start() function
returns non-zero or zero status depending on whether the operation needs to
block or not. And a _cont() function, which takes the "result" parameter, the
MYSQL object pointer, and the set of events that occured, and which returns
the same status value.

This regular structure makes it easy for the application programmer to know
how to use the new functions based on the original blocking API, which should
hopefully make it easier to learn and use.

Just like with normal blocking operation, it is not possible to have multiple
queries active at the same time with a single MYSQL object and connection,
however any number of MYSQL objects and connections can be used
simultaneously. It would be nice to support multiple queries on a single
connection, but this requires extensive changes to the server and protocol
also to tag each connection packet with a query ID, and is thus a rather
larger project to do.

Also note that once an operation is started, eg. mysql_real_query_start(), it
has to finish before a new operation can be started. This is similar to the
normal blocking API, however the non-blocking API adds the requirement to
continue calling mysql_real_query_cont() until it returns zero; only then can
other operations proceed. As a later extension, we could implement the
possibility to abort an operation early by passing MYSQL_WAIT_ABORT as the
event that occured; this would cause the library to close the connection and
finish the ongoing operation as quickly as possible.
 Low-Level Design
 File Attachments
 NameTypeSizeByDate
 User Comments
 Time Estimates
NameHours WorkedLast Updated
Knielsen12626 Sep 2011
Monty622 Sep 2011
Total132 
 Hrs WorkedProgressCurrentOriginal
This Task132050
Total132050
 
 Funding and Votes
Votes: 1: 100%
 Change vote: Useless    Nice to have    Important    Very important    

Funding: 0 offers, total 0 Euro
 Progress Reports
(Knielsen - Tue, 28 Feb 2012, 12:30
    
Category updated.
--- /tmp/wklog.192.old.14260	2012-02-28 12:30:19.000000000 +0000
+++ /tmp/wklog.192.new.14260	2012-02-28 12:30:19.000000000 +0000
@@ -1,2 +1,2 @@
-Client-RawIdeaBin
+Client-Sprint
 

(Knielsen - Tue, 28 Feb 2012, 12:30
    
Status updated.
--- /tmp/wklog.192.old.14260	2012-02-28 12:30:19.000000000 +0000
+++ /tmp/wklog.192.new.14260	2012-02-28 12:30:19.000000000 +0000
@@ -1,2 +1,2 @@
-Code-Review
+Complete
 

(Knielsen - Tue, 28 Feb 2012, 12:30
    
Documentation updated:  -> Knielsen

(Knielsen - Tue, 28 Feb 2012, 12:29
    
High Level Description modified.
--- /tmp/wklog.192.old.14058	2012-02-28 12:29:30.000000000 +0000
+++ /tmp/wklog.192.new.14058	2012-02-28 12:29:30.000000000 +0000
@@ -5,3 +5,608 @@
 
 Is this possible?
 
+=======================================================================
+
+Below is user documentation for this feature.
+
+
+Non-blocking client library
+---------------------------
+
+MariaDB from version 5.5.21 support _non-bloking_ operation in the
+client-library. This allow an application to start a query or other operation
+against the database, and then continue to do other work (in the same thread)
+while the request is sent over the network, query is processed in the server,
+and result travels back. As parts of the result become ready, the application
+can - at its leisure - call back into the library to continue processing,
+repeating this until the operation is completed.
+
+Non-blocking operation is entirely implemented in the client library. This
+means that no special server support is necessary; non-blocking operation
+works with any version of the MariaDB or MySQL server, same as the normal
+blocking API. It also means that it is not possible to have two queries
+running at the same time on the same connection (this is a protocol
+limitation). But a single thread can have any number of non-blocking queries
+running at the same time, each using its own MYSQL connection object.
+
+Non-blocking operation is useful when an application needs to run a number of
+independent queries in parallel at the same time, to speed up operation
+compared to running then sequentially one after the other. This could be
+multiple queries against a single server (to better utilise multiple CPU cores
+and/or high-capacity I/O system on the server), or it could be queries against
+multiple servers (eg. SHOW STATUS against all running servers for monitoring,
+or map/reduce-like operation against a big sharded database).
+
+Non-blocking operation is also very useful in applications that are already
+written in a non-blocking style, for example using a framework like libevent
+(http://libevent.org/), or eg. a GUI-application using an event loop. Using
+the non-blocking client library allows to integrate database queries into such
+applications, without risking that long-running queries will "hang" the user
+interface or stall the event loop, and without having to manually spawn
+separate threads to run the queries and re-synchronise with the threads to get
+the results back.
+
+In this context, "blocking" means the situation where communication on the
+network socket to the server has to wait while processing the query. Waiting
+can be necessary because the server has not yet had time to process the query,
+or because we must wait for the data to travel over the network from the
+server, or even because we need to wait for the first part of a large request
+to be sent out on the network before local socket buffers can accept the last
+part. Whenever such wait is necessary, control returns to the application. The
+application will then run select() or poll() or similar to detect when any
+wait condition is satisfied, and then call back into the library to continue
+processing.
+
+An example program is available in the MariaDB source tree:
+
+    tests/async_queries.c
+
+It uses libevent to run a set of queries in parallel from within a single
+thread / event loop. This is a good example for how to integrate non-blocking
+query processing into an event-based framework.
+
+The non-blocking API in the client library is entirely optional. The new
+library is completely ABI- and source-compatible with existing
+applications. Applications not using non-blocking operations are not affected,
+nor is there any significant performance penalty for having support for
+non-blocking operations in the library for applications that do not use them.
+
+
+Using the non-blocking library
+------------------------------
+
+The MariaDB non-blocking client API is modelled after the normal blocking
+library calls. This makes it easy to learn/remember, makes it easier
+to translate code using the blocking API to do non-blocking operation (or vice
+versa), and also makes it simple to mix blocking and non-blocking calls in the
+same code path.
+
+For every library call that may block on socket I/O, such as
+int mysql_real_query(MYSQL, query, query_length), two additional non-blocking calls
+are introduced:
+
+    int mysql_real_query_start(&status, MYSQL, query, query_length)
+    int mysql_real_query_start(&status, MYSQL, wait_status)
+
+To do non-blocking operation, an application first calls
+mysql_real_query_start() instead of mysql_real_query(), passing the same
+parameters.
+
+If mysql_real_query_start() returns zero, then the operation completed without
+blocking, and `status' is set to the value that would normally be returned
+from mysql_real_query().
+
+Else, the return value from mysql_real_query_start() is a bitmask of events
+that the library is waiting on. This can be MYSQL_WAIT_READ / MYSQL_WAIT_WRITE
+/ MYSQL_WAIT_EXCEPT, corresponding to the similar flags for select() or
+poll(); and it can include MYSQL_WAIT_TIMEOUT when waiting for a timeout to
+occur (eg. connection timeout).
+
+In this case, the application continues other processing and eventually checks
+for the appropriate condition(s) to occur on the socket (or for timeout). When
+this occurs, the application can resume the operation by calling
+mysql_real_query_cont(), passing in `wait_status' a bitmask of the events that
+actually occured.
+
+Just like mysql_real_query_start(), mysql_real_query_cont() returns zero when
+done, or bitmask of events it needs to wait on. Thus the application continues
+to repeatedly call mysql_real_query_cont(), intermixed with other processing
+of its choice; until zero is returned, after which the result of the operation
+is stored in `status'.
+
+Some calls, like mysql_option(), do not do any socket I/O, and so can never
+block. For these, there are no separate _start() or _cont() calls. See the
+section "Non-blocking API reference" below for a full list of what functions
+can and can not block.
+
+The checking for events on the socket / timeout can be done with select() or
+poll() or similar mechanism. Though often it will be done using a higher-level
+framework (such as libevent), which supplies facilities for registering and
+acting on such conditions.
+
+The descriptor of the socket on which to check for events can be obtained by
+calling mysql_get_socket(). The duration of any timeout can be obtained from
+mysql_get_timeout_value().
+
+Here is a trivial but full example of running a query with the non-blocking
+API. The example is found in the MariaDB source tree as
+client/async_example.c. (A larger, more realistic example using libevent is
+found as tests/async_queries.c in the source):
+
+  static void run_query(const char *host, const char *user, const char *password) {
+    int err, status;
+    MYSQL mysql, *ret;
+    MYSQL_RES *res;
+    MYSQL_ROW row;
+
+    mysql_init(&mysql);
+    mysql_options(&mysql, MYSQL_OPT_NONBLOCK, 0);
+
+    status = mysql_real_connect_start(&ret, &mysql, host, user, password, NULL,
+0, NULL, 0);
+    while (status) {
+      status = wait_for_mysql(&mysql, status);
+      status = mysql_real_connect_cont(&ret, &mysql, status);
+    }
+
+    if (!ret)
+      fatal(&mysql, "Failed to mysql_real_connect()");
+
+    status = mysql_real_query_start(&err, &mysql, SL("SHOW STATUS"));
+    while (status) {
+      status = wait_for_mysql(&mysql, status);
+      status = mysql_real_query_cont(&err, &mysql, status);
+    }
+    if (err)
+      fatal(&mysql, "mysql_real_query() returns error");
+
+    /* This method cannot block. */
+    res= mysql_use_result(&mysql);
+    if (!res)
+      fatal(&mysql, "mysql_use_result() returns error");
+
+    for (;;) {
+      status= mysql_fetch_row_start(&row, res);
+      while (status) {
+        status= wait_for_mysql(&mysql, status);
+        status= mysql_fetch_row_cont(&row, res, status);
+      }
+      if (!row)
+        break;
+      printf("%s: %s\n", row[0], row[1]);
+    }
+    if (mysql_errno(&mysql))
+      fatal(&mysql, "Got error while retrieving rows");
+    mysql_free_result(res);
+    mysql_close(&mysql);
+  }
+
+  /* Helper function to do the waiting for events on the socket. */
+  static int wait_for_mysql(MYSQL *mysql, int status) {
+    struct pollfd pfd;
+    int timeout, res;
+
+    pfd.fd = mysql_get_socket(mysql);
+    pfd.events =
+      (status & MYSQL_WAIT_READ ? POLLIN : 0) |
+      (status & MYSQL_WAIT_WRITE ? POLLOUT : 0) |
+      (status & MYSQL_WAIT_EXCEPT ? POLLPRI : 0);
+    if (status & MYSQL_WAIT_TIMEOUT)
+      timeout = 1000*mysql_get_timeout_value(mysql);
+    else
+      timeout = -1;
+    res = poll(&pfd, 1, timeout);
+    if (res == 0)
+      return MYSQL_WAIT_TIMEOUT;
+    else if (res < 0)
+      return MYSQL_WAIT_TIMEOUT;
+    else {
+      int status = 0;
+      if (pfd.revents & POLLIN) status |= MYSQL_WAIT_READ;
+      if (pfd.revents & POLLOUT) status |= MYSQL_WAIT_WRITE;
+      if (pfd.revents & POLLPRI) status |= MYSQL_WAIT_EXCEPT;
+      return status;
+    }
+  }
+
+
+Setting MYSQL_OPT_NONBLOCK
+--------------------------
+
+Before using any non-blocking operation, it is necessary to enable it first
+by setting the MYSQL_OPT_NONBLOCK option:
+
+    mysql_options(&mysql, MYSQL_OPT_NONBLOCK, 0);
+
+This call can be made at any time - typically it will be done at the start,
+before mysql_real_connect(), but it can be done at any time to start using
+non-blocking operations.
+
+If a non-blocking operation is attempted without setting the
+MYSQL_OPT_NONBLOCK option, the program will typically crash with a NULL
+pointer exception.
+
+The argument for MYSQL_OPT_NONBLOCK is the size of the stack used to save the
+state of a non-blocking operation while it is waiting for I/O and the
+application is doing other processing. Normally, applications will not have to
+change this, and it can be passed as zero to use the default value.
+
+
+Mixing blocking and non-blocking operation
+------------------------------------------
+
+It is possible to freely mix blocking and non-blocking calls on the same MYSQL
+connection.
+
+Thus, an application can do a normal blocking mysql_real_connect() and
+subsequently do a non-blocking mysql_real_query_start(). Or vice versa, do a
+non-blocking mysql_real_connect_start(), and later do a blocking
+mysql_real_query() on the resulting connection.
+
+Mixing can be useful to allow code to use the simpler blocking API in parts of
+the program where waiting is not a problem. For example establishing the
+connection(s) at program startup, or doing small quick queries between large,
+long-running ones.
+
+The only restriction is that any previous non-blocking operation must have
+finished before starting a new blocking (or non-blocking) operation, see
+"Terminating a non-blocking operation early" below.
+
+
+Terminating a non-blocking operation early
+------------------------------------------
+
+When a non-blocking operation is started with mysql_real_query_start() or
+other _start() function, it must be allowed to finish before starting a new
+operation. Thus, the application must continue calling mysql_real_query_cont()
+until zero is returned, indicating that the operation is completed. It is not
+allowed to leave one operation "hanging" in the middle of processing and then
+start a new one on top of it.
+
+It is, however, permissible to terminate the connection completely with
+mysql_close() in the middle of processing a non-blocking call. A new
+connection must then be initiated with mysql_real_connect before new queries
+can be run, either with a new MYSQL object or re-using the old one.
+
+In the future, we may implement an abort facility to force an on-going
+operation to terminate as quickly as possible (but it will still be necessary
+to call mysql_real_query_cont() one last time after abort, allowing it to
+clean up the operation and return immediately with an appropriate error code).
+
+
+Restrictions
+------------
+
+When mysql_real_connect_start() is passed a hostname (as opposed to a local
+unix socket or an IP address, it may need to look up the hostname in DNS,
+depending on local host configuration (ie. if the name is not in /etc/hosts or
+cached). Such DNS lookup does _not_ happen in a non-blocking way. This means
+that mysql_real_connect_start() will not return control to the application
+while waiting for the DNS response. Thus the application may "hang" for some
+time if DNS is slow or non-functional.
+
+If this is a problem, the application can pass an IP address to
+mysql_real_connect_start() instead of a hostname, which avoids the
+problem. The IP address can be obtained by the application with whatever
+non-blocking DNS loopup operation is available to it from the operating system
+or event framework used. Alternatively, a simple solution may be to just add
+the hostname to the local host lookup file (/etc/hosts on Posix/Unix/Linux
+machines).
+
+There is no support in the non-blocking API for connections using Windows
+named pipes.
+
+Named pipes can still be used, using either the blocking or the non-blocking
+API. However, operations that need to wait on I/O on the named pipe will not
+return control to the application; instead they will "hang" waiting for the
+operation to complete, just like the normal blocking API calls.
+
+
+Non-blocking API reference
+--------------------------
+
+Here is a list of all functions in the non-blocking client API and their
+parameters. Apart from operating in the non-blocking way, they all work
+exactly the same as their non-blocking counterpart, so their exact semantics
+can be obtained from the documentation of the normal client API.
+
+The API contains additionally the following two functions used to get the
+socket fd and timeout value when waiting for events to occur:
+
+  my_socket mysql_get_socket(const MYSQL *mysql)
+
+  Return the descriptor of the socket used for the connection.
+
+
+  unsigned int STDCALL mysql_get_timeout_value(const MYSQL *mysql)
+
+  This should only be called when a _start() or _cont() function returns a
+  value with the MYSQL_WAIT_TIMEOUT flag set. In this case, it returns the
+  value, in seconds, after which a timeout has occured and the application
+  should call the appropriate _cont() function passing MYSQL_WAIT_TIMEOUT as
+  the event that occured.
+
+  This is used to handle connection and read timeouts.
+
+
+At the end is a list of all functions from the normal API that can be used
+safely in a non-blocking program, since they never need to block.
+
+
+int mysql_real_connect_start(MYSQL **ret, MYSQL *mysql, const char *host,
+                         const char *user, const char *passwd, const char *db,
+                         unsigned int port, const char *unix_socket,
+                         unsigned long client_flags)
+int mysql_real_connect_cont(MYSQL **ret, MYSQL *mysql, int ready_status)
+
+mysql_real_connect_start() initiates a non-blocking connection request to a
+server.
+
+When mysql_real_connect_start() or mysql_real_connect_cont() returns zero, a
+copy of the passed `mysql' argument is stored in *ret.
+
+
+int mysql_real_query_start(int *ret, MYSQL *mysql, const char *stmt_str,
+unsigned long length)
+int mysql_real_query_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_fetch_row_start(MYSQL_ROW *ret, MYSQL_RES *result)
+int mysql_fetch_row_cont(MYSQL_ROW *ret, MYSQL_RES *result, int ready_status)
+
+Initiate fetch of another row from a SELECT query.
+
+If the MYSQL_RES was obtained from mysql_use_result(), then this function
+allows stream processing, where initial rows are returned to the application
+while the server is still sending subsequent rows. When no more data is
+available on the socket, mysql_fetch_row_start() or mysql_fetch_row_cont()
+will return MYSQL_WAIT_READ (or possibly MYSQL_WAIT_WRITE if using SSL and SSL
+re-negotiation is needed; also MYSQL_WAIT_TIMEOUT may be set if read timeout
+is enabled). When data becomes available, more rows can be fetched with
+mysql_fetch_row_cont().
+
+If the MYSQL_RES was obtained from mysql_store_result() /
+mysql_store_result_start() / mysql_store_result_cont(), then this function
+cannot block - mysql_fetch_row_start() will always return 0 (and if desired,
+plain mysql_fetch_row() may be used instead with equivalent effect).
+
+
+int mysql_set_character_set_start(int *ret, MYSQL *mysql, const char *csname)
+int mysql_set_character_set_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+mysql_select_db_start(int *ret, MYSQL *mysql, const char *db)
+int mysql_select_db_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_send_query_start(int *ret, MYSQL *mysql, const char *q, unsigned long
+length)
+int mysql_send_query_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_store_result_start(MYSQL_RES **ret, MYSQL *mysql)
+int mysql_store_result_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_free_result_start(MYSQL_RES *result)
+int mysql_free_result_cont(MYSQL_RES *result, int ready_status)
+
+This function can need to wait if not all rows were fetched before it was
+called (then it needs to consume any pending rows sent from the server so they
+do not interfere with any subsequent queries sent).
+
+If all rows were already fetched, then this function will not need to
+wait. mysql_free_result_start() will return zero (or if so desired, plain
+mysql_free_result() may be used instead).
+
+Note that mysql_free_result() returns no value, so there is no extra `ret'
+parameter for mysql_free_result_start() or mysql_free_result_cont().
+
+
+int mysql_close_start(MYSQL *sock)
+int mysql_close_cont(MYSQL *sock, int ready_status)
+
+mysql_close() sends a COM_QUIT request to the server, though it does not wait
+for any reply.
+
+Thus teoretically it can block (if the socket buffer is full), though in
+practise it is probably unlikely to occur frequently.
+
+The non-blocking version of mysql_close() is provided for completeness; for
+many applications using the normal mysql_close() is probably sufficient (and
+may be simpler).
+
+Note that mysql_close() returns no value, so there is no extra `ret' parameter
+for mysql_close_start() or mysql_close_cont().
+
+
+int mysql_change_user_start(my_bool *ret, MYSQL *mysql, const char *user, const
+char *passwd, const char *db)
+int mysql_change_user_cont(my_bool *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_query_start(int *ret, MYSQL *mysql, const char *q)
+int mysql_query_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_shutdown_start(int *ret, MYSQL *mysql, enum mysql_enum_shutdown_level
+shutdown_level)
+int mysql_shutdown_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_dump_debug_info_start(int *ret, MYSQL *mysql)
+int mysql_dump_debug_info_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_refresh_start(int *ret, MYSQL *mysql, unsigned int refresh_options)
+int mysql_refresh_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_kill_start(int *ret, MYSQL *mysql, unsigned long pid)
+int mysql_kill_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_set_server_option_start(int *ret, MYSQL *mysql,
+                              enum enum_mysql_set_option option)
+int mysql_set_server_option_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_ping_start(int *ret, MYSQL *mysql)
+int mysql_ping_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_stat_start(const char **ret, MYSQL *mysql)
+int mysql_stat_cont(const char **ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_list_dbs_start(MYSQL_RES **ret, MYSQL *mysql, const char *wild)
+int mysql_list_dbs_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_list_tables_start(MYSQL_RES **ret, MYSQL *mysql, const char *wild)
+int mysql_list_tables_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_list_processes_start(MYSQL_RES **ret, MYSQL *mysql)
+int mysql_list_processes_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_list_fields_start(MYSQL_RES **ret, MYSQL *mysql, const char *table,
+                        const char *wild)
+int mysql_list_fields_cont(MYSQL_RES **ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_read_query_result_start(my_bool *ret, MYSQL *mysql)
+int mysql_read_query_result_cont(my_bool *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_stmt_prepare_start(int *ret, MYSQL_STMT *stmt, const char *query,
+                         unsigned long length)
+int mysql_stmt_prepare_cont(int *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_stmt_execute_start(int *ret, MYSQL_STMT *stmt)
+int mysql_stmt_execute_cont(int *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_stmt_fetch_start(int *ret, MYSQL_STMT *stmt)
+int mysql_stmt_fetch_cont(int *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_stmt_store_result_start(int *ret, MYSQL_STMT *stmt)
+int mysql_stmt_store_result_cont(int *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_stmt_close_start(my_bool *ret, MYSQL_STMT *stmt)
+int mysql_stmt_close_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_stmt_reset_start(my_bool *ret, MYSQL_STMT *stmt)
+int mysql_stmt_reset_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_stmt_free_result_start(my_bool *ret, MYSQL_STMT *stmt)
+int mysql_stmt_free_result_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_stmt_send_long_data_start(my_bool *ret, MYSQL_STMT *stmt,
+                                unsigned int param_number,
+                                const char *data, unsigned long length)
+int mysql_stmt_send_long_data_cont(my_bool *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+int mysql_commit_start(my_bool *ret, MYSQL *mysql)
+int mysql_commit_cont(my_bool *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_rollback_start(my_bool *ret, MYSQL *mysql)
+int mysql_rollback_cont(my_bool *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_autocommit_start(my_bool *ret, MYSQL *mysql, my_bool auto_mode)
+int mysql_autocommit_cont(my_bool *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_next_result_start(int *ret, MYSQL *mysql)
+int mysql_next_result_cont(int *ret, MYSQL *mysql, int ready_status)
+
+
+int mysql_stmt_next_result_start(int *ret, MYSQL_STMT *stmt)
+int mysql_stmt_next_result_cont(int *ret, MYSQL_STMT *stmt, int ready_status)
+
+
+The following client API functions never need to do I/O and thus can never
+block. Therefore, they can be used as normal in programs using non-blocking
+operations; no need to call any special _start() variant. (Even if a _start()
+variant was available, it would always return zero, so no _cont() call would
+ever be needed).
+
+    mysql_num_rows()
+    mysql_num_fields()
+    mysql_eof()
+    mysql_fetch_field_direct()
+    mysql_fetch_fields()
+    mysql_row_tell()
+    mysql_field_tell()
+    mysql_field_count()
+    mysql_affected_rows()
+    mysql_insert_id()
+    mysql_errno()
+    mysql_error()
+    mysql_sqlstate()
+    mysql_warning_count()
+    mysql_info()
+    mysql_thread_id()
+    mysql_character_set_name()
+    mysql_init()
+    mysql_ssl_set()
+    mysql_get_ssl_cipher()
+    mysql_use_result()
+    mysql_get_character_set_info()
+    mysql_set_local_infile_handler()
+    mysql_set_local_infile_default()
+    mysql_get_server_info()
+    mysql_get_server_name()
+    mysql_get_client_info()
+    mysql_get_client_version()
+    mysql_get_host_info()
+    mysql_get_server_version()
+    mysql_get_proto_info()
+    mysql_options()
+    mysql_data_seek()
+    mysql_row_seek()
+    mysql_field_seek()
+    mysql_fetch_lengths()
+    mysql_fetch_field()
+    mysql_escape_string()
+    mysql_hex_string()
+    mysql_real_escape_string()
+    mysql_debug()
+    myodbc_remove_escape()
+    mysql_thread_safe()
+    mysql_embedded()
+    mariadb_connection()
+    mysql_stmt_init()
+    mysql_stmt_fetch_column()
+    mysql_stmt_param_count()
+    mysql_stmt_attr_set()
+    mysql_stmt_attr_get()
+    mysql_stmt_bind_param()
+    mysql_stmt_bind_result()
+    mysql_stmt_result_metadata()
+    mysql_stmt_param_metadata()
+    mysql_stmt_errno()
+    mysql_stmt_error()
+    mysql_stmt_sqlstate()
+    mysql_stmt_row_seek()
+    mysql_stmt_row_tell()
+    mysql_stmt_data_seek()
+    mysql_stmt_num_rows()
+    mysql_stmt_affected_rows()
+    mysql_stmt_insert_id()
+    mysql_stmt_field_count()
+    mysql_more_results()
+    mysql_get_socket()
+    mysql_get_timeout_value()
+
+

(Knielsen - Mon, 26 Sep 2011, 06:51
    
Fix last Windows test failures, prepare patch for code review, review discussions.
Worked 7 hours and estimate 0 hours remain (original estimate decreased by 12 hours).

(Knielsen - Thu, 22 Sep 2011, 16:06
    
Status updated.
--- /tmp/wklog.192.old.7401	2011-09-22 16:06:19.000000000 +0000
+++ /tmp/wklog.192.new.7401	2011-09-22 16:06:19.000000000 +0000
@@ -1,2 +1,2 @@
-Un-Assigned
+Code-Review
 

(Knielsen - Thu, 22 Sep 2011, 16:06
    
Implementor updated:  -> Knielsen
Implementor signoff
Code Review updated:  -> Monty
Code Review signoff

(Monty - Thu, 22 Sep 2011, 10:52
    
Review

(Knielsen - Mon, 19 Sep 2011, 13:21
    
Fix a problem when using multiple connections in debug builds.
Libevent example cleaned up and ready.
Fix missing handling of read and write timeouts.
Clean up the optimised x86_64 my_context implementation and make a similar optimisation for i386
(32-bit).
Debug and fix some Windows failures.
Worked 20 hours and estimate 19 hours remain (original estimate increased by 24 hours).

(Knielsen - Wed, 14 Sep 2011, 13:05
    
High-Level Specification modified.
--- /tmp/wklog.192.old.14233	2011-09-14 13:05:48.000000000 +0000
+++ /tmp/wklog.192.new.14233	2011-09-14 13:05:48.000000000 +0000
@@ -13,7 +13,7 @@
     MYSQL mysql, *ret;
     MYSQL_RES *res;
     MYSQL_ROW row;
-    MYSQL_ASYNC_STATUS status;
+    int status;
 
     mysql_init(&mysql);
     /* Returns 0 when done, or flag for what to wait for when need to block. */
@@ -96,20 +96,20 @@
 The mysql_real_query_start() function takes the same parameters as the
 original mysql_real_query(), and in addition one extra parameter "result". The
 mysql_real_query_cont() takes three parameters: the MYSQL object, the extra
-"result" parameter, and a MYSQL_ASYNC_STATUS value of events that
-occured. They both return a status value of type MYSQL_ASYNC_STATUS:
+"result" parameter, and a MYSQL_ASYNC_STATUS flag of events that
+occured. They both return a status value of MYSQL_ASYNC_STATUS flags:
 
 
 int
 mysql_real_query(MYSQL *mysql, const char *stmt_str, unsigned long length)
 
-MYSQL_ASYNC_STATUS
+int
 mysql_real_query_start(int *result,
                        MYSQL *mysql, const char *stmt_str, unsigned long length)
 
-MYSQL_ASYNC_STATUS
+int
 mysql_real_query_cont(int *result,
-                       MYSQL *mysql, MYSQL_ASYNC_STATUS ready_status)
+                       MYSQL *mysql, int ready_status)
 
 
 The application first calls mysql_real_query_start(). When libmysqlclient
-- View All Progress Notes (23 total) --


Report Generator:
 
Saved Reports:

WorkLog v4.0.0
  © 2010  Sergei Golubchik and Monty Program AB
  © 2004  Andrew Sweger <yDNA@perlocity.org> and Addnorya
  © 2003  Matt Wagner <matt@mysql.com> and MySQL AB