From 46897254d6ccdb61f4f2deed067119c8184ed80c Mon Sep 17 00:00:00 2001 From: Ray Strode Date: Mon, 30 Apr 2007 01:34:46 -0400 Subject: add braindump on how to keep programs sequential but still asynchronous --- KEEPING-PROGRAMS-LINEAR | 173 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100644 KEEPING-PROGRAMS-LINEAR diff --git a/KEEPING-PROGRAMS-LINEAR b/KEEPING-PROGRAMS-LINEAR new file mode 100644 index 0000000..224187c --- /dev/null +++ b/KEEPING-PROGRAMS-LINEAR @@ -0,0 +1,173 @@ +One of the essential peices of most UI programs is the event loop. +Blocking the event loop in one subsystem of a program will cause other +subsystems to not receive events. This is obviously not desirable, so it +is important to make sure that operations don't block the event loop. +Instead they must be handled asynchronously. i.e., post a request for the +operation to get done and wait for notification that it is finished. +Factoring the code to work in these situations can be cumbersome and +unclear. Something as simple as: + +do_an_operation () +do_another_operation () +do_a_third_operation () + +has to be factored into a non-linear program, where one operation is +requested and the reply is waited for in the event loop then on reply +the next operation is requested and so one. + +I guess the goal should be come up with an api that makes a +program as simple as or nearly as simple as the above but still +returns to the event loop between steps. + +Maybe something like: + +transaction = transaction_new () +transaction_add_action (transaction, do_an_operation) +transaction_add_action (transaction, do_another_operation) +transaction_add_action (transaction, do_a_third_operation) +transaction_commit () + +The above api would invoke each added actions in turn after the +previously invoked action finished. A big hole in the above +api, though, is failure handling. If the second action fails, +there should be a way say it failed and cancel the third action +from running. + +If it isn't clear, what we need is a state machine where there +are two possible state transitions: goto failure state, goto to +next action state. Another interesting point is states have +data associated with them and there isn't any mechanism provided +in the above api for associating data with the states. + +Each action should be able to query information about the +transaction it's currently in, and also be able to keep its own +state information. One interesting point is the transaction +object itself is generic above. It probably needs to be, +because having specialized transaction implementations for every +possible type of transaction would take a lot of coding time. +Transactions need to be easy to define. Since the transaction +objects themselves are generic, transaction specific information +must be defined by the actions in the transaction. Early +actions should define the state data needed by later +transactions. + +A common idiom in glib for callbacks is to provide a "user data" +pointer and a destroy notification function. The user data pointer is +passed to the callback when the callback is called and the +destroy notification function is invoked when the operation +associated with the callback is complete. The idea is you can +allocate some state data for use by the callback and then free +it from the destroy notification function. If we were to apply +that idea to the transaction example above we would end up with: + +transaction = transaction_new () +transaction_add_action (transaction, do_an_operation, + transaction_state_data, free_transaction_state_data) +transaction_add_action (transaction, do_another_operation, + transaction_state_data, free_transaction_state_data) +transaction_add_action (transaction, do_a_third_operation, + transaction_state_data, free_transaction_state_data) +transaction_commit () + +Where transaction_state_data would encode the current state of +the transaction along with an transaction specific data. The +problem with this approach is their is a lot of redundancy. We +want the same data available from every action along the way, so +we should only have to specify it at most once. + +Another idea might be something like: + +transaction = transaction_new (transaction_state_data, free_transaction_state_data) +transaction_add_action (transaction, do_an_operation) +transaction_add_action (transaction, do_another_operation) +transaction_add_action (transaction, do_a_third_operation) +transaction_commit () + +and then could pass the state data to each action automatically +or via an accessor function on the transaction object. A +problem with this approach (and the approach before this one) is +it enforces a tight binding between all the actions in the +transaction. If transaction_state_data is statically defined +then all the actions "know" about all the data that all the +other actions need and use. It means an action is tightly +coupled to its transaction. This is undesirable because an +action really only cares about the state data it's operating on. +It only needs to "know" about the data it depends on and on the +data its outputing for later actions to use. By limiting the +data available to an action we can make sure that it is usable +in multiple transactions. + +One way to do this, at the loss of type safety, is make +transaction_state_data be a generic container type like a +GHashTable. An action would then rely on certain named keys +being set to do its work and would set other named keys for +later actions to do their work. This could be done in the above +suggested api without changes, but the api could be simplified +by removing the user_data argument entirely and having the hash +table be provided by the transaction object. + +Maybe something like: + +from do_an_operation: + transaction_set_state_data (transaction, "an-operation-key", key, + free_an_operation_key) +from do_another_operation: + key = transaction_get_state_data (transaction, "an-operation-key") + copy_key (key, copy) + transaction_delete_state_data (transaction, "an-operation-key") + +In this way, the current state is determined from the current +action and defined transaction state data, and all state data is +cleaned up automatically when the transaction finishes or is +explicitly removed from the transaction object with +delete_state_data. + +Another important consideration is that actions need to only +chip away at a problem, not block. So they will need to be +invoked multiple times and notify the transaction when they are +done. Each action handler could return one of three possible +values: + +NOT_FINISHED, +FAILED, +SUCCEEDED + +If an action isn't finished it will be called again until it is +finished. If an action fails, then the whole transaction fails +and all state is cleaned up. If it succeed, the whole +transaction succeeds and all state is cleaned up. + +In either the failed or success cases, there should probably be +a handler invoked to note that the transaction has finished. +The handler should get passed the transaction state. + +Other thoughts... + +It would be nice to be able to not call an incomplete +action's handler more than necessary. Maybe some sort of api +like, "don't call me again until this fd is ready" would be +useful. We probably want to be able to set that from within +the action since the action won't have an idea what the fd is, +until its run once. Maybe an api like: + +void transaction_pause_for_fd (Transaction *transaction, int fd, GIOCondition io_condition) + +Of course there should be a way to cancel a transaction as well +(maybe transaction_cancel (transaction)) and when a transaction +is canceled there should be a way to clean up the fd. It +might be sufficient, to expect the idiom + +transaction_set_state_data (transaction, "my-action-fd", fd, close_fd) +transaction_pause_for_fd (transaction, fd, G_IO_READ) + +But that might be error prone, so it's probably better to +enforce the state directly in the pause call. + +transaction_pause_for_fd (transaction, "my-action-fd", fd, G_IO_READ, close_fd) + +We should post a g_warning if a transaction is pausing on more +than fd also. + +Another interesting point is actions may sometimes be compound. +In those cases it might be useful to allow an action to prepend +actions before it in the queue. -- cgit v1.2.3