add braindump on how to keep programs sequential

but still asynchronous

1 files changed, 173 insertions, 0 deletions

diff --git a/KEEPING-PROGRAMS-LINEAR b/KEEPING-PROGRAMS-LINEAR
new file mode 100644
index 0000000..224187c
--- /dev/null
+++ b/KEEPING-PROGRAMS-LINEAR
@@ -0,0 +1,173 @@
+One of the essential peices of most UI programs is the event loop.
+Blocking the event loop in one subsystem of a program will cause other
+subsystems to not receive events.  This is obviously not desirable, so it
+is important to make sure that operations don't block the event loop.  
+Instead they must be handled asynchronously.  i.e., post a request for the
+operation to get done and wait for notification that it is finished.
+Factoring the code to work in these situations can be cumbersome and
+unclear.  Something as simple as:
+
+do_an_operation ()
+do_another_operation ()
+do_a_third_operation ()
+
+has to be factored into a non-linear program, where one operation is
+requested and the reply is waited for in the event loop then on reply
+the next operation is requested and so one.
+
+I guess the goal should be come up with an api that makes a
+program as simple as or nearly as simple as the above but still
+returns to the event loop between steps.
+
+Maybe something like:
+
+transaction = transaction_new ()
+transaction_add_action (transaction, do_an_operation)
+transaction_add_action (transaction, do_another_operation)
+transaction_add_action (transaction, do_a_third_operation)
+transaction_commit ()
+
+The above api would invoke each added actions in turn after the
+previously invoked action finished.  A big hole in the above
+api, though, is failure handling.  If the second action fails,
+there should be a way say it failed and cancel the third action
+from running.
+
+If it isn't clear, what we need is a state machine where there
+are two possible state transitions: goto failure state, goto to
+next action state.  Another interesting point is states have
+data associated with them and there isn't any mechanism provided
+in the above api for associating data with the states.  
+
+Each action should be able to query information about the
+transaction it's currently in, and also be able to keep its own 
+state information.  One interesting point is the transaction
+object itself is generic above.  It probably needs to be,
+because having specialized transaction implementations for every
+possible type of transaction would take a lot of coding time.
+Transactions need to be easy to define.  Since the transaction
+objects themselves are generic, transaction specific information
+must be defined by the actions in the transaction.  Early
+actions should define the state data needed by later
+transactions.
+
+A common idiom in glib for callbacks is to provide a "user data"
+pointer and a destroy notification function.  The user data pointer is
+passed to the callback when the callback is called and the
+destroy notification function is invoked when the operation
+associated with the callback is complete.  The idea is you can
+allocate some state data for use by the callback and then free
+it from the destroy notification function.  If we were to apply
+that idea to the transaction example above we would end up with:
+
+transaction = transaction_new ()
+transaction_add_action (transaction, do_an_operation,
+                        transaction_state_data, free_transaction_state_data)
+transaction_add_action (transaction, do_another_operation,
+                        transaction_state_data, free_transaction_state_data)
+transaction_add_action (transaction, do_a_third_operation,
+                        transaction_state_data, free_transaction_state_data)
+transaction_commit ()
+
+Where transaction_state_data would encode the current state of
+the transaction along with an transaction specific data.  The
+problem with this approach is their is a lot of redundancy.  We
+want the same data available from every action along the way, so
+we should only have to specify it at most once.
+
+Another idea might be something like:
+
+transaction = transaction_new (transaction_state_data, free_transaction_state_data)
+transaction_add_action (transaction, do_an_operation)
+transaction_add_action (transaction, do_another_operation)
+transaction_add_action (transaction, do_a_third_operation)
+transaction_commit ()
+
+and then could pass the state data to each action automatically
+or via an accessor function on the transaction object.  A
+problem with this approach (and the approach before this one) is
+it enforces a tight binding between all the actions in the
+transaction. If  transaction_state_data is statically defined
+then all the actions "know" about all the data that all the
+other actions need and use.  It means an action is tightly
+coupled to its transaction.  This is undesirable because an
+action really only cares about the state data it's operating on.
+It only needs to "know" about the data it depends on and on the
+data its outputing for later actions to use.  By limiting the
+data available to an action we can make sure that it is usable
+in multiple transactions.
+
+One way to do this, at the loss of type safety, is make
+transaction_state_data be a generic container type like a
+GHashTable.  An action would then rely on certain named keys
+being set to do its work and would set other named keys for
+later actions to do their work.  This could be done in the above
+suggested api without changes, but the api could be simplified
+by removing the user_data argument entirely and having the hash
+table be provided by the transaction object.
+
+Maybe something like:
+
+from do_an_operation: 
+  transaction_set_state_data (transaction, "an-operation-key", key,
+                              free_an_operation_key)
+from do_another_operation:
+  key = transaction_get_state_data (transaction, "an-operation-key")
+  copy_key (key, copy)
+  transaction_delete_state_data (transaction, "an-operation-key")
+
+In this way, the current state is determined from the current
+action and defined transaction state data, and all state data is
+cleaned up automatically when the transaction finishes or is
+explicitly removed from the transaction object with
+delete_state_data.
+
+Another important consideration is that actions need to only
+chip away at a problem, not block.  So they will need to be
+invoked multiple times and notify the transaction when they are
+done.  Each action handler could return one of three possible
+values:
+
+NOT_FINISHED,
+FAILED,
+SUCCEEDED
+
+If an action isn't finished it will be called again until it is
+finished.  If an action fails, then the whole transaction fails
+and all state is cleaned up.  If it succeed, the whole
+transaction succeeds and all state is cleaned up.
+
+In either the failed or success cases, there should probably be
+a handler invoked to note that the transaction has finished.
+The handler should get passed the transaction state.
+
+Other thoughts...
+
+It would be nice to be able to not call an incomplete
+action's handler more than necessary.  Maybe some sort of api
+like, "don't call me again until this fd is ready" would be
+useful.  We probably want to be able to set that from within
+the action since the action won't have an idea what the fd is,
+until its run once.  Maybe an api like:
+
+void transaction_pause_for_fd (Transaction *transaction, int fd, GIOCondition io_condition)
+
+Of course there should be a way to cancel a transaction as well
+(maybe transaction_cancel (transaction)) and when a transaction
+is canceled there should be a way to clean up the fd.  It
+might be sufficient, to expect the idiom
+
+transaction_set_state_data (transaction, "my-action-fd", fd, close_fd)
+transaction_pause_for_fd (transaction, fd, G_IO_READ)
+
+But that might be error prone, so it's probably better to
+enforce the state directly in the pause call.
+
+transaction_pause_for_fd (transaction, "my-action-fd", fd, G_IO_READ, close_fd)
+
+We should post a g_warning if a transaction is pausing on more
+than fd also.
+
+Another interesting point is actions may sometimes be compound.
+In those cases it might be useful to allow an action to prepend
+actions before it in the queue.