TODO

1 files changed, 425 insertions, 1 deletions

diff --git a/TODO b/TODO
index 10523b7..353ba89 100644
--- a/TODO
+++ b/TODO
@@ -12,6 +12,430 @@ types, because the alternative is to believe the XML from the service,
 which would mean a malicious service could make us send random stuff
 from the stack.
 
-Write tool that reads the introspection XML and generates service code.
+Write tool that reads the introspection XML and generates service
+code.
 
+Things from glib:
 
+  - Consistent API between things like GCheckSum and GBase64.
+    GCheckSum looks better to me offhand. Note, these should also be
+    able to take nul_byte_queue_t's.
+
+       nul_checksum_t
+       nul_base64_t
+       nul_iconv_t (iconv wrapper)
+
+    Consider making them objects inheriting some sort of stream
+    interface. This could also be done separately, and in a way that
+    would allow errors to propagate. Ie., build a stream out of
+    smaller streams.
+
+    	nul_compound_stream (
+            nul_convert_stream ("LATIN-1", "UTF-8"),
+            nul_checksum_new (G_CHECK_SUM_SHA1),
+	    NULL);
+
+	nul_object_connect (stream, "event", on_event, NULL);
+
+	nul_stream_feed (stream, "asdfasdf");
+	nul_stream_end (stream);
+
+    The event system here could really benefit from extensible unions
+    so that you can have inherited streams with more complex events.
+
+  - Note GData is used in GObject, and by GScanner, which is also used
+    for various things. GHook is also used in GObject (signals), plus,
+    weirdly, an enum from ghook.h is used in gmain.c. Also, gpattern
+    is used in gtestutils.c
+
+  - These things should be improved:
+
+	  - ghash.h		open addressing, no primes
+	  - gmain.h		simpler API, better threads
+	    			Better child watch
+	  - gmarkup.h		cut and paste expat
+	  - grand.h		Delete the context version
+	  - gsequence.h		Should gain aggregators
+	  - gstring.h		Should probably be replaced with byte array 
+	    			(which in turn must get equivalent API to
+				 gstring)
+				Is GStringChunk useful?
+          - gthreadpool.h	maybe (replace with executor?)
+          - gasyncqueue.h	probably
+	  - G*Array		Direct pointers, byte queue
+
+  - New things:
+
+    	  - sparse table	maybe
+	  - byte queue
+	  - sfile
+	  - block sequence	maybe, or could become default for gsequence.
+	  - FreeList		maybe
+	  - watch API
+
+  - GObject should go away, replaced by new, simpler object system
+    (See below)
+
+  - Note: GIO depends on GObject, so the right order is:
+    - Add new object system
+    - Port GIO to it
+    - delete GObject
+
+  - GIO in separate library?
+
+  - GThread integrated into main library, and simplified
+
+  - Careful API review of what remains
+
+  - These things are crack:
+
+      - gbacktrace.h	though G_BREAKPOINT is useful
+      - gdataset.h
+      - ghook.h
+      - giochannel.h
+      - gnode.h
+      - gpattern.h
+      - gprimes.h
+      - gquark.h		possibly
+      - gslice.h
+      - gscanner.h
+      - galloca.h
+      - gmem.h		the malloc wrappers are useful, chunks most go
+      - gcache.h
+      - gcompletion.h
+      - gtree.h
+      - grel.h
+
+-=-=- Notes on main loop
+
+New, even simpler design
+
+Basically, having one main thread is good enough. JobQueues/MainContexts
+can be implemented on top of that. Executor could be called ThreadPool.
+JobQueues should have a reference to an executor. 
+
+	  job_queue_set_read (job_queue_t *job, int fd, read_func, data);
+	  job_queue_set_write (job_queue_t *job, ...);
+	  job_queue_add_idle/timeout().
+
+Whenever something is queued up for a job queue, the jobqueue's
+run-next time is computed, and if it needs to run at some point, it is
+added to the executor along with the timeout. The executor will
+maintain a sorted list of things that need to run (sorted by when they
+need to run). When there is nothing to do it waits for that timeout to
+expire (or for something to happen), otherwise it schedules things in
+order.
+
+When something is added that should run "as soon as possible", it is
+added with the current time. This ensures that timeouts etc. will
+eventually run (because their deadline will eventually be in the
+past).
+
+So when an fd fires for a job queue, this will happen:
+
+   1. callback is added to the job queue's queue. (Insert sorted with
+      current time as deadline).
+
+   2. Current run-time is computed, and the job queue is added to the
+      executor.
+
+The job queue queue probably needs its own mutex, separate from the
+job queue's.
+
+
+New, simpler design:
+
+There is an 'Engine' which is both Executor and Mainloop. It uses an
+Epoll object to handle the polling. This epoll object does not do any
+locking by itself.
+
+Contexts can be created; they are like jobqueues, except they also
+allow you to add fd's and idles and timeouts. They can do that since
+they have access to the engine. These are completely unlocked; if you
+want to access a context from more than one thread, you must lock them
+yourself. It is guaranteed that only one callback will be active for a
+context at the same time. From within that callback it is then safe to
+remove other callbacks etc. In general such a context can act pretty
+much like one thread.
+
+A web server will consist of a listening context, which will create
+other contexts as necessary and add the fd's to them with the
+appropriate callback. Or maybe just call back with the new fd. I guess
+there should be a generic listener object that will create its own
+context and call back. Then an http module can just create a listener
+and do whatever it wants in response.
+
+One possible problem. The thread that creates a context and adds the
+fd not necessarily the one that will execute the first callback. Can
+these interfere? Maybe the solution is just that the creating thread
+should not do anything with the context after it adds the fd.  A
+better idea may be to just have a context_dispatch() function that you
+must call after creating the context. Until this is called, nothing
+the context is not active.
+
+Beginning of this new design is implemented and can be built with
+build2.sh.
+
+
+
+Old design (referenced above)
+
+Highlevel design for a web server:
+
+        - Executor
+                takes callbacks and executes them, possibly in parallell
+                or out of order. The callbacks are likely to be called in
+                a different thread than the one that queued them.
+
+		Should support both push_back() and push_front(). Push_front()
+		is needed to do parallel requests with good latency. If 
+		a request can be parallellized, the parallel tasks should
+		be put at the front of the queue.
+
+        - EPoll
+                Simple class that wraps epoll()/poll(). Note: must support
+		oneshot polling and rearming.
+
+        - JobQueue
+                Maintain a queue of jobs, uses executor to execute them
+                in order. Two jobs in the same JobQueue will never run
+                at the same time.
+
+        - Mainloop
+                Uses executor, epoll.
+                - Takes filedescriptors and callbacks. calls
+                        back when descriptor is readable/writable/etc.
+		- The callbacks are put on a job queue, which is passed in
+		- Each callback is a oneshot - ie., after calling, the
+		  filedescriptor is not polled again until it is rearmed.
+		  The mainloop will have a method to rearm filedescriptors.
+		  This ensures that the mainloop can start a new poll()
+		  whenever it wants without waiting for all the callbacks
+		  to finish.
+                - Also takes timeouts that can be canceled. Timeouts also
+		  need to be put on a queue.
+
+		Sketch of implementation:
+
+			polljob()
+			{
+				timeout = compute min_timeout ();
+				if (timeout > 0)
+					poll_armed_describtors(timeout);
+				call all timeouts (ie., put them on queues);
+				call all callbacks (ie., put them on queues);
+				schedule (polljob);
+			}
+
+	- MainContext
+
+		- The thing clients will deal with
+
+		- Uses the main loop
+
+		- Filedescriptors
+			maincontext is responsible for rearming the
+			descriptor after the client callback has been
+			called.
+
+		- Timeouts
+
+		- Idle handlers
+
+		- Everything associated with a main context happens 
+		  serialized - ie., as if only one thread executed it.
+
+		- This means a client structure doesn't need to be locked.
+
+		- Has a get_executor() method so that clients can parallelize
+		  if they want to.
+
+		- Current thinking is that if we have main contexts, who
+		  really needs a main loop? The only thing you could do
+		  with it is to pass it to main contexts.
+
+		  OTOH that's true of several of the objects here.
+		  see notes at top of maincontext.c
+
+		- Worth noting that stuff that has to be passed in to
+		  create a client object must be available to the 
+		  listener callback. But see http_server.c for an example
+
+		- Err, the epoll in itself is not enough for maincontext,
+		  since it needs to be shared between maincontexts. Ie., 
+		  who would call epoll_wait(). We do really need a main
+		  loop that will call back.
+
+        - Listener
+                Listens on a port. Calls back with a file descriptor when
+		someone connects.
+
+	- Connection
+		initialized with a MainContext and a filedescriptor.
+		Creates events when something happens
+
+	- Http:
+		Has a Connection
+		parses http, emits events such as
+			"get hostname pagename query"
+			"post etc etc etc"
+	
+	- ContentProvider
+
+
+
+
+-=-=- Notes on object/component system:
+
+/*
+  signals	(no class handlers, no return values).
+
+  ssp_object_emit (object, "birnan", x, y, z);
+  ssp_object_block (object, "birnan");
+  ssp_object_unblock (object, "birnan");
+
+  ssp_object_connect (object, "birnan", on_birnan, DATA);
+  
+*/
+
+const ssp_type_t *
+ssp_toggle_button_get_type(void)
+{
+    return ssp_define_class (
+	"ssp_toggle_button_t",
+	ssp_extends (ssp_fish_get_type()),
+	ssp_implements (ssp_scrollable_get_type()),
+	ssp_doc (
+	    ssp_doc_since (2, 14, 0),
+	    ssp_doc_class_blurb (
+		"A type of button that retains its state"),
+	    ssp_doc_class_long ("				\
+Yeah, it's going to be quite annoying to type this stuff asd	\
+But then, it's hopefully not that much documentation that must  \
+be written this way.						\
+								\
+<p>A better scheme may be to have ToggleButton.txt that are run	\
+through a preprocesser which generates a header file which	\
+defines a macro TOGGLE_BUTTON_LONG_DOC_STRING, which we then	\
+just use here.							\
+								\
+<p>Or maybe writing documentation as C strings is not		\
+<em>that</em> bad -- the main issue is likely going to be	\
+changing existing documentation and manually putting in		\
+newlines. Maybe we don't actually need newlines. We can just	\
+require &lt;p&gt; tags.						\
+								\
+<p>Yeah, not that bad, actually.				\
+								\
+<example>							\
+<title>Creating two <structname>GtkToggleButton</structname>	\
+widgets.</title>						\
+<programlisting>						\
+void make_toggles (void) {					\
+   GtkWidget *dialog, *toggle1, *toggle2;			\
+
+   dialog = gtk_dialog_new (<!-- -->);				\
+   toggle1 = gtk_toggle_button_new_with_label (\"Hi, i'm a toggle button.\"); \
+								\
+   /* Makes this toggle button invisible */				\
+   gtk_toggle_button_set_mode (GTK_TOGGLE_BUTTON (toggle1), TRUE);	\
+									\
+   g_signal_connect (toggle1, \"toggled\",				\
+                     G_CALLBACK (output_state), NULL);			\
+   gtk_box_pack_start (GTK_BOX (GTK_DIALOG (dialog)->action_area),	\
+                       toggle1, FALSE, FALSE, 2);			\
+									\
+   toggle2 = gtk_toggle_button_new_with_label (\"Hi, i'm another toggle button.\"); \
+   gtk_toggle_button_set_mode (GTK_TOGGLE_BUTTON (toggle2), FALSE);	\
+   g_signal_connect (toggle2, \"toggled\",				\
+                     G_CALLBACK (output_state), NULL);			\
+   gtk_box_pack_start (GTK_BOX (GTK_DIALOG (dialog)->action_area),	\
+                       toggle2, FALSE, FALSE, 2);			\
+									\
+   gtk_widget_show_all (dialog);					\
+}									\
+									\
+</programlisting>							\
+</example>								\
+									\
+<p>Ok, that's more painful, especially since you don't get any		\
+syntax highlighting. It also distracts from the rest of the class	\
+definition.								\
+									\
+Also, MSVC does not support longer literals than 2048; an ANSI C	\
+is only required to support 509 characters.				\
+									\
+So a preprocessor that generates a macro that expands to a comma	\
+separate list of literals.						\
+"),
+	    NULL),
+	    NULL);
+	ssp_define_constructor (
+	    ...);
+	ssp_define_property_rw ("fish", ssp_type_int32_limits (0, 0, 200)),
+	ssp_define_property_r ("moose", ssp_type_int32()),
+	ssp_define_property_cr ("fwr", ssp_type_string()),
+	ssp_define_method (
+	    "set_label",
+	    ssp_return_type (SSP_TYPE_VOID),
+	    ssp_arg_in ("label", SSP_TYPE_LABEL),
+	    ssp_arg_out ("size", SSP_TYPE_INT),
+	    ssp_arg_in ("fisH", SSP_TYPE_FISH),
+	    NULL);
+	ssp_define_method (
+	    "birnan",
+	    SSP_TYPE_INT,
+	    ssp_type_closure (
+		SSP_TYPE_VOID,
+		SSP_TYPE_INT,
+		SSP_TYPE_DOUBLE,
+		NULL)),
+	ssp_define_virtual_method (
+	    "..."),
+	ssp_define_signal (
+	    "toggled",
+	    NULL);
+	NULL);
+}
+
+ssp_toggle_button_t *
+ssp_toggle_button_new_with_label (const char *label)
+{
+    return ssp_object_new (ssp_toggle_button_get_type(), NULL);
+}
+	
+/* ... */
+const ssp_introspect_info_t *
+ssp_introspect (void)
+{
+    /* This should be updated everytime
+     * new types are added
+     */
+    return ssp_introspection_info (
+	ssp_introspect_types (
+	    ssp_object_get_type,
+	    ssp_toggle_button_get_type,
+	    ssp_int_get_type,
+	    ssp_rectangle_get_type,
+	    NULL),
+	ssp_introspect_functions (
+	    ...));
+}
+
+
+types:
+	objects
+	structs		(represented as pointers)
+	unions?		(tagged unions, that can extend other unions,
+			 useful for extensible events, represented as
+			 pointers, but contents is a full type. Ie.,
+			 a union of structs will be a pointer to
+			 a (tag + struct)).
+			 If we add them, make sure they can represent a
+			 dbus variant.
+        basic types
+	arrays		(represented in C as an integer and pointer to 
+			 contents; or as a pointer to nul_queue? with
+			 similar semantics as boxed types)
+
+	You only get arrays of basic types probably.