diff options
author | Mathieu Duponchelle <mathieu.duponchelle@opencreed.com> | 2016-06-05 23:32:50 +0200 |
---|---|---|
committer | Mathieu Duponchelle <mathieu.duponchelle@opencreed.com> | 2016-06-05 23:36:01 +0200 |
commit | c26379435d2baa57505f928c7b0cd263a37fa36f (patch) | |
tree | 96d7932cdfde65c5a8b8415a2ddf069732c819c1 /sdk-android-tutorial-a-running-pipeline.md | |
parent | 6cdcdcd2d9fc6e12803d1a3151084fbf9e631819 (diff) |
Make naming consistent
Diffstat (limited to 'sdk-android-tutorial-a-running-pipeline.md')
-rw-r--r-- | sdk-android-tutorial-a-running-pipeline.md | 1061 |
1 files changed, 1061 insertions, 0 deletions
diff --git a/sdk-android-tutorial-a-running-pipeline.md b/sdk-android-tutorial-a-running-pipeline.md new file mode 100644 index 0000000..fe31f91 --- /dev/null +++ b/sdk-android-tutorial-a-running-pipeline.md @@ -0,0 +1,1061 @@ +# Android tutorial 2: A running pipeline + +# Goal ![](attachments/thumbnails/2687063/2654324) + +The tutorials seen in the [Basic](Basic%2Btutorials.html) and +[Playback](Playback%2Btutorials.html) sections are intended for Desktop +platforms and, therefore, their main thread is allowed to block (using +`gst_bus_pop_filtered()`) or relinquish control to a GLib main loop. On +Android this would lead to the application being tagged as +non-responsive and probably closed. + +This tutorial shows how to overcome this problem. In particular, we will +learn: + + - How to move the native code to its own thread + - How to allow threads created from C code to communicate with Java + - How to access Java code from C + - How to allocate a `CustomData` structure from C and have Java host + it + +# Introduction + +When using a Graphical User Interface (UI), if the application waits for +GStreamer calls to complete the user experience will suffer. The usual +approach, with the [GTK+ toolkit](http://www.gtk.org) for example, is to +relinquish control to a GLib `GMainLoop` and let it control the events +coming from the UI or GStreamer. + +This approach can be very cumbersome when GStreamer and the Android UI +communicate through the JNI interface, so we take a cleaner route: We +use a GLib main loop, and move it to its own thread, so it does not +block the application. This simplifies the GStreamer-Android +integration, and we only need to worry about a few inter-process +synchronization bits, which are detailed in this tutorial. + +Additionally, this tutorial shows how to obtain, from any thread, the +[JNI Environment +pointer](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/design.html#wp16696) +required to make JNI calls. This is necessary, for example, to call Java +code from callbacks in threads spawned deep within GStreamer, which +never received this pointer directly. + +Finally, this tutorial explains how to call Java methods from native C +code, which involves locating the desired method’s ID in the class. +These IDs never change, so they are cached as global variables in the C +code and obtained in the static initializer of the class. + +The code below builds a pipeline with an `audiotestsrc` and an +`autoaudiosink` (it plays an audible tone). Two buttons in the UI allow +setting the pipeline to PLAYING or PAUSED. A TextView in the UI shows +messages sent from the C code (for errors and state changes). + +# A pipeline on Android \[Java code\] + +**src/org/freedesktop/gstreamer/tutorials/tutorial\_2/Tutorial2.java** + +``` lang=java +package org.freedesktop.gstreamer.tutorials.tutorial_2; + +import android.app.Activity; +import android.os.Bundle; +import android.util.Log; +import android.view.View; +import android.view.View.OnClickListener; +import android.widget.ImageButton; +import android.widget.TextView; +import android.widget.Toast; + +import com.gstreamer.GStreamer; + +public class Tutorial2 extends Activity { + private native void nativeInit(); // Initialize native code, build pipeline, etc + private native void nativeFinalize(); // Destroy pipeline and shutdown native code + private native void nativePlay(); // Set pipeline to PLAYING + private native void nativePause(); // Set pipeline to PAUSED + private static native boolean nativeClassInit(); // Initialize native class: cache Method IDs for callbacks + private long native_custom_data; // Native code will use this to keep private data + + private boolean is_playing_desired; // Whether the user asked to go to PLAYING + + // Called when the activity is first created. + @Override + public void onCreate(Bundle savedInstanceState) + { + super.onCreate(savedInstanceState); + + // Initialize GStreamer and warn if it fails + try { + GStreamer.init(this); + } catch (Exception e) { + Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show(); + finish(); + return; + } + + setContentView(R.layout.main); + + ImageButton play = (ImageButton) this.findViewById(R.id.button_play); + play.setOnClickListener(new OnClickListener() { + public void onClick(View v) { + is_playing_desired = true; + nativePlay(); + } + }); + + ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop); + pause.setOnClickListener(new OnClickListener() { + public void onClick(View v) { + is_playing_desired = false; + nativePause(); + } + }); + + if (savedInstanceState != null) { + is_playing_desired = savedInstanceState.getBoolean("playing"); + Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired); + } else { + is_playing_desired = false; + Log.i ("GStreamer", "Activity created. There is no saved state, playing: false"); + } + + // Start with disabled buttons, until native code is initialized + this.findViewById(R.id.button_play).setEnabled(false); + this.findViewById(R.id.button_stop).setEnabled(false); + + nativeInit(); + } + + protected void onSaveInstanceState (Bundle outState) { + Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired); + outState.putBoolean("playing", is_playing_desired); + } + + protected void onDestroy() { + nativeFinalize(); + super.onDestroy(); + } + + // Called from native code. This sets the content of the TextView from the UI thread. + private void setMessage(final String message) { + final TextView tv = (TextView) this.findViewById(R.id.textview_message); + runOnUiThread (new Runnable() { + public void run() { + tv.setText(message); + } + }); + } + + // Called from native code. Native code calls this once it has created its pipeline and + // the main loop is running, so it is ready to accept commands. + private void onGStreamerInitialized () { + Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired); + // Restore previous playing state + if (is_playing_desired) { + nativePlay(); + } else { + nativePause(); + } + + // Re-enable buttons, now that GStreamer is initialized + final Activity activity = this; + runOnUiThread(new Runnable() { + public void run() { + activity.findViewById(R.id.button_play).setEnabled(true); + activity.findViewById(R.id.button_stop).setEnabled(true); + } + }); + } + + static { + System.loadLibrary("gstreamer_android"); + System.loadLibrary("tutorial-2"); + nativeClassInit(); + } + +} +``` + +As usual, the first bit that gets executed is the static initializer of +the class: + +``` lang=java +static { + System.loadLibrary("gstreamer_android"); + System.loadLibrary("tutorial-2"); + nativeClassInit(); +} +``` + +As explained in the previous tutorial, the two native libraries are +loaded and their `JNI_OnLoad()` methods are executed. Here, we also call +the native method `nativeClassInit()`, previously declared with the +`native` keyword in line 19. We will later see what its purpose is + +In the `onCreate()` method GStreamer is initialized as in the previous +tutorial with `GStreamer.init(this)`, and then the layout is inflated +and listeners are setup for the two UI buttons: + +``` lang=java +ImageButton play = (ImageButton) this.findViewById(R.id.button_play); +play.setOnClickListener(new OnClickListener() { + public void onClick(View v) { + is_playing_desired = true; + nativePlay(); + } +}); +ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop); +pause.setOnClickListener(new OnClickListener() { + public void onClick(View v) { + is_playing_desired = false; + nativePause(); + } +}); +``` + +Each button instructs the native code to set the pipeline to the desired +state, and also remembers this state in the +`is_playing_desired` variable. This is required so, when the +application is restarted (for example, due to an orientation change), it +can set the pipeline again to the desired state. This approach is easier +and safer than tracking the actual pipeline state, because orientation +changes can happen before the pipeline has moved to the desired state, +for example. + +``` lang=java +if (savedInstanceState != null) { + is_playing_desired = savedInstanceState.getBoolean("playing"); + Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired); +} else { + is_playing_desired = false; + Log.i ("GStreamer", "Activity created. There is no saved state, playing: false"); +} +``` + +Restore the previous playing state (if any) from `savedInstanceState`. +We will first build the GStreamer pipeline (below) and only when the +native code reports itself as initialized we will use +`is_playing_desired`. + +``` lang=java +nativeInit(); +``` + +As will be shown in the C code, `nativeInit()` creates a dedicated +thread, a GStreamer pipeline, a GLib main loop, and, right before +calling `g_main_loop_run()` and going to sleep, it warns the Java code +that the native code is initialized and ready to accept commands. + +This finishes the `onCreate()` method and the Java initialization. The +UI buttons are disabled, so nothing will happen until native code is +ready and `onGStreamerInitialized()` is called: + +``` lang=java +private void onGStreamerInitialized () { + Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired); +``` + +This is called by the native code when its main loop is finally running. +We first retrieve the desired playing state from `is_playing_desired`, +and then set that state: + +``` lang=java +// Restore previous playing state +if (is_playing_desired) { + nativePlay(); +} else { + nativePause(); +} +``` + +Here comes the first caveat, when re-enabling the UI buttons: + +``` lang=java +// Re-enable buttons, now that GStreamer is initialized +final Activity activity = this; +runOnUiThread(new Runnable() { + public void run() { + activity.findViewById(R.id.button_play).setEnabled(true); + activity.findViewById(R.id.button_stop).setEnabled(true); + } +}); +``` + +This method is being called from the thread that the native code created +to run its main loop, and is not allowed to issue UI-altering commands: +Only the UI thread can do that. The solution is easy though: Android +Activities have a handy +[runOnUiThread()](http://developer.android.com/reference/android/app/Activity.html#runOnUiThread\(java.lang.Runnable\)) +method which lets bits of code to be executed from the correct thread. A +[Runnable](http://developer.android.com/reference/java/lang/Runnable.html) +instance has to be constructed and any parameter can be passed either by +sub-classing +[Runnable](http://developer.android.com/reference/java/lang/Runnable.html) +and adding a dedicated constructor, or by using the `final` modifier, as +shown in the above snippet. + +The same problem exists when the native code wants to output a string in +our TextView using the `setMessage()` method: it has to be done from the +UI thread. The solution is the same: + +``` lang=java +private void setMessage(final String message) { + final TextView tv = (TextView) this.findViewById(R.id.textview_message); + runOnUiThread (new Runnable() { + public void run() { + tv.setText(message); + } + }); +} +``` + +Finally, a few remaining bits: + +``` lang=java +protected void onSaveInstanceState (Bundle outState) { + Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired); + outState.putBoolean("playing", is_playing_desired); +} +``` + +This method stores the currently desired playing state when Android is +about to shut us down, so next time it restarts (after an orientation +change, for example), it can restore the same state. + +``` lang=java +protected void onDestroy() { + nativeFinalize(); + super.onDestroy(); +} +``` + +And this is called before Android destroys our application. We call the +`nativeFinalize()`method to exit the main loop, destroy its thread and +all allocated resources. + +This concludes the UI part of the tutorial. + +# A pipeline on Android \[C code\] + +**jni/tutorial-2.c** + +``` lang=c +#include <string.h> +#include <jni.h> +#include <android/log.h> +#include <gst/gst.h> +#include <pthread.h> + +GST_DEBUG_CATEGORY_STATIC (debug_category); +#define GST_CAT_DEFAULT debug_category + +/* + * These macros provide a way to store the native pointer to CustomData, which might be 32 or 64 bits, into + * a jlong, which is always 64 bits, without warnings. + */ +#if GLIB_SIZEOF_VOID_P == 8 +# define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(*env)->GetLongField (env, thiz, fieldID) +# define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)data) +#else +# define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(jint)(*env)->GetLongField (env, thiz, fieldID) +# define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)(jint)data) +#endif + +/* Structure to contain all our information, so we can pass it to callbacks */ +typedef struct _CustomData { + jobject app; /* Application instance, used to call its methods. A global reference is kept. */ + GstElement *pipeline; /* The running pipeline */ + GMainContext *context; /* GLib context used to run the main loop */ + GMainLoop *main_loop; /* GLib main loop */ + gboolean initialized; /* To avoid informing the UI multiple times about the initialization */ +} CustomData; + +/* These global variables cache values which are not changing during execution */ +static pthread_t gst_app_thread; +static pthread_key_t current_jni_env; +static JavaVM *java_vm; +static jfieldID custom_data_field_id; +static jmethodID set_message_method_id; +static jmethodID on_gstreamer_initialized_method_id; + +/* + * Private methods + */ + +/* Register this thread with the VM */ +static JNIEnv *attach_current_thread (void) { + JNIEnv *env; + JavaVMAttachArgs args; + + GST_DEBUG ("Attaching thread %p", g_thread_self ()); + args.version = JNI_VERSION_1_4; + args.name = NULL; + args.group = NULL; + + if ((*java_vm)->AttachCurrentThread (java_vm, &env, &args) < 0) { + GST_ERROR ("Failed to attach current thread"); + return NULL; + } + + return env; +} + +/* Unregister this thread from the VM */ +static void detach_current_thread (void *env) { + GST_DEBUG ("Detaching thread %p", g_thread_self ()); + (*java_vm)->DetachCurrentThread (java_vm); +} + +/* Retrieve the JNI environment for this thread */ +static JNIEnv *get_jni_env (void) { + JNIEnv *env; + + if ((env = pthread_getspecific (current_jni_env)) == NULL) { + env = attach_current_thread (); + pthread_setspecific (current_jni_env, env); + } + + return env; +} + +/* Change the content of the UI's TextView */ +static void set_ui_message (const gchar *message, CustomData *data) { + JNIEnv *env = get_jni_env (); + GST_DEBUG ("Setting message to: %s", message); + jstring jmessage = (*env)->NewStringUTF(env, message); + (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage); + if ((*env)->ExceptionCheck (env)) { + GST_ERROR ("Failed to call Java method"); + (*env)->ExceptionClear (env); + } + (*env)->DeleteLocalRef (env, jmessage); +} + +/* Retrieve errors from the bus and show them on the UI */ +static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) { + GError *err; + gchar *debug_info; + gchar *message_string; + + gst_message_parse_error (msg, &err, &debug_info); + message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message); + g_clear_error (&err); + g_free (debug_info); + set_ui_message (message_string, data); + g_free (message_string); + gst_element_set_state (data->pipeline, GST_STATE_NULL); +} + +/* Notify UI about pipeline state changes */ +static void state_changed_cb (GstBus *bus, GstMessage *msg, CustomData *data) { + GstState old_state, new_state, pending_state; + gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state); + /* Only pay attention to messages coming from the pipeline, not its children */ + if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) { + gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state)); + set_ui_message(message, data); + g_free (message); + } +} + +/* Check if all conditions are met to report GStreamer as initialized. + * These conditions will change depending on the application */ +static void check_initialization_complete (CustomData *data) { + JNIEnv *env = get_jni_env (); + if (!data->initialized && data->main_loop) { + GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop); + (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id); + if ((*env)->ExceptionCheck (env)) { + GST_ERROR ("Failed to call Java method"); + (*env)->ExceptionClear (env); + } + data->initialized = TRUE; + } +} + +/* Main method for the native code. This is executed on its own thread. */ +static void *app_function (void *userdata) { + JavaVMAttachArgs args; + GstBus *bus; + CustomData *data = (CustomData *)userdata; + GSource *bus_source; + GError *error = NULL; + + GST_DEBUG ("Creating pipeline in CustomData at %p", data); + + /* Create our own GLib Main Context and make it the default one */ + data->context = g_main_context_new (); + g_main_context_push_thread_default(data->context); + + /* Build pipeline */ + data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error); + if (error) { + gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message); + g_clear_error (&error); + set_ui_message(message, data); + g_free (message); + return NULL; + } + + /* Instruct the bus to emit signals for each received message, and connect to the interesting signals */ + bus = gst_element_get_bus (data->pipeline); + bus_source = gst_bus_create_watch (bus); + g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL); + g_source_attach (bus_source, data->context); + g_source_unref (bus_source); + g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data); + g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data); + gst_object_unref (bus); + + /* Create a GLib Main Loop and set it to run */ + GST_DEBUG ("Entering main loop... (CustomData:%p)", data); + data->main_loop = g_main_loop_new (data->context, FALSE); + check_initialization_complete (data); + g_main_loop_run (data->main_loop); + GST_DEBUG ("Exited main loop"); + g_main_loop_unref (data->main_loop); + data->main_loop = NULL; + + /* Free resources */ + g_main_context_pop_thread_default(data->context); + g_main_context_unref (data->context); + gst_element_set_state (data->pipeline, GST_STATE_NULL); + gst_object_unref (data->pipeline); + + return NULL; +} + +/* + * Java Bindings + */ + +/* Instruct the native code to create its internal data structure, pipeline and thread */ +static void gst_native_init (JNIEnv* env, jobject thiz) { + CustomData *data = g_new0 (CustomData, 1); + SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data); + GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "Android tutorial 2"); + gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG); + GST_DEBUG ("Created CustomData at %p", data); + data->app = (*env)->NewGlobalRef (env, thiz); + GST_DEBUG ("Created GlobalRef for app object at %p", data->app); + pthread_create (&gst_app_thread, NULL, &app_function, data); +} + +/* Quit the main loop, remove the native thread and free resources */ +static void gst_native_finalize (JNIEnv* env, jobject thiz) { + CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id); + if (!data) return; + GST_DEBUG ("Quitting main loop..."); + g_main_loop_quit (data->main_loop); + GST_DEBUG ("Waiting for thread to finish..."); + pthread_join (gst_app_thread, NULL); + GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app); + (*env)->DeleteGlobalRef (env, data->app); + GST_DEBUG ("Freeing CustomData at %p", data); + g_free (data); + SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL); + GST_DEBUG ("Done finalizing"); +} + +/* Set pipeline to PLAYING state */ +static void gst_native_play (JNIEnv* env, jobject thiz) { + CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id); + if (!data) return; + GST_DEBUG ("Setting state to PLAYING"); + gst_element_set_state (data->pipeline, GST_STATE_PLAYING); +} + +/* Set pipeline to PAUSED state */ +static void gst_native_pause (JNIEnv* env, jobject thiz) { + CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id); + if (!data) return; + GST_DEBUG ("Setting state to PAUSED"); + gst_element_set_state (data->pipeline, GST_STATE_PAUSED); +} + +/* Static class initializer: retrieve method and field IDs */ +static jboolean gst_native_class_init (JNIEnv* env, jclass klass) { + custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J"); + set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V"); + on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V"); + + if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) { + /* We emit this message through the Android log instead of the GStreamer log because the later + * has not been initialized yet. + */ + __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods"); + return JNI_FALSE; + } + return JNI_TRUE; +} + +/* List of implemented native methods */ +static JNINativeMethod native_methods[] = { + { "nativeInit", "()V", (void *) gst_native_init}, + { "nativeFinalize", "()V", (void *) gst_native_finalize}, + { "nativePlay", "()V", (void *) gst_native_play}, + { "nativePause", "()V", (void *) gst_native_pause}, + { "nativeClassInit", "()Z", (void *) gst_native_class_init} +}; + +/* Library initializer */ +jint JNI_OnLoad(JavaVM *vm, void *reserved) { + JNIEnv *env = NULL; + + java_vm = vm; + + if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) { + __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv"); + return 0; + } + jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_2/Tutorial2"); + (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods)); + + pthread_key_create (¤t_jni_env, detach_current_thread); + + return JNI_VERSION_1_4; +} +``` + +Let’s start with the `CustomData` structure. We have seen it in most of +the basic tutorials, and it is used to hold all our information in one +place, so we can easily pass it around to +callbacks: + +``` lang=c +/* Structure to contain all our information, so we can pass it to callbacks */ +typedef struct _CustomData { + jobject app; /* Application instance, used to call its methods. A global reference is kept. */ + GstElement *pipeline; /* The running pipeline */ + GMainContext *context; /* GLib context used to run the main loop */ + GMainLoop *main_loop; /* GLib main loop */ + gboolean initialized; /* To avoid informing the UI multiple times about the initialization */ +} CustomData; +``` + +We will see the meaning of each member as we go. What is interesting now +is that `CustomData` belongs to the application, so a pointer is kept in +the Tutorial2 Java class in the `private long +native_custom_data` attribute. Java only holds this pointer for us; it +is completely handled in C code. + +From C, this pointer can be set and retrieved with the +[SetLongField()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp16613) +and +[GetLongField()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp16572) +JNI functions, but two convenience macros have been defined, +`SET_CUSTOM_DATA` and `GET_CUSTOM_DATA`. These macros are handy because +the `long` type used in Java is always 64 bits wide, but the pointer +used in C can be either 32 or 64 bits wide. The macros take care of the +conversion without warnings. + +``` lang=c +/* Library initializer */ +jint JNI_OnLoad(JavaVM *vm, void *reserved) { + JNIEnv *env = NULL; + + java_vm = vm; + + if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) { + __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv"); + return 0; + } + jclass klass = (*env)->FindClass (env, "org/freedesktop/gstreamer/tutorials/tutorial_2/Tutorial2"); + (*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods)); + + pthread_key_create (¤t_jni_env, detach_current_thread); + + return JNI_VERSION_1_4; +} +``` + +The `JNI_OnLoad` function is almost the same as the previous tutorial. +It registers the list of native methods (which is longer in this +tutorial). It also +uses [pthread\_key\_create()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_key_create.html) +to be able to store per-thread information, which is crucial to properly +manage the JNI Environment, as shown later. + +``` lang=c +/* Static class initializer: retrieve method and field IDs */ +static jboolean gst_native_class_init (JNIEnv* env, jclass klass) { + custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J"); + set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V"); + on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V"); + + if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) { + /* We emit this message through the Android log instead of the GStreamer log because the later + * has not been initialized yet. + */ + __android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods"); + return JNI_FALSE; + } + return JNI_TRUE; +} +``` + +This method is called from the static initializer of the Java class, +which is passed as a parameter (since this is called from a static +method, it receives a class object instead of an instance object). In +order for C code to be able to call a Java method, it needs to know the +method’s +[MethodID](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/types.html#wp1064). +This ID is obtained from the method’s name and signature and can be +cached. The purpose of the `gst_native_class_init()` function is to +obtain the IDs of all the methods and fields that the C code will need. +If some ID cannot be retrieved, the calling Java class does not offer +the expected interface and execution should halt (which is not currently +done for simplicity). + +Let’s review now the first native method which can be directly called +from Java: + +### `gst_native_init()` (`nativeInit()` from Java) + +This method is called at the end of Java's `onCreate()`. + +``` lang=c +static void gst_native_init (JNIEnv* env, jobject thiz) { + CustomData *data = g_new0 (CustomData, 1); + SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data); +``` + +It first allocates memory for the `CustomData` structure and passes the +pointer to the Java class with `SET_CUSTOM_DATA`, so it is remembered. + +``` lang=c +data->app = (*env)->NewGlobalRef (env, thiz); +``` + +A pointer to the application class (the `Tutorial2` class) is also kept +in `CustomData` (a [Global +Reference](http://developer.android.com/guide/practices/jni.html#local_and_global_references) +is used) so its methods can be called later. + +``` lang=c +pthread_create (&gst_app_thread, NULL, &app_function, data); +``` + +Finally, a thread is created and it starts running the +`app_function()` method. + +### `app_function()` + +``` lang=c +/* Main method for the native code. This is executed on its own thread. */ +static void *app_function (void *userdata) { + JavaVMAttachArgs args; + GstBus *bus; + CustomData *data = (CustomData *)userdata; + GSource *bus_source; + GError *error = NULL; + + GST_DEBUG ("Creating pipeline in CustomData at %p", data); + + /* Create our own GLib Main Context and make it the default one */ + data->context = g_main_context_new (); + g_main_context_push_thread_default(data->context); +``` + +It first creates a GLib context so all `GSource` are kept in the same +place. This also helps cleaning after GSources created by other +libraries which might not have been properly disposed of. A new context +is created with `g_main_context_new()` and then it is made the default +one for the thread with +`g_main_context_push_thread_default()`. + +``` lang=c +data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error); +if (error) { + gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message); + g_clear_error (&error); + set_ui_message(message, data); + g_free (message); + return NULL; +} +``` + +It then creates a pipeline the easy way, with `gst-parse-launch()`. In +this case, it is simply an `audiotestsrc` (which produces a continuous +tone) and an `autoaudiosink`, with accompanying adapter elements. + +``` lang=c +bus = gst_element_get_bus (data->pipeline); +bus_source = gst_bus_create_watch (bus); +g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL); +g_source_attach (bus_source, data->context); +g_source_unref (bus_source); +g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data); +g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data); +gst_object_unref (bus); +``` + +These lines create a bus signal watch and connect to some interesting +signals, just like we have been doing in the basic tutorials. The +creation of the watch is done step by step instead of using +`gst_bus_add_signal_watch()` to exemplify how to use a custom GLib +context. + +``` lang=c +GST_DEBUG ("Entering main loop... (CustomData:%p)", data); +data->main_loop = g_main_loop_new (data->context, FALSE); +check_initialization_complete (data); +g_main_loop_run (data->main_loop); +GST_DEBUG ("Exited main loop"); +g_main_loop_unref (data->main_loop); +data->main_loop = NULL; +``` + +Finally, the main loop is created and set to run. When it exits (because +somebody else calls `g_main_loop_quit()`) the main loop is disposed of. +Before entering the main loop, though, +`check_initialization_complete()` is called. This method checks if all +conditions are met to consider the native code “ready” to accept +commands. Since having a running main loop is one of the conditions, +`check_initialization_complete()` is called here. This method is +reviewed below. + +Once the main loop has quit, all resources are freed in lines 178 to +181. + +### `check_initialization_complete()` + +``` lang=c +static void check_initialization_complete (CustomData *data) { + JNIEnv *env = get_jni_env (); + if (!data->initialized && data->main_loop) { + GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop); + (*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id); + if ((*env)->ExceptionCheck (env)) { + GST_ERROR ("Failed to call Java method"); + (*env)->ExceptionClear (env); + } + data->initialized = TRUE; + } +} +``` + +This method does not do much in this tutorial, but it will also be used +in the next ones, with progressively more complex functionality. Its +purpose is to check if the native code is ready to accept commands, and, +if so, notify the UI code. + +In tutorial 2, the only conditions are 1) the code is not already +initialized and 2) the main loop is running. If these two are met, the +Java `onGStreamerInitialized()` method is called via the +[CallVoidMethod()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp4256) +JNI call. + +Here comes a tricky bit. JNI calls require a JNI Environment, **which is +different for every thread**. C methods called from Java receive a +`JNIEnv` pointer as a parameter, but this is not the situation with +`check_initialization_complete()`. Here, we are in a thread which has +never been called from Java, so we have no `JNIEnv`. We need to use the +`JavaVM` pointer (passed to us in the `JNI_OnLoad()` method, and shared +among all threads) to attach this thread to the Java Virtual Machine and +obtain a `JNIEnv`. This `JNIEnv` is stored in the [Thread-Local +Storage](http://en.wikipedia.org/wiki/Thread-local_storage) (TLS) using +the pthread key we created in `JNI_OnLoad()`, so we do not need to +attach the thread anymore. + +This behavior is implemented in the `get_jni_env()` method, used for +example in `check_initialization_complete()` as we have just seen. Let’s +see how it works, step by step: + +### `get_jni_env()` + +``` lang=c +static JNIEnv *get_jni_env (void) { + JNIEnv *env; + if ((env = pthread_getspecific (current_jni_env)) == NULL) { + env = attach_current_thread (); + pthread_setspecific (current_jni_env, env); + } + return env; +} +``` + +It first retrieves the current `JNIEnv` from the TLS using +[pthread\_getspecific()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_getspecific.html) +and the key we obtained from +[pthread\_key\_create()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_key_create.html). +If it returns NULL, we never attached this thread, so we do now with +`attach_current_thread()` and then store the new `JNIEnv` into the TLS +with +[pthread\_setspecific()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_setspecific.html). + +### `attach_current_thread()` + +This method is simply a convenience wrapper around +[AttachCurrentThread()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#attach_current_thread) +to deal with its parameters. + +### `detach_current_thread()` + +This method is called by the pthreads library when a TLS key is deleted, +meaning that the thread is about to be destroyed. We simply detach the +thread from the JavaVM with +[DetachCurrentThread()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#detach_current_thread). + +Let's now review the rest of the native methods accessible from Java: + +### `gst_native_finalize()` (`nativeFinalize()` from Java) + +``` lang=c +static void gst_native_finalize (JNIEnv* env, jobject thiz) { + CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id); + if (!data) return; + GST_DEBUG ("Quitting main loop..."); + g_main_loop_quit (data->main_loop); + GST_DEBUG ("Waiting for thread to finish..."); + pthread_join (gst_app_thread, NULL); + GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app); + (*env)->DeleteGlobalRef (env, data->app); + GST_DEBUG ("Freeing CustomData at %p", data); + g_free (data); + SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL); + GST_DEBUG ("Done finalizing"); +} +``` + +This method is called from Java in `onDestroy()`, when the activity is +about to be destroyed. Here, we: + + - Instruct the GLib main loop to quit with `g_main_loop_quit()`. This + call returns immediately, and the main loop will terminate at its + earliest convenience. + - Wait for the thread to finish with + [pthread\_join()](http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_join.html). + This call blocks until the `app_function()` method returns, meaning + that the main loop has exited, and the thread has been destroyed. + - Dispose of the global reference we kept for the Java application + class (`Tutorial2`) in `CustomData`. + - Free `CustomData` and set the Java pointer inside the + `Tutorial2` class to NULL with +`SET_CUSTOM_DATA()`. + +### `gst_native_play` and `gst_native_pause()` (`nativePlay` and `nativePause()` from Java) + +These two simple methods retrieve `CustomData` from the passed-in object +with `GET_CUSTOM_DATA()` and set the pipeline found inside `CustomData` +to the desired state, returning immediately. + +Finally, let’s see how the GStreamer callbacks are handled: + +### `error_cb` and `state_changed_cb` + +This tutorial does not do much in these callbacks. They simply parse the +error or state changed message and display a message in the UI using the +`set_ui_message()` method: + +### `set_ui_message()` + +``` lang=c +static void set_ui_message (const gchar *message, CustomData *data) { + JNIEnv *env = get_jni_env (); + GST_DEBUG ("Setting message to: %s", message); + jstring jmessage = (*env)->NewStringUTF(env, message); + (*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage); + if ((*env)->ExceptionCheck (env)) { + GST_ERROR ("Failed to call Java method"); + (*env)->ExceptionClear (env); + } + (*env)->DeleteLocalRef (env, jmessage); +} +``` + + + +This is the other method (besides `check_initialization_complete()`) +that needs to call a Java function from a thread which never received an +`JNIEnv` pointer directly. Notice how all the complexities of attaching +the thread to the JavaVM and storing the JNI environment in the TLS are +hidden in the simple call to `get_jni_env()`. + +The desired message (received in +[ASCII](http://en.wikipedia.org/wiki/ASCII), or modified +[UTF8](http://en.wikipedia.org/wiki/Modified_UTF-8#Modified_UTF-8)), is +converted to [UTF16](http://en.wikipedia.org/wiki/UTF-16) as required by +Java using the +[NewStringUTF()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp17220) +JNI call. + +The `setMessage()` Java method is called via the JNI +[CallVoidMethod()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#wp4256) +using the global reference to the class we are keeping in +`CustomData` (`data->app`) and the `set_message_method_id` we cached in +`gst_native_class_init()`. + +We check for exceptions with the JNI +[ExceptionCheck()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#exception_check) +method and free the UTF16 message with +[DeleteLocalRef()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/functions.html#DeleteLocalRef). + +# A pipeline on Android \[Android.mk\] + +**jni/Android.mk** + +``` lang=ruby +LOCAL_PATH := $(call my-dir) + +include $(CLEAR_VARS) + +LOCAL_MODULE := tutorial-2 +LOCAL_SRC_FILES := tutorial-2.c +LOCAL_SHARED_LIBRARIES := gstreamer_android +LOCAL_LDLIBS := -llog +include $(BUILD_SHARED_LIBRARY) + +ifndef GSTREAMER_ROOT +ifndef GSTREAMER_ROOT_ANDROID +$(error GSTREAMER_ROOT_ANDROID is not defined!) +endif +GSTREAMER_ROOT := $(GSTREAMER_ROOT_ANDROID) +endif +GSTREAMER_NDK_BUILD_PATH := $(GSTREAMER_ROOT)/share/gst-android/ndk-build/ +include $(GSTREAMER_NDK_BUILD_PATH)/plugins.mk +GSTREAMER_PLUGINS := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_SYS) +include $(GSTREAMER_NDK_BUILD_PATH)/gstreamer-1.0.mk +``` + +Notice how the required `GSTREAMER_PLUGINS` are now +`$(GSTREAMER_PLUGINS_CORE)` (For the test source and converter elements) +and `$(GSTREAMER_PLUGINS_SYS)` (for the audio sink). + +And this is it\! This has been a rather long tutorial, but we covered a +lot of territory. Building on top of this one, the following ones are +shorter and focus only on the new topics. + +# Conclusion + +This tutorial has shown: + + - How to manage multiple threads from C code and have them interact + with java. + - How to access Java code from any C thread + using [AttachCurrentThread()](http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#attach_current_thread). + - How to allocate a CustomData structure from C and have Java host it, + so it is available to all threads. + +Most of the methods introduced in this tutorial, like `get_jni_env()`, +`check_initialization_complete()`, `app_function()` and the API methods +`gst_native_init()`, `gst_native_finalize()` and +`gst_native_class_init()` will continue to be used in the following +tutorials with minimal modifications, so better get used to them\! + +As usual, it has been a pleasure having you here, and see you soon\! + +## Attachments: + +![](images/icons/bullet_blue.gif) +[tutorial2-screenshot.png](attachments/2687063/2654325.png) +(image/png) +![](images/icons/bullet_blue.gif) +[tutorial2-screenshot.png](attachments/2687063/2654412.png) +(image/png) +![](images/icons/bullet_blue.gif) +[tutorial2-screenshot.png](attachments/2687063/2654417.png) +(image/png) +![](images/icons/bullet_blue.gif) +[tutorial2-screenshot.png](attachments/2687063/2654324.png) +(image/png) |