diff options
| author | Tim-Philipp Müller <tim@centricular.com> | 2024-03-04 23:42:09 +0000 |
|---|---|---|
| committer | Tim-Philipp Müller <tim@centricular.com> | 2024-03-05 01:00:39 +0000 |
| commit | 31f4b63243a02c665528952cb4e1887f9df45f8b (patch) | |
| tree | 26f7031fad0b402752fd8c487bce37073d9a6dc0 | |
| parent | a16c82069bcf300dcc54d85c4bd897dee8c4aa1f (diff) | |
releases: add release notes for 1.24
Co-authored-by: Edward Hervey <edward@centricular.com>
Co-authored-by: Matthew Waters <matthew@centricular.com>
Co-authored-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Co-authored-by: Nirbheek Chauhan <nirbheek@centricular.com>
Co-authored-by: Olivier Crête <olivier.crete@collabora.com>
Co-authored-by: Sebastian Dröge <sebastian@centricular.com>
Co-authored-by: Seungha Yang <seungha@centricular.com>
Co-authored-by: Thibault Saunier <tsaunier@igalia.com>
Co-authored-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
| -rw-r--r-- | src/htdocs/releases/1.24/release-notes-1.24.md | 1486 |
1 files changed, 1422 insertions, 64 deletions
diff --git a/src/htdocs/releases/1.24/release-notes-1.24.md b/src/htdocs/releases/1.24/release-notes-1.24.md index ae16c8a4..3e191533 100644 --- a/src/htdocs/releases/1.24/release-notes-1.24.md +++ b/src/htdocs/releases/1.24/release-notes-1.24.md @@ -1,94 +1,996 @@ # GStreamer 1.24 Release Notes -GStreamer 1.24 has not been released yet. It is scheduled for release ASAP. - -GStreamer 1.23.90 is the first release candidate (rc1) for 1.24. - -1.24 will be backwards-compatible to the stable 1.22, 1.20, 1.18, 1.16, 1.14, -1.12, 1.10, 1.8, 1.6,, 1.4, 1.2 and 1.0 release series. +GStreamer 1.24.0 was originally released on 4 March 2024. See [https://gstreamer.freedesktop.org/releases/1.24/][latest] for the latest version of this document. -*Last updated: Friday 23 February 2024, 13:00 UTC [(log)][gitlog]* +*Last updated: Monday 4 March 2024, 23:00 UTC [(log)][gitlog]* [latest]: https://gstreamer.freedesktop.org/releases/1.24/ [gitlog]: https://gitlab.freedesktop.org/gstreamer/www/commits/main/src/htdocs/releases/1.24/release-notes-1.24.md ## Introduction -The GStreamer team is proud to announce a new major feature release in the stable 1.x API series of your favourite cross-platform multimedia framework! +The GStreamer team is proud to announce a new major feature release in the +stable 1.x API series of your favourite cross-platform multimedia framework! As always, this release is again packed with many new features, bug fixes and other improvements. ## Highlights -- This section will be completed in due course +- New [Discourse forum][discourse] and [Matrix chat space][matrix] +- New Analytics and Machine Learning abstractions and elements +- Playbin3 and decodebin3 are now stable and the default in gst-play-1.0, GstPlay/GstPlayer +- The va plugin is now preferred over gst-vaapi and has higher ranks +- GstMeta serialization/deserialization and other GstMeta improvements +- New GstMeta for SMPTE ST-291M HANC/VANC Ancillary Data +- New unixfd plugin for efficient 1:N inter-process communication on Linux +- cudaipc source and sink for zero-copy CUDA memory sharing between processes +- New intersink and intersrc elements for 1:N pipeline decoupling within the same process +- Qt5 + Qt6 QML integration improvements including qml6glsrc, qml6glmixer, qml6gloverlay, and qml6d3d11sink elements +- DRM Modifier Support for dmabufs on Linux +- OpenGL, Vulkan and CUDA integration enhancements +- Vulkan H.264 and H.265 video decoders +- RTP stack improvements including new RFC7273 modes and more correct header extension handling in depayloaders +- WebRTC improvements such as support for ICE consent freshness, and a new webrtcsrc element to complement webrtcsink +- WebRTC signallers and webrtcsink implementations for LiveKit and AWS Kinesis Video Streams +- WHIP server source and client sink, and a WHEP source +- Precision Time Protocol (PTP) clock support for Windows and other additions +- Low-Latency HLS (LL-HLS) support and many other HLS and DASH enhancements +- New W3C Media Source Extensions library +- Countless closed caption handling improvements including new cea608mux and cea608tocea708 elements +- Translation support for awstranscriber +- Bayer 10/12/14/16-bit depth support +- MPEG-TS support for asynchronous KLV demuxing and segment seeking, plus various new muxer features +- Capture source and sink for AJA capture and playout cards +- SVT-AV1 and VA-API AV1 encoders, stateless AV1 video decoder +- New uvcsink element for exporting streams as UVC camera +- DirectWrite text rendering plugin for windows +- Direct3D12-based video decoding, conversion, composition, and rendering +- AMD Advanced Media Framework AV1 + H.265 video encoders with 10-bit and HDR support +- AVX/AVX2 support and NEON support on macOS on Apple ARM64 CPUs via new liborc +- GStreamer C# bindings have been updated +- Rust bindings improvements and many new and improved Rust plugins +- Rust plugins now shipped in packages for all major platforms including Android and iOS +- Lots of new plugins, features, performance improvements and bug fixes ## Major new features and changes -- This section will be completed in due course +### Discourse forum and Matrix chat space + +- The new [Discourse forum][discourse] and [Matrix chat space][matrix] are + now our preferred communication channels for support and developer chat. + +- The mailing lists and IRC channel are on their way to being phased out, + but Discourse can be used [via e-mail][discourse-email] as well. + +- For release announcements please subscribe to the News + Announcements + category on Discourse, although we will continue to also send announcements + to the mailing list for the time being. + +[discourse]: https://discourse.gstreamer.org +[discourse-email]: https://discourse.gstreamer.org/t/using-discourse-via-email/109 +[matrix]: https://discourse.gstreamer.org/t/new-gstreamer-matrix-chat-space/675 + +### Playbin3, decodebin3 now stable and default + +- After a year of stability, testing and more improvements, `playbin3`, and its + various components (`uridecodebin3`, `decodebin3` and `urisourcebin`), are now + the recommended playback components. + +- Some playback components have now switched to defaulting to `playbin3`: + `gst-play-1.0` and the `GstPlay` / `GstPlayer` libraries. Application developers are + strongly recommended to switch to using those components instead of the + legacy `playbin` and `(uri)decodebin`. + +Improvements in this cycle: + +- Better support missing/faulty decoders, attempt to use another one or + gracefully un-select the stream. + +- Many fixes for more complex gapless and instant-switching scenarios + +- Lower latency for live pipelines + +- Fix for "chained" streams (ex: Ogg, or PMT update in MPEG-TS) + +- Fixes for hardware-accelerated playback with subtitles (provided the sink + can handle offloading composition). This was also partly due to a historical + confusion between subtitle "decoders" (which decode the format to text + and "parsers" (which only do timing detection and optional seeking). + +### GstMeta serialization/deserialization and other GstMeta improvements + +- **GstMeta serialization/deserialization** allows metas to be transmitted or + stored. This is used by the `unixfd` and `cudaipc` plugins for inter-process + communication (IPC). Implemented so far for `GstCustomMeta`, `GstVideoMeta`, + `GstAudioMeta` and `GstReferenceTimestampMeta`. + +- **Simplified `GstCustomMeta` registration** with + `gst_meta_register_custom_simple()` for the simple + case where tags and transform functions are not needed. + +- `GstMetaClearFunction` clears the content of the meta. This will be + called by buffer pools when a pooled buffer is returned to the pool. + +- Add `gst_meta_info_new()` and `gst_meta_info_register()` to + **register a GstMeta in two steps** for easier extensibility. + +### New unixfd plugin for efficient 1:N inter-process communication on Linux + +- **unixfdsink** and **unixfdsrc** are elements that, inspired by + `shmsink` andn `shmsrc`, send UNIX file descriptors (e.g. memfd, dmabuf) + from one sink to multiple source elements in other processes on Linux. + +- The `unixfdsink` proposes a memfd/shm allocator to upstream elements which + allows for example `videotestsrc` to write directly into memory that can be + transfered to other processes without copying. + +### New GstMeta for SMPTE ST-291M HANC/VANC Ancillary Data + +- Previously only various specific `GstMeta` for ancillary data were + provided, such as `GstVideoCaptionMeta` and `GstVideoAFDMeta`. The + new `GstAncillaryMeta` allows passing arbitrary ancillary data between + elements, including custom and non-standard ancillary data. + See [`GstAncillaryMeta`][anc-meta] for details. + +- Add with `gst_buffer_add_ancillary_meta()` and retrieve with + `gst_buffer_get_ancillary_meta()` or `gst_buffer_iterate_ancillary_meta()`. + +- Supported by the newly added AJA sink and source elements + +[anc-meta]: https://gstreamer.freedesktop.org/documentation/video/gstvideoanc.html#GstAncillaryMeta + +### DSD audio support + +- [DSD audio][dsd-audio] is a non-PCM raw audio format representation and the + GstAudio library gained support for this in form of new [`GstDsdInfo`][dsd-info] + and [`GstDsdFormat`][dsd-format] API. + +- Support for DSD audio has been implemented in `alsasink` as well as the + `GstAudioSink` and `GstAudioRingBuffer` base classes, and the gst-libav + plugin to enable FFmpeg-based DSD elements and functionality. + +[dsd-audio]: https://en.wikipedia.org/wiki/Direct_Stream_Digital#DSD_signal_format +[dsd-format]: https://gstreamer.freedesktop.org/documentation/audio/gstdsdformat.html#GstDsdFormat +[dsd-info]: https://gstreamer.freedesktop.org/documentation/audio/gstdsd.html#GstDsdInfo + +### Analytics and Machine Learning + +- A new library, [GstAnalytics][analytics], has been added. It defines a + [GstAnalyticsRelationMeta][analytics-relation-meta] that can efficiently hold + a large number of observations from a data analysis process, for example + from machine learning. It also contains a matrix of the relationship + between those observations. + +- Three types of metadata are already defined in the library: object + detection, classification and tracking. + +- A new `objectdetectionoverlay` element has been merged that draws the + bounding boxes and the classes from the object detection and + classification metadata types. + +- The `onnxinference` element has been split into two parts. The first part + works with the ONNX Runtime library to do the actual inference, while the + second part called `ssdobjectdetector` interprets the produced tensor. + This new element creates [GstAnalyticsRelationMeta][analytics-relation-meta]. + +- The `onnxinference` element now accepts video frames without + transformation if the module declares that it accepts the "Image" type + and the format is something that GStreamer knows. + +- In the next release, tensor decoders such as `ssdobjectdetector` will + live outside of the ONNX plugin so they can be used with other machine + learning acceleration frameworks. + +[analytics]: https://gstreamer.freedesktop.org/documentation/analytics/ +[analytics-relation-meta]: https://gstreamer.freedesktop.org/documentation/analytics/gstanalyticsmeta.html#GstAnalyticsMtdImpl + +### Qt5 + Qt6 QML integration improvements + +- The Qt5 **qmlglsink**, **qmlgloverlay**, **qmlglmixer** received support + for directly consuming `BGRA` and `YV12` video frames without a + prior `glcolorconvert`. + +- New **qml6glsrc**, **qml6glmixer**, and **qml6gloverlay** elements as + Qt6 counterparts to the existing Qt5 elements, also with support for directly + consuming `BGRA` and `YV12` video frames without a prior `glcolorconvert`. + +- **qml6d3d11sink** is a new **Direct3D11 Qt6 QML sink** for Windows as + an alternative to the existing qml6glsink. + +### DRM Modifier Support for dmabufs on Linux + +The [Linux dmabuf subsystem][kernel-dmabuf] provides buffer sharing across +different hardware device drivers and subsystems, and is used extensively by +the DRM subsystem to exchange buffers between processes, contexts, and +library APIs within the same process, and also to exchange buffers with +other subsystems such as Linux Media. + +In GStreamer, it's used on the capture side (v4l2src, pipewire), +hardware-accelerated video decoders and encoders, OpenGL integration, +Wayland renderers, etc. + +GStreamer has had support for dmabufs for a long time and was able to negotiate +"zero-copy" paths between different components, however it only supported and +assumed simple linear formats and was not able to negotiate complex non-linear +formats. This meant that dmabuf support actually had to be disabled in many +scenarios to avoid "garbled video". + +With GStreamer 1.24 there is now full DRM modifier support and complex +non-linear formats can be supported and negotiated between components. + +This is achieved with an extra `drm_format` field in +`video/x-raw(memory:DMABuf), format=(string)DMA_DRM` caps, e.g. +`drm-format=(string)NV12:0x0x0100000000000001`. + +See the [GStreamer DMA buffers design documentation][gstreamer-dmabuf] for +more details. + +This is used in the VA-API `va` plugin, `waylandsink`, the MSDK plugin, +and the OpenGL integration. Video4Linux support is expected to land in +one of the next minor releases. + +New API has been added for easy handling of these new caps: + + - [`GstVideoInfoDmaDrm`][video-info-dma-drm] plus associated functions, + similar to `GstVideoInfo`, including conversion to and from `GstVideoInfo` + with `gst_video_info_dma_drm_from_video_info()` and + `gst_video_info_dma_drm_to_video_info()` + + - `GST_VIDEO_DMA_DRM_CAPS_MAKE` + + - `GST_VIDEO_FORMAT_DMA_DRM` + +[gstreamer-dmabuf]: https://gstreamer.freedesktop.org/documentation/additional/design/dmabuf.html +[kernel-dmabuf]: https://docs.kernel.org/driver-api/dma-buf.html + +[video-info-dma-drm]: https://gstreamer.freedesktop.org/documentation/video/video-info-dma-drm.html#GstVideoInfoDmaDrm + +### OpenGL integration enhancements + +- When using EGL, if both OpenGL ES and OpenGL are available, OpenGL ES is preferred + over OpenGL. OpenGL ES supports some necessary features required for dmabuf support. + This does not apply if an external library/application chooses an OpenGL API first. + +- Improved support for dmabuf use cases. The `glupload` element now supports + the new and improved dmabuf negotiation with explicit modifiers. + +- Base classes for mixing with OpenGL are now public API. `GstGLBaseMixer` + and `GstGLMixer` are exposed matching the existing filter-based + `GstGLBaseFilter` and `GstGLFilter` base classes. The new OpenGL mixer base + classes are based on `GstVideoAggregator`. + +- Add support for a 'surfaceless' EGL context using `EGL_MESA_platform_surfaceless`. + +- Expose Vivante Framebuffer build-related files (pkg-config, gir) as public API + +- Add support for more video formats: + - A420 8/10/12/16-bit. + - A422 8/10/12/16-bit. + - A444 8/10/12/16-bit. + - I420 10/12 bit. + - RBGA. + +- Add support for tiled video formats + - `NV12_16L32S` (Mediatek format) + - `NV12_4L4` (Verisilicon Hantro format) + +- `glcolorconvert` now has API for retrieving shader strings for: + - swizzling (reordering components). + - YUV->RGB conversion. + - RGB->YUV conversion. + +- Add more helpers for information about a particular video and/or GL format + e.g. number of components, bytes used, or pixel ordering. + +- `glvideomixer` has new sink pad properties `sizing-policy`, `xalign`, + `yalign` matching `compositor`. + +- `GstGLBufferPool` now has a configuration option for allowing a number of buffers + to be always outstanding allowing for reducing the potential synchronisation delay + when reusing OpenGL memory backed buffers. + +### Vulkan integration enhancements + +- Add support for the Vulkan H.264 and H.265 decoders. + +- Add support for timeline semaphores. + +- Optionally use newer Vulkan functions for format selection. + +- Add support for GPU-assisted validation. + +- Vulkan/Wayland: add support for `xdg_wm_base` protocol for creating a + visible debug window. Required as the previous `wl_shell` interface is + being removed from compositors. + +### CUDA / NVCODEC integration and feature additions + +- New **cudaipcsrc** and **cudaipcsink** elements for zero-copy CUDA memory + sharing between processes + +- New `nvJPEG` library based **nvjpegenc** JPEG encoder element + +- The NVIDIA desktop GPU decoder `nvh264sldec`, `nvh265sldec`, `nvvp8sldec`, + and `nvvp9sldec` elements were renamed to `nvh264dec`, `nvh265dec`, + `nvvp8dec`, and `nvvp9dec`, respectively. + +- GStreamer **NVIDIA H.264 and H.265 encoders** except for `nvh264enc` and + `nvh265enc` **gained support for CEA708 Closed Caption inserting**. + +- **OpenGL memory support** is added to `nv{cuda,autogpu}h264enc` and + `nv{cuda,autogpu}h265enc` elements + +- **CUDA stream integration**: As of 1.24, CUDA stream synchronization is an + application’s responsibility, and GStreamer will not execute unnecessary + synchronization operations. If an application needs direct access to CUDA + memory via `GST_MAP_CUDA` map flag, `cuStreamSynchronize()` or + [gst_cuda_memory_sync()][gst-cuda-memory-sync] call is required unless + application-side CUDA operation is executed with the GstCudaMemory’s + [associated CUDA stream][gst-cuda-memory-get-stream]. + +[gst-cuda-memory-sync]: https://gstreamer.freedesktop.org/documentation/cuda/gst-libs/gst/cuda/gstcudamemory.html?gi-language=c#gst_cuda_memory_sync +[gst-cuda-memory-get-stream]: https://gstreamer.freedesktop.org/documentation/cuda/gst-libs/gst/cuda/gstcudamemory.html?gi-language=c#gst_cuda_memory_get_stream + +### RTP stack improvements + +- New **rtppassthroughpay** element which just passes RTP packets through + unchanged, but appears like an RTP payloader element. This is useful for + relaying an RTP stream as-is through gst-rtsp-server, which expects an RTP + payloader with certain properties at the end of an RTSP media sub-pipeline. + +- New "timeout-inactive-rtp-sources" property on **rtpbin**, **sdpdemux** and + **rtpsession** to allow applications to disable automatic timeout of sources + from which no data has been received for a while. + +- **rtpvp8pay**, **rtpvp9pay**: expose "picture-id" as a property, and add + a "picture-id-offset" property to the VP9 payloader to bring it in line + with the VP8 payloader. + +- **rtpjitterbuffer** has seen improved media clock handling (clock equality + and clock setting/resetting), as well as two new properties that allow + reconstruction of absolute PTP timestamps without actually syncing to the + PTP clock, which can be useful in scenarios where one wants to reconstruct + the absolute PTP clock timestamps on a machine that doesn't have access to + the network of the PTP clock provider. The two new properties are: + + - "rfc7273-use-system-clock": allows the jitter buffer to assume that the + system clock is synced sufficiently close to the media clock used by an + RFC7273 stream. By default the property is disabled and the jitter buffer + will create a media clock and try to sync to it, but this is only required + to determine in which wraparound period from the media clock's Epoch the + current RTP timestamps refer to (and thus to reconstruct absolute time + stamps from them). If the property is enabled the wraparound period and + current offset from the Epoch will be determined based on the local + system clock, which means that no direct network connection to the media + clock provider is needed to reconstruct absolute timestamps. There is + also no start-up delay, because there's no clock sync that needs to be + established first. + + - "rfc7273-reference-timestamp-meta-only": If this property is enabled then + the jitter buffer will do the normal timestamp calculations for the output + buffers according to the configured mode instead of making use of the + RFC7273 media clock for that. It will still calculate RFC7273 media clock + timestamps, but only attach them to the output buffers in form of a + clock reference meta. + +- RTP payloaders and depayloaders now have an "extensions" property + for retrieving the list of currently enabled RTP header extensions. + +- **rtpbin** and **webrtcbin** no longer blindly set properties on the + jitter buffer assuming it's a standard `rtpjitterbuffer`, but instead + check if the property is available first, to better support non-standard + jitterbuffers or even an `identity` element in lieu of a jitter buffer. + +- **RTP header extension handling fixes for depayloaders** that + aggregate multiple input buffers into a single output buffer. Before, + only the last RTP input buffer was checked for header extensions. Now the + depayloader remembers all RTP packets pushed before an output buffer is + produced and checks all RTP input buffers for header extensions. + + - Affected depayloaders: `rtph264depay`, `rtph265depay`, `rtpvp8depay`, + `rtpvp9depay`, `rtpxqtdepay`, `rtpasfdepay`, `rtpmp4gdepay`, + `rtpsbcdepay`, `rtpvorbisdepay`, `rtpmp4vdepay`, `rtptheoradepay`, + `rtpsv3vdepay`, `rtpmp4adepay`, `rtpklvdepay`, `rtpjpegdepay`, + `rtpj2kdepay`, `rtph263pdepay`, `rtph263depay`, `rtph261depay`. + `rtpgstdepay`. + +### WebRTC improvements + +- Add support for **ICE consent freshness** (RFC 7675). + This requires libnice >= 0.1.22. + +- Advertise the local side of an end-of-candidates with an empty + candidate string. + +- Add the number of Data Channels opened and closed to `webrtcbin`'s statistics. + +- Various improvements and feature additions in the Rust `webrtc` plugin, which provides + `webrtcsrc` and `webrtcsink` elements as well as specific elements for different + WebRTC signalling protocols. See the Rust plugins section below for more details. + +### Adaptive Streaming improvements and Low-Latency HLS (LL-HLS) support + +- **hlsdemux2** now supports Low-Latency HLS (LL-HLS) + +- **hlsdemux2** asynchronous playlist download and update improves + responsiveness and bandwith usage. + +- **hlsdemux2** handles fallback variant URLs. + +- **hlsdemux2** is more responsive and accurate when handling seeks. + +- **dashdemux2** and **hlsdemux2** have a new "start-bitrate" property, + improving the decision for which initial stream variant that will be used. + +- **dashdemux2**, **hlsdemux2**, **mssdemux2** have received many + improvements regarding seeking, along with support for "early-seek" + which allows playback to start immediately from the requested + position without any previous download. + +- **dashdemux2**, **hlsdemux2**, **mssdemux2** better handle errors + on or near the live edge. + +- **dashsink** can now use the `dashmp4mux` muxer from the Rust plugins + and will also produce better and RFC 6381-compatible codec strings. The + "suggested-presentation-delay" property allows to set the suggested + presentation delay in the MPD. + +- No development took place on the legacy demuxers (`dashdemux`, `hlsdemux`, + `mssdemux`). Application developers are reminded to use the new demuxers + instead. They are automatically picked up when using `urisourcebin`, `uridecodebin3` + or `playbin3`. + +### W3C Media Source Extensions library + +- A new GStreamer library ([mse][gst-mse]) implementing the + W3C Media Source Extensions specification was added. + +- Applications can embed this library along with GStreamer in order to + integrate software that uses the Media Source APIs without relying on + a web browser engine. Typically an application consuming this library + will wrap the C API with JavaScript bindings that match the Media Source + API so their existing code can integrate with this library. + +[gst-mse]: https://gstreamer.freedesktop.org/documentation/mselib/index.html + +### Closed Caption handling improvements + +- **ccconverter** supports converting between the two CEA-608 fields. + +- New **cea608mux** element for muxing multiple CEA-608 streams together. + +- Various improvements and feature additions in the Rust-based closed caption + elements. Check out the Rust plugins section below for more details. + +### Precision Time Protocol (PTP) clock improvements + +- Many fixes and compatibility/interoperability improvements. + +- Better support for running on devices with multiple network interfaces. + +- Allow sync to master clock on same host. + +- PTP clock support is now also available on Windows. + +- The standalone `ptp-helper` binary has been rewritten in Rust for + portability and security. This works on Linux, Android, Windows, + macOS, FreeBSD, NetBSD, OpenBSD, DragonFlyBSD, Solaris and Illumos. + Newly supported compared to the C version is Windows. Compared to the + C version various error paths are handled more correctly and a couple + of memory leaks are fixed. Otherwise it should work identically. + The minimum required Rust version for compiling this is 1.48, i.e. the + version currently in Debian stable. On Windows, Rust 1.54 is needed at least. + +- New `ptp-helper` Meson build option so PTP support can be disabled or required. + +- `gst_ptp_init_full()` allows for a more fine-grained and extensible + configuration and initialization of the GStreamer PTP subsystem, including + TTL configuration. + +### Bayer 10/12/14/16-bit depth support + +- **bayer2rgb** and **rgb2bayer** now support bayer with 10/12/14/16 bit depths + +- **v4l2src** and **videotestsrc** now support bayer with 10/12/14/16 bit depths + +- **imagefreeze** gained bayer support as well + +### MPEG-TS improvements + +- **mpegtsdemux** gained support for + - **segment seeking** for seamless non-flushing looping, and + - **synchronous KLV** + +- **mpegtsmux** now + - allows attaching PCR to non-PES streams + - allows **setting of the PES stream number** for AAC audio and AVC video streams + via a new "stream-number" property on the muxer sink pads. Currently the + PES stream number is hard-coded to zero for these stream types. + - allows writing **arbitrary Opus channel mapping families and up to 255 channels** + - separate handling of DVB and ATSC AC3 descriptors ## New elements and plugins -- This section will be completed in due course +- **analyticsoverlay** visualises object-detection metas on a video stream. + +- **autovideoflip** and **autodeinterlace** are two new auto elements. + +- **AJA source and sink** elements plus device provider for AJA capture + and playout cards, including support for HANC/VANC ancillary data. + +- New **cea608mux** element for muxing multiple CEA-608 streams together. + +- The **codec2json** plugin adds `av12json`, `h2642json`, `h2652json` and + `vp82json` elements which convert AV1, H.264, H.265 and VP8 frame parameters + into human readable JSON data, which is useful for debugging and testing + purposes. + +- New **lc3** plugin with a decoder and encoder for the Bluetooth LC3 audio codec. + +- New **onnxinference** element to run ONNX inference models on video buffers. + +- New **rtppassthroughpay** element which just passes RTP packets through + unchanged, but appears like an RTP payloader element. This is mostly useful + for medias that simply pass through an existing RTP stream in `gst-rtsp-server`. + +- Qt6: **qml6glsrc**, **qml6glmixer**, **qml6gloverlay**, and **qml6d3d11sink** + +- New **SVT-AV1 encoder** plugin, imported from SVT-AV1 but with many fixes. + +- Many exciting **new Rust elements**, see Rust section below. + +- New **DirectWrite text rendering** and **Direct3D12** plugins (see Windows section below). + +- New **vaav1enc** element for encoding video in AV1 (See VA-API section) + +- New **uvcsink** element for exporting streams as UVC camera ## New element features and additions -- This section will be completed in due course +- **alphacombine** supports `I420_10LE` now for 10-bit WebM/alpha support. + +- The **amfcodec** for **hardware-accelerated video encoding using** + **the Advanced Media Framework (AMF) SDK for AMD GPUs** gained some new + features: + - 10-bit and HDR support for H.265 / HEVC and AV1 video encoders + - B-frame support in the H.264 encoder + - Initial support of pre-analysis and pre-encoding + - Initial support of Smart Access Video for optimal distribution + amongst multiple AMD hardware instances. + +- **appsink**: new "propose-allocation" signal so applications can provide + a buffer pool or allocators to the upstream elements, as well as "max-time" + and "max-buffers" properties to configure the maximum size of the + appsink-internal queue in addition to the existing "max-bytes" property. + +- **autovideoconvert** exposes colorspace and scaler elements for well know elements + +- **avtp**: add AVTP Raw Video Format payload and de-payload support. + +- **cacasink**'s output driver can now be selected via the "driver" property. + +- **camerabin**: various fixes and stability improvements + +- **clocksync**: "QoS" property to optionally send QoS events upstream like + a synchronising sink would. + +- **cutter**: can add `GstAudioLevelMeta` on output buffers, which can + be enabled via the new "audio-level-meta" property. + +- **dashdemux2** has a new "start-bitrate" property. + +- **dashsink** can now use the `dashmp4mux` muxer from the Rust plugins + and will also produce better and RFC 6381-compatible codec strings. The + "suggested-presentation-delay" property allows to set the suggested + presentation delay in the MPD. + +- **deinterlace**: Add support for 10/12/16-bit planar YUV formats + +- The **dvdspu** subpicture overlay now implements `GstVideoOverlayComposition` + support to make it work better with hardware decoders where the video data + should ideally stay on the GPU/VPU and the overlay blitting be delegated to + the renderer. + +- **encodebin** now automatically autoplugs timestamper elements such as + `h264timestamper` or `h265timestamper`, based on new "Timestamper" element + factory type and rank. + +- New **fakevideodec** element (see debugging section below). + +- **filesink**: "file-mode" property to allow the ability to specify `rb+` + file mode, which overwrites an existing file. This is useful in combination + with `splitmuxsink` so that files can be pre-allocated which can be useful + to reduce disk fragmentation over time. + +- **flvmux**: add "enforce-increasing-timestamps" property to allow disabling + a hack that was added back in the day because librtmp as used in `rtmpsink` + would get confused by timestamps going backwards, but this is no longer + required with `rtmpsink2`. If set to true (still the default, for backwards + compatibility), `flvmux` will modify buffers timestamps to ensure they are + always strictly increasing, inside one stream and also between the audio + and video streams. + +- **giostreamsink**: Add a property to close the stream on stop(). + +- **h264parse** improved its AU boundary detection. + +- **h264parse**, **h265parse**, **mpegvideoparse** now support multiple + unregistered user data SEI messages. + +- **insertbin** is now a registered element and available via the registry, + so can be constructed via parse-launch and not just via the insertbin API. + +- **jack**: libjack is now loaded dynamically at runtime instead of linking it + at build time. This means the plugin can be shipped on Windows and macOS + and will work if there's a user-installed JACK server/library setup. + +- **jpegparse** now has a rank so it will be autoplugged if needed. + +- **kmssink**: Add auto-detection for NXP i.MX8M Plus LCDIFv3, ST STM32 LTDC, + and Texas Instruments TIDSS display controllers. + +- **matroskademux** and **matroskamux** gained support for more raw video + formats, namely `RGBA64_LE`, `BGRA64_LE`, `GRAY10_BE32`, `GRAY16_LE` + +- **mpg123audiodec**'s rank was changed from MARGINAL to PRIMARY so it's now + higher than `avdec_mp3`, as it works better with "freeformat" MP3s. + +- **msdk**: + - DRM modifier support on Linux + - only expose codecs and capabilities actually supported by the platform + - **msdkvpp** video post-processing: + - new "hdr-tone-mapping" property to enable HDR-to-SDR tone mapping + - new compute scaling mode + - **Decoders** sport D3D11 and VA integration, and the VP9 decoder + supports certain resolution changes. + + - **Encoders**: + - **msdkh264enc, **msdkh265enc**: "pic-timing-sei" property to insert pic timing SEI + - **msdkh264enc, **msdkh265enc**: Add properties to allow different max/min-qp values for I/P/B frames + - **msdkh264enc**: Added BGRx format DMABuf support + - Advertise special image formats in low power mode + +- **mxfdemux** gained support for FFV1 demuxing + +- **opusenc**, **opusdec** now support decoding and encoding more than + 8 channels, and can also handle unknown/unpositioned channel layouts. + +- The **oss** plugin gained a device provider for audio device discovery + +- **pcapparse** learned how to handle the Linux "cooked" capture encapsulation v2 + +- Intel Quick Sync plugin improvements: + - **qsvh264enc** gained more encoding options + - **qsvh265dec** now supports GBR decoding and HEVC RExt profiles + +- **qtdemux** now adds audio clipping meta when playing gapless m4a content, + supports CENC sample grouping, as well as the SpeedHQ video codec. + +- **ristsrc** gained support for dynamic payloads via the new "caps" and + "encoding-name" properties. These can be used to make the `ristsrc` receive + other payload types than MPEG-TS. + +- **rtmp2src**: a new "no-eof-is-error" property was added: There is currently + no way for applications to know if the stream has been properly terminated + by the server or if the network connection was disconnected, as an EOS is + sent in both cases. With the property set, connection errors will be reported + as errors, allowing applications to distinguish between both scenarios. + +- **rtspsrc**: new "extra-http-request-headers" property for adding custom + http request headers when using http tunnelling. + +- **sdpdemux** now supports SDP source filters as per RFC 4570; audio-only or + video-only streaming can be selected via the new "media" property, and RTCP + feedback can be disabled via the "rtcp-mode" property. + +- **splitmuxsrc** uses natural ordering to sort globbed filenames now, i.e. + 0, 1, 2, 10, 11 instead of 0, 1, 10, 11, 2, ... + +- **srt**: Add more fields to the statistics to see how many packets + were retransmitted and how many were dropped. + +- **switchbin**: many improvements, especially for caps handling and + passthrough. + +- **taginject**: a "scope" property was added to allow injection of global + tags in addition to the current default which is stream tags. + +- **timeoverlay**: add `buffer-count` and `buffer-offset` time modes. + +- **udpsrc**: new "multicast-source" property to support IGMPv3 Source + Specific Muliticast (SSM) as per RFC 4604. + +- **videoconvertscale**, **videoconvert**: add a "converter-config" + property to allow fine-tuning conversion parameters that are not + exposed directly as property. + +- **videoflip**: many orientation tag handling fixes and improvements + +- **videorate**: add "drop-out-of-segment" property to force dropping + of out-of-segment buffers. + +- **volume** now supports arbitrarily-large positive gains via + a new "volume-full-range" property (it was not possibly to just + allow a bigger maximum value for the existing "volume" property + for `GstController`-related backwards-compatibility reasons). + +- **waylandsink**, **gtkwaylandsink**: improved frame scheduling + reducing frame drops and improve throughput. + +- **webpenc** now has support for **animated WebP** which can be enabled + via the new "animated" property. By default it will just output a + stand-alone WebP image for each input buffer, just like before. + +- **wpe**: added a `WebProcess` crash handler; gained WPEWebKit 2.0 API support. + +- **x264enc** gained support for 8-bit monochrome video (`GRAY8`). + +- **ximagesrc** gained navigation support (mouse and keyboard events). + +- **y4mdec** now parses extended headers to support high bit depth video. ## Plugin and library moves -- This section will be completed in due course +- The **AMR-NB** and **AMR-WB** plugins have been moved from -bad to -good. -## Plugin removals +## Plugin and element removals -- This section will be completed in due course +- The entire **gst-omx** package and plugin has been retired. + See the OMX section below for more details. + +- The **RealServer RTSP extension**, RDT handling and PNM source + have been removed from the realmedia plugin. + +- The **kate subtitle plugin** has been removed. ## Miscellaneous API additions -- This section will be completed in due course +### GStreamer Core + +- `gst_pipeline_get_configured_latency()` and `gst_pipeline_is_live()` + **convenience functions to query liveness and configured latency** + **of a pipeline**. + +- Plugins can now provide **status info messages for plugins** that will be + displayed in `gst-inspect-1.0` and is useful for dynamic plugins that + register features at runtime. They are now able to provide information + to the user why features might not be available. This is now used in + the `amfcodec`, `nvcodec`, `qsv`, and `va` plugins. + +- `GST_OBJECT_AUTO_LOCK()` and `GST_PAD_STREAM_AUTO_LOCK()` are + **`g_autoptr(GMutexLocker)`-based helpers for `GstPad` and `GstObject`** that + unlock the mutex automatically when the helper goes out of scope. This is + not portable so should not be used in GStreamer code that needs to be + portable to e.g. Windows with MSVC. + +- `gst_clear_context()`, `gst_clear_promise()`, `gst_clear_sample()` + +- `gst_util_ceil_log2()` and `gst_util_simplify_fraction()` utility functions + +- New `TAG_CONTAINER_SPECIFIC_TRACK_ID` **tag for container specific track ID** + as used in an HTML5 context, plus basic support in `matroskademux`, `qtdemux`, + `dashdemux` and `dashdemux2` + +- New **utility functions to create a stream-id without a pad** for elements: + - `gst_element_decorate_stream_id()` + - `gst_element_decorate_stream_id_printf_valist()` + - `gst_element_decorate_stream_id_printf()` + +- `GstQueueArray` gained API for sorting and sorted insertion + +- Add **strict GstStructure serialisation** with `gst_structure_serialize_full()` + in combination with `GST_SERIALIZE_FLAG_STRICT` which only succeeds if the + result can later be fully deserialised again. + +- **`GstBaseSrc` enhancements**: the "automatic-eos" property can be used to + do the equivalent to `gst_base_src_set_automatic_eos()`. + `gst_base_src_push_segment()` sends a segment event right away which can be + useful for subclasses like `appsrc` which have their own internal queuing. + +- `GstBaseSink` gained a new custom **`GST_BASE_SINK_FLOW_DROPPED`** flow return + which can be used by subclasses from the virtual `::render` method to signal + to the base class that a frame is not being rendered. This is used in e.g. + `waylandsink` and ensures that elements such as `fpsdisplaysink` will + correctly report the rate of frames rendered and dropped. + +### GstDiscoverer + +- New "load-serialized-info" signal to retrieve a serialized `GstDiscovererInfo` + +### GstSDP + +- Add `gst_sdp_message_remove_media()` + +### Video Library + +#### DRM Modifier Support for dmabufs on Linux + +See section above. + +#### List of Video Formats for Passthrough + +New helper API was added to get a list of all supported video formats, +including DMA_DRM formats, and can be used to advertise all supported +formats for passthrough purposes: + + - `GST_VIDEO_FORMATS_ANY_STR`, `GST_VIDEO_FORMATS_ANY` + - `gst_video_formats_any()` which can be used by bindings or for code that + prefers `GstVideoFormat` values instead of strings. + +#### New Video Formats + +- 12-bit and 16-bit A420 / A422 / A444 (YUV with alpha channel) variants: + - `A444_12BE`, `A444_12LE` + - `A422_12BE`, `A422_12LE` + - `A420_12BE`, `A420_12LE` + - `A444_16BE`, `A444_16LE` + - `A422_16BE`, `A422_16LE` + - `A420_16BE`, `A420_16LE` + +- 8-bit A422 / A444 (YUV with alpha channel) variant: + - `A422` + - `A444` + +- Planar 16-bit 4:4:4 RGB formats: + - `GBR_16BE` + - `GBR_16LE` + +- `RBGA`, intended to be used by hardware decoders where `VUYA` is only + supported 4:4:4 decoding surface but the stream is encoded with GBR + color space, such as HEVC and VP9 GBR streams for example. + +- Two tiled Mediatek 10-bit formats: + - `MT2110T` + - `MT2110R` + +- Tiled 10-bit NV12 format `NV12_10LE40_4L4` (Verisilicon Hantro) ## Miscellaneous performance, latency and memory optimisations -- This section will be completed in due course +- liborc 0.4.35 (latest: 0.4.38) adds support for AVX/AVX2 and contains + improvements for the SSE backend. + +- liborc 0.4.37 adds support for NEON on macOS on Apple ARM64 CPUs. -- liborc 0.4.35 (latest: 0.4.37) adds support for AVX/AVX2 and contains improvements for the SSE backend. +- Most direct use of the GLib `GSLice` allocator has been removed, as there is + little evidence that it actually still provides much advantage over the standard + system allocator on Linux or Windows in 2024. There *is* strong evidence however + that it causes memory fragmentation for standard GStreamer workloads such as + RTSP/RTP/WebRTC streaming. -- as always there have been plenty of performance, latency and memory optimisations all over the place. +- As always there have been plenty of performance, latency and memory optimisations + all over the place. + +<!-- ## Miscellaneous other changes and enhancements -- This section will be completed in due course +- Nothing that hasn't been mentioned elsewhere already. + +--> ## Tracing framework and debugging improvements +- The **gst-stats** tool can now be passed a custom regular expression + +- **gst-debug-viewer** from the devtools module has seen minor improvements and fixes + ### New tracers -- This section will be completed in due course +- None in this release. ### Debug logging system improvements -- This section will be completed in due course +- Nothing major in this cycle. + +### Fake video decoder + +- The new **fakevideodec** element does not decode the input bitstream, + it only reads video width, height and framerate from the caps and then + pushes out raw video frames of the expected size in RGB format. + +- It draws a snake moving from left to right in the middle of the frame, + which is reasonably light weight and still provides an idea about how + smooth the rendering is. ## Tools -- This section will be completed in due course +- **gst-launch-1.0** gained a new `--prog-name` command line option to set + the program name, which will be used by GTK and GStreamer to set the + class or app-id. + +- **gst-play-1.0** now defaults to using `playbin3`, but can still be made to + use the old `playbin` by passing the `--use-playbin2` command line argument. + +## GStreamer FFmpeg wrapper + +- New **avvideocompare** element to compare two incoming video buffers + using a specified comparison method (e.g. SSIM or PSNR). + +- **Prefer using FFmpeg Musepack decoder/demuxer** over `musepackdec` + as they work better with `decodebin3` and `playbin3` which likes to + have parsers and decoders separate. + +- Added **codec mappings** for AV1, MxPEG, FFVHuff video -## GStreamer FFMPEG wrapper +- Added **raw video format** support for `P010`, `VUYA`, `Y410`, + `P012`, `Y212` and `Y412`. -- This section will be completed in due course +- Newer, non-deprecated APIs are used by the plugin when built with FFmpeg 6.0 or newer. + +- The FFmpeg meson subproject wrap has been updated to v6.1 + +- Note: see Known Issues section below for known issues with FFmpeg 6.0 ## GStreamer RTSP server -- This section will be completed in due course +- New "ensure-keyunit-on-start" property: While the suspend modes NONE + and PAUSED provided a low startup latency for connecting clients, it + did not ensure that streams started on fresh data. With this new + property it is possible to maintain the low startup latency of those + suspend modes while also ensuring that a stream starts on a key unit. + Furthermore, by setting the new "ensure-keyunit-on-start-timeout" + property it is also possible to accept a key unit of a certain age, + but discard it if too much time has passed and instead force a new + key unit. + +- rtspclientsink: apply "port-range" property for RTCP port selection as well ## GStreamer VA-API support -- This section will be completed in due course +### GstVA + +- `vah264dec`, `vah265dec`, `vavp8dec`, `vavp9dec`, `vampeg2dec` and + `vaav1dec` were promoted to rank PRIMARY+1 on Linux + +- Improved support for dmabuf use cases. All `va` elements now negotiate + the new and improved dmabuf capabilities with explicit modifiers. + This supports both import and export of dmabufs. + +- Added `vaav1enc` element, available in recent Intel and AMD GPUs + +- Added support for the experimental [VA-Win32 backend][va-win32]. + It needs at least libva 1.18 + +- Improved handling of multi-GPU systems. Still, sharing buffers among them + is not advised. + +- Bumped minimum libva version to 1.12 + +- Enhanced support for RadeonSI Mesa driver for 10bit decoding + +- Register elements only for allowed drivers (Intel and Mesa, for the moment) + +[va-win32]: https://devblogs.microsoft.com/directx/video-acceleration-api-va-api-now-available-on-windows/ + +### GStreamer-VAAPI + +- The new GstVA elements (see above) should be preferred when possible. + +- Ranks of decoders were demoted to `NONE` so they won't be used + automatically by `playbin` and similar elements anymore. + +- Clean-ups and minimal fixes. + +- **gstreamer-vaapi should be considered deprecated** and may be + discontinued as soon as the `va` plugin is fully feature equivalent. + Users who rely on gstreamer-vaapi are encouraged to migrate and + test the `va` elements at the earliest opportunity. ## GStreamer Video4Linux2 support -- This section will be completed in due course +- **New `uvcsink` element**, based on v4l2sink allow streaming your pipeline + as a UVC camera using Linux UVC Gadget driver. + +- `v4l2src` now supports **10/12/14/16-bit bayer formats**. + +- Stateful decoders now pass too large encoded frames over multiple buffers. + +- **AV1 Stateless video decoder**. + +- Stateless decoders now tested using Virtual driver (visl), making it possible to run the tests in the cloud based CI ## GStreamer OMX @@ -98,109 +1000,565 @@ As always, this release is again packed with many new features, bug fixes and ot to the Video4Linux-based video encoders and decoders which have been the standard on embedded Linux for quite some time now. +- Hardware vendors which still use OpenMAX are known to have non-standard + forks and it is recommended that they maintain it while planning their + move to the Video4Linux API. + ## GStreamer Editing Services and NLE -- This section will be completed in due course +- Implement a `gesvideoscale` effect which gives user the ability to chooses where + a clip has to be scaled in the chain of effects. By default scaling is done in + the `compositor`. + +- Add support for `gessrc` as sub-timeline element so third party can implement their own + formatters and use their timelines as sub-timelines. Before that, only timelines serialized + as files on the filesystem could be loaded as sub-timelines (using `gesdemux`). + +- Implement a new `GESDiscovererManager` singleton object making management of the discoverers used + to discoverer media files cleaner and allowing to expose the following APIs: + - `load-serialize-info` signal so `GstDiscovererInfo` can be serialized by users the way they like and load them without requiring discovering the file when reloading a project. + - `source-setup` signal so user can tweak source elements during discovery + +- Expose `GESFrameCompositionMeta` in public API so user can implement their own effects + targetting GES which take into account that meta. + +- Expose `audioconvert:mix-matrix` property in audio sources + +- Port `GESPipeline` rendering to use `encodebin2`. This allows rendering timelines directly with + a muxing sink (like `hlssinkX` etc..) and leverage all new features of that new element. + +### ges-launch + +- Fix setting keyframes + +- Add an `ignore-eos` option + +- Allow overriding container profile so that the user can build encoding profiles + following the media format of a specific media file, for example, but ensuring + it is muxed using a specific format + +- Ensure sink elements are inside a `GstBin` and never in a `GstPipeline` + +- Move `+effect` stack effects from source to last effect added, so it feels more natural + to user as adding them at the beginning of the chain while the syntax is `+effect` + felt wrong ## GStreamer validate -- This section will be completed in due course +- In action types, add a way to avoid checking property value after setting it, + in case elements do it async for example. + +- Add a vmethod to free `GstValidateActionParameter`s to be more binding friendly. + +- Allow scenarios to define the pipeline state target in the metadata + instead of assuming `PLAYING` state. + +- Add support to run sub-pipelines/scenarios + - Added support to forward buffers from appsink to appsrc + +- Add a way to set pipeline base-time, start-time and force using the system clock. + +- Add a 'fill-mode' to the `appsrc-push` action type so we can create some type + of streams easily using an appsrc, giving control when writing scenarios + without requiring files with the content. + +- Add a "select-streams" action type to test "stream aware" elements. + +- Add a way to wait for a property to reach a specified value before executing + an action. For example it is possible to wait for a pad to get some specific + caps set before executing an action. + +- validate: Add support to replace variables in deeply nested structures in + particular for more complex action types where some of the properties are + inside structures. + +- Fixed compatibility with Python 3.12. ## GStreamer Python Bindings -- This section will be completed in due course +gst-python is an extension of the regular GStreamer Python bindings based on +gobject-introspection information and PyGObject, and provides "syntactic sugar" +in form of overrides for various GStreamer APIs that makes them easier to use +in Python and more pythonic; as well as support for APIs that aren't available +through the regular gobject-introspection based bindings, such as e.g. +GStreamer's fundamental GLib types such as `Gst.Fraction`, `Gst.IntRange` etc. + +- Added a **`GstTagList` override that makes a tag list act like a dict** + +- Fix build and usage in Windows + +- Various fixes for Python >= 3.12 + +- Rework libpython loading to be relocatable + +- Fix libpython dlopen on macOS ## GStreamer C# Bindings -- This section will be completed in due course +- The GStreamer C# bindings have been updated to a more recent + version of GtkSharp and the bindings have been regenerated + with that version. + +- GStreamer API added in recent GStreamer releases is now available + +- GstRtspServer bindings have been added, plus an RTSP server example ## GStreamer Rust Bindings and Rust Plugins -The GStreamer Rust bindings are released separately with a different release -cadence that's tied to `gtk-rs`, but the latest release has already been -updated for the new GStreamer 1.24 API. +The GStreamer Rust bindings and plugins are released separately with a different release +cadence that's tied to the twice-a-year GNOME release cycle. + +The latest release of the bindings (0.22) has already been updated for the new GStreamer 1.24 APIs, and works with any GStreamer version starting at 1.14. + +`gst-plugins-rs`, the module containing GStreamer plugins written in Rust, +has also seen lots of activity with many new elements and plugins. The GStreamer 1.24 binaries track the 0.12 release series of `gst-plugins-rs`, and fixes from newer versions will be backported as needed to the 0.12 brach for future 1.24.x bugfix releases. + +Rust plugins can be used from any programming language. To applications +they look just like a plugin written in C or C++. + +### WebRTC + +- New element `webrtcsrc` that can act as a recvonly WebRTC client. Just like the opposite direction, `webrtcsink`, this can support various different WebRTC signalling protocols. Some are included with the plugin and provide their own element factory for easier usage but it is also possible for applications to provide new signalling protocol implementations. + +- `webrtcsink` now exposes the signaller as property and allows implementing a custom signaller by connecting signal handlers to the default signaller. + +- A new signaller and `webrtcsink` implementation for Janus' VideoRoom implementation. The corresponding `webrtcsrc` signaller implementation is currently in a merge request in GitLab. + +- New `whepsrc` element that can receive WHEP WebRTC streams. This is currently not based on `webrtcsrc` but in the future a new element around `webrtcsrc` will be added. + +- New `whipserversrc` element around `webrtcsrc` for ingesting WHIP WebRTC streams in GStreamer. + +- New `whipclientsink` element around `webrtcsink` for publishing WHIP WebRTC streams from GStreamer. This deprecates the old `whipsink` element. + +- A new signaller and `webrtcsink` implementation for LiveKit. The corresponding `webrtcsrc` signaller implementation was merged into the git repository recently. + +- A new signaller and `webrtcsink` implementation for AWS Kinesis Video Streams + +- `webrtcsink` has a new `payloader-setup` signal to allow the application more fine grained control over the RTP payloader configuration, similar to the already existing `encoder-setup` signal for encoders. + +- `webrtcsrc` gained support for a custom navigation event protocol over the data channel, which is compatible with the navigation event protocol supported by `webrtcsink`. + +- `webrtcsink` supports encoded streams as input. Using encoded streams will disable `webrtcsink`s congestion control changing any encoded stream parameters. + +- `webrtcsink` and `webrtcsrc` have a new signal 'request-encoded-filter' to allow transformations of the encoded stream. This can be used, for example, for the same use-cases as the WebRTC Insertable Streams API. + +- gstwebrtc-api: JavaScript API for interacting with the default signalling protocol used by `webrtcsink` / `webrtcsrc`. + +... and various other smaller improvements! + +### RTSP + +- **New `rtspsrc2` element**. Only a subset of RTSP features are implemented so far: + - RTSP 1.0 support + - TCP, UDP, UDP-Multicast lower transports + - RTCP SR, RTCP RR, RTCP-based A/V sync + - Tested for correctness in multicast cases too + - Lower transport selection and order (NEW!) + - The existing `rtspsrc` has a hard-coded order list for lower transports + - Many advanced features are not implemented yet, such as non-live support. + See the [README][rtspsrc2-readme] for the current status. + +[rtspsrc2-readme]: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/blob/main/net/rtsp/README.md?ref_type=heads + +### GTK4 + +- Support for **rendering GL textures** on X11/EGL, X11/GLX, Wayland, macOS, and WGL/EGL on Windows. + +- Create a window for testing purposes when running in `gst-launch-1.0` or if `GST_GTK4_WINDOW=1` is set. + +- New `background-color` property for **setting the color of the background of the frame and the borders**, if any. This also allows setting a fully transparent background. + +- New `scale-filter` property for defining how to scale the frames. + +- Add Python example application to the repository. + +- Various bugfixes, including support for the new GTK 4.14 GL renderer. The plugin needs to be built with at least the `gtk_v4_10` feature to work with the new GTK 4.14 GL renderer, and will work best if built with the `gtk_v4_14` feature. + +### Closed Caption + +- Add `cea608tocea708` element for **upconverting CEA-608 captions to their CEA-708 representation**. + +- Add **support for translations** within `transcriberbin`. + +- `awstranscriber` supports **translating the transcribed text into different languages**, including **multiple languages at the same time**. + +- `awstranscriber` is using the new HTTP/2-based API now instead of the WebSocket-based one. + +### Other new elements + +- New `awss3putobjectsink` that works similar to `awss3sink` but with a different upload strategy. + +- New `hlscmafsink` element for **writing HLS streams with CMAF/ISOBMFF fragments**. + +- New `inter` plugin with **new `intersink` and `intersrc` elements that allow to 1:N connect different pipelines in the same process**. This is implemented around the `appsrc` / `appsink`-based `StreamProducer` API that is provided as part of the GStreamer Rust bindings, and is also used inside `webrtcsrc` and `webrtcsink`. + +- New `livesync` element that allows **maintaining a contiguous live stream without gaps from a potentially unstable source**. -`gst-plugins-rs`, the module containing GStreamer plugins written in Rust, has also seen lots of activity with many new elements and plugins +- New `isomp4mux` **non-fragmented MP4 muxer** element. -- Rust plugins can be used from any programming language. To the outside they look just like a plugin written in C or C++. +### Other improvements -## New Rust plugins and elements +- audiornnoise + - Attach audio level meta to output buffers. + - Add voice detection threshold property -- This section will be completed in due course +- fmp4mux + - Add support for CMAF-style chunking, e.g. low-latency / LL HLS and DASH + - Add support for muxing Opus, VP8, VP9 and AV1 streams + - Add 'offset-to-zero' property and make media/track timescales configurable + +- hlssink3 + - Allow adding `EXT-X-PROGRAM-DATE-TIME` tag to the manifest. + - Allow generating I-frame-only playlist + +- ndi + - Closed Caption support in `ndisrc` / `ndisink` + - Zero-copy output support in `ndisrc` for raw video and audio + +- spotifyaudiosrc: Support configurable bitrate + +For a full list of changes in the Rust plugins see the +[gst-plugins-rs ChangeLog][rs-changelog] between versions 0.9 (shipped with GStreamer 1.22) and 0.12 (shipped with GStreamer 1.24). + +[rs-changelog]: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/blob/main/CHANGELOG.md ## Cerbero Rust support -- This section will be completed in due course +- As of GStreamer 1.24, the GStreamer Rust plugins are shipped as part of + our binary packages on all major platforms. This includes Android and iOS + now in addition to macOS and Windows/MSVC. ## Build and Dependencies -- This section will be completed in due course +- Meson >= 1.1 is now required for all modules -### Monorepo build (neé gst-build) +- The GLib requirement has been bumped to >= 2.64 -- This section will be completed in due course +- liborc >= 0.4.38 is strongly recommended -### Cerbero +- libnice >= 0.1.22 is strongly recommended, as it is required + for WebRTC **ICE consent freshness** (RFC 7675). -Cerbero is a meta build system used to build GStreamer plus dependencies on platforms where dependencies are not readily available, such as Windows, Android, iOS, and macOS. +- gst-libav was updated for FFmpeg API deprecations and removals -#### General improvements +- libwebpmux is required for the animated WebP support + +- The wpe plugin gained support for the WPEWebKit 2.0 API version + +- Bumped minimum libva version to 1.12 for the `va` plugin. + +- zxing: added support for the zxing-c++ 2.0 API + +- The `ptp-helper` for Precision Time Protocol (PTP) support in GStreamer core + has been rewritten in Rust, and the minimum required Rust version for building + this is 1.48, i.e. the version currently in Debian stable. On Windows, at least + Rust 1.54 is needed. There is a new `ptp-helper` Meson feature option that can + be used to make sure everything needed for PTP support is available (if set to + `ptp-helper=enabled`). `cargo` is not required for building. + +- `gst-plugins-rs` requires Rust 1.70 or newer. + +- Link to libsoup at build time in all cases on non-Linux, and only load it + dynamically on Linux systems where we might need to support a mix of applications + with dependencies that might be using libsoup2 or libsoup3. + A "soup-version" meson build option was added to prefer a specific version. + Distros should make sure that libsoup is still a package dependency, since it's + still required at runtime for the `soup` and `adaptivedemux2` plugins to function. + +- libjack is now dynamically loaded at runtime by the JACK audio plugin, and no + longer a hard build dependency. However, it still is a runtime dependency, so + distros should make sure it remains a package dependency. + +### Monorepo build (née gst-build) + +- There is now a top-level meson build option to enable/require `webrtc` + +- The FFmpeg subproject wrap was udpated to 6.1 + +- A libvpx wrap was added (for VP8/VP9 software encoding/decoding) + +### gstreamer-full + +- Add **full static build** support, including on Windows: Allow a project to use + gstreamer-full as a static library and link to create a binary without + dependencies. Introduce the meson build option `gst-full-target-type` to + select the build type: `dynamic` (default) or `static`. + +- Registers all full features in a plugin now to offer the possibility to + get information at the plugin level and get it from the registry. All the + full features are now registered in a `fullstaticfeatures` meta plugin + instead of having a `NULL` plugin. + +### Development environment + +- add VSCode IDE integration -- This section will be completed in due course +- gst-env.py: Output a setting for the prompt with `--only-environment` -#### macOS / iOS +### Cerbero + +Cerbero is a meta build system used to build GStreamer plus dependencies +on platforms where dependencies are not readily available, such as Windows, +Android, iOS, and macOS. + +#### General improvements -- This section will be completed in due course +- New plugins have been added + - `codecalpha` `dvbsubenc` `rtponvif` `switchbin` `videosignal` `isac` `ivfparse` `inter` `rtspsrc2` +- JACK plugin is now available all platforms (previously only Linux), and will be loaded if the JACK library is found at plugin load time +- Several recipes were ported to meson, leading to faster builds and better MSVC support + - ffmpeg, gperf, lame, libvpx, ogg, opencore-amr, sbc, speex, tiff, webrtc-audio-processing + - For more information, please see [the tracker issue](https://gitlab.freedesktop.org/gstreamer/cerbero/-/issues/215) +- Some recipes are now outdated or unnecessary and have been removed: + - intltool, libkate +- Various recipe updated to their latest versions +- Rust toolchain updated to 1.76.0 (latest as of writing) +- Rust plugins are now stripped and debug info split out correctly, reducing their size +- Fix several spurious build issues, especially with the Rust toolchain +- CMake is picked up from the system if available +- Cerbero will no longer OOM or consume excessive resources on low-end systems +- Python recipes have been moved from setuptools to virtualenv +- Fixed support for Python 3.12+ + +#### macOS + +- Minimum macOS version has been increased to 10.13 (High Sierra) + - Released 5½ years ago, >95% marketshare +- Fix macOS app store validation issue caused by absolute RPATHs +- Rosetta is automatically installed if required for universal builds on Apple Silicon +- The official macOS binaries now also include static libs for the GStreamer Rust plugins + +#### iOS + +- Minimum iOS version has been increased to 12.0 +- The iOS binary packages now include the GStreamer Rust plugins +- `tremor` and `ivorbisdec` plugins are no longer shipped on iOS #### Windows -- This section will be completed in due course +- New features shipped with the official binaries: + - plugins: `dwrite` `d3d12` (MSVC) `soundtouch` (MSVC) `taglib` (MSVC) `webrtcdsp` (MSVC) + - plugin features: `d3d11-wgc` (Windows Graphics Capture Support) + - libraries: `gstdxva-1.0` +- New `qml6` plugin can be built on Windows with the `qt6` variant enabled + - Similar to qt5, this plugin cannot be included in the official binaries +- GLib process handling helpers for Windows are now shipped +- Windows 11 SDK is now required for builds + - Visual Studio 2019 and newer ship this SDK +- MSYS is now deprecated for building Windows binaries, will be removed in the next release + - MSYS2 is required, and the bootstrap script `tools/bootstrap-windows.ps` can install it for you +- Windows bootstrap script `tools/bootstrap-windows.ps1` is much more interactive and user-friendly now +- Fixed Pango crash on 32-bit Windows +- WiX packaging now works with cross-windows builds from linux #### Linux -- This section will be completed in due course +- Linux packages will now also include static libs for the GStreamer Rust plugins +- Add Python support for multiarch distributions +- Build fixes for various recipes on multiarch distributions +- Use arch-specific libdir correctly on multiarch distributions +- gst-omx was removed from gstreamer, and hence is no longer shipped +- Fixed Gentoo support +- Added support for RHEL 9 +- Added support for Rocky Linux +- Added support for Manjaro Linux #### Android -- This section will be completed in due course +- Android NDK has been updated to r25c + - Only the Clang toolchain is used from the NDK now (both target and host) + - gnustl has been completely removed +- The Android binary packages now include the GStreamer Rust plugins +- `tremor` and `ivorbisdec` plugins are no longer shipped on Android +- `openh264` plugin no longer enables ASM optimizations on Android x86 due to relocation errors ## Platform-specific changes and improvements ### Android -- This section will be completed in due course +- Add NDK implementation of Android MediaCodec. This reduces the amount of + Java <-> native calls, which should reduce overhead. + +- Add support for AV1 to the `androidmedia` video encoder and decoder. ### Apple macOS and iOS -- This section will be completed in due course +- osxaudio: **audio clock improvements** (interpolate based on system time) + +- Set activation policy in `gst_macos_main()` and in **osxvideosink** and + **glimagesink**. Setting the policy to `NSApplicationActivationPolicyAccessory` + by default makes sure that we can activate windows programmatically or + by clicking on them. Without that, windows would disappear if you clicked + outside them and there would be no way to bring them to front again. This + change also allows osxvideosink to receive navigation events correctly. ### Windows -- This section will be completed in due course +- New **DirectWrite text rendering plugin** with **dwriteclockoverlay**, + **dwritetimeoverlay**, **dwritetextoverlay**, **dwritesubtitlemux**, + and **dwritesubtitleoverlay** elements, including closed caption overlay + support in dwritetextoverlay. + +- **PTP clock support** is now also available on Windows + +- **qml6d3d11sink** is a new **Direct3D11 Qt6 QML sink** for Windows + as an alternative to the existing qml6glsink. + +- **wasapi2** audio plugin: + - Added an option to monitor a loopback device's mute state + - Allows process loopback capture on Windows 10 + +- **win32ipc** supports zero-copy rendering now through a shared bufferpool. + +- Add a **Windows-specific plugin loader implementation** (`gst-plugin-scanner`), + so plugin loading during registry updates happens in an external process + on Windows as well. + +- `gst_video_convert_sample()` which is often used for thumbnailing + gained a D3D11 specific conversion path. + +- **d3d11** plugin: + - `d3d11mpeg2dec` element is promoted to `PRIMARY` + 1 rank + - Added `d3d11ipcsrc` and `d3d11ipcsink` elements for zero-copy GPU memory sharing between multiple processes. + - Added HLSL shader pre-compilation (at binary build time) support in MSVC build. + - `d3d11videosink` and `d3d11convert` elements support 3D transform, MSAA (MultiSampling Anti-Aliasing) and anisotropic sampling method. + - Added support for more raw video formats by using compute shader. A list of supported raw video formats can be found in the [`d3d11videosink` plugin documentation][d3d11videosink-docs]. + - Added `d3d11overlay` element for applications to be able to draw/blend an overlay image on Direct3D11 texture directly. + +[d3d11videosink-docs]: https://gstreamer.freedesktop.org/documentation/d3d11/d3d11videosink.html#pad-templates + +- **New Direct3D12 plugin**: From a video decoding, conversion, composition, and rendering point of view, this new `d3d12` plugin is feature-wise near equivalent to the `d3d11` plugin. Notable differences between `d3d12` and `d3d11` are: + - The GStreamer Direct3D12 integration layer is not exposed as a GStreamer API yet. Thus, other plugins such as `amfcodec`, `nvcodec`, `qsv`, and `dwrite` are not integrated with `d3d12` yet. + - H.264 video encoding support via `d3d12h264enc` element. + - Direct3D12 video encoding API requires Windows 11 or [DirectX 12 Agility SDK][directx-12-agility-sdk] + - IPC, overlay, and deinterlace elements are not implemented in `d3d12` + - [Windows Graphics Capture][windows-graphics-capture] API based screen capturing is not implemented in `d3d12` + - In this release, MSVC is the only officially supported toolchain for the `d3d12` plugin build. + - All `d3d12` elements are zero ranked for now. Users will need to adjust rank of each `d3d12` element via `GST_PLUGIN_RANK` environment or appropriate plugin feature APIs if they want these elements autoplugged. + +[directx-12-agility-sdk]: https://devblogs.microsoft.com/directx/gettingstarted-dx12agility/ +[windows-graphics-capture]: https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture + +<!-- ### Linux -- This section will be completed in due course +- Many improvements which are described in other sections. + +--> ## Documentation improvements -- This section will be completed in due course +- hotdoc has been updated to the latest version, and the theme has also + been updated, which should fix various usability issues. ## Possibly Breaking Changes -- This section will be completed in due course +- `gst_plugin_feature_check_version()` has been updated to fix unexpected + version check behaviour for git versions. It would return TRUE if the + plugin version is for a git development version (e.g. 1.24.0.1) and the + check is for the "next" micro version (1.24.1). Now it no longer does this. + This just brings the runtime version check in line with the build time + version checks which have been fixed some time ago. + +- `GstAudioAggregator` and subclasses such as **audiomixer** now sync + property values to output timestamp, which is what `GstVideoAggregator` + has been doing already since 2019, and which makes sense, as the properties + need to change at every output frame based on the output time because they + may change even though the input frame is not changing. + +- **rtpac3depay** now outputs `audio/x-ac3` instead of `audio/ac3` as that + is the canonical media format in GStreamer. `audio/ac3` is still sometimes + accepted as input format for backwards compatibility (e.g. in `rtpac3pay` + or `ac3parse`), but shouldn't be output. + +- **timecodestamper**: The "drop-frame" property now defaults to TRUE + +- The NVIDIA desktop GPU decoders `nvh264sldec`, `nvh265sldec`, `nvvp8sldec` + and `nvvp9sldec` were renamed to `nvh264dec`, `nvh265dec`, `nvvp8dec` and + `nvvp9dec`, respectively. ## Known Issues -- This section will be completed in due course +- There are known issues with FFmpeg version 6.0.0 due to opaque passing + being broken in that version. This affects at least `avdec_h264`, but + may affect other decoders as well. Versions before 6.0.0, and 6.0.1 or higher + are not affected. ## Statistics -- This section will be completed in due course +- 4643 commits +- 2405 Merge Requests +- 850 Issues +- 290+ Contributors +- ~25% of all commits and Merge Requests were in Rust modules + +- 4747 files changed +- 469633 lines added +- 209842 lines deleted +- 259791 lines added (net) ## Contributors -- This section will be completed in due course +Aaron Boxer, Aaron Huang, Acky Xu, adixonn, Adrian Fiergolski, +Adrien De Coninck, Akihiro Sagawa, Albert Sjölund, Alessandro Bono, +Alexande B, Alexander Slobodeniuk, Alicia Boya García, amindfv, +Amir Naghdinezhad, anaghdin, Anders Hellerup Madsen, Andoni Morales Alastruey, +Antonio Kevo, Antonio Rojas, Arnaud Rebillout, Arnaud Vrac, Arun Raghavan, +badcel, Balló György, Bart Van Severen, Bastien Nocera, Benjamin Gaignard, +Bilal Elmoussaoui, Brad Hards, Camilo Celis Guzman, Carlo Cabrera, +Carlos Falgueras García, Carlos Rafael Giani, Célestin Marot, Chao Guo, +Charlie Blevins, Cheah, Vincent Beng Keat, Chris Degawa, Chris Spencer, +Christian Curtis Veng, Christian Meissl, Christopher Degawa, Chris Wiggins, +Cidana-Developers, Colin Kinloch, Damian Hobson-Garcia, Daniel Almeida, +Daniel Knobe, Daniel Moberg, Daniel Morin, Daniel Pendse, Daniel Stone, +Daniel Ulery, Dan Searles, Dario Marino Saccavino, Dave Patrick Caberto, +David Craven, David Revay, David Rosca, David Svensson Fors, Detlev Casanova, +Diego Nieto, Dominique Leroux, Dongyun Seo, Doug Nazar, Edward Hervey, +Ekwang Lee, elenril, Elliot Chen, Enrique Ocaña González, Erik Fröbrant, +Eva Pace, Evgeny Pavlov, Fabian Orccon, Felix Yan, Fernando Jiménez Moreno, +Florian Zwoch, François Laignel, Frank Dana, Georges Basile Stavracas Neto, +Guillaume Desmottes, Guillaume Gomez, Gwyn Ciesla, Haihua Hu, Hassene Tmar, +hassount, Heiko Becker, He Junyan, hguermaz, Hiero32, Hosang Lee, Hou Qi, +Hugo Svirak, Hugues Fruchet, Hu Qian, Hyung Song, Ignazio Pillai, Ilie Halip, +Itamar Marom, Ivan Molodetskikh, Ivan Tishchenko, JackZhouVCD, Jacob Johnsson, +jainl28patel, Jakub Adam, James Cowgill, James Hilliard, James Oliver, +Jan Alexander Steffens (heftig), Jan Beich, Jan Schmidt, Jan Vermaete, +Jayson Reis, Jeff Wilson, Jeongki Kim, Jeri Li, Jimmi Holst Christensen, +Jimmy Ohn, Jochen Henneberg, Johan Adam Nilsson, Johan Sternerup, John King, +Jonas Danielsson, Jonas K Danielsson, Jonas Kvinge, Jordan Petridis, +Jordan Yelloz, Josef Kolář, Juan Navarro, Julien Isorce, Jun Zhao, +Jurijs Satcs, Kalev Lember, Kamal Mostafa, kelvinhu325, Kevin Song, Khem Raj, +Kourosh Firoozbakht, Leander Schulten, Leif Andersen, L. E. Segovia, +Lieven Paulissen, lijing0010, Lily Foster, Link Mauve, Li Yuanheng, +Loïc Le Page, Loïc Molinari, Lukas Geiger, Luke McGartland, Maksym Khomenko, +Ma, Mingyang, Manel J, Marcin Kolny, Marc Leeman, Marc Solsona, +Marc Wiblishauser, Marek Vasut, Marijn Suijten, Mark Hymers, Markus Ebner, +Martin Nordholts, Martin Robinson, Mart Raudsepp, Marvin Schmidt, +Mathieu Duponchelle, Matt Feury, Matthew Waters, Matthias Fuchs, Matthieu Volat, +Maxim P. Dementyev, medithe, Mengkejiergeli Ba, Michael Bunk, Michael Gruner, +Michael Grzeschik, Michael Olbrich, Michael Tretter, Michiel Westerbeek, +Mihail Ivanchev, Ming Qian, Nader Mahdi, naglis, Nick Steel, Nicolas Beland, +Nicolas Dufresne, Nirbheek Chauhan, Olivier Babasse, Olivier Blin, +Olivier Crête, Omar Khlif, Onur Demiralay, Patricia Muscalu, Paul Fee, +Pawel Stawicki, Peter Stensson, Philippe Normand, Philipp Zabel, +PhoenixWorthVCD, Piotr Brzeziński, Priit Laes, Qian Hu, Rabindra Harlalka, +Rafał Dzięgiel, Rahul T R, rajneeshksoni, Ratchanan Srirattanamet, renjun wang, +Rhythm Narula, Robert Ayrapetyan, Robert Mader, Robert Rosengren, +Robin Gustavsson, Roman Lebedev, Rouven Czerwinski, Ruben Gonzalez, +Ruslan Khamidullin, Ryan Pavlik, Sanchayan Maity, Sangchul Lee, Scott Kanowitz, +Scott Moreau, SeaDve, Sean DuBois, Sebastian Dröge, Sebastian Szczepaniak, +Sergey Radionov, Seungha Yang, Shengqi Yu, Simon Himmelbauer, Slava Andrejev, +Slawomir Pawlowski, soak, Stefan Brüns, Stéphane Cerveau, Stephan Seitz, +Stijn Last, Talha Khan, Taruntej Kanakamalla, Jin Chung Teng, Théo Maillart, +Thibault Saunier, Thomas Schneider, Tim-Philipp Müller, Tobias Rapp, Tong Wu, +Tristan van Berkom, ttrigui, U. Artie Eoff, utzcoz, Víctor Manuel Jáquez Leal, +Vivia Nikolaidou, Wang Chuan, William Manley, Wojciech Kapsa, +Xabier Rodriguez Calvar, Xavier Claessens, Xuchen Yang, Yatin Maan, Yinhang Liu, +Yorick Smilda, Yuri Fedoseev, Gang Zhao, Jack Zhou, ... + +... and many others who have contributed bug reports, translations, +sent suggestions or helped testing. Thank you all! ## Stable 1.24 branch @@ -214,7 +1572,7 @@ which will be a stable branch. ### 1.24.0 -1.24.0 has not yet been released. +GStreamer 1.24.0 was released on 4 March 2024. ## Schedule for 1.26 @@ -223,12 +1581,12 @@ development version leading up to the stable 1.26 release. The development of 1.25/1.25 will happen in the git `main` branch of the GStreamer mono repository. -The schedule for 1.26 is yet to be confirmed. We're still busy getting 1.24 out! +The schedule for 1.26 is yet to be decided. 1.26 will be backwards-compatible to the stable 1.24, 1.22, 1.20, 1.18, 1.16, 1.14, 1.12, 1.10, 1.8, 1.6, 1.4, 1.2 and 1.0 release series. - - - -*These release notes have been prepared by Tim-Philipp Müller.* +*These release notes have been prepared by Tim-Philipp Müller with contributions from Edward Hervey, Matthew Waters, Nicolas Dufresne, Nirbheek Chauhan, Olivier Crête, Sebastian Dröge, Seungha Yang, Thibault Saunier, and Víctor Manuel Jáquez Leal.* *License: [CC BY-SA 4.0](http://creativecommons.org/licenses/by-sa/4.0/)* |