For systems with Intel integrated graphics, hardware GPU decoding with the gstreamer VAAPI plugin is preferable. VAAPI is open-source, and in addition to Intel, can support some AMD GPU’s (the open-source “Nouveau” drivers for NVIDIA graphics are also in principle supported when VAAPI is supplemented with firmware extracted from the proprietary NVIDIA drivers).
For NVIDIA graphics with the proprietary drivers, the nvdec plugin (recently renamed nvh264dec) can be used for accelerated video decoding on the NVIDIA GPU with CUDA. The nvdec plugin is part of gstreamer1.0-plugins-bad, but is generally not included in binary packages, as NVIDIA’s proprietary Video Codec SDK must be downloaded, and three header files from it must be added to the gstreamer source before the plugin can be compiled. Users must do this themselves: see these instructions, and adapt them as necessary for your GStreamer installation. This plugin should be used with the -vd nvdec (or nvh264dec) and -vs glimagesink uxplay options.
Support for Raspberry Pi (not yet).
-The Raspberry Pi (e.g., model 4B) is powerful enough to run UxPlay, but only with hardware-accelerated h264 decoding by its Broadcom GPU (software decoding with uxplay option -avdec works, but with unacceptable latency). Raspberry Pi OS (Bullseye) has abandoned the omx (OpenMAX) GPU driver used by RPiPlay, and the corresponding GStreamer plugin omxh264dec has been deprecated and broken for some time. The replacement is the Video4Linux (v4l2) plugin v4l2h264dec from gstreamer1.0-plugins-good, which works well for playing mp4 files on the Pi. Unfortunately, features needed by UxPlay are currently broken in this plugin, see the open Issue. The (undocumented) UxPlay option -rpi will use the Pi’s GPU for video decoding, but will only work when an updated v4l2h264dec plugin becomes available. There is now work on this in the GStreamer development cycle: hopefully this might get backported to GStreamer-1.18 or 1.20 for the Pi, otherwise it will appear in v1.22.
GPU Support for Raspberry Pi
+Some Raspberry Pi models (e.g., model 4B) are powerful enough to run UxPlay, but work best with hardware-accelerated h264 decoding by its Broadcom GPU (software decoding with combined uxplay options -rpi -avdec may also work with acceptable latency on Pi model 4). Raspberry Pi OS (Bullseye) has recently abandoned the older omx (OpenMAX) GPU driver used by RPiPlay, and the corresponding GStreamer plugin omxh264dec has been deprecated and broken for some time. The replacement is the Video4Linux (v4l2) plugin v4l2h264dec from gstreamer1.0-plugins-good, which works well for playing mp4 files on the Pi. Unfortunately, features needed by UxPlay are broken in current releases of this plugin, but are fixed as of the v1.21 development branch. See the open Issue for a patch against the current stable release 1.20.0. The UxPlay option -rpi by itself will use the Pi’s GPU for video decoding with the patched plugin.
Some Linux distributions such as Debian do not allow distribution of compiled GPL code linked to OpenSSL-1.1.1 because its “dual OpenSSL/SSLeay” license has some incompatibilities with GPL, unless all code authors have explicitly given an “exception” to allow such linking (the historical origins of UxPlay make this impossible to obtain). Other distributions treat OpenSSL as a “System Library” which the GPL allows linking to.
@@ -43,7 +43,7 @@If you intend to modify the code, use a separate “build” directory: replace “cmake [ ] .” by “mkdir build ; cd build ; cmake [ ] ..”; you can then clean the build directory with “rm -rf build/*” (run from within the UxPlay source directory) without affecting the source directories which contain your modifications.
The above script installs the executable file “uxplay” to /usr/local/bin, (and installs a manpage to somewhere like /usr/local/share/man/man1 and README files to somewhere like /usr/local/share/doc/uxplay). It can also be found in the build directory after the build processs.
Finally, run uxplay in a terminal window. If it is not seen by the iOS client’s drop-down “Screen Mirroring” panel, check that your DNS-SD server (usually avahi-daemon) is running: do this in a terminal window with systemctl status avahi-daemon. If this shows the avahi-daemon is not running, control it with sudo systemctl [start,stop,enable,disable] avahi-daemon (or avahi-daemon.service). If UxPlay is seen, but the client fails to connect when it is selected, there may be a firewall on the server that prevents UxPlay from receiving client connection requests unless some network ports are opened. See Troubleshooting below for help with this or other problems. See Usage for run-time options. For OpenGL support (option -vs glimagesink), needed for NVIDIA GPU-based video decoding, make sure gstreamer1.0-gl is installed.
Finally, run uxplay in a terminal window. If it is not seen by the iOS client’s drop-down “Screen Mirroring” panel, check that your DNS-SD server (usually avahi-daemon) is running: do this in a terminal window with systemctl status avahi-daemon. If this shows the avahi-daemon is not running, control it with sudo systemctl [start,stop,enable,disable] avahi-daemon (or avahi-daemon.service). If UxPlay is seen, but the client fails to connect when it is selected, there may be a firewall on the server that prevents UxPlay from receiving client connection requests unless some network ports are opened. See Troubleshooting below for help with this or other problems. See Usage for run-time options. For OpenGL support (option -vs glimagesink), needed for Raspberry Pi and NVIDIA GPU-based video decoding, make sure gstreamer1.0-gl is installed.
Red Hat, Fedora, CentOS (now continued as Rocky Linux or Alma Linux): (sudo yum install) openssl-devel libplist-devel avahi-compat-libdns_sd-devel (some from the “PowerTools” add-on repository) (+libX11-devel for ZOOMFIX). The required GStreamer packages (some from rpmfusion.org) are: gstreamer1-devel gstreamer1-plugins-base-devel gstreamer1-libav gstreamer1-plugins-bad-free (+ gstreamer1-vaapi for intel graphics).
OpenSUSE: (sudo zypper install) libopenssl-devel libplist-devel avahi-compat-mDNSResponder-devel (+ libX11-devel for ZOOMFIX). The required GStreamer packages (you may need to use versions from Packman) are: gstreamer-devel gstreamer-plugins-base-devel gstreamer-plugins-libav gstreamer-plugins-bad (+ gstreamer-plugins-vaapi for Intel graphics).
-vc converter chooses the GStreamer pipeline’s videoconverter element, instead of the default value “videoconvert”. When using video4linux hardware decoding by a GPU,-vc v4l2convert will also use the GPU for video conversion. Using quotes “…” allows some parameters to be included with the converter name.
-vs videosink chooses the GStreamer videosink, instead of letting autovideosink pick it for you. Some videosink choices are: ximagesink, xvimagesink, vaapisink (for intel graphics), gtksink, glimagesink, waylandsink, osximagesink (for macOS), or fpsdisplaysink (which shows the streaming framerate in fps). Using quotes “…” allows some parameters to be included with the videosink name. For example, fullscreen mode is supported by the vaapisink plugin, and is obtained using -vs "vaapisink fullscreen=true"; this also works with waylandsink. The syntax of such options is specific to a given plugin, and some choices of videosink might not work on your system.
-vs 0 suppresses display of streamed video, but plays streamed audio. (The client’s screen is still mirrored at a reduced rate of 1 frame per second, but is not rendered or displayed.) This feature (which streams audio in AAC audio format) is now probably unneeded, as UxPlay can now stream superior-quality Apple Lossless audio without video in Airplay non-mirror mode.
+-rpi Video settings for Raspberry Pi, for hardware h264 video decoding in the GPU (requires the video4linux2 plugin from GStreamer-1.21.0 or later, or a backported patched version of an earlier release. (If this is unavailable, use uxplay -rpi -avdec). Uses the glimagesink videosink.
-avdec forces use of software h264 decoding using Gstreamer element avdec_h264 (libav h264 decoder). This option should prevent autovideosink choosing a hardware-accelerated videosink plugin such as vaapisink.
-as audiosink chooses the GStreamer audiosink, instead of letting autoaudiosink pick it for you. Some audiosink choices are: pulsesink, alsasink, osssink, oss4sink, and osxaudiosink (for macOS). Using quotes “…” might allow some parameters to be included with the audiosink name. (Some choices of audiosink might not work on your system.)
-as 0 (or just -a) suppresses playing of streamed audio, but displays streamed video.
@@ -125,7 +126,7 @@This triggers an unending stream of error messages, and means that the audio decryption key (also used in video decryption) was not correctly extracted from data sent by the client. This should not happen for iOS 9.3 or later clients. However, if a client uses the same older version of the protocol that is used by the Windows-based AirPlay client emulator AirMyPC, the protocol can be switched to the older version by the setting OLD_PROTOCOL_CLIENT_USER_AGENT_LIST in lib/global.h. UxPlay reports the client’s “User Agent” string when it connects. If some other client also fails to decrypt all audio and video, try adding its “User Agent” string in place of “xxx” in the entry “AirMyPC/2.0;xxx” in global.h and rebuild uxplay.
Note that Uxplay declares itself to be an AppleTV3,2 with a sourceVersion 220.68; this can also be changed in global.h. It had been thought that it was necessary for UxPlay to claim to be an older 32 bit AppleTV model that cannot run modern 64bit tvOS, in order for the client to use a “legacy” protocol for pairing with the server (see the “Notes on AirPlay protocol versions” at the end of this README). However, UxPlay still works if it declares itself as an AppleTV6,2 with sourceVersion 380.20.1 (an AppleTV 4K 1st gen, introduced 2017, running tvOS 12.2.1), so it is unclear what setting prompts the client to use the “legacy” protocol needed by UxPlay.
1.48 2022-02-24 Made the video pipeline fully configurable, for use with hardware h264 decoding.
+1.48 2022-02-24 Made the video pipeline fully configurable, for use with hardware h264 decoding. Support for Raspberry Pi.
1.47 2022-02-05 Added -FPSdata option to display (in the terminal) regular reports sent by the client about video streaming performance. Internal cleanups of processing of video packets received from the client. Added -reset n option to reset the connection after n ntp timeouts (also reset after “connection reset by peer” error in video stream).
1.46 2022-01-20 Restore pre-1.44 behavior (1.44 may have broken hardware acceleration): once again use decodebin in the video pipeline; introduce new option “-avdec” to force software h264 decoding by libav h264, if needed (to prevent selection of vaapisink by autovideosink). Update llhttp to v6.0.6. UxPlay now reports itself as AppleTV3,2. Restrict connections to one client at a time (second client must now wait for first client to disconnect).
1.45 2022-01-10 New behavior: close video window when client requests “stop mirroring”. (A new “no close” option “-nc” is added for users who wish to retain previous behavior that does not close the video window).
diff --git a/README.md b/README.md index 863a788..50c4832 100644 --- a/README.md +++ b/README.md @@ -76,17 +76,18 @@ Users must do this themselves: see [these instructions](https://gist.github.com/ adapt them as necessary for your GStreamer installation. This plugin should be used with the `-vd nvdec` (or nvh264dec) and `-vs glimagesink` uxplay options. -* **Support for Raspberry Pi (not yet).** +* **GPU Support for Raspberry Pi** - The Raspberry Pi (e.g., model 4B) is powerful enough to run UxPlay, but only with hardware-accelerated h264 decoding by - its Broadcom GPU (software decoding with uxplay option `-avdec` works, but with unacceptable latency). Raspberry Pi OS (Bullseye) - has abandoned the omx (OpenMAX) GPU driver used by [RPiPlay](http://github.com/FD-/RPiPlay), and the corresponding + Some Raspberry Pi models (e.g., model 4B) are powerful enough to run UxPlay, but work best with hardware-accelerated h264 decoding by + its Broadcom GPU (software decoding with combined uxplay options `-rpi -avdec` may also work with acceptable + latency on Pi model 4). Raspberry Pi OS (Bullseye) + has recently abandoned the older omx (OpenMAX) GPU driver used by [RPiPlay](http://github.com/FD-/RPiPlay), and the corresponding GStreamer plugin omxh264dec has been deprecated and broken for some time. The replacement is the Video4Linux (v4l2) plugin v4l2h264dec from gstreamer1.0-plugins-good, which works well for playing mp4 files on the Pi. Unfortunately, features needed - by UxPlay are currently broken in this plugin, see the open [Issue](https://github.com/FDH2/UxPlay/issues/70). The (undocumented) - UxPlay option `-rpi` will use the Pi's GPU for video decoding, but will only work when an updated v4l2h264dec plugin becomes available. - There is now work on this in the GStreamer development cycle: hopefully this might get backported to GStreamer-1.18 or 1.20 for the Pi, - otherwise it will appear in v1.22. + by UxPlay are broken in current releases of this plugin, but are fixed as of the v1.21 development branch. See the + open [Issue](https://github.com/FDH2/UxPlay/issues/70) for a patch against the current stable release 1.20.0. The + UxPlay option `-rpi` by itself will use the Pi's GPU for video decoding with the patched plugin. + ### Note to packagers: OpenSSL-3.0.0 solves GPL v3 license issues. @@ -148,7 +149,7 @@ If this shows the avahi-daemon is not running, control it with ```sudo systemctl If UxPlay is seen, but the client fails to connect when it is selected, there may be a firewall on the server that prevents UxPlay from receiving client connection requests unless some network ports are opened. See [Troubleshooting](#troubleshooting) below for help with this or other problems. See [Usage](#usage) for run-time options. For OpenGL support (option -vs glimagesink), needed for -NVIDIA GPU-based video decoding, make sure gstreamer1.0-gl is installed. +Raspberry Pi and NVIDIA GPU-based video decoding, make sure gstreamer1.0-gl is installed. * **Red Hat, Fedora, CentOS (now continued as Rocky Linux or Alma Linux):** (sudo yum install) openssl-devel libplist-devel avahi-compat-libdns_sd-devel (some from the "PowerTools" add-on repository) @@ -369,6 +370,10 @@ Also: image transforms that had been added to RPiPlay have been ported to UxPlay feature (which streams audio in AAC audio format) is now probably unneeded, as UxPlay can now stream superior-quality Apple Lossless audio without video in Airplay non-mirror mode. +**-rpi** Video settings for Raspberry Pi, for hardware h264 video decoding in the GPU (requires the + video4linux2 plugin from GStreamer-1.21.0 or later, or a backported patched version of an earlier release. + (If this is unavailable, use `uxplay -rpi -avdec`). Uses the glimagesink videosink. + **-avdec** forces use of software h264 decoding using Gstreamer element avdec_h264 (libav h264 decoder). This option should prevent autovideosink choosing a hardware-accelerated videosink plugin such as vaapisink. @@ -526,7 +531,7 @@ tvOS 12.2.1), so it is unclear what setting prompts the client to use the "legacy" protocol needed by UxPlay. # ChangeLog -1.48 2022-02-24 Made the video pipeline fully configurable, for use with hardware h264 decoding. +1.48 2022-02-24 Made the video pipeline fully configurable, for use with hardware h264 decoding. Support for Raspberry Pi. 1.47 2022-02-05 Added -FPSdata option to display (in the terminal) regular reports sent by the client about video streaming performance. Internal cleanups of processing of video packets received from the client. Added -reset n option diff --git a/README.txt b/README.txt index ceac569..51a2a27 100644 --- a/README.txt +++ b/README.txt @@ -100,26 +100,25 @@ and adapt them as necessary for your GStreamer installation. This plugin should be used with the `-vd nvdec` (or nvh264dec) and `-vs glimagesink` uxplay options. -- **Support for Raspberry Pi (not yet).** +- **GPU Support for Raspberry Pi** - The Raspberry Pi (e.g., model 4B) is powerful enough to run UxPlay, - but only with hardware-accelerated h264 decoding by its Broadcom GPU - (software decoding with uxplay option `-avdec` works, but with - unacceptable latency). Raspberry Pi OS (Bullseye) has abandoned the - omx (OpenMAX) GPU driver used by + Some Raspberry Pi models (e.g., model 4B) are powerful enough to run + UxPlay, but work best with hardware-accelerated h264 decoding by its + Broadcom GPU (software decoding with combined uxplay options + `-rpi -avdec` may also work with acceptable latency on Pi model 4). + Raspberry Pi OS (Bullseye) has recently abandoned the older omx + (OpenMAX) GPU driver used by [RPiPlay](http://github.com/FD-/RPiPlay), and the corresponding GStreamer plugin omxh264dec has been deprecated and broken for some time. The replacement is the Video4Linux (v4l2) plugin v4l2h264dec from gstreamer1.0-plugins-good, which works well for playing mp4 - files on the Pi. Unfortunately, features needed by UxPlay are - currently broken in this plugin, see the open - [Issue](https://github.com/FDH2/UxPlay/issues/70). The - (undocumented) UxPlay option `-rpi` will use the Pi's GPU for video - decoding, but will only work when an updated v4l2h264dec plugin - becomes available. There is now work on this in the GStreamer - development cycle: hopefully this might get backported to - GStreamer-1.18 or 1.20 for the Pi, otherwise it will appear in - v1.22. + files on the Pi. Unfortunately, features needed by UxPlay are broken + in current releases of this plugin, but are fixed as of the v1.21 + development branch. See the open + [Issue](https://github.com/FDH2/UxPlay/issues/70) for a patch + against the current stable release 1.20.0. The UxPlay option `-rpi` + by itself will use the Pi's GPU for video decoding with the patched + plugin. ### Note to packagers: OpenSSL-3.0.0 solves GPL v3 license issues. @@ -206,8 +205,9 @@ connect when it is selected, there may be a firewall on the server that prevents UxPlay from receiving client connection requests unless some network ports are opened. See [Troubleshooting](#troubleshooting) below for help with this or other problems. See [Usage](#usage) for run-time -options. For OpenGL support (option -vs glimagesink), needed for NVIDIA -GPU-based video decoding, make sure gstreamer1.0-gl is installed. +options. For OpenGL support (option -vs glimagesink), needed for +Raspberry Pi and NVIDIA GPU-based video decoding, make sure +gstreamer1.0-gl is installed. - **Red Hat, Fedora, CentOS (now continued as Rocky Linux or Alma Linux):** (sudo yum install) openssl-devel libplist-devel @@ -479,6 +479,12 @@ streams audio in AAC audio format) is now probably unneeded, as UxPlay can now stream superior-quality Apple Lossless audio without video in Airplay non-mirror mode. +**-rpi** Video settings for Raspberry Pi, for hardware h264 video +decoding in the GPU (requires the video4linux2 plugin from +GStreamer-1.21.0 or later, or a backported patched version of an earlier +release. (If this is unavailable, use `uxplay -rpi -avdec`). Uses the +glimagesink videosink. + **-avdec** forces use of software h264 decoding using Gstreamer element avdec\_h264 (libav h264 decoder). This option should prevent autovideosink choosing a hardware-accelerated videosink plugin such as @@ -693,7 +699,7 @@ ChangeLog ========= 1.48 2022-02-24 Made the video pipeline fully configurable, for use with -hardware h264 decoding. +hardware h264 decoding. Support for Raspberry Pi. 1.47 2022-02-05 Added -FPSdata option to display (in the terminal) regular reports sent by the client about video streaming performance. diff --git a/uxplay.1 b/uxplay.1 index 2494c06..53d2e99 100644 --- a/uxplay.1 +++ b/uxplay.1 @@ -58,6 +58,8 @@ UxPlay 1.48: An open\-source AirPlay mirroring server based on RPiPlay .TP \fB\-vs\fR 0 Streamed audio only, with no video display window. .TP +\fB\-rpi\fR Video settings for Raspberry Pi (for GPU h264 decoding). +.TP \fB\-avdec\fR Force software h264 video decoding with libav decoder. .TP \fB\-as\fI sink\fR Choose the GStreamer audiosink; default "autoaudiosink" diff --git a/uxplay.cpp b/uxplay.cpp index f66d216..211b602 100644 --- a/uxplay.cpp +++ b/uxplay.cpp @@ -237,8 +237,8 @@ static void print_info (char *name) { printf("-vs ... Choose the GStreamer videosink; default \"autovideosink\"\n"); printf(" some choices: ximagesink,xvimagesink,vaapisink,glimagesink,\n"); printf(" gtksink,waylandsink,osximagesink,fpsdisplaysink, etc.\n"); - // printf("-rpi Video settings for Raspberry Pi 4 (h264 decoding in GPU).\n"); printf("-vs 0 Streamed audio only, with no video display window\n"); + printf("-rpi Video settings for Raspberry Pi (for GPU h264 decoding).\n"); printf("-avdec Force software h264 video decoding with libav decoder\n"); printf("-as ... Choose the GStreamer audiosink; default \"autoaudiosink\"\n"); printf(" choices: pulsesink,alsasink,osssink,oss4sink,osxaudiosink\n"); @@ -483,9 +483,13 @@ int main (int argc, char *argv[]) { } else if (arg == "-nc") { new_window_closing_behavior = false; } else if (arg == "-avdec") { + video_parser.erase(); + video_parser = "h264parse"; video_decoder.erase(); video_decoder = "avdec_h264"; - } else if (arg == "-rpi") { /*undocumented option, may be removed */ + video_converter.erase(); + video_converter = "videoconvert"; + } else if (arg == "-rpi") { video_parser.erase(); video_parser = "h264parse ! capssetter caps=\"video/x-h264, colorimetry=bt709\""; video_decoder.erase();