KWin 'too many open files' error - pw and KDE actually working!
- PipeWire version (
pipewire --version
): Built from Master Compiled with libpipewire 1.1.82 Linked with libpipewire 1.1.82 - Distribution and distribution version (
PRETTY_NAME
from/etc/os-release
): openSUSE Tumbleweed - Desktop Environment: KDE Plasma 6.0.5
- Kernel version (
uname -r
): 6.9.1-1-default
Description of Problem:
TL;DR for the busy:
Stock KDE interaction with stock pipewire, demands a greater number of open file descriptors, than stock systemd permits.
This also effects other applications, of course. Some repeated examples seen here and elsewhere (eg reddit) are: Pro audio apps with high IO counts, notification sounds which are repeated in close succession (libcanberra spam), creating and destroying objects rapidly (eg mouseover Task Manager panels, open volume control)
Related issues:
Details:
General instability from KDE whenever interfacing with pipewire has been ongoing for a couple of years now. Apps crash/become unresponsive, and often pipewire crashes, but always errors, produces broken audio, and needs a restart. Occurs when screen sharing, opening the volume control plasma widget, viewing application thumbnails in Task Manager widget, etc. Probably the most common example I see around here, are notification sound effects breaking pipewire (I use a custom terminal bell script, to avoid libcanberra breaking it. There's an existing issue for that).
Recently (since KDE 6) I've seen several fixes go into pipewire itself, kwin, plasma, qt....and it's almost there, it's improved a LOT, but screen sharing and the volume widget remain as broken as ever. Until today, because I have an urgent demand to get screen sharing working right, so I chased this down.
Many errors are generated during the various failures, or sometimes no error at all, just a locked-up pipewire and app. The one that I chased today was this:
Jun 10 17:31:06 Pallas wireplumber[11044]: wp-event-dispatcher: <WpAsyncEventHook:0x55a635d01830> failed: <WpSiStandardLink:0x55a6363d5710> link failed: some node was destroyed before the link was created
Jun 10 17:31:31 Pallas kwin_wayland[10171]: <Unknown File>:97: TypeError: Cannot read property 'translucencyWindowTypeAnimation' of undefined
Jun 10 17:31:40 Pallas rtkit-daemon[10203]: Successfully made thread 14785 of process 13840 owned by 'pallaswept' RT at priority 10.
Jun 10 17:31:42 Pallas kded6[10607]: Registering ":1.107/StatusNotifierItem" to system tray
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4051e4930: can't DUP fd:920 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4051e4930: can't DUP fd:920 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4051e4930: can't DUP fd:920 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:939 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:942 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:941 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:946 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:945 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:948 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:947 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:952 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:949 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:954 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:953 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:956 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:955 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:960 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:959 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:962 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:961 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:966 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: mod.protocol-native: connection 0x55e4079ce410: can't DUP fd:963 Too many open files
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:6: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 8: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b14720: error seq:77 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 8: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b14720: error seq:78 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 8: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b14720: error seq:79 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:7: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 9: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a7c850: error seq:83 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 9: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a7c850: error seq:84 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 9: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a7c850: error seq:85 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:8: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 10: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a81870: error seq:89 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 10: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a81870: error seq:90 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 10: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a81870: error seq:91 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:9: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 11: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a86890: error seq:95 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 11: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a86890: error seq:96 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 11: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407a86890: error seq:97 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:10: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 12: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b18790: error seq:101 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 12: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b18790: error seq:102 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 12: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b18790: error seq:103 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:11: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 13: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b1d7b0: error seq:107 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 13: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b1d7b0: error seq:108 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 13: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b1d7b0: error seq:109 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:12: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 14: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b227d0: error seq:113 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 14: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b227d0: error seq:114 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 14: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b227d0: error seq:115 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:13: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 15: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b277f0: error seq:119 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 15: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b277f0: error seq:120 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 15: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b277f0: error seq:121 -2 (set_activation: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -22 for resource 0: can't import mem id:14: Invalid argument
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 16: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b2c810: error seq:125 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 16: node_set_io failed: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b2c810: error seq:126 -2 (node_set_io failed: No such file or directory)
Jun 10 17:31:42 Pallas pipewire[11043]: pw.core: 0x55e4045dece0: error -2 for resource 16: set_activation: No such file or directory
Jun 10 17:31:42 Pallas pipewire[11043]: mod.client-node: 0x55e407b2c810: error seq:127 -2 (set_activation: No such file or directory)
Jun 10 17:31:44 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:44 Pallas pipewire[11043]: pw.buffers: 0x55e4051f6118: can't alloc buffers: Too many open files
Jun 10 17:31:44 Pallas pipewire[11043]: pw.link: (680.0.0 -> 711.0.0) allocating -> error (error alloc buffers: Too many open files) (ready-ready)
Jun 10 17:31:44 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:44 Pallas pipewire[11043]: pw.buffers: 0x55e406e780f8: can't alloc buffers: Too many open files
Jun 10 17:31:44 Pallas pipewire[11043]: pw.link: (682.0.0 -> 713.0.0) allocating -> error (error alloc buffers: Too many open files) (ready-ready)
Jun 10 17:31:44 Pallas pipewire[11043]: pw.mem: 0x55e4045f7ab0: Failed to create memfd: Too many open files
Jun 10 17:31:44 Pallas pipewire[11043]: pw.buffers: 0x55e406e78a88: can't alloc buffers: Too many open files
Jun 10 17:31:44 Pallas pipewire[11043]: pw.link: (684.0.0 -> 715.0.0) allocating -> error (error alloc buffers: Too many open files) (ready-ready)
Jun 10 17:31:44 Pallas kwin_wayland[10171]: kwin_screencast: Stream error: error alloc buffers: Too many open files
Jun 10 17:31:44 Pallas kwin_wayland[10171]: kwin_screencast: PipeWire remote error: error alloc buffers: Too many open files
Jun 10 17:31:44 Pallas kwin_wayland[10171]: kwin_screencast: Stream error: error alloc buffers: Too many open files
Jun 10 17:31:44 Pallas kwin_wayland[10171]: kwin_screencast: PipeWire remote error: error alloc buffers: Too many open files
Jun 10 17:31:44 Pallas kwin_wayland[10171]: kwin_screencast: Stream error: error alloc buffers: Too many open files
Jun 10 17:31:44 Pallas kwin_wayland[10171]: kwin_screencast: PipeWire remote error: error alloc buffers: Too many open files
Jun 10 17:31:44 Pallas pipewire[11043]: mod.protocol-native: 0x55e404610ce0: connection_data: client 0x55e405171370 error -71 (Protocol error)
Jun 10 17:31:44 Pallas kwin_wayland[10171]: kwin_screencast: PipeWire remote error: connection error
Jun 10 17:31:52 Pallas xdg-desktop-portal-kde[11110]: xdp-kde-screencast: Invalid window! QMap((0, QVariant(QString, "— Konsole"))(1, QVariant(QIcon, QIcon("utilities-terminal",availableSizes[normal,Off]=QList(QSize(16, 16), QSize(22, 22), QSize(32, 32), QSize(48, 48),>
Jun 10 17:31:52 Pallas kded6[10607]: Service ":1.107" unregistered
This is what is currently seen in the journal, when attempting a screenshare. Many of the other failures (such as the volume control) do not show any errors, they just fail/lock up/distort audio/etc....
... but apparently, they are ALL fixed by fixing the screensharing. In a quick shot at fixing this, I did the obvious, and increased the number of file handlers allowed to the user. Instant fix. And not only for the screensharing thing I intended to fix, but now all kinds of broken things are starting to work. Even my 'litmus test', broken since day 1 on linux, the KDE volume plasmoid.
Going through the issues here, I see a few rather old issues that seem related, they all mention running lsof
on pipewire/pipewire-pulse, so I have (attached), and I find it interesting that the number of file handles allocated is ~500 + ~1500-1700. I feel like this is noteworthy because:
A) My system is nothing unusual. It's a fairly normal gaming desktop with a few soundcards, one PCIe on the motherboard, a few USB and bluetooth, some loopback devices for bussing, some filters for noise cancelling the mic and EQ'ing headphones. It's a stock gamer/streamer setup. The use case here is simple; watch a video, share your screen to your chat room, change the volume.
B) The stock systemd file handle soft limit is 1024 (hard limit is 524288)
It was only after raising this limit that I could see what was required, for this simple use-case to be functional.
Perhaps relevant? During testing, I saw the following results:
System limits
soft 10485760 hard unlimited
User limits
soft 10485760 hard unlimited
Result: Everything works
System limits
soft 1024 hard 524288 (stock)
User limits
soft 10485760 hard unlimited (result: soft 524288 hard 524288)
Result: Screen share works, volume meters cause distortion
Stock limits:
soft 1024 hard 524288 (system and user)
Result: Screen share crashes firefox and plasma and pipewire, volume control breaks audio and pipewire.
The interesting thing here is that I still see failures with a very large number of file handlers (524288) allowed which is far higher than the number actually in use. Sitting here watching a video and typing this up, I get:
> lsof |wc -l
lsof: WARNING: can't stat() tracefs file system /sys/kernel/debug/tracing
Output information may be incomplete.
445220
So I'm still fairly well clear of 500K. Opening the volume control only adds about 1000 to this number.
So now I'm unsure what's broken here? KDE? Distro packaging? systemd? pipewire defaults? Something deeper? Maybe this is not a configuration problem excessively limiting fds, maybe this is a pw bug excessively using fds? I don't know! Enlightening thoughts are appreciated
I hope this is useful. KDE has been super annoying me by being mostly awesome and yet totally hating doing any A/V work beyond a single audio stream, for a couple of years, now. This is a huge weight off my shoulders. I'd like to see if, with your help, we can squeeze a 'real fix' out of what I've found here today, to share the love.
How Reproducible:
100%
Steps to Reproduce:
- Use KDE
- Apply this config: https://bugsfiles.kde.org/attachment.cgi?id=162904
- Try to change volume.
- Try to screen share (eg WebRTC)
Actual Results:
KDE creates a node to capture the audio, to show volume meters on the volume controls. It creates one for every node in the graph, so, a lot, all at once. When this happens, The widget becomes unresponsive, and all of the nodes for the volume widget become 'stuck' until pw is restarted.
For screen capture, the nodes also become 'stuck' and the application becomes unresponsive (eg webrtc client firefox tab cannot be closed without quitting the entire browser session, plasma panel becomes unresponsive)
Expected Results:
Volume control and screen sharing and other KDE-pipewire interactions as normal.
Additional Info (as attachments):
relevant journalctl snippet memfd_journal.txt
lsof -p pidof pipewire
memfd_pw-lsof.txt
lsof -p pidof pipewire-pulse
memfd_pw-pulse-lsof.txt
I did take a pw-dump but I did not attach it yet as I'm pretty sure it's not relevant, mine is a mess right now and would only serve as a distraction. I've tested with stock config extensively, but didn't save logs at the time. You can see screenshots in the KDE issue linked above where I have taken a live tumbleweed ISO and booted it, added the attached config (30 loopback devices), and reproduced the error. I did the same with kubuntu and neon, so I'm very confident that anyone could repro this behaviour with that config. But if you really want to see a big ugly pw-dump.txt just say the word
Thanks!
Edit: added TLDR and related issues, at the top.