Effects do not apply to JACK apps, is this intentional and how to enable?

[nyanpasu64]

EasyEffects on WirePlumber currently does not route JACK apps through the effect chain. Is this deliberate, and how can I force it to apply speaker filtering to JACK apps?

[wwmm]

Is this deliberate, and how can I force it to apply speaker filtering to JACK apps?

Definitely not intentional. In theory it should work. But as I do not use Jack I am not sure about what may be happening. Kill EasyEffects easyeffects -q and restart it in debug mode G_MESSAGES_DEBUG=easyeffects easyeffects. After this start the Jack app and show me what is printed by EasyEffects in the terminal.

Do effects work if you force the Jack app to use our virtual device? If yes then the problem is EasyEffects not being able to move the Jack stream to our virtual device. What may be a WirePlumber limitation. All we do is asking the session manager, in this case WirePlumber, to move the stream to our virtual de vice.

[nyanpasu64]

If JACK clients lack SPA_KEY_MEDIA_CLASS = "media.class", do you think this is a pipewire/pipewire-jack/wireplumber bug or a deliberate decision? I asked on IRC though Wim probably didn't see my message.

Is there a deliberate distinction between JACK and other clients (https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/FAQ#does-desktop-audio-interfere-with-pro-audio-using-pipewire, https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/425), and is this built around omitting media.class? That would seem odd.

I'll have to look into how pipewire-alsa/pulse/jack/Audacious set SPA_KEY_MEDIA_CLASS.

[wwmm]

If JACK clients lack SPA_KEY_MEDIA_CLASS = "media.class", do you think this is a pipewire/pipewire-jack/wireplumber bug or a deliberate decision?

Considering that the media role is set to DSP I do not think it is a bug. One condition for a stream to be visible by Pulseaudio clients like Pavucontrol is having the property media.class set to a suitable value. If the stream/node does not have it Pavucontrol and other Pulseaudio apps won't be able to see it. That is why you can not see EasyEffects filters in Pavucontrol. If they do not want DSP nodes to be visible by Pulseaudio apps it makes sense that there is no media.class set.

I can make them available in EasyEffects. But there is the problem I talked about before. It does not make sense for us to show the same node both in the Players and the Recorders tab. Otherwise the last one to enable effects to the node may undo what was done by the other. We need some way to know if the DSP node is intended to play audio to a device or recording from it.

[wwmm]

We need some way to know if the DSP node is intended to play audio to a device or recording from it.

Actually it is worse than this. EasyEffects would try to move all DSP nodes to its virtual devices. What would break any pipeline using these nodes. I think we will need deeper changes to deal with them. It is not the same as handling streams.

[nyanpasu64]

I found out that media.role = DSP is likely not why pipewire-jack lacks a PW_KEY_MEDIA_CLASS = SPA_KEY_MEDIA_CLASS = media.class.

In doc/tutorial4.c, pw_stream_new_simple() isn't passed a PW_KEY_MEDIA_CLASS = media.class, but the client (main() -> src/pipewire/stream.c#pw_stream_connect, find PW_KEY_MEDIA_CLASS) synthesizes a media.class = "Stream/Output/{media.type}". Changing PW_KEY_MEDIA_TYPE to a different string changed the last field of media.class and broke audio playback.

pipewire-jack/src/pipewire-jack.c sets:

	if (pw_properties_get(client->props, PW_KEY_MEDIA_ROLE) == NULL)
		pw_properties_set(client->props, PW_KEY_MEDIA_ROLE, "DSP");

Adding media.class to pipewire-jack?

I think the discrepancy is that pipewire-jack simply doesn't call pw_stream_new_simple (though C debuggers can never prove the absence of function calls, only their presence), and creates a stream some other way but omits PW_KEY_MEDIA_CLASS. But I don't find any functions mentioning stream in the entire file. Maybe it's worth adding in an equivalent to pw_stream_new_simple:

	if ((pw_properties_get(stream->properties, PW_KEY_MEDIA_CLASS) == NULL)) {
		const char *media_type = pw_properties_get(stream->properties, PW_KEY_MEDIA_TYPE);
		pw_properties_setf(stream->properties, PW_KEY_MEDIA_CLASS, "Stream/%s/%s",
				direction == PW_DIRECTION_INPUT ? "Input" : "Output",
				media_type ? media_type : get_media_class(impl));
	}

I'm not sure if this is correct or where it should go in JACK.

• jack_client_open might be the wrong spot since direction could be input or output or both, and we don't know any spa_direction until jack_port_register (a public JACK function called by the client app, twice by Audacious's stereo output streams, which is probably not what we want).
• I don't see any examples of playing and capturing audio simultaneously in PipeWire.

I tried testing how existing duplex apps handle media.class: webrtc-pulse audacious-jack jack_iodelay.txt

• Discord voice calls open "id": 109, "type": "PipeWire:Interface:Client", ..."application.name": "WEBRTC VoiceEngine",.
  • Two child nodes:
  • "id": 132, "type": "PipeWire:Interface:Node", ... "n-input-ports": 0, "n-output-ports": 2, ... "client.id": 109, "node.name": "WEBRTC VoiceEngine", "media.class": "Stream/Output/Audio",
    • two output (right) ports linked by node.id (don't care)
  • "id": 133, "type": "PipeWire:Interface:Node", ... "n-input-ports": 2, "n-output-ports": 2, ... "client.id": 109, "node.name": "WEBRTC VoiceEngine", "media.class": "Stream/Input/Audio",
    • two input (left) ports and two monitor output ports linked by node.id (don't care)
• Audacious stereo playback opens "id": 114, "type": "PipeWire:Interface:Client", ... "application.name": "audacious",.
  • Two child nodes:
  • "id": 113, "type": "PipeWire:Interface:Node", ... "n-input-ports": 0, "n-output-ports": 2, ... "client.id": 114, "node.name": "audacious", no media.class
    • two output (right) ports linked by node.id.
• jack_iodelay mono capture/playback opens "id": 124, "type": "PipeWire:Interface:Client", ... "application.name": "jack_iodelay",
  • One child node:
  • "id": 128, "type": "PipeWire:Interface:Node", ... "n-input-ports": 1, "n-output-ports": 1, ... "client.id": 124, "node.name": "jack_delay", no media.class
    • Two child ports, no input monitor interestingly.

No road forward?

I'm beginning to suspect JACK apps are not built to work well with EasyEffects, since many (not all) of them expose input and output audio ports, don't connect them to speakers, and expect the user to route them as needed.

• Is there a specification for media.class names, or a list of valid ones?

One condition for a stream to be visible by Pulseaudio clients like Pavucontrol is having the property media.class set to a suitable value.

• Is there a media.class for nodes which have both input and output ports? Should one be added?
  • Does PulseAudio even have such a concept? What about EasyEffects? (EasyEffects should not process JACK nodes unless they autoconnect to speakers, and whether to autoconnect or have the user route audio where needed is a point of social contention I don't fully understand.)
• Should JACK nodes have input monitors? Should they be split into separate input+monitor and output nodes? (Does this break compatibility with existing JACK apps? IDK.)
• Is there an API with JACK-like latency, but designed to connect directly to speakers and properly apply global mic/speaker filtering by default? Perhaps PipeWire (though few apps support it so far)? I hear that PulseAudio is push-based and inherently not fixed-latency, but I'm not sure, and pipewire-alsa has far more latency than necessary (or even recommended output latency to avoid app malfunctions).
  • Audacity JACK depends on autoconnecting, since it destroys and recreates audio playback ports (though not the node itself) when I start and stop playback (presumably the same applies to recording).

(Incidentally I hit https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2726 during testing.)

[wwmm]

I think the discrepancy is that pipewire-jack simply doesn't call pw_stream_new_simple

Probably because audacity is not creating a stream. There are others ways to create a node. For example in EasyEffects we create our filters through pw_filter objects. No streams involved in this case. And we also do not set a media.class to them.

I'm beginning to suspect JACK apps are not built to work well with EasyEffects, since many (not all) of them expose input and output audio ports, don't connect them to speakers, and expect the user to route them as needed.

They are definitely not implemented with EasyEffects in mind. Jack apps are older than even PipeWire after all. But the main point is that Jack apps usually have general purpose hosts like Ardour or Carla in mind. Like you have noticed the idea is that the user will link the nodes according to what has to be done.

Is there a specification for media.class names, or a list of valid ones?

Not that I am aware of. But you can see here https://github.com/wwmm/easyeffects/blob/master/include/tags_pipewire.hpp the ones we make use in EasyEffects. There are other tags we do not check for like Audio/Duplex.

Is there a media.class for nodes which have both input and output ports?

Yes. It would be Audio/Duplex https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Virtual-Devices#create-a-duplex-node. But I think it is intended for nodes that behave like sink/source and not filters or similar. They are visible in Pavucontrol for example.

Should JACK nodes have input monitors?

Probably not. Monitors are usually only useful in sinks. A DSP node should have the proper output ports if it makes sense to get audio from it.

Is there an API with JACK-like latency, but designed to connect directly to speakers and properly apply global mic/speaker filtering by default? Perhaps PipeWire (though few apps support it so far)?

As far as I can see the most an app can do is suggesting to PipeWire the latency it wants. It is up to PipeWire to decide if it will switch to that latency or not.