-
Notifications
You must be signed in to change notification settings - Fork 6.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
usb: device_next: add new MIDI 2.0 device class #81197
base: main
Are you sure you want to change the base?
Conversation
e9d967c
to
3845704
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you planning on supporting MIDI 1.0 later? The specification strongly recomments, but does not mandate the MIDI 1.0 backwards compatible interface.
Do you have idea about how to support System Exclusive commands? Should "slicing" the SysEx message be application duty or class?
7daaed1
to
45becfb
Compare
Thank you @tmon-nordic for your first comments !
I decided to go for USB-MIDI2.0, because it transports Universal MIDI Packets (UMPs). The intended benefit is that this is an externally defined spec, independent of the transport layer, supporting both MIDI1 and MIDI2, and carrying "routing" (MIDI group) metadata. I believe these packets could be readily used on other transports such as Bluetooth or IP (but this goes beyond my knowledge). On the contrary, USB-MIDI1.0 has a packet format that is specific to this transport, and therefore somehow "leaks" some implementation details that are specific to USB-MIDI. Moreover, I decided to NOT use a stream-based API ( int usbd_midi_send_sysex(const struct device *dev,
const uint8_t *data, size_t size); Overall, sending/receiving simple MIDI events (note on/off, control changes, system messages, short sysex, etc...) is straightforward with this proposal. Long SysEx are already some kind of "advanced" use cases, so I wouldn't try to do more about it here. Even though USB-MIDI1.0 transport is not supported, I noticed that I needed to add that "dummy" backward compatible interface before the USB-MIDI2.0 one, otherwise the USBD MIDI Sample wouldn't enumerate properly. I tested both with Linux 6.11/alsa 1.2.12 and Mac OS X 15.1, but I'd be glad to hear about any experience with other hosts (like Android, iOS or Windows). For the record, here is how it looks on Mac OS: And on Linux:
I haven't planned on implementing a USB-MIDI1.0 stack, because 2.0 works for me, and its features are a superset of 1.0. Moreover, I haven't a clear idea on how it would look like for application code (are the 1.0/2.0 altsetting distinguably usable from dt/code ? How does the application code know which is selected, and how to format packets ? Otherwise do we have a translation layer 1.0<->2.0 depending on the selected altsetting ?). Finally, I will dig into the full-speed/high-speed descriptors and amend the PR accordingly. Thank you again for looking at this ! |
It wouldn't enumerate properly because it is mandatory for the MIDI 2.0 interface to be on alternate setting 1. Take a look at USB MIDI 2.0 Specification and note where "should" (recommended but optional) is used and where "shall" (mandatory) is used.
I don't quite know about MIDI 2.0 but in MIDI 1.0 the most significant bit is reserved for Real Time messages. All SysEx commands (and responses) do have the most significant bit cleared to enable having the Real Time message while the SysEx is being transmitted. To what degree this is a problem is unknown to me, but USB-MIDI 1.0 chunks everything into 4 byte USB-MIDI Event Packets that can be freely interleaved. While this API would be useful for applications that use SysEx messages exclusively (e.g. Digitech effect pedals, see https://github.com/desowin/gdigi for open-source host-side implementation), I have no idea if would introduce problems when chunk interleaving is necessary.
This is something that would have to be solved if we wanted proper backwards compatibility. The alt setting is only known at runtime and is entirely host-dependent. The class would have to somehow provide the information about selected protocol version to the application. About the format conversion I have no idea. |
45becfb
to
b54c0bf
Compare
Pxl.20241119.152833728.mp4@titouanc this is really cool :) |
fbe792a
to
373525c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wrap macro values in parentheses to avoid potential issues on macro use.
373525c
to
14074e6
Compare
@titouanc wondering if you missed some of my comments regarding the code sample? Thanks! |
a9781ec
to
ea3c042
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
refreshing +1 for docs/sample - thanks!
ad22d57
to
dfea98b
Compare
Hello again @jfischer-no, and thank you for your latest review ! I've once again addressed your (and other reviewers) comments, and rebased on top of There are still a couple of open points, for which I would really like to have a clear answer:
|
@titouanc about this file: |
c8d88a1
to
e37ff03
Compare
@@ -11,3 +11,4 @@ New USB device support APIs | |||
usbd_hid_device.rst | |||
uac2_device.rst | |||
usbd_msc_device.rst | |||
usb_midi.rst |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like you used markdown formatting(?) in the commit message, what is the intent behind that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because I initially had one commit, and because Github uses the commit message by default for the PR description, this made for a nice description.
Will rewrite the commit message with the bare links only 👍
.. _usb_midi: | ||
|
||
MIDI 2.0 Class device API | ||
######################### | ||
|
||
USB MIDI 2.0 device specific API defined in :zephyr_file:`include/zephyr/usb/class/usbd_midi.h`. | ||
|
||
API Reference | ||
************* | ||
|
||
.. doxygengroup:: usb_midi | ||
.. doxygengroup:: usb_midi_ump |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding it. usb_midi_ump should be just midi_ump, if I am correct there are no USB dependencies in "Universal MIDI Packet definitions". IMO still okay to link it here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think a similar reasoning can be applied to the recently renamed include/zephyr/audio/midi.h. There are many references to USB there, which I think could be removed since that could be used for transports other than USB.
include/zephyr/usb/class/midi.h
Outdated
#include <stdint.h> | ||
|
||
/** | ||
* @brief Universal MIDI Packet definitions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is better to place "Universal MIDI Packet definitions" header in include/zephyr/audio because unlike HID it is not a USB standard and there are no USB dependencies.
include/zephyr/usb/class/usbd_midi.h
Outdated
* -EIO if MIDI2.0 is not enabled by the host | ||
* -EAGAIN if there isn't room in the transmission buffer | ||
*/ | ||
int usbd_midi_send(const struct device *dev, const uint32_t ump[static 4]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, what you call a sample is half of the driver. It is not an application or sample code just because it is placed in samples. That is part of the MIDI driver. I put it there as a requirement to get it in.
include/zephyr/usb/class/usbd_midi.h
Outdated
* -EIO if MIDI2.0 is not enabled by the host | ||
* -EAGAIN if there isn't room in the transmission buffer | ||
*/ | ||
int usbd_midi_send(const struct device *dev, const struct midi_ump *ump); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have expected to see int usbd_midi_send(const struct device *dev, const uint32_t ump[static 4]);
here. struct midi_ump is just unnecessary dependency. Not sure why it is different now, and how it helps to mark unresolved conversation resolved or in general in review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As per your suggestion, I had in an intermediate version const uint32_t ump[static 4]
(which is also my favorite version by the way). However, because C++ does not understand T [static N]
, this would prevent any C++ application to use the USB-MIDI class, because it would fail to compile any file that includes usbd_midi.h
.
While I agree this adds a layer of indirection (struct midi_ump->data
), this at least allows both C and C++ application to work with the USB-MIDI class
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry to over-write on this, but I just want to make sure that I fully understand your intent. When you say
what you call a sample is half of the driver.
this is because usbd_midi_send
is not a syscall, and therefore the caller is not isolated from kernelspace ? Is there any other reason ?
Like I said, I prefer the simplicity of uint32_t [static 4]
, but this is functionally equivalent to struct {uint32_t [4]}
, and they have the same memory layout. Besides esthetic considerations, is there any other advantage of the former over the struct form ?
In my project using usb-midi, I implement UMP Stream (discovery and dynamic function block configuration). I do the following, which just works fine (in plain C), so I don't need to bother with structs at all:
uint32_t reply[4];
/* build reply */
usbd_midi_send(dev, (const struct midi_ump *) reply);
If we are to go for the static array form, how are C++ users supposed to do ? If I understand correctly, they would need to add a some boilerplate (to compile in C) that understands uint32_t [static 4]
and expose a C++ friendly interface, which sounds like an extra burden without clear advantage to me. Do you have any other suggestion ?
Thanks again for the review, and sorry for the lengthy discussion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, because the sample is actualy the driver. You are using the Zephyr driver model and DT to implement a MIDI driver with a USB interface. In Zephyr, the drivers are written in C. If someone for whatever reason decides to write them in C++ or some other language in downstream work, it is their responsibility to provide the proper interface. I do not see why the code here should suffer if someone uses a different language. There are no real dependencies on UMP in the USB MIDI2 spec, other than determining the length. The macro to get the length can be defined in usbd_midi.c, and there would be no need to include midi.h. Together with int usbd_midi_send(const struct device *dev, const uint32_t ump[static 4]);
, this implementation would provide a tiny, clean and safe interface, independent of any future changes to midi.h.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feel free to correct me if I am on a complete wrong track, but an application will send midi data with usbd_midi_send. Therefore it has to include include/zephyr/usb/class/usbd_midi.h
According to this information from the zephyr doc page (https://docs.zephyrproject.org/latest/develop/languages/cpp/index.html#header-files-and-incompatibilities-between-c-and-c) some care has to be taken to allow headers to be included from C and C++.
If "uint32_t [static 4]" is not available in C++ it should not appear in a header file that has to be included in order to use a certain functionality. This is not writing a driver in another language, this is simply ensuring that the driver can be used from C and C++.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@stemschmidt It might help if you read my comments where I said that it is still the driver that calls usbd_midi_send(). That is the design behind using the Zephyr device model and devicetree. Limitations of the language you use must not weaken the upstream code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jfischer-no I read it, but I did not understand it. :-) And still do not understand it: How should an application send MIDI messages?
#define ALT_USB_MIDI_1 0x00 | ||
#define ALT_USB_MIDI_2 0x01 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did not understand what is the benefit of these macros, you can only have 0 and 1 as alternate, from code readability perspective there is no need to have them, but even if then
#define MIDI1_ALTERNATE 0
#define MIDI2_ALTERNATE 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find it more readable to have a name in the API implementation, especially because alt0=MIDI1 and alt1=MIDI2 (off-by-one; easy to confuse 1 for MIDI1), but this is definitely a matter of taste.
I have no opinion on ALT_USB_MIDI_1 vs. MIDI1_ALTERNATE, so I will apply your suggestion
include/zephyr/audio/midi.h
Outdated
struct midi_ump { | ||
uint32_t data[4]; /**< Raw content */ | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@titouanc Not sure how other interfaces would require it, e.g. Bluetooth. You could change usbd_midi_send() to int usbd_midi_send(const struct device *dev, const struct midi_ump ump);
, so it would be more safe, and then I would also be happy.
Also, please add a comment that the variables must be in the CPU native endianness.
include/zephyr/audio/midi.h
Outdated
|
||
/** | ||
* @brief Universal MIDI Packet definitions | ||
* @defgroup usb_midi_ump USB MIDI 2.0 Universal MIDI Packet definitions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please make it interface neutral, remove all the usb_
prefixes and USB
.
This adds a new USB device class (based on usb/device_next) that implements revision 2.0 of the MIDIStreaming interface, a sub-class of the USB audio device class. In practice, the MIDI interface is much more simple and has little in common with Audio, so it makes sense to have it as a separate class driver. MIDI inputs and outputs are configured through the device tree, under a node `compatible = "zephyr,usb-midi"`. As per the USB-MIDI2.0 spec, a single usb-midi interface can convey up to 16 Universal MIDI groups, comprising 16 channels each. Data is carried from/to the host via so-called Group Terminals, that are organized in Group Terminal Blocks. They are represented as children of the usb-midi interface in the device tree. From the Zephyr application programmer perspective, MIDI data is exchanged with the host through the device associated with the `zephyr,usb-midi` interface, using the following API: * Send a Universal MIDI Packet to the host: `usb_midi_send(device, pkt)` * Universal MIDI Packets from the host are delivered to the function passed in `usb_midi_set_ops(device, &{.rx_packet_cb = handler})` Compliant USB-MIDI 2.0 devices are required to expose a USB-MIDI1.0 interface as alt setting 0, and the 2.0 interface on alt setting 1. To avoid the extra complexity of generating backward compatible USB descriptors and translating Universal MIDI Packets from/to the old USB-MIDI1.0 format, this driver generates an empty MIDI1.0 interface (without any input/output); and therefore will only be able to exchange MIDI data when the host has explicitely enabled MIDI2.0 (alt setting 1). This implementation is based on the following documents, which are referred to in the inline comments: * `midi20`: Universal Serial Bus Device Class Definition for MIDI Devices Release 2.0 https://www.usb.org/sites/default/files/USB%20MIDI%20v2_0.pdf * `ump112`: Universal MIDI Packet (UMP) Format and MIDI 2.0 Protocol With MIDI 1.0 Protocol in UMP Format Document Version 1.1.2 https://midi.org/universal-midi-packet-ump-and-midi-2-0-protocol-specification Signed-off-by: Titouan Christophe <moiandme@gmail.com>
Add a sample application that demonstrates how to use the new USB-MIDI 2.0 device class. This shows how to set up the device tree, how to exchange MIDI data with the host, and how to use the data accessors provided by the new API. Signed-off-by: Titouan Christophe <moiandme@gmail.com>
e37ff03
to
1cc95e2
Compare
Thanks again for you latest review @jfischer-no I just applied your comments, and rebased on top of main:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a few inconsistencies, please fix them.
# Copyright (c) 2024 Titouan Christophe | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
description: USB MIDI Class |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
description: MIDI2 device
|
||
description: USB MIDI Class | ||
|
||
compatible: "zephyr,usb-midi" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
compatible: "zephyr,midi2-device"
|
||
child-binding: | ||
description: | | ||
USB MIDI Group terminal block. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MIDI2 Group terminal block?
First MIDI Group number (address) and number of Group Terminals (size) | ||
in this USB-MIDI Group Terminal Block. | ||
The MIDI Groups 1 to 16 corresponds to address 0x0 to 0xf. There are at | ||
most 16 addressable groups (of 16 channels each) per USB-MIDI interface. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
per MIDI2 interface.
type: array | ||
required: true | ||
description: | | ||
First MIDI Group number (address) and number of Group Terminals (size) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First MIDI2 Group ?
#include <zephyr/usb/class/usbd_midi.h> | ||
#include <zephyr/usb/usbd.h> | ||
|
||
#include "usbd_uac2_macros.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you still need to include it?
#include "usbd_uac2_macros.h" | ||
|
||
#include <zephyr/logging/log.h> | ||
LOG_MODULE_REGISTER(usbd_midi, CONFIG_USBD_MIDI_LOG_LEVEL); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LOG_MODULE_REGISTER(usbd_midi2, CONFIG_USBD_MIDI2_LOG_LEVEL);
struct usbd_midi_data *data = dev->data; | ||
struct udc_buf_info *info = udc_get_buf_info(buf); | ||
|
||
LOG_DBG("USB-MIDI request for %s ep=%d len=%d err=%d", dev->name, info->ep, buf->len, err); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Endpoints should always be logged as ep 0x%02x
LOG_DBG("MIDI2 request for %s ep 0x%02x len %u err %d",
dev->name, info->ep, buf->len, err);
data->ops.status_change_cb(dev, USBD_MIDI_2_ENABLED); | ||
} | ||
|
||
LOG_DBG("USB-MIDI enable for %s", dev->name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prefixing logging message with USB-MIDI does not make any sense, they are already prefixed with usbd_midi2:
LOG_DBG("Enable %s", dev->name);
+ DT_INST_CHILD_NUM_STATUS_OKAY(inst) \ | ||
* sizeof(struct usb_midi_grptrm_block_descriptor) | ||
|
||
#define USBD_MIDI_DEFINE_DESCRIPTORS(inst) \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer just n
and want it to be consistent with other class implementations #define USBD_MIDI_DEFINE_DESCRIPTORS(n)
Add implementation for the 2nd revision of the USB MIDI device class (the MIDIStreaming subclass of the Audio device class), based on the new USB device stack. Moreover, add a sample application to demonstrate usage of this new device class.
This implementation is based on the following documents:
midi20
: Universal Serial Bus Device Class Definition for MIDI Devices Release 2.0ump112
: Universal MIDI Packet (UMP) Format and MIDI 2.0 Protocol With MIDI 1.0 Protocol in UMP Format Document Version 1.1.2In this initial implementation, the device class only supports USB-MIDI2.0 (which can convey MIDI1 and MIDI2 data). However, the USB-MIDI2.0 specification requires to define a valid USB-MIDI1.0 interface before the 2.0 one for backward compatibility. Therefore a single interface defined in the device tree will yield 2 USB interfaces (with altsetting 0 and 1). The first one is always a "dummy" one (the minimal interface without any input/output), and the inputs/outputs of the second one are the ones defined in the device tree. As such, data exchange is only possible if the MIDI2.0 (altsetting 1) has been enabled by the host.
Zephyr application code can exchange MIDI data with the host in the form of Universal MIDI Packets through "Group Terminals", as illustrated in
midi20: 4. Operational model
: