rest-proxy: allow reading messages with avro-value and string-key

[jclarysse]

It is not uncommon that Kafka users produce messages with String-key and Avro-value. In this case Karapace REST typically fails to read the message since it only supports one format for deserializing both the key and the value. For more convenience, I suggest to fallback to binary deserializer for the key (only) when reading it using avro fails.

For the manual (and integration) test we actually don't care about the value. We just assume that it exists in Avro-format, important is that we produce a message with Base64 encoded String-key:

$ echo "testkey" | base64
dGVzdGtleQo=
$ curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" \
  -d '{"records": [{"key": "dGVzdGtleQ=="}]}' \
  http://localhost:8082/topics/my_topic
{"key_schema_id":null,"offsets":[{"offset":0,"partition":0}],"value_schema_id":null}

Now we want to consumer the whole message, knowing that the value is in Avro format, so we create the consumer accordingly, and subscribe to the topic:

$ curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" -H "Accept: application/vnd.kafka.v2+json" \
  --data '{"name": "my_consumer", "format": "avro", "auto.offset.reset": "earliest"}' \
  http://localhost:8082/consumers/avro_consumers
$ curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" --data '{"topics":["my_topic"]}' \
  http://localhost:8082/consumers/avro_consumers/instances/my_consumer/subscription

Reading records BEFORE the change:

$ curl -X GET -H "Accept: application/vnd.kafka.avro.v2+json" \
  http://localhost:8082/consumers/avro_consumers/instances/my_consumer/records\?timeout\=1000
{"error_code":422,"message":"key deserialization error for format avro: unpack requires a buffer of 5 bytes"}

Reading records AFTER the change:

$ curl -X GET -H "Accept: application/vnd.kafka.avro.v2+json" \
  http://localhost:8081/consumers/avro_consumers/instances/my_consumer/records\?timeout\=1000
[{"key":"dGVzdGtleQ==","offset":0,"partition":0,"timestamp":1738667083672,"topic":"test-topic","value":null}]

Note that I also tested with a record with a String-key and a non-empty Avro-value that was produced in Java.