Struct with Key and Value Converts into a Map in Search Request

[LuisaWetzel]

Describe the bug
When you have an array of a struct with two fields named "key" and "value," it gets automatically converted into a map<string, string> when queried via the search API. This conversion does not occur when using the document API, and it can lead to data loss in the search response if there are duplicate keys.

To Reproduce
Steps to reproduce the behavior:

1. Used sample application:

schema doc {
    document doc {
        struct label {
            field key type string {
            }
            field value type string {
            }
        }
        field data type array<label> {
            indexing: summary
            struct-field key    { indexing: attribute }
            struct-field value  { indexing: attribute }
        }
    }
    fieldset default {
        fields: data
    }
}

2. Feed the following Document:

{
	"fields": 
		{
		"data":
			[
				{
					"key": "Same Key",
					"value": "Some value"
				},
				{
					"key": "Same Key",
					"value": "Some other value"
				}
			]
	}
}

3. Retrieve the document using the document API and verify it returns the correct array<label> format:

{
	"pathId": "/document/v1/default/doc/group/testing",
	"documents": [
		{
			"id": "id:default:doc:g=testing:testing::1",
			"fields": {
				"data": [
					{
						"value": "Some value",
						"key": "Same Key"
					},
					{
						"value": "Some other value",
						"key": "Same Key"
					}
				]
			}
		}
	],
	"documentCount": 1
}

4. Query the document via the search API:

{
	"yql": "select * from sources doc where true",
	"tracelevel": 1
}

5. Inspect the search response, where the array<label> has been converted into a map<string, string>, resulting in the loss of the value "Some value" for the "Same Key" key.

{
	"trace": {
		"children": [
			{
				"message": "No query profile is used"
			},
			{
				"children": [
					{
						"children": [
							{
								"message": "text.num0 search to dispatch: query=[TRUE] timeout=482ms offset=0 hits=10 groupingSessionCache=true sessionId=6272e70a-6b42-411d-8a71-7e4d3fcf8ce5.1739517406795.7.default restrict=[doc]"
							},
							{
								"message": "text.num0 dispatch response: Result (1 of total 1 hits)"
							},
							{
								"message": "text.num0 fill to dispatch: query=[TRUE] timeout=482ms offset=0 hits=10 groupingSessionCache=true sessionId=6272e70a-6b42-411d-8a71-7e4d3fcf8ce5.1739517406795.7.default restrict=[doc] summary=[null]"
							}
						]
					}
				]
			}
		]
	},
	"root": {
		"id": "toplevel",
		"relevance": 1.0,
		"fields": {
			"totalCount": 1
		},
		"coverage": {
			"coverage": 100,
			"documents": 1,
			"full": true,
			"nodes": 1,
			"results": 1,
			"resultsFull": 1
		},
		"children": [
			{
				"id": "id:default:doc:g=testing:testing::1",
				"relevance": 0.0,
				"source": "text",
				"fields": {
					"sddocname": "doc",
					"documentid": "id:default:doc:g=testing:testing::1",
					"data": {
						"Same Key": "Some other value"
					}
				}
			}
		]
	}
}

Expected behavior
I would expect the search response to behave similarly to the document API response and return the array<label> struct, rather than converting it to a map<string, string>. If I wanted this behavior, I would have specified the data type accordingly.

Environment (please complete the following information):
Dockerized vespa

Vespa version
Vespa version: 8.475.11

Additional context
This issue does not occur if the struct fields have different names than "key" and "value".

[kkraune]

Yes, this can be confusing behavior, see the note at https://docs.vespa.ai/en/reference/schema-reference.html#array :

Important: key and value are reserved words in an array, as these are used to implement map. Do not use these as struct-field names.

I am not sure if it is easy to change this behavior, a workaround is not using "key".

[LuisaWetzel]

Ah, thank you! I overlooked the note.