WP_Query: sort more arrays to improve cache hits.

[peterwilsoncc]

Lighter touch alternative to #5347

What

Improves the cache hits for WP_Query

How

It sorts query variable whenever possible. This results in the SQL queries being normalised so that it doesn't produce different SQL for the same effective arguments. For example category__in => [1,2] and category__in => [2,1] produce the same IN() clause.

Why

Currently the SQL queries and the generated cache key differ for the same effective arguments.

Trac ticket: https://core.trac.wordpress.org/ticket/59516

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• The Plugin and Theme Directories cannot be accessed within Playground.
• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[kt-12]

@peterwilsoncc I was concerned that sorting might effect the order of the output too, looks like it doesn't. Can we also ensure, https://core.trac.wordpress.org/ticket/59492#comment:3 is handled (if not), please. If there is only one element in the array it should be converted into string. I remember getting that scenario from a live site, but I can't recall the details.

[peterwilsoncc]

I was concerned that sorting might effect the order of the output too, looks like it doesn't.

The order should only have an effect for items that can be used for the orderby clause, for example post__in. It's fine to sort for the where clause so the key differs only in the instances where the item is used for ordering.

Can we also ensure, https://core.trac.wordpress.org/ticket/59492#comment:3 is handled (if not), please. If there is only one element in the array it should be converted into string. I remember getting that scenario from a live site, but I can't recall the details.

My recollection is that in some instances the use of a string vs an array can result in a different where clause beyond just the use of = vs IN. At the very least, I know it is complicated so I'm trying to get something lighter touch in initially so it's possible to spend additional time comparing the effect of passing a string or an array.

[peterwilsoncc]

These two are failing. cat is normalised in the generated SQL query but not in the arguments passed to ::generate_cache_key().

I can't figure out why as the query vars are passed to ::parse_term_query() by reference so sorting it there should work. @joemcgill Can you see what I am missing?

[peterwilsoncc]

These two are failing. cat is normalised in the generated SQL query but not in the arguments passed to ::generate_cache_key().

I can't figure out why as the query vars are passed to ::parse_term_query() by reference so sorting it there should work. @joemcgill Can you see what I am missing?

nvm, the tests were testing the wrong thing so I've started testing the cache key generated by WP_Query in bc24627

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props peterwilsoncc, spacedmonkey, joemcgill, thekt12.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[spacedmonkey]

LGTM. Reviewed all parameters and this PR look solid to me.

[peterwilsoncc]

@spacedmonkey I've pushed a few additional tests and bugs they highlighted since your last review so I'll dismiss it as stale. Because WP_Query is such a fundamental and low level feature, I don't want to yolo even the smallest of changes.

State: code changes pushed since review.

[joemcgill]

Thanks for the additional test coverage here. I left one question but am pre-approving as everything looks good to me.

[spacedmonkey]

We can't sort post__in, post_name__in or post_parent__in. These can be orderby these values. This would mean the sort of the orderby would be wrong.

[peterwilsoncc]

We can't sort post__in, post_name__in or post_parent__in. These can be orderby these values. This would mean the sort of the orderby would be wrong.

These are only sorted for the WHERE clause and cache key generation, they are untouched for the orderby clause. If used for ordering the SQL query will differ, generating a different hash for the cache key.

See

wordpress-develop/src/wp-includes/class-wp-query.php

Lines 2232 to 2237 in 62db25a

	// Duplicate array before sorting to allow for the orderby clause.
	$post__in_for_where = $q['post__in'];
	$post__in_for_where = array_unique( array_map( 'absint', $post__in_for_where ) );
	sort( $post__in_for_where );
	$post__in = implode( ',', array_map( 'absint', $post__in_for_where ) );
	$where .= " AND {$wpdb->posts}.ID IN ($post__in)";

wordpress-develop/src/wp-includes/class-wp-query.php

Lines 2215 to 2220 in 62db25a

	$q['post_name__in'] = array_map( 'sanitize_title_for_query', $q['post_name__in'] );
	// Duplicate array before sorting to allow for the orderby clause.
	$post_name__in_for_where = array_unique( $q['post_name__in'] );
	sort( $post_name__in_for_where );
	$post_name__in = "'" . implode( "','", $post_name__in_for_where ) . "'";
	$where .= " AND {$wpdb->posts}.post_name IN ($post_name__in)";

wordpress-develop/src/wp-includes/class-wp-query.php

Lines 2247 to 2252 in 62db25a

	// Duplicate array before sorting to allow for the orderby clause.
	$post_parent__in_for_where = $q['post_parent__in'];
	$post_parent__in_for_where = array_unique( array_map( 'absint', $post_parent__in_for_where ) );
	sort( $post_parent__in_for_where );
	$post_parent__in = implode( ',', array_map( 'absint', $post_parent__in_for_where ) );
	$where .= " AND {$wpdb->posts}.post_parent IN ($post_parent__in)";

The test Test_Query_CacheResults::test_orderby_clauses_are_not_normalized() has been introduced in this pull request to ensure the cache keys differ.

[peterwilsoncc]

@spacedmonkey I've pushed a change to the orderby tests to include tests that the order of posts returned is as expected. As the post IDs change for each test run, I needed to split the tests in to three rather than use the data provider.

[spacedmonkey]

Thanks for the clarification @peterwilsoncc

This pr looks great

[github-actions]

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 59766
GitHub commit: c22e267

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.