cudaFuncSetCacheConfig questions

[zjin-lcf]

Quite a few cuda code show the functions (https://userweb.cs.txstate.edu/~burtscher/research/ECL-BH/)

cudaFuncSetCacheConfig(BoundingBoxKernel, cudaFuncCachePreferShared);
cudaFuncSetCacheConfig(TreeBuildingKernel, cudaFuncCachePreferL1);
cudaFuncSetCacheConfig(ClearKernel1, cudaFuncCachePreferL1);
cudaFuncSetCacheConfig(ClearKernel2, cudaFuncCachePreferL1);
cudaFuncSetCacheConfig(SummarizationKernel, cudaFuncCachePreferShared);
cudaFuncSetCacheConfig(SortKernel, cudaFuncCachePreferL1);
cudaFuncSetCacheConfig(ForceCalculationKernel, cudaFuncCachePreferEqual);
cudaFuncSetCacheConfig(IntegrationKernel, cudaFuncCachePreferL1);

I am not familiar with these functions; do they have equivalents in SYCL ? Thanks.

[steffenlarsen]

My understanding is that these functions are intended to give a hint about shared memory vs. L1 cache usage of the kernels. This is useful on modern CUDA GPUs as they have L1 cache and shared memory in shared dynamically adjusted resources.

Sadly the CUDA backend does not currently support setting such preferences. I have an old patch which adds these hints at a context level, but I never got around to making a PR for it and it might have rotted slightly now. However, what you're asking about is more granular.

One solution would be to have CUDA-backend specific properties for these to be used when online compiling the kernels in a kernel_bundle. This would however require both online compiling your kernels and setting the preferences when compiling the kernels. Consequently if you wanted to change the preference at any time during execution you would have to compile a version with the new preference, despite the preferences not having anything to do with the kernel compilation.

To my knowledge kernel_bundle doesn't currently work with the CUDA backend, but even when it is implemented I still think the drawbacks of the above solution makes it suboptimal. @AerialMantis and @gmlueck might be able to think of alternative solutions.

[gmlueck]

If these cache configuration options are related to the way the kernel is submitted (as opposed to affecting the way the kernel is compiled), then we might be able to express these as properties passed to the parallel_for or single_task functions. Tagging @Pennycook because he is contemplating a DPC++ extension that allows applications to pass properties to parallel_for and single_task.

[Pennycook]

Even if they affect the way that the kernel is compiled, the proposal still allows for those properties to be passed to parallel_for or single_task. The distinction that the extension currently makes is that properties can only affect the way that a kernel is compiled if the property is a compile-time property.

Either way, I agree that the SYCL_EXT_ONEAPI_KERNEL_PROPERTIES extension could solve this. We just need to know whether a run-time or compile-time property is desired.

[steffenlarsen]

The cache preferences mentioned here shouldn't affect how the kernel would be compiled. The kernel_bundle was just the only way I could think of to configure the kernels as-is, but the mentioned proposal fits the bill perfectly.

[zjin-lcf]

Thank you for your answers and links!