[Feature] TensorDictMap

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] MCTS Scoring functions #2358
• [Feature] TensorSpec.enumerate() #2354
• [Feature] MCTSForest #2307
• -> [Feature] TensorDictMap #2306

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2306

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 4 Unrelated Failures

As of commit f4d223c with merge base 4736fac ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  [W1015 13:29:53.567820338 CudaIPCTypes.cpp:16] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 61cb36a40c9f0e14ebe8890daf36ad188e15b4d060a8b684d4f4906766451743 /exec failed with exit code 134
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  test/test_rb.py::TestEnsemble::test_rb[SamplerWithoutReplacement-48-None-None-Tensor-ListStorage]
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  ##[error]fatal: couldn't find remote ref refs/pull/2306/merge

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_4 (gh) (similar failure)
  ##[error]Unable to find any artifacts for the associated workflow

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cpu (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_1 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 143. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_simple	0.4147s	0.4136s	2.4178 Ops/s	2.4068 Ops/s	$\color{#35bf28}+0.46\%$
test_transformed	0.6781s	0.6047s	1.6538 Ops/s	1.7024 Ops/s	$\color{#d91a1a}-2.85\%$
test_serial	1.4189s	1.3502s	0.7407 Ops/s	0.7524 Ops/s	$\color{#d91a1a}-1.56\%$
test_parallel	1.2136s	1.1955s	0.8365 Ops/s	0.8148 Ops/s	$\color{#35bf28}+2.66\%$
test_step_mdp_speed[True-True-True-True-True]	0.3542ms	28.8509μs	34.6609 KOps/s	34.5949 KOps/s	$\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-True-True-True-False]	50.2340μs	17.8194μs	56.1186 KOps/s	56.8803 KOps/s	$\color{#d91a1a}-1.34\%$
test_step_mdp_speed[True-True-True-False-True]	55.9750μs	16.2683μs	61.4694 KOps/s	61.9319 KOps/s	$\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-True-False-False]	44.9040μs	9.8774μs	101.2415 KOps/s	104.2812 KOps/s	$\color{#d91a1a}-2.91\%$
test_step_mdp_speed[True-True-False-True-True]	83.8960μs	31.1524μs	32.1003 KOps/s	31.6349 KOps/s	$\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-True-False-True-False]	51.9170μs	19.8509μs	50.3755 KOps/s	51.2701 KOps/s	$\color{#d91a1a}-1.74\%$
test_step_mdp_speed[True-True-False-False-True]	44.8840μs	18.3786μs	54.4111 KOps/s	55.0110 KOps/s	$\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-True-False-False-False]	46.6980μs	12.1375μs	82.3895 KOps/s	86.2348 KOps/s	$\color{#d91a1a}-4.46\%$
test_step_mdp_speed[True-False-True-True-True]	77.6740μs	33.3702μs	29.9669 KOps/s	30.1845 KOps/s	$\color{#d91a1a}-0.72\%$
test_step_mdp_speed[True-False-True-True-False]	54.5920μs	21.8080μs	45.8547 KOps/s	46.1966 KOps/s	$\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-True-False-True]	83.6160μs	18.4847μs	54.0989 KOps/s	54.8467 KOps/s	$\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-False-True-False-False]	38.6620μs	11.9123μs	83.9471 KOps/s	85.5458 KOps/s	$\color{#d91a1a}-1.87\%$
test_step_mdp_speed[True-False-False-True-True]	74.9290μs	34.8829μs	28.6673 KOps/s	28.4729 KOps/s	$\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-False-False-True-False]	65.1810μs	23.8323μs	41.9599 KOps/s	41.9622 KOps/s	$-0.01\%$
test_step_mdp_speed[True-False-False-False-True]	60.9530μs	20.3475μs	49.1461 KOps/s	49.3999 KOps/s	$\color{#d91a1a}-0.51\%$
test_step_mdp_speed[True-False-False-False-False]	46.8770μs	14.0418μs	71.2162 KOps/s	73.1139 KOps/s	$\color{#d91a1a}-2.60\%$
test_step_mdp_speed[False-True-True-True-True]	74.6890μs	32.9442μs	30.3544 KOps/s	29.9045 KOps/s	$\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-True-True-True-False]	56.1040μs	21.8416μs	45.7843 KOps/s	45.9548 KOps/s	$\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-True-True-False-True]	53.9300μs	21.3749μs	46.7838 KOps/s	46.7179 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[False-True-True-False-False]	2.2732ms	13.5578μs	73.7581 KOps/s	75.0554 KOps/s	$\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-True-False-True-True]	82.6040μs	35.4321μs	28.2230 KOps/s	28.0106 KOps/s	$\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-True-False-True-False]	60.0230μs	24.0293μs	41.6158 KOps/s	42.3901 KOps/s	$\color{#d91a1a}-1.83\%$
test_step_mdp_speed[False-True-False-False-True]	61.0730μs	23.7355μs	42.1310 KOps/s	42.8834 KOps/s	$\color{#d91a1a}-1.75\%$
test_step_mdp_speed[False-True-False-False-False]	55.0230μs	15.5770μs	64.1973 KOps/s	64.8034 KOps/s	$\color{#d91a1a}-0.94\%$
test_step_mdp_speed[False-False-True-True-True]	85.8100μs	37.6582μs	26.5546 KOps/s	26.6559 KOps/s	$\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-False-True-True-False]	57.2060μs	26.2118μs	38.1508 KOps/s	38.7904 KOps/s	$\color{#d91a1a}-1.65\%$
test_step_mdp_speed[False-False-True-False-True]	63.8390μs	23.7139μs	42.1694 KOps/s	43.1335 KOps/s	$\color{#d91a1a}-2.24\%$
test_step_mdp_speed[False-False-True-False-False]	45.5850μs	15.6811μs	63.7709 KOps/s	64.8464 KOps/s	$\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-False-False-True-True]	77.5740μs	39.8635μs	25.0856 KOps/s	25.6016 KOps/s	$\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-False-False-True-False]	66.8240μs	28.0950μs	35.5935 KOps/s	36.5417 KOps/s	$\color{#d91a1a}-2.59\%$
test_step_mdp_speed[False-False-False-False-True]	66.4040μs	25.1367μs	39.7824 KOps/s	39.5954 KOps/s	$\color{#35bf28}+0.47\%$
test_step_mdp_speed[False-False-False-False-False]	48.8310μs	17.6984μs	56.5022 KOps/s	57.5977 KOps/s	$\color{#d91a1a}-1.90\%$
test_values[generalized_advantage_estimate-True-True]	9.6439ms	9.3077ms	107.4383 Ops/s	104.3386 Ops/s	$\color{#35bf28}+2.97\%$
test_values[vec_generalized_advantage_estimate-True-True]	40.6807ms	35.8548ms	27.8903 Ops/s	27.8491 Ops/s	$\color{#35bf28}+0.15\%$
test_values[td0_return_estimate-False-False]	0.2367ms	0.1661ms	6.0198 KOps/s	5.8742 KOps/s	$\color{#35bf28}+2.48\%$
test_values[td1_return_estimate-False-False]	24.1458ms	23.0970ms	43.2957 Ops/s	41.7809 Ops/s	$\color{#35bf28}+3.63\%$
test_values[vec_td1_return_estimate-False-False]	41.0209ms	35.8693ms	27.8790 Ops/s	27.8664 Ops/s	$\color{#35bf28}+0.05\%$
test_values[td_lambda_return_estimate-True-False]	38.8005ms	33.4810ms	29.8677 Ops/s	28.7105 Ops/s	$\color{#35bf28}+4.03\%$
test_values[vec_td_lambda_return_estimate-True-False]	40.7573ms	35.9028ms	27.8530 Ops/s	27.8347 Ops/s	$\color{#35bf28}+0.07\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	12.5335ms	8.2844ms	120.7092 Ops/s	118.3576 Ops/s	$\color{#35bf28}+1.99\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.4513ms	1.9801ms	505.0232 Ops/s	494.9671 Ops/s	$\color{#35bf28}+2.03\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4981ms	0.3559ms	2.8100 KOps/s	2.7611 KOps/s	$\color{#35bf28}+1.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	50.9813ms	49.2045ms	20.3234 Ops/s	20.3470 Ops/s	$\color{#d91a1a}-0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.9891ms	3.0410ms	328.8342 Ops/s	328.6753 Ops/s	$\color{#35bf28}+0.05\%$
test_dqn_speed[False-None]	1.6013ms	1.3364ms	748.2874 Ops/s	743.9998 Ops/s	$\color{#35bf28}+0.58\%$
test_dqn_speed[False-backward]	1.8743ms	1.8209ms	549.1711 Ops/s	552.9859 Ops/s	$\color{#d91a1a}-0.69\%$
test_dqn_speed[True-None]	0.6500ms	0.4678ms	2.1376 KOps/s	2.1218 KOps/s	$\color{#35bf28}+0.74\%$
test_dqn_speed[True-backward]	0.9161ms	0.8756ms	1.1421 KOps/s	1.0867 KOps/s	$\textbf{\color{#35bf28}+5.10\%}$
test_dqn_speed[reduce-overhead-None]	0.6717ms	0.4663ms	2.1446 KOps/s	2.1453 KOps/s	$\color{#d91a1a}-0.03\%$
test_dqn_speed[reduce-overhead-backward]	0.9663ms	0.8802ms	1.1361 KOps/s	1.1332 KOps/s	$\color{#35bf28}+0.25\%$
test_ddpg_speed[False-None]	3.6221ms	2.8026ms	356.8138 Ops/s	353.0611 Ops/s	$\color{#35bf28}+1.06\%$
test_ddpg_speed[False-backward]	5.2806ms	3.9964ms	250.2235 Ops/s	254.6995 Ops/s	$\color{#d91a1a}-1.76\%$
test_ddpg_speed[True-None]	1.2055ms	1.0033ms	996.7182 Ops/s	998.4410 Ops/s	$\color{#d91a1a}-0.17\%$
test_ddpg_speed[True-backward]	1.9260ms	1.8855ms	530.3672 Ops/s	526.8959 Ops/s	$\color{#35bf28}+0.66\%$
test_ddpg_speed[reduce-overhead-None]	1.2218ms	1.0081ms	991.9617 Ops/s	989.9909 Ops/s	$\color{#35bf28}+0.20\%$
test_ddpg_speed[reduce-overhead-backward]	2.0743ms	1.9073ms	524.3118 Ops/s	526.0986 Ops/s	$\color{#d91a1a}-0.34\%$
test_sac_speed[False-None]	9.1100ms	7.9192ms	126.2756 Ops/s	125.9552 Ops/s	$\color{#35bf28}+0.25\%$
test_sac_speed[False-backward]	12.9526ms	10.6974ms	93.4809 Ops/s	94.0134 Ops/s	$\color{#d91a1a}-0.57\%$
test_sac_speed[True-None]	2.4894ms	1.8552ms	539.0341 Ops/s	540.9485 Ops/s	$\color{#d91a1a}-0.35\%$
test_sac_speed[True-backward]	3.5756ms	3.5210ms	284.0141 Ops/s	263.3693 Ops/s	$\textbf{\color{#35bf28}+7.84\%}$
test_sac_speed[reduce-overhead-None]	2.1395ms	1.8605ms	537.4991 Ops/s	537.3334 Ops/s	$\color{#35bf28}+0.03\%$
test_sac_speed[reduce-overhead-backward]	3.6104ms	3.5399ms	282.4937 Ops/s	283.0375 Ops/s	$\color{#d91a1a}-0.19\%$
test_redq_speed[False-None]	18.6037ms	13.0105ms	76.8612 Ops/s	74.4331 Ops/s	$\color{#35bf28}+3.26\%$
test_redq_speed[False-backward]	24.7023ms	22.0591ms	45.3327 Ops/s	45.0156 Ops/s	$\color{#35bf28}+0.70\%$
test_redq_speed[True-None]	5.2536ms	4.5401ms	220.2612 Ops/s	219.6929 Ops/s	$\color{#35bf28}+0.26\%$
test_redq_speed[True-backward]	12.9574ms	12.0061ms	83.2907 Ops/s	79.9692 Ops/s	$\color{#35bf28}+4.15\%$
test_redq_speed[reduce-overhead-None]	5.2920ms	4.5839ms	218.1540 Ops/s	147.0346 Ops/s	$\textbf{\color{#35bf28}+48.37\%}$
test_redq_speed[reduce-overhead-backward]	13.5059ms	12.0092ms	83.2694 Ops/s	66.0188 Ops/s	$\textbf{\color{#35bf28}+26.13\%}$
test_redq_deprec_speed[False-None]	15.9081ms	12.6669ms	78.9461 Ops/s	61.1424 Ops/s	$\textbf{\color{#35bf28}+29.12\%}$
test_redq_deprec_speed[False-backward]	20.1166ms	18.3998ms	54.3483 Ops/s	43.3038 Ops/s	$\textbf{\color{#35bf28}+25.50\%}$
test_redq_deprec_speed[True-None]	3.9892ms	3.5842ms	279.0014 Ops/s	232.8823 Ops/s	$\textbf{\color{#35bf28}+19.80\%}$
test_redq_deprec_speed[True-backward]	8.1003ms	7.9705ms	125.4632 Ops/s	124.7352 Ops/s	$\color{#35bf28}+0.58\%$
test_redq_deprec_speed[reduce-overhead-None]	3.7011ms	3.5758ms	279.6593 Ops/s	278.6130 Ops/s	$\color{#35bf28}+0.38\%$
test_redq_deprec_speed[reduce-overhead-backward]	8.0619ms	7.9769ms	125.3613 Ops/s	125.1136 Ops/s	$\color{#35bf28}+0.20\%$
test_td3_speed[False-None]	23.3066ms	8.1139ms	123.2447 Ops/s	126.9281 Ops/s	$\color{#d91a1a}-2.90\%$
test_td3_speed[False-backward]	10.8061ms	10.2687ms	97.3834 Ops/s	97.3096 Ops/s	$\color{#35bf28}+0.08\%$
test_td3_speed[True-None]	2.1245ms	1.7939ms	557.4325 Ops/s	561.9825 Ops/s	$\color{#d91a1a}-0.81\%$
test_td3_speed[True-backward]	3.4547ms	3.3905ms	294.9435 Ops/s	295.2689 Ops/s	$\color{#d91a1a}-0.11\%$
test_td3_speed[reduce-overhead-None]	2.0251ms	1.7759ms	563.0934 Ops/s	561.7661 Ops/s	$\color{#35bf28}+0.24\%$
test_td3_speed[reduce-overhead-backward]	3.9615ms	3.4342ms	291.1854 Ops/s	295.2446 Ops/s	$\color{#d91a1a}-1.37\%$
test_cql_speed[False-None]	40.3850ms	35.9639ms	27.8057 Ops/s	27.7216 Ops/s	$\color{#35bf28}+0.30\%$
test_cql_speed[False-backward]	53.0353ms	47.3338ms	21.1266 Ops/s	21.4051 Ops/s	$\color{#d91a1a}-1.30\%$
test_cql_speed[True-None]	16.5944ms	15.6148ms	64.0418 Ops/s	64.5679 Ops/s	$\color{#d91a1a}-0.81\%$
test_cql_speed[True-backward]	23.3068ms	21.9602ms	45.5370 Ops/s	45.1577 Ops/s	$\color{#35bf28}+0.84\%$
test_cql_speed[reduce-overhead-None]	19.1213ms	15.7140ms	63.6376 Ops/s	64.1377 Ops/s	$\color{#d91a1a}-0.78\%$
test_cql_speed[reduce-overhead-backward]	23.7850ms	21.9072ms	45.6471 Ops/s	44.5558 Ops/s	$\color{#35bf28}+2.45\%$
test_a2c_speed[False-None]	8.8939ms	7.1340ms	140.1741 Ops/s	138.4637 Ops/s	$\color{#35bf28}+1.24\%$
test_a2c_speed[False-backward]	14.5218ms	14.1204ms	70.8193 Ops/s	69.5256 Ops/s	$\color{#35bf28}+1.86\%$
test_a2c_speed[True-None]	3.7783ms	3.3642ms	297.2457 Ops/s	297.0227 Ops/s	$\color{#35bf28}+0.08\%$
test_a2c_speed[True-backward]	10.4970ms	9.8165ms	101.8691 Ops/s	103.2897 Ops/s	$\color{#d91a1a}-1.38\%$
test_a2c_speed[reduce-overhead-None]	3.7561ms	3.3579ms	297.8039 Ops/s	296.5529 Ops/s	$\color{#35bf28}+0.42\%$
test_a2c_speed[reduce-overhead-backward]	10.1498ms	9.8071ms	101.9671 Ops/s	102.2528 Ops/s	$\color{#d91a1a}-0.28\%$
test_ppo_speed[False-None]	8.9741ms	7.3808ms	135.4864 Ops/s	134.4296 Ops/s	$\color{#35bf28}+0.79\%$
test_ppo_speed[False-backward]	16.2423ms	14.5128ms	68.9049 Ops/s	67.7838 Ops/s	$\color{#35bf28}+1.65\%$
test_ppo_speed[True-None]	4.2583ms	3.7206ms	268.7727 Ops/s	269.1811 Ops/s	$\color{#d91a1a}-0.15\%$
test_ppo_speed[True-backward]	10.5923ms	9.5937ms	104.2348 Ops/s	103.6060 Ops/s	$\color{#35bf28}+0.61\%$
test_ppo_speed[reduce-overhead-None]	4.2375ms	3.7284ms	268.2120 Ops/s	267.3994 Ops/s	$\color{#35bf28}+0.30\%$
test_ppo_speed[reduce-overhead-backward]	9.9072ms	9.4931ms	105.3398 Ops/s	101.3284 Ops/s	$\color{#35bf28}+3.96\%$
test_reinforce_speed[False-None]	7.7961ms	6.4377ms	155.3352 Ops/s	152.6747 Ops/s	$\color{#35bf28}+1.74\%$
test_reinforce_speed[False-backward]	11.1540ms	9.6714ms	103.3972 Ops/s	101.5271 Ops/s	$\color{#35bf28}+1.84\%$
test_reinforce_speed[True-None]	3.4080ms	2.6568ms	376.3967 Ops/s	363.4979 Ops/s	$\color{#35bf28}+3.55\%$
test_reinforce_speed[True-backward]	8.9117ms	8.5166ms	117.4184 Ops/s	116.7969 Ops/s	$\color{#35bf28}+0.53\%$
test_reinforce_speed[reduce-overhead-None]	3.3504ms	2.6418ms	378.5252 Ops/s	373.3707 Ops/s	$\color{#35bf28}+1.38\%$
test_reinforce_speed[reduce-overhead-backward]	9.5538ms	8.6196ms	116.0141 Ops/s	114.4721 Ops/s	$\color{#35bf28}+1.35\%$
test_iql_speed[False-None]	34.5149ms	31.6944ms	31.5513 Ops/s	31.3188 Ops/s	$\color{#35bf28}+0.74\%$
test_iql_speed[False-backward]	47.3615ms	44.7497ms	22.3465 Ops/s	22.3065 Ops/s	$\color{#35bf28}+0.18\%$
test_iql_speed[True-None]	11.7117ms	10.4961ms	95.2734 Ops/s	94.0284 Ops/s	$\color{#35bf28}+1.32\%$
test_iql_speed[True-backward]	23.2416ms	21.7690ms	45.9369 Ops/s	46.2218 Ops/s	$\color{#d91a1a}-0.62\%$
test_iql_speed[reduce-overhead-None]	11.8975ms	10.6585ms	93.8220 Ops/s	94.5590 Ops/s	$\color{#d91a1a}-0.78\%$
test_iql_speed[reduce-overhead-backward]	27.1846ms	21.9388ms	45.5813 Ops/s	46.2273 Ops/s	$\color{#d91a1a}-1.40\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.3608ms	4.8312ms	206.9864 Ops/s	207.8809 Ops/s	$\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.6993ms	0.4830ms	2.0703 KOps/s	2.0720 KOps/s	$\color{#d91a1a}-0.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6975ms	0.4599ms	2.1743 KOps/s	2.1818 KOps/s	$\color{#d91a1a}-0.34\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.6152ms	4.7528ms	210.4027 Ops/s	212.4602 Ops/s	$\color{#d91a1a}-0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.1827ms	0.4895ms	2.0427 KOps/s	2.0907 KOps/s	$\color{#d91a1a}-2.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	1.2687ms	0.4789ms	2.0881 KOps/s	2.2196 KOps/s	$\textbf{\color{#d91a1a}-5.92\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.2951ms	1.5928ms	627.8291 Ops/s	621.0824 Ops/s	$\color{#35bf28}+1.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	8.8763ms	1.5578ms	641.9254 Ops/s	642.9512 Ops/s	$\color{#d91a1a}-0.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.8481ms	5.1679ms	193.5020 Ops/s	203.1666 Ops/s	$\color{#d91a1a}-4.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.6645ms	0.6600ms	1.5151 KOps/s	1.6017 KOps/s	$\textbf{\color{#d91a1a}-5.41\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9842ms	0.6294ms	1.5887 KOps/s	1.6765 KOps/s	$\textbf{\color{#d91a1a}-5.24\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.4419ms	4.9552ms	201.8080 Ops/s	210.8827 Ops/s	$\color{#d91a1a}-4.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.7376ms	0.4883ms	2.0480 KOps/s	2.0685 KOps/s	$\color{#d91a1a}-0.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7752ms	0.4685ms	2.1346 KOps/s	2.1465 KOps/s	$\color{#d91a1a}-0.56\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.0395ms	4.9265ms	202.9836 Ops/s	211.1429 Ops/s	$\color{#d91a1a}-3.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.5544ms	0.4857ms	2.0588 KOps/s	2.0711 KOps/s	$\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6966ms	0.4545ms	2.2003 KOps/s	2.2291 KOps/s	$\color{#d91a1a}-1.29\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.6846ms	5.0288ms	198.8536 Ops/s	205.4556 Ops/s	$\color{#d91a1a}-3.21\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8404ms	0.6198ms	1.6134 KOps/s	1.6174 KOps/s	$\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	7.7611ms	0.6052ms	1.6524 KOps/s	1.6669 KOps/s	$\color{#d91a1a}-0.87\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.4521s	13.2209ms	75.6376 Ops/s	36.7421 Ops/s	$\textbf{\color{#35bf28}+105.86\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	7.5336ms	2.2853ms	437.5831 Ops/s	433.0512 Ops/s	$\color{#35bf28}+1.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	5.2319ms	1.2796ms	781.5180 Ops/s	726.4915 Ops/s	$\textbf{\color{#35bf28}+7.57\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	5.8659ms	4.2747ms	233.9337 Ops/s	223.9468 Ops/s	$\color{#35bf28}+4.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	7.7519ms	2.3189ms	431.2386 Ops/s	446.2438 Ops/s	$\color{#d91a1a}-3.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.2413ms	1.2816ms	780.2512 Ops/s	759.8183 Ops/s	$\color{#35bf28}+2.69\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.4089s	12.6339ms	79.1521 Ops/s	224.5306 Ops/s	$\textbf{\color{#d91a1a}-64.75\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	9.1191ms	2.6631ms	375.4999 Ops/s	414.7350 Ops/s	$\textbf{\color{#d91a1a}-9.46\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.8537ms	1.3537ms	738.6915 Ops/s	670.7844 Ops/s	$\textbf{\color{#35bf28}+10.12\%}$