question about DeepCoord updates scheduling tables

[burnCalories]

After reading the author's paper, I have some confusion about how the schedule works.
How to calculate the scheduling probability when the flow arrives? Is it a random probability or is it based on the load degree of the node?
Which part of the code corresponds to it? I'm interested in how to make a schedule. I hope I can learn more ：）

[stefanbschneider]

Hi @burnCalories , happy to hear that you read our paper and want to learn more. I'll try to answer your questions.

How to calculate the scheduling probability when the flow arrives? Is it a random probability or is it based on the load degree of the node?

The scheduling probabilities are the actions of the DeepCoord reinforcement learning agent. This means they are controlled by the RL agent. Initially, during training, it starts with random probabilities, and then gradually learns to pick suitable probabilities.

This also means, there are no hard-coded rules how the probabilities are set since they are controlled by the agent. In the end, it should learn to set the probabilities, among others, based on the load degree, but also any other relevant factors (eg, delay).

Which part of the code corresponds to it?

Here are some pointers:

• As a I said, the specific scheduling probabilities are the actions, ie, output, of the RL agent. The size of the action space is defined here: https://github.com/RealVNF/DeepCoord/blob/master/src/rlsp/envs/environment_limits.py#L40
• The RL agent itself is set up and trained here: https://github.com/RealVNF/DeepCoord/blob/master/src/rlsp/agents/rlsp_ddpg.py
• As described in Sec IV.A in the paper, we postprocess the raw probabilities returned by the agent and also derive placement decisions from it. This happens here: https://github.com/RealVNF/DeepCoord/blob/master/src/rlsp/envs/simulator_wrapper.py#L112
• The placement decisions and scheduling tables are then given to the environment/the network, where the scheduling probabilities are applied at each node to decide how to schedule incoming flows. We use coord-sim as environment; the corresponding code for flow scheduling is here: https://github.com/RealVNF/coord-sim/blob/master/src/coordsim/decision_maker/default_decision_maker.py

Let me know if this answers your questions.

[burnCalories]

This method is great. Thank you for your reply. I haven't understood the content and code of this part before

[stefanbschneider]

Happy to hear :) I'll mark this question as answered for now. Feel free to reopen or add a new question if something comes up.