DPGEN dispatcher: Node control within a queue using PBS scheduler

[srimayee-27-jnc]

Hi,
I am trying to dispatch DPGEN jobs to specific nodes using a submission script for the PBS scheduler. However, I am unable to restrict or control the nodes the jobs enter. I have consistently specified the nodes of our choice both in the submission script and in machine.json. Kindly advise me on how to fix this. Thanks and regards.

Given below is the submission script:

#!/bin/sh
#PBS -N dpgen
#PBS -q AGR
#PBS -l select=1:ncpus=48:host=nodes12-ib+1:ncpus=48:host=nodes17-ib
#PBS -l walltime=150:00:00
#PBS -l place=scatter

cd $PBS_O_WORKDIR

module load vasp-VTST-intel

module load python-3.9

#source activate /home/srimayee/.conda/envs/dpgen
source ~/deepmd_source.sh

dpgen run param.json machine.json

Given below is one example relevant portion of machine.json :

"train":[
      {  
        "command": "/home/srimayee/mysoft/deepmd-kit-2.2.5/bin/dp",
        "machine": {
            "batch_type": "PBS",
            "context_type": "local",
            "local_root": "./",
            "remote_root": "./"  
        },
        "resources": {
            "batch_type": "PBS",
            "number_node": 1,
            "cpu_per_node": 8,
            "gpu_per_node": 0,
            "queue_name": "AGR",
            "group_size": 1,
            "custom_flags": [
                "#PBS -l walltime=150:00:00",
                "#PBS -l select=1:ncpus=48:host=node12-ib+1:ncpus=48:host=node17-ib",
                "#PBS -l place=scatter",
                "cd $PBS_O_WORKDIR"
            ]
        }
      }
    ]

[njzjz]

I guess PBS doesn't allow you to override the previous flag (that is generated by dpdispatcher). I don't have a PBS environment to validate my guess. If it is true, you can use `customized_script_header_template_file` to customize the whole script header (see https://docs.deepmodeling.com/projects/dpdispatcher/en/latest/examples/template.html ).

…

On Sat, Feb 10, 2024 at 7:52 AM srimayee-27-jnc ***@***.***> wrote: Hi, I am trying to dispatch DPGEN jobs to specific nodes using a submission script for the PBS scheduler. However, I am unable to restrict or control the nodes the jobs enter. I have consistently specified the nodes of our choice both in the submission script and in machine.json. Kindly advise me on how to fix this. Thanks and regards. Given below is the submission script: #!/bin/sh #PBS -N dpgen #PBS -q AGR #PBS -l select=1:ncpus=48:host=nodes12-ib+1:ncpus=48:host=nodes17-ib #PBS -l walltime=150:00:00 #PBS -l place=scatter cd $PBS_O_WORKDIR module load vasp-VTST-intel module load python-3.9 #source activate /home/srimayee/.conda/envs/dpgen source ~/deepmd_source.sh dpgen run param.json machine.json Given below is one example relevant portion of machine.json : "train":[ { "command": "/home/srimayee/mysoft/deepmd-kit-2.2.5/bin/dp", "machine": { "batch_type": "PBS", "context_type": "local", "local_root": "./", "remote_root": "./" }, "resources": { "batch_type": "PBS", "number_node": 1, "cpu_per_node": 8, "gpu_per_node": 0, "queue_name": "AGR", "group_size": 1, "custom_flags": [ "#PBS -l walltime=150:00:00", "#PBS -l select=1:ncpus=48:host=node12-ib+1:ncpus=48:host=node17-ib", "#PBS -l place=scatter", "cd $PBS_O_WORKDIR" ] } } ] — Reply to this email directly, view it on GitHub <#1467>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACIOQ7RCP6M6WXZQJRK4AOTYS5UPDAVCNFSM6AAAAABDCWMEAWVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGIYDGMZRGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[srimayee-27-jnc]

Thank you, this worked!