an error happened when using the enhanced sampling in dpgen #494

343333333 · 2021-08-10T12:55:52Z

343333333
Aug 10, 2021

dpgen version:0.10.0
deepmd version :1.3.3
i use the example in https://github.com/deepmodeling/dpgen/tree/master/examples/run/dp-lammps-enhance_sampling, after running the MTD in lammps the error happened saying that
RuntimeError: do not find download file /home/dpgen/test3/temp/5f54663f-8f0e-47b0-9d1c-4c66c8cf4c76/task.000.000000/dump.0.xyz
and i check the input.lammps in modeldevi document in temp , it seems dont output the dump.0.xyz.

here is the log file and the input file

log file:
`NOTE: Some configuration directories are not set up yet: 
	/home/chem-wangyg/lvxinm/.gromacswrapper
	/home/chem-wangyg/lvxinm/.gromacswrapper/qscripts
	/home/chem-wangyg/lvxinm/.gromacswrapper/templates
NOTE: You can create the configuration file and directories with:
	>>> import gromacs
	>>> gromacs.config.setup()
DeepModeling
Version: 0.10.0
Date:    Aug-09-2021
Path:    /home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen

Dependency
numpy     1.21.1   /home/chem-wangyg/lvxinm/deepmd-kit/lib/python3.9/site-packages/numpy
 dpdata      0.2.1   /home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpdata
  pymatgen            unknown version or path
     monty   2021.7.8   /home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/monty
       ase     3.22.0   /home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/ase
  paramiko      2.7.2   /home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/paramiko
 custodian   2021.2.8   /home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/custodian

Reference

Please cite:
Yuzhi Zhang, Haidi Wang, Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang, and Weinan E,
DP-GEN: A concurrent learning platform for the generation of reliable deep learning
based potential energy models, Computer Physics Communications, 2020, 107206.


Description

INFO:dpgen:-------------------------iter.000000 task 01--------------------------
/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/generator/run.py:565: UserWarning: the dpdispatcher will be updated to new version.And the interface may be changed. Please check the documents for more details
  warnings.warn(f"the dpdispatcher will be updated to new version."
INFO:dpgen:new submission of fefb8f48-a430-499f-bfb6-21105464c138 for chunk 8aefb06c426e07a0a671a1e2488b4858d694a730
INFO:dpgen:new submission of f7568ce6-ebb6-4637-a3f8-3a48fce42982 for chunk e193a01ecf8d30ad0affefd332ce934e32ffce72
INFO:dpgen:new submission of 69d4ebfc-1ad8-4815-a539-c1ecd6f3a301 for chunk 6fc978af728d43c59faa400d5f6e0471ac850d4c
INFO:dpgen:new submission of d14aa1c6-230e-4d61-9244-a1d036090d73 for chunk 221407c03ae5c73109cce71d27e24637824f3333
INFO:dpgen:job fefb8f48-a430-499f-bfb6-21105464c138 finished
INFO:dpgen:job d14aa1c6-230e-4d61-9244-a1d036090d73 finished
INFO:dpgen:job f7568ce6-ebb6-4637-a3f8-3a48fce42982 finished
INFO:dpgen:job 69d4ebfc-1ad8-4815-a539-c1ecd6f3a301 finished
INFO:dpgen:-------------------------iter.000000 task 02--------------------------
INFO:dpgen:-------------------------iter.000000 task 03--------------------------
INFO:dpgen:-------------------------iter.000000 task 04--------------------------
/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/generator/run.py:1232: UserWarning: the dpdispatcher will be updated to new version.And the interface may be changed. Please check the documents for more details
  warnings.warn(f"the dpdispatcher will be updated to new version."
INFO:dpgen:new submission of 5f54663f-8f0e-47b0-9d1c-4c66c8cf4c76 for chunk 2a332aeea901b380d1755cc375d3fcb7e993ecae
INFO:dpgen:job 5f54663f-8f0e-47b0-9d1c-4c66c8cf4c76 finished
Traceback (most recent call last):
  File "/home/chem-wangyg/lvxinm/.local/bin/dpgen", line 8, in <module>
    sys.exit(main())
  File "/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/main.py", line 175, in main
    args.func(args)
  File "/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/generator/run.py", line 2714, in gen_run
    run_iter (args.PARAM, args.MACHINE)
  File "/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/generator/run.py", line 2689, in run_iter
    run_model_devi (ii, jdata, mdata)
  File "/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/generator/run.py", line 1235, in run_model_devi
    dispatcher.run_jobs(mdata['model_devi_resources'],
  File "/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/dispatcher/Dispatcher.py", line 95, in run_jobs
    while not self.all_finished(job_handler, mark_failure) :
  File "/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/dispatcher/Dispatcher.py", line 230, in all_finished
    rjob['context'].download(task_chunks[idx], backward_task_files)
  File "/home/chem-wangyg/lvxinm/.local/lib/python3.9/site-packages/dpgen/dispatcher/LocalContext.py", line 114, in download
    raise RuntimeError('do not find download file ' + rfile)
**RuntimeError: do not find download file /home/chem-wangyg/lvxinm/dpgen/test3/temp/5f54663f-8f0e-47b0-9d1c-4c66c8cf4c76/task.000.000000/dump.0.xyz`**

here is my para.json :

{
    "type_map": [
        "C",
        "H"
    ],
    "mass_map": [
        12,
        1
    ],
    "init_data_prefix": "data",
    "init_data_sys": [
        "raw_data"
    ],
    "sys_configs_prefix": "data",
    "sys_configs": [
        [
            "modeldevi/POSCAR"

        ],
        [
            "modeldevi/POSCAR"

        ]

    ],
    "_comment": " that's all ",
    "numb_models": 4,
    "default_training_param": {
          "_comment": " model parameters",
    "model": {
	"type_map":	["C", "H"],
	"descriptor" :{
	    "type":		"se_a",   
	    "sel":		[5, 5],
	    "rcut_smth":	4.00,
	    "rcut":		5.00,
	    "neuron":		[25, 50, 100],
	    "resnet_dt":	false,
	    "axis_neuron":	16,
	    "seed":		1,
	    "_comment":		" that's all"
	},
	"fitting_net" : {
	    "neuron":		[240, 240, 240],
	    "resnet_dt":	true,
	    "seed":		1,
	    "_comment":		" that's all"
	},
	"_comment":	" that's all"
    },

    "learning_rate" :{
	"type":		"exp",
	"decay_steps":	500,
	"decay_rate":	0.95,
	"start_lr":	0.001,	
	"stop_lr":	3.51e-8,
	"_comment":	"that's all"
    },

    "loss" :{
	"start_pref_e":	0.02,
	"limit_pref_e":	1,
	"start_pref_f":	1000,
	"limit_pref_f":	1,
	"start_pref_v":	0,
	"limit_pref_v":	0,
	"_comment":	" that's all"
    },

    "_comment": " traing controls",
    "training" : {
	"systems":	["data/deepmd"],
	"set_prefix":	"set",    
	"stop_batch":	1000,
	"batch_size":	"auto",

	"seed":		1,

	"_comment": " display and restart",
	"_comment": " frequencies counted in batch",
	"disp_file":	"lcurve.out",
	"disp_freq":	100,
	"numb_test":	10,
	"save_freq":	1000,
	"save_ckpt":	"model.ckpt",
	"load_ckpt":	"model.ckpt",
	"disp_training":true,
	"time_training":true,
	"profiling":	false,
	"profiling_file":"timeline.json",
	"_comment":	"that's all"
    },
`
`
    "_comment":		"that's all"
    },
    "model_devi_dt": 0.0005,
    "model_devi_skip": 0,
    "model_devi_f_trust_lo": 0.08,
    "model_devi_f_trust_hi": 0.25,
    "model_devi_e_trust_lo": 10000000000.0,
    "model_devi_e_trust_hi": 10000000000.0,
    "model_devi_clean_traj": false,
    "model_devi_plumed"  :  true,
    "model_devi_jobs":  [
        {   "sys_idx": [0],"traj_freq": 10,"_idx": "00",
            "template":{ "lmp": "lmp/input.lammps", "plm": "lmp/input.plumed" },
            "rev_mat":{ "lmp": {"V_NSTEPS": [2000], "V_TEMP": [300], "V_PRES": [1]},
                        "plm": {"V_TEMP":  [300], "V_STRIDE": [10]}
                      }
        },
        {   "sys_idx": [0],"traj_freq": 10,"_idx": "01",
            "template":{ "lmp": "lmp/input.lammps", "plm": "lmp/input.plumed" },
            "rev_mat":{ "lmp": {"V_NSTEPS": [5000], "V_TEMP": [300], "V_PRES": [1]},
                        "plm": {"V_TEMP":  [300], "V_STRIDE": [10]}
                      }
        },
        {   "sys_idx": [1],"traj_freq": 10,"_idx": "01",
            "template":{ "lmp": "lmp/input.lammps", "plm": "lmp/input.plumed" },
            "rev_mat":{ "lmp": {"V_NSTEPS": [3000], "V_TEMP": [300], "V_PRES": [1]},
                        "plm": {"V_TEMP":  [300], "V_STRIDE": [10]}
                      }
        }
  ],
    "fp_style": "vasp",
    "shuffle_poscar": false,
    "fp_task_max": 100,
    "fp_task_min": 5,
    "fp_pp_path": "vasp/",
    "fp_pp_files": [
        "POTCAR-c",
        "POTCAR-h"
    ],
    "fp_incar": "vasp/INCAR"
}

and this is the machine.json

{
"train": [
	{
		"command": "/deepmd-kit/bin/dp",
	"machine": {
		"batch": "pbs",
		"_hostname": "localhost",
		"_port" : 22,
		"username" :"ben" ,
		"work_path": "./temp"
		},
	"resources": {
		"numb_gpu": 0,
		"numb_node" :1 ,
		"task_per_node": 24,
		"partition": "short",
		"exclude_list": [],
		"mem_limit": 8,
		"source_list": [],
     	                "time_limit": "168:0:0",
		"module_list": []
		}
	}
],
"model_devi": [
    {
    "machine": {
        "batch": "pbs",
        "work_path": "./temp",
        "_comment": "that's all"
      },
    "resources": {
		"numb_gpu": 0,
		"numb_node" :1 ,
		"task_per_node": 24,
		"partition": "short",
		"exclude_list": [],
		"mem_limit": 8,
     	                "time_limit": "168:0:0",
		"source_list": [],
		"module_list": []
      },
      "command": "/deepmd-kit/bin/lmp",
      "_comment" : "DP-GEN will put 5 tasks together in one submitting script.",  
      "group_size": 20
    }
  ], 
"fp": [
	{
	"machine": {
		"batch": "pbs",
		"work_path": "./temp"
	},
	"resources": {
		"numb_gpu": 0,
		"numb_node" :1,
		"task_per_node": 24,
		"partition": "short",
     	                "time_limit": "168:0:0",
		"with_mpi": false,
		"_source_list": ["/home/ben/intel/parallel_studio_xe_2019.5.075/psxevars.sh"],
		"module_list": [],
		"_envs" : {"PATH" : "/root/vasp/bin:$PATH"}
	},
	"command": "NP=`cat $PBS_NODEFILE | wc -l` ; mpiexec.hydra  -machinefile $PBS_NODEFILE -n $NP vasp_std > stdout 2>&1",
	"group_size": 20
	}
	]
}

Answered by 343333333

Aug 12, 2021

it turns out that the input.plumed in https://github.com/deepmodeling/dpgen/tree/master/examples/run/dp-lammps-enhance_sampling, has something wrong .
i change the file into this and now work.

UNITS LENGTH=A TIME=0.0005  #Amstroeng, hartree, fs
#RESTART
MOLINFO STRUCTURE=c2h6.pdb
ene: ENERGY
phi: TORSION ATOMS=5,1,2,8

METAD ...
 LABEL=metad
 ARG=phi
 PACE=50
 HEIGHT=1.0
 SIGMA=0.1
 FILE=HILLS
 BIASFACTOR=20
 TEMP=V_TEMP
... METAD


PRINT STRIDE=V_STRIDE FILE=COLVAR ARG=phi,metad.*,ene
DUMPATOMS FILE=dump.0.xyz ATOMS=@mdatoms
FLUSH STRIDE=V_STRIDE

View full answer

343333333 · 2021-08-12T08:21:33Z

343333333
Aug 12, 2021
Author

it turns out that the input.plumed in https://github.com/deepmodeling/dpgen/tree/master/examples/run/dp-lammps-enhance_sampling, has something wrong .
i change the file into this and now work.

UNITS LENGTH=A TIME=0.0005  #Amstroeng, hartree, fs
#RESTART
MOLINFO STRUCTURE=c2h6.pdb
ene: ENERGY
phi: TORSION ATOMS=5,1,2,8

METAD ...
 LABEL=metad
 ARG=phi
 PACE=50
 HEIGHT=1.0
 SIGMA=0.1
 FILE=HILLS
 BIASFACTOR=20
 TEMP=V_TEMP
... METAD


PRINT STRIDE=V_STRIDE FILE=COLVAR ARG=phi,metad.*,ene
DUMPATOMS FILE=dump.0.xyz ATOMS=@mdatoms
FLUSH STRIDE=V_STRIDE

6 replies

343333333 Aug 12, 2021
Author

btw, could you tell me what this file is used for? If.xyz is only used to record coordinates, the.traj file can also do that. It feels a bit redundant

AnguseZhang Aug 12, 2021
Maintainer

Do you mean input.plumed ? It's a parameter file used in plumed for controlling parameters for enhanced sampling, such as how to select CV.

343333333 Aug 12, 2021
Author

i mean dump.0.xyz file . if i just touch dump.0.xyz in the temp/mission/task.000.000000, it also can work well and select the candidate configuration and do the fp calculation .

fqgong Aug 12, 2021

Sorry for your inconvenience! In dpgen, if you set the use_plm as True, it needs a dump.0.xyz to be a back forward file by default. I think it might be a bug. I will fix it latter.

343333333 Aug 13, 2021
Author

Sorry for your inconvenience! In dpgen, if you set the use_plm as True, it needs a dump.0.xyz to be a back forward file by default. I think it might be a bug. I will fix it latter.

that's alright .thx for replying.

LiangMD-BGI · 2022-04-24T12:05:18Z

LiangMD-BGI
Apr 24, 2022

I use dpgen (version 0.9.2) and meet the same problem. So I dump twice in input.lammps file: one is for the traj files, and the other one is to generate the dump.0.xyz. I am not sure this makes sense because I still do not quite understand what is the dump.0.xyz for.

In addition, I have another confusion about the temperature setting for lmp and plm. I need to perform the simulations at two T (say 300 and 400 K). So I set "rev_mat":{ "lmp": {"V_NSTEPS": [5000], "V_TEMP": [300,400], "V_PRES": [1]}, "plm": {"V_TEMP": [300,400], "V_STRIDE": [10]}. I found that dpgen submit four but not two simulations becaue half of the jobs are repeated.

2 replies

LiangMD-BGI Apr 24, 2022

I suppose "V_TEMP": [300,400] for plm can be removed.

343333333 Apr 24, 2022
Author

I forget who said that the XYZ file generated by lammps in dpgen is used for gromacs, so this file can be ignored if only LMP and PLM are used

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

an error happened when using the enhanced sampling in dpgen #494

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 8 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

an error happened when using the enhanced sampling in dpgen #494

343333333 Aug 10, 2021

here is the log file and the input file

here is my para.json :

and this is the machine.json

Replies: 2 comments · 8 replies

343333333 Aug 12, 2021 Author

343333333 Aug 12, 2021 Author

AnguseZhang Aug 12, 2021 Maintainer

343333333 Aug 12, 2021 Author

fqgong Aug 12, 2021

343333333 Aug 13, 2021 Author

LiangMD-BGI Apr 24, 2022

LiangMD-BGI Apr 24, 2022

343333333 Apr 24, 2022 Author

343333333
Aug 10, 2021

Replies: 2 comments 8 replies

343333333
Aug 12, 2021
Author

343333333 Aug 12, 2021
Author

AnguseZhang Aug 12, 2021
Maintainer

343333333 Aug 12, 2021
Author

343333333 Aug 13, 2021
Author

LiangMD-BGI
Apr 24, 2022

343333333 Apr 24, 2022
Author