How to Create a Custom Description Dataset for HumanML3D?

[mustafizur-r]

Hi,
Great work!
I'm currently working on a project where I need to generate a custom description dataset similar to the one used in HumanML3D. I noticed that you've made changes to your own combat descriptions, and I'm curious about the best practices and steps involved in creating such a dataset. could you help me?

[fyyakaxyy]

Hi, Great work! I'm currently working on a project where I need to generate a custom description dataset similar to the one used in HumanML3D. I noticed that you've made changes to your own combat descriptions, and I'm curious about the best practices and steps involved in creating such a dataset. could you help me?

Thanks for you appreciate,
About descriptions, our main work can be summarized as constructing a vocabulary, processing with LLM, and manually filtering. You can find the detailed process in this link: https://zhuanlan.zhihu.com/p/691984079 "§2.2 制作过程"

[mustafizur-r]

Thank you for share the link. could you give the dataset creating code?

[fyyakaxyy]

Thank you for share the link. could you give the dataset creating code?

We created the dataset according to the processing flow of HumanML3D. For text processing, you can refer to: https://github.com/EricGuo5513/HumanML3D/blob/main/text_process.py

[mustafizur-r]

Thank you for send me the HumanML3d text process code link.

def process_humanml3d(corpus):
text_save_path = './dataset/pose_data_raw/texts'
desc_all = corpus
for i in tqdm(range(len(desc_all))):
caption = desc_all.iloc[i]['caption']
start = desc_all.iloc[i]['from']
end = desc_all.iloc[i]['to']
name = desc_all.iloc[i]['new_joint_name']
word_list, pose_list = process_text(caption)
tokens = ' '.join(['%s/%s'%(word_list[i], pose_list[i]) for i in range(len(word_list))])
with cs.open(pjoin(text_save_path, name.replace('npy', 'txt')), 'a+') as f:
f.write('%s#%s#%s#%s\n'%(caption, tokens, start, end))

if name == "main":
corpus = pd.read_csv('./dataset/kit_mocap_dataset/desc_final.csv')
process_humanml3d(corpus)

How to make desc_final.csv?
i am concern with this 'new_joint_name'.
could you help me? can i see yours one?

[fyyakaxyy]

Thank you for send me the HumanML3d text process code link.

def process_humanml3d(corpus): text_save_path = './dataset/pose_data_raw/texts' desc_all = corpus for i in tqdm(range(len(desc_all))): caption = desc_all.iloc[i]['caption'] start = desc_all.iloc[i]['from'] end = desc_all.iloc[i]['to'] name = desc_all.iloc[i]['new_joint_name'] word_list, pose_list = process_text(caption) tokens = ' '.join(['%s/%s'%(word_list[i], pose_list[i]) for i in range(len(word_list))]) with cs.open(pjoin(text_save_path, name.replace('npy', 'txt')), 'a+') as f: f.write('%s#%s#%s#%s\n'%(caption, tokens, start, end))

if name == "main": corpus = pd.read_csv('./dataset/kit_mocap_dataset/desc_final.csv') process_humanml3d(corpus)

How to make desc_final.csv? i am concern with this 'new_joint_name'. could you help me? can i see yours one?

We used Excel when processing text, with the first column containing animation ID and the second column containing text annotations, then processed into id.txt.
My suggestion is to read this paper and the code of HumanML3D. They contain all the code and steps.

[mustafizur-r]

Hi,
To create a .npy file for animation and pair it with a suitable text description for use in the HumanML3D dataset. could you tell me the process?
Actually, I want to make my own animation .npy file and then use into HumanML3d after that i want to use that dataset to your project.