You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello, thank you for sharing your NP-FKGC work and code.
While experimenting with the provided settings, I encountered some confusion regarding the configuration. In the paper and README, you mention different data settings such as "Pre-Train" and "In-Train." However, I noticed that even when using the --data_form Pre-Train option, the code still loads files with the _in_train suffix (e.g., train_tasks_in_train.json). As a result, it appears that the background KG is included in the training tasks.
To verify this, I replaced all _in_train files with their original counterparts (e.g., using train_tasks.json instead of train_tasks_in_train.json), thus creating a "pure Pre-Train" setting without including the background KG in training. Surprisingly, under this strict Pre-Train setup, the performance I achieved was even higher than the metrics reported in the paper. I initially expected that incorporating the background KG (In-Train) would boost performance, so obtaining better results without the background KG was unexpected.
I would appreciate clarification on the following points:
Why does the code load _in_train files even when --data_form Pre-Train is specified? Is this the intended behavior, or is it a discrepancy that arose during code updates?
When adhering strictly to the Pre-Train setting as described in the paper (i.e., excluding the background KG), is it normal to observe better performance than the reported metrics, or is this an unforeseen outcome?
Here are the experimental results obtained by training exclusively on the data without the _in_train suffix, which I verified directly using your code.
So far, everything I have verified pertains to the NELL dataset.
Thank you in advance for your response.
The text was updated successfully, but these errors were encountered:
Hello, thank you for sharing your NP-FKGC work and code.
While experimenting with the provided settings, I encountered some confusion regarding the configuration. In the paper and README, you mention different data settings such as "Pre-Train" and "In-Train." However, I noticed that even when using the
--data_form Pre-Train
option, the code still loads files with the_in_train
suffix (e.g.,train_tasks_in_train.json
). As a result, it appears that the background KG is included in the training tasks.To verify this, I replaced all
_in_train
files with their original counterparts (e.g., usingtrain_tasks.json
instead oftrain_tasks_in_train.json
), thus creating a "pure Pre-Train" setting without including the background KG in training. Surprisingly, under this strict Pre-Train setup, the performance I achieved was even higher than the metrics reported in the paper. I initially expected that incorporating the background KG (In-Train) would boost performance, so obtaining better results without the background KG was unexpected.I would appreciate clarification on the following points:
Why does the code load
_in_train
files even when--data_form Pre-Train
is specified? Is this the intended behavior, or is it a discrepancy that arose during code updates?When adhering strictly to the Pre-Train setting as described in the paper (i.e., excluding the background KG), is it normal to observe better performance than the reported metrics, or is this an unforeseen outcome?
Here are the experimental results obtained by training exclusively on the data without the
_in_train
suffix, which I verified directly using your code.So far, everything I have verified pertains to the NELL dataset.
Thank you in advance for your response.
The text was updated successfully, but these errors were encountered: