Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About T1K's evaluation results #1

Open
mourisl opened this issue Mar 20, 2024 · 1 comment
Open

About T1K's evaluation results #1

mourisl opened this issue Mar 20, 2024 · 1 comment

Comments

@mourisl
Copy link

mourisl commented Mar 20, 2024

Greetings. Thank you for developing Geny to expand the potential of the KIR genotyping with next-generation sequencing data!

In your manuscript, you also compared the accuracy with T1K, and I appreciate the sharing of the genotyping results of the HPRC samples on the github repository. As the developer, I can't help to check how T1K behaves in these experiments :). I noticed that many errors T1K made are the missing of alleles marked as "partial" or the alleles with no intron data in the IPD-KIR database. For example, for the KIR2DL3 in the HG00438 sample, T1K misses the 023 allele, which has no intronic sequence in the IPD-KIR v2.12.0. In T1K's older versions, these alleles are excluded when creating the reference sequence for WGS data. This constraint is addressed in T1K v1.0.4 and a relevant bug is fixed in v1.05 later. In our current test on the HPRC sample, I can get the KIR2DL3*023 in the outcome. I have uploaded the results for T1K v1.0.5 on the IPD-KIR v2.12.0 database at: https://github.com/mourisl/T1K_manuscript_evaluation/tree/master/T1K_1.0.5 for you convenience. By comparing the results with the excel file in your github repo, I found several missing/wrong predictions in T1K have been resolved. Though in the manuscript, you mentioned that the T1K was tested with v1.0.5, I guess the extracted reference sequence was still generated with an older version (t1k-build). Could you please check whether this is the case and maybe give T1K another round of test? Since the default output from T1K does not infer copy numbers, I expect Geny will still be more sensitive than T1K considering the copy number. But the new test may reflect the true precision of T1K v1.0.5.

Thank you for considering this.

@inumanag
Copy link
Contributor

inumanag commented May 6, 2024

Hi @mourisl

Thanks for the comment!

IIRC we've run everything (including the database prep) w/ 1.0.5 or 1.0.4. I will check this again for sure---might reach out if we get stuck. The commands are available here: https://github.com/0xTCG/geny/blob/master/paper/experiments.ipynb

Cheers,
Ibrahim

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants