Exporting Metadata/Matrices/etc. from an ArchR Project

[edt711]

Hello ArchR team,

Thank you for creating such an awesome analysis tool! We are in the process of submitting a manuscript that features a lot of ArchR-analyzed scATAC-seq data, and we would like to upload the data to UCSC's Cell Browser. They generally accept .rds files, but my understanding (please correct me if I'm wrong) is that all the analyses occur within the individual arrow files, and the ArchRProject (and the .rds file it is saved as) simply calls back to those? Is there a way to save the ArchR project as an .rds that contains all of the metadata/matrix information, or would the best way to upload/share this be to share the entire output directory?

Alternatively, is there an easy way to export peak x cell matrices (corresponding to both GeneScore and GeneIntegration), metadata, and UMAP coordinates?

Thanks,
Eric Thomas

[rcorces]

This is not currently available in ArchR but is a good idea. I think it would be good to create a function where you pass it which matricies you want and it creates a new RDS object with all of the metadata and those matrices together.
For now, you will have to use the function getMatrixFromProject() to get individual matrices and manually cobble together the metadata or just use the ArchRProject object as the metadata object.

I'm changing this post to a feature request.

[edt711]

Thanks, Ryan!

I see that I can get a matrix of UMAP coordinates with the getEmbedding() function, but the getMatrixFromProject() function returns an S4 object (which seemingly still has all the relevant metadata in it). Would the best thing to do be to get the matrices I need (in this case, probably GeneScore, GeneIntegration, and Peak) and save them as individual .rds files? Or would it be better to convert those S4s to a matrix?

For the metadata, you mentioned that the ArchRProject itself could be used (i.e. the Save-ArchR-Project.rds)? Is that better than trying to extract from the CellColData?

Thanks,
Eric

[rcorces]

You should do whatever works for you. The S4 object you are talking about is a SummarizedExperiment. You can pull the individual matrix from there if that is what you want to do.

[edt711]

How would I go about doing that? My apologies, I'm still pretty green at general coding stuff.

[rcorces]

Sorry - we dont have bandwidth to provide that level of support. Just google around about SummarizedExperiments or post to more general help forums.

[edt711]

Thanks!

[cschmidl]

Dear Ryan, I think it would be very useful to get a relatively easy option to export your analysis/processed data so you can share/upload when publishing a paper. Due to data protection laws in Europe it's impossible here to share primary sequencing data without going through a cumbersome data access committee or the like, and even then many of the patients/donors do not consent to sharing outside the EU, hence we need to dump processed data along the raw data. so there people can use it easily. The nicest thing would be getting a fragment file that can be read into Seurat etc. but that may be hardly possible due to the different ways ArchR and Seurat work? For now I will use the getMatrixFromProject() and getFragmentsFromProject() functions to put sth together but they are restricted on the 500bp tile matrix I guess. Thanks again for this awesome package!

[rcorces]

I'm not sure I follow the logic of some of this. The best way to share your data is by saving your ArchRProject with saveArchRProject(). The output folder could be tar-gz'd and shared in its entirety. This would give end-users access to everything in your project. Is there a reason why this doesnt suit your use case? Is it because it is too large in size?

If all you want is a cell x feature matrix, then we already provide export utilities to do this (for ex. getMatrixFromProject() or getGroupSE()) and we have recently added some to the dev branch (for ex. getPBGroupSE() which allows export based on pseudo-bulk groups). These SummarizedExperiment exports already include all of the cellular metadata in colData.

but they are restricted on the 500bp tile matrix I guess

A 500-bp tile matrix has ~6 million features. The vector size limit in R is 2^31 - 1. This means that you cannot create an R object for a tile matrix for more than 358 cells.

The nicest thing would be getting a fragment file that can be read into Seurat etc.

If you used ArchR for your analysis, why export your data so that end-users import it into a different tool? Regardless, ArchR already provides getFragmentsFromProject(). Is there a reason that this doesnt satisfy your use case?