etl

Extract Transform Load (ETL) data into BigData storage is the central purpose of this repository.

pip install -r requirements.txt

python tem_to_parq.py --tiff your_tiff.tif --hdfs hdfs://localhost:9000

Name		Name	Last commit message	Last commit date
Latest commit History 17 Commits
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
Recon_NanoParticle_doi_10.1021-nl103400a.tif		Recon_NanoParticle_doi_10.1021-nl103400a.tif
TiltSeries_NanoParticle_doi_10.1021-nl103400a.parq		TiltSeries_NanoParticle_doi_10.1021-nl103400a.parq
TiltSeries_NanoParticle_doi_10.1021-nl103400a.tif		TiltSeries_NanoParticle_doi_10.1021-nl103400a.tif
recon_to_parq.py		recon_to_parq.py
recon_to_xyzv_parq.py		recon_to_xyzv_parq.py
requirements.txt		requirements.txt
run_pyspark_jupyter.sh		run_pyspark_jupyter.sh
spark-defaults.conf		spark-defaults.conf
tem_to_iv.py		tem_to_iv.py
tem_to_parq.py		tem_to_parq.py
tem_to_xyav_parq.py		tem_to_xyav_parq.py
verify_hdfs_import.ipynb		verify_hdfs_import.ipynb

Provide feedback