Alternative to cargo fetch
for use in CI or other "clean" environments that you want to quickly bootstrap
with the necessary crates to compile/test etc your project(s).
- You run CI jobs inside of GCP and you want faster crates.io and git downloads so that your compute resources can be spent on the things that you actually care about.
- You don't run CI inside of GCP. Currently
cargo-fetcher
only supports storing crates/git snapshots inside of GCS which means they can be located closer to the compute resources your CI is running on. PRs are of course welcome for adding additional storage backends though! cargo-fetcher
should not be used in a typical user environment as it completely disregards various safety mechanisms that are built into cargo, such as file-based locking.
The gcs
feature enables the use of Google Cloud Storage as a backend.
- Must provide a url to the
-u | --url
parameter with the gsutil syntaxgs://<bucket_name>(/<prefix>)?
- Must provide GCP service account credentials either with
--credentials
or via theGOOGLE_APPLICATION_CREDENTIALS
environment variable
The s3
feature enables the use of Amazon S3 as a backend.
- Must provide a url to the
-u | --url
parameter, it must of the formhttp(s)?://<bucket>.s3(-<region>).<host>(/<prefix>)?
- Must provide AWS credentials by the default mechanism(s) described here
This is an example from our CI for an internal project.
- 424 crates.io crates: cached - 38MB, unpacked - 214MB
- 13 crates source from 10 git repositories: db - 27MB, checked out - 38MB
The following CI jobs are run in parallel, each in a Kubernetes Job running on GKE. The container base is roughly the same as the official rust:1.39.0-slim image.
- Build modules for WASM
- Build modules for native
- Build host client for native
~ wait for all jobs to finish ~
- Run the tests for both the WASM and native modules from the host client
All 3 build jobs take around 1m2s each to do cargo fetch --target x86_64-unknown-linux-gnu
All 3 build jobs take 3-4s each to do cargo fetcher --include-index mirror
followed by 5-7s to
do cargo fetch --target x86_64-unknown-linux-gnu
.
We welcome community contributions to this project.
Please read our Contributor Guide for more information on how to get started.
Licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.