Feature Request: Add option to copy plots and don't stop the new plot creation process

[amauryvh]

Hello, I have been testing bladebit and it works very nice. I think something that could make it much better than madmax is if you add a new parameter to bladebit, that will make bladebit copy the plot to the final destination and continue plotting at the same time (I guess it'ill work only for people with more than 416gb + 102gb of ram)
Copying a plot to the final destination could take a couple of minutes easily and I have noticed that my processors are not working while the plot is being copied to the final destination. Thank you so much for all the hard work you have done, this is something that is definitely a game changer for the chia project (Nobody will have to complain anymore about chia destroying ssds and plots could be created veeery fast)

[koiesmith]

I second this request. Have several servers at 768gb or 1024gb running bladebit. Would be very helpful.

…

On Mon, Sep 6, 2021, 8:36 AM amauryvh ***@***.***> wrote: Hello, I have been testing bladebit and it works very nice. I think something that could make it much better than madmax is if you add a new parameter to bladebit, that will make bladebit copy the plot to the final destination and continue plotting at the same time (I guess it'ill work only for people with more than 416gb + 102gb of ram) Copying a plot to the final destination could take a couple of minutes easily and I have noticed that my processors are not working while the plot is being copyied to the final destination. Thank you so much for all the hard work you have done, this is something that is definitely a game changer for the chia project (Nobody will have to complain anymore about chia destroying ssds and plots could be created veeery fast) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#83>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMWGF5T4GTWDDZF2D246PNTUASYUDANCNFSM5DQLXJUQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[maxcom99]

Does anyone have a script i can run on a schedule to at least move the completed files to NAS?

[amauryvh]

I copy mines directly to the disk, but this is from Mike (xorinox), in this publication:
https://chiaforum.com/t/bladebit-plotting-at-220-280-plots-per-day-on-one-system/11987

He also added this comment and script:
"Don’t mind at all, use on your own risk. I truncated the repetitive aspects of the script. Lot’s of hard coded values in there as my setup is standardized. All farmer hard drives are 18 TB, 5 buffers of 5.5 TB each made of 2x 3.2 TB nvme flash drives in raid-0."

#!/bin/bash
log="/chia/logs/chia-plotter.log"

pool() {
./bladebit -t "$1" -n "$2" -f -c "$3" |& tee "$4"
}

rem_plots() {
plots=$(find "$1" -type f -size +108700000000c |wc -l)
rem=$((33 - plots))
echo $rem
}

for i in {1..5}; do
buff="/chia/scratch/disk01/"
rem=$(rem_plots "$buff")
echo "...info: $rem plots going to be created"
if [ "$rem" -gt "0" ]; then
echo 3 > /proc/sys/vm/drop_caches
pool "128" "$rem" "$buff" "$log.disk01"
fi

disk="/chia/plots/disk098/"
(rclone move "$buff" "$disk" --transfers 1 --include *.plot --min-size 101G) &
echo "...info: kicked off rclone to move plots from $buff to $disk"
.
.
#And so on, 4 more blocks as above, for a total of 5 disks. Could have made it more granular...
.
.
done

[amauryvh]

another guy published this one
#!/bin/bash

#Watches for newly created plots and moves them to farmer 1 by 1.
#Requires inotify-tools: sudo apt-get install -y inotify-tools

#Directory paths
SRCDIR="/mnt/nvme1/"
DESTDIR="/mnt/storage/poolplots/"
#Limit transfer speed to minimise long lookup times whilst farming.
BWLIMIT="140000"

echo "Watching for new plots in: $SRCDIR"
echo "Copying plots to: $DESTDIR"

inotifywait -m $SRCDIR -e create -e moved_to |
while read path action file; do
if [[ "$file" =~ .*plot$ ]]; then
echo "Found new plot!"
echo "Copying plot to $DESTDIR"
rsync --bwlimit="$BWLIMIT" --preallocate --remove-source-files --skip-compress=plot --whole-file -avP "$SRCDIR/$file" "$DESTDIR/$file"
echo -e "Copy complete, waiting for new plots...\n"
fi
done