Re-downloading deleted videos.

[mikeyounge]

Hey meeb! I updated my container last night to whatever had been committed up to 10PM MST on Feb 13th and when the tasks were run this morning, it triggered a re-download of everything that had been deleted from my library. Plex manages my files as I watch them, so that I don't have Tubesync delete something that I haven't seen. It hasn't been an issue in the past, but today it decided to download 300+ videos that had been deleted from the library by Plex. The previous update to my container was I think on Feb 2nd. Let me how to export the logs you need and I'll get them for you. Thanks.

[tcely]

This is the most likely change to be involved in what you are seeing:

tubesync/tubesync/sync/signals.py

Lines 182 to 206 in f083ad6

	existing_media_download_task = get_media_download_task(str(instance.pk))
	# If the media has not yet been downloaded schedule it to be downloaded
	if not (instance.media_file_exists or existing_media_download_task):
	# The file was deleted after it was downloaded, skip this media.
	if instance.can_download and instance.downloaded:
	skip_changed = True != instance.skip
	instance.skip = True
	instance.downloaded = False
	instance.media_file = None
	if (instance.source.download_media and instance.can_download) and not (
	instance.skip or instance.downloaded or existing_media_download_task):
	verbose_name = _('Downloading media for "{}"')
	download_media(
	str(instance.pk),
	queue=str(instance.source.pk),
	priority=10,
	verbose_name=verbose_name.format(instance.name),
	remove_existing_tasks=True
	)
	# Save the instance if any changes were required
	if skip_changed or can_download_changed:
	post_save.disconnect(media_post_save, sender=Media)
	instance.save()
	post_save.connect(media_post_save, sender=Media)

Using docker logs -t [container name or ID] is my preferred way to view container logs. You can redirect that output to a file and upload that if you prefer to have someone else look at it.

[mikeyounge]

docker logs -t didn't print to the file all the info that I'm seeing output to the command line. It's only showing the web requests, not the task log.
docker logs -t tubesync > /opt/tubesync/docker.log

EDIT: found an alternate way with Portainer. Sorry for all the Uptime Kuma requests

_tubesync_logs.txt

[tcely]

docker logs -t didn't print to the file all the info that I'm seeing output to the command line. It's only showing the web requests, not the task log.

It should contain both types of logs. Moving web requests into a file (#713) is a way to make it easier to read.

found an alternate way with Portainer. Sorry for all the Uptime Kuma requests

Thanks! I'll be making use of grep andsed to remove those parts, so no problem.

Your logs already proved useful. #725

2025-02-14 05:49:58,662 [tubesync/INFO] Media: 2muchcolinfurze / AcmKeh0lnTQ has changed skip setting to False

tubesync/tubesync/sync/filtering.py

Lines 42 to 52 in f083ad6

	# Check if skipping
	if instance.skip != skip:
	instance.skip = skip
	log.info(
	f"Media: {instance.source} / {instance} has changed skip setting to {skip}"
	)
	return True
	return False

I'm still looking through them.

[mikeyounge]

Thanks tcely. Your contributions have been great! I appreciate the recent Sponsor block changes. They work great!

[tcely]

The logs contain a lot of lines without a date/time stamp, such as:

ERROR: ^M[download] Got error: <urllib3.connection.HTTPSConnection object at 0x7fcaf7bf18d0>: Failed to establish a new connection: [Errno 111] Connection refused

Search for common.errors.DownloadFailedException: Failed to download media: arACyG0jS2w and scroll up to find these.

Do you have any idea what might be happening?

[mikeyounge]

_tubesync_logs (1).txt

Updated logs with timestamps turned on. I have 5 errors in my tasks list. Three are "Float() argument must be a string or a real number, not 'NoneType'" for media sources. Two are failed downloads that are probably rate limited or something. I haven't cancelled any of these downloads, except for two, so they are just running and I'm deleting them manually from Plex. Your example "arACyG0jS2w" is one of those failed downloads.

[tcely]

Ok. I've looked through the logs and saw much the same things.

One odd thing, your log file, in addition to Windows line endings, contained ^B^@^@^@^@^@^A in a few places. Any idea what may be adding those?

It's not a huge deal, I simply had to filter out the nul characters with tr so that the rest of my commands would work on the entire file.

For anyone interested, this was the command used to view the log files:

< _tubesync_logs.1.txt tr -d '\0\r' | sed -e 's@[0-9-]*T[0-9:.]*Z 172\..* "Uptime-Kuma/1.23.11"@@g' | grep -v '^$' | less -FRX

[mikeyounge]

I downloaded the logs from a browser using Portainer. It must have added a bunch of irrelevant characters. If there is a better way to call the full logs from the command line, I'm all for it. docker logs -t tubesync > /opt/tubesync/docker.log didn't output all the tasks, just the Django output. The terminal was filled with the tasks logs, so I assumed it was also writing, but it did not.

[tcely]

I downloaded the logs from a browser using Portainer. It must have added a bunch of irrelevant characters.

Possibly.

If there is a better way to call the full logs from the command line, I'm all for it. docker logs -t tubesync > /opt/tubesync/docker.log didn't output all the tasks, just the Django output. The terminal was filled with the tasks logs, so I assumed it was also writing, but it did not.

Try this to write both standard output and standard error to the file:

docker logs -t tubesync > /opt/tubesync/docker.log 2>&1

[mikeyounge]

Much better

docker.log

[tcely]

Yep, no nul characters in that file and proper line endings too. 🙂

[mikeyounge]

@meeb Any chance this can get committed to main branch? It keeps loading these deleted videos to my queue on every refresh of a source.

@tcely I can test the fix if you need, but I don't know how to do that. I'm an enthusiastic amateur at best.

[mikeyounge]

Hopefully this fixed it for the future, but because I believe it reset everything to missing, I’ll have to wait for it to process everything, delete it all again and wait for the refreshes. Previously, it was re-queuing items I deleted last night, so we will see about tomorrow’s refresh.

[mikeyounge]

Can confirm this is fixed. Thank you. It didn’t end up re-downloading.