immich-go not uploading most of my 202Gig Takeout

[phands]

Hi all,

Sorry if this is a daft question.
I have a large-ish (just over 202Gigs) Takeout directory (extracted from the compressed archives to a directory structure), and I'm trying to use immich-go to upload it all.

It seems to work partially, but I only see a tiny fraction of the files in the immich server. There are tens of thousand of photos and videos. but less than 25 in Immich.

I know that the uploader is changing rapidly, and I'd love to help with testing if I can.
My Takeout directory is structured as shown in the screenshot...

Immich-go says it's seeing 31553 images...see this screenshot...

my iimmich-go command is ....
/immich-go upload from-google-photos /local-12T/Google\ Photos/Takeout --api-key 853oJrfwOpiGB8XhASoAraVsQ8USDS156Eu1nLaNWZY --server http://10.0.0.37:2283

The log file seems OK, but shows that while it scanned 31553 images, it only uploaded 35 the first time, and now they are duplicates. Here are the last few lines of 102900...

2025-01-24 12:31:34 INF Tagged file=Takeout:Photos from 2024/Simon Durrant and Anna and Alan Kidd and wife.jpg tags=[Takeout]
2025-01-24 12:31:34 INF added to an album file=Takeout:Photos from 2024/Simon Durrant and Anna and Alan Kidd and wife.jpg Album=TATC 11th Entry
2025-01-24 12:33:39 INF
2025-01-24 12:33:39 INF Input analysis:
2025-01-24 12:33:39 INF ---------------
2025-01-24 12:33:39 INF scanned image file : 31553
2025-01-24 12:33:39 INF scanned video file : 2926
2025-01-24 12:33:39 INF scanned sidecar file : 33311
2025-01-24 12:33:39 INF discarded file : 0
2025-01-24 12:33:39 INF unsupported file : 468
2025-01-24 12:33:39 INF file duplicated in the input : 7
2025-01-24 12:33:39 INF associated metadata file : 42
2025-01-24 12:33:39 INF missing associated metadata file : 34437
2025-01-24 12:33:39 INF
2025-01-24 12:33:39 INF Uploading:
2025-01-24 12:33:39 INF ----------
2025-01-24 12:33:39 INF uploaded : 0
2025-01-24 12:33:39 INF upload error : 0
2025-01-24 12:33:39 INF file not selected : 0
2025-01-24 12:33:39 INF server's asset upgraded with the input : 0
2025-01-24 12:33:39 INF server has same asset : 35
2025-01-24 12:33:39 INF server has a better asset : 0
2025-01-24 12:33:39 INF
2025-01-24 12:33:39 INF Check the log file: /home/phands/.cache/immich-go/immich-go_2025-01-24_12-31-01.log

Can anyone point me at where to start debugging or what is wrong with my immich-go command? I'm also available for live debugging (USA Eastern time).

Thanks,

Paul

[simulot]

The JSON files have an unusual name:

photo.jpg-supplemental-metadata.json 

This explain this missing associated metadata file : 34437

You're the first to report such case.
Could you grab the list of files? See the page https://github.com/simulot/immich-go/blob/main/docs/how-to-send-debug-data.md

The takeout isn't easy to parse. Long file names are truncated using unclear rules. So, I'm surprised to have much longer name for the JSON files. I'm particularly curious how are named JSON files attached to photos with very long names...

I have requested a fresh takeout, to see if something has changed.

[phands]

Here's another view of the list...

list2.txt

[simulot]

Thank you for the list. This is new for me.

Some weird cases

Screenshot_20231027_123303_Facebook(1).jpg
Screenshot_20231027_123303_Facebook.jpg.supple.json
Screenshot_20231027_123303_Facebook.jpg.supple(1).json
Screenshot_20231027_123303_Facebook.jpg
Screenshot_20231027_123303_Facebook-edited.jpg
original_c576ba15-9661-4bd4-b0bf-a7bf2ce3d354_.json
original_c576ba15-9661-4bd4-b0bf-a7bf2ce3d354_2.jpg
original_c576ba15-9661-4bd4-b0bf-a7bf2ce3d354_2(1).jpg

I'll add the support for those case very soon

[simulot]

Issue #652

Could you provide the content of some of the supplemental-metadata.json files ?

[phands]

I'll get on it today...do you want the metadata for the particular files you list above, or just any metadata?

[simulot]

any metadata is good.