Allow inference on multiple individual videos via sleap-track

[roomrys]

Currently, to run inference on multiple videos via the sleap-track command, users need to call this command many times either manually or in a script. It might be a nice feature to allow multiple videos/inputs.

[talmo]

This is especially useful for projects with many videos where it takes forever to generate predictions on each one since it requires so many subprocess startups.

But how do we do this without breaking console character limits (4096 - 8191 depending on OS)? This is especially tricky since absolute paths may end up quite long.

That said, I agree that for scenarios that fit in the command line length constraints, we should support that. For calls with more paths, perhaps we should be able to pass in a YAML file, CSV, text file with one path per line, etc. so that we can generate that file temporarily just to pass the paths in.

[roomrys]

I like the yaml idea... I just wish it was more user friendly. A "hack" that I thought might work would be adding all the videos into an slp and then running inference on that, but also not ideal. Even just a directory option to predict on all videos in a directory would be helpful.

[talmo]

Agreed -- directory of videos is a common pattern.

Maybe more user friendly than the yaml idea would be plaintext with one path per line (easy to generate, even manually), or CSV so researchers can just paste paths in Excel or Google Sheets (we just need to be smart about ignoring column name).

This is probably a great starter enhancement PR!

[khatch1323]

Similarly it would be helpful if in the predict on dropdown there was an option to run inference on all untracked videos in a project. This would be helpful for individuals who want to add multiple untracked videos to an established project and run inference without rerunning tracking on previously tracked videos (by selecting all videos) or individually running tracking on each newly added video (by selecting entire current video for each new video).

I'm not familiar with Python, but a possible idea from something I do in MATLAB would be to add code that checks which file names in the project already have tracking, skips those, and run inference on those that do not.

[WeissShahaf]

can sleap-track "[data_path]" be modified to have a "--f" key to mark folder instead of file?
to allow batch inference of files in a folder, making SLEAP ignore the -o names but only take the folder
?

[majaneubauer]

I have written a small script for my project that might help with that. After I've trained my model and was happy with its performance, I needed it to predict on about 50 videos. I did not want to go through each video individually, so this is what I came up with:

# Imports
import os
import sleap
from tqdm import tqdm

input_dir = r"C:\Users\Cystein\Maja\videos" # path to the directory where videos are stored
models_dir = r"C:\Users\Cystein\Maja\sleap_gui\models" # path to models directory

video_files = [files for files in os.listdir(input_dir) if files.endswith('.mp4')] 

# for id model
for video_file in tqdm(video_files):
    # Construct the command
    command = (
        f"sleap-track {os.path.join(input_dir, video_file)} "
        "--max_instances 2 "
        "--tracking.tracker simplemaxtracks "
        "--tracking.max_tracking true "
        "--tracking.max_tracks 2 "
        "--tracking.target_instance_count 2 "
        "--tracking.post_connect_single_breaks 1 "
        "--tracking.match hungarian "
        "--tracking.track_window 5 " 
        "--tracking.similarity iou "
        "--no-empty-frames "
        f"-o model_td_id_{video_file[:-4]}.predictions.slp " # [:-4] gets rid of .mp4 file extension
        f"-m {os.path.join(models_dir, '240404_141957.centroid.n=145')} "
        f"-m {os.path.join(models_dir, '240404_142457.multi_class_topdown.n=145')} "
    )
    
    # Execute the command
    os.system(command)

This was for the specific top-down-id-model I was training, but the CLI commands can just be adapted to whatever you need. Maybe this helps :)

Best,
Maja

[eberrigan]

This is in a future release! See #1777.