Newbie at sleap, need some help :) (skeleton design)

[Haaapppy]

Hello,
I started experimenting with SLEAP as part of my master's degree in behavioral biology. I want to track two birds in cages filmed from above and am encountering difficulties. I designed the first models using the SLEAP instructions on the developer site and have the feeling that I have done something fundamentally wrong. When I export the data I only get one bird out even if both are tracked.

My question, when creating the Skeleton, how do I create two skeletons? I can't cope somehow. I have the feeling that the data points are overwriting each other. What do you need to help me? I would be very happy about a nudge in the right direction. thank you very much

[roomrys]

Hi @Haaapppy,

SLEAP currently only supports having a single Skeleton for each slp file (this Skeleton will be re-used across all animals/Instances that SLEAP predicts). When you say that "both are tracked", do you mean that both birds have pose estimations on a frame (which SLEAP refers to as "predictions") - or that both birds have Tracks assigned (which is what SLEAP refers to as "tracking")?

THanks,
Liezl

[roomrys]

It sounds like the issue might be that you haven't performed tracking on your birds (which will assign a Tracks object to each bird/Instance).

Note: ID models assign tracks during model inference, but also require Track annotations by the user. ID models essentially learn animal ID's, but can be difficult to annotate (and accurately predict) if the animals look similar.

When exporting to an .analysis.slp file, SLEAP checks each Instance for an associated Track. If no Instances have an associated Track, then SLEAP assumes this is a single animal project and only exports the first Instance it finds in each frame.

[roomrys]

@Haaapppy

If you are not using a supervised ID model (which predicts the Track during model inference), then "tracking" can be run automatically after inference by

1. Open the Predict > Run Inference dialog
2. Select a "Tracker (cross-frame identity) Method" other than None
3. Predict On: <a selection of sequential frames>

Note: it is important to run "tracking" on sequential frames because the temporal tracker used by non-ID models compares the distance between instances in temporally adjacent frames to assign Tracks.

Please let us know if this is helpful or if you need more assistance.

Thanks,
Liezl

[Haaapppy]

@roomrys

first of all thanks in advance,for that detailed reply.
I will try to answer the question from the beginning to the end of your text.
Exactly I get pose predictions from each bird,yes. The information that only one skeleton can be used right now is what i wanted to hear.
It makes sense now. I use a non-id multi animal top down model.
So far i only got predictions on 20 random frames.I think that's how I understand your reply is the problem,because sleap ai can't make any sense of it ,so there is no track_id and only one animal tracked in the exported analysis file,right?

Next step I will as suggest do a inference on 20 subsequently frames and let you know what I got out of it.

Thank you very much.

Greetings from Germany

Tim

[roomrys]

Hi @Haaapppy,

Early on in the project, we recommend just doing iterations of predicting on "suggested frames" (without tracking), fixing the predictions, adding more frames to the Labels > "Labeling Suggestions", and re-iterating until the pose estimations look good.

Then, once you are happy with the pose estimations, you can tune in your tracking parameters by selecting a small clip of sequential frames and running inference+tracking on the "selected clip". You can select a clip by shift-dragging in the seekbar (or shift + other movement key). To deselect a clip, shift+click on the seekbar.

Once you are pretty happy with your tracking parameters, run inference+tracking on the entire video.

Lastly, export the results to an .analysis.slp file.

Thanks,
Liezl

[Haaapppy]

@roomrys
Hello, it's me again, I tried to implement your recommendations today and what can I say?
It works, I just got the first file out for analysis, with track_id and everything I was missing for further analysis. Thank you very much for the great feedback.