Optimization time

[zinuok]

❓ Questions and Help

Hello, really thanks for sharing your great work.

For the homography estimation case in the provided example, the optimization itself takes 2 [secs] (approximately), even if the number of max iterations is set to 10, which is reasonable in real case.
It's too slow compared to my model pipeline, which originally took about 134 [ms] per epoch.

Is there any guide or tip to speed up optimization when training in combination with PyTorch's learning model?
I wonder if there's something I'm missing out on.

Thank you

[ddetone]

Hello, thanks for reaching out. How is "my model pipeline" implemented? Are you using the analytical jacobian as a baseline?

[luisenp]

Hi @zinuok, we are working some on improving aspects of our optimization time, particularly related to automated Jacobian computation (see #268); so, there is definitely room for speeding things up.

That being said, I'm also curious about the model pipeline that you referred to in your post. It's difficult to offer more useful suggestions w/o knowing what are the key differences.

[zinuok]

Thank you for your kind reply.

The model I mentioned above is just a simple Depth estimation model based on CNN.
I have not yet incorporated the Theseus layer into my model, and just measured the iteration time of a given homography estimation code (which is implemented with AutoDiff), for feasibility check. Sorry for misleading you guys.

What I meant to say was that the optimization time of Theseus was relatively large compared to the training epoch time of a typical deep learning model, so it was a concern that it might slow down the overall learning time.

[luisenp]

Hi @zinuok, thanks for clarifying, I see what you mean. There will definitely be some unavoidable overhead, since adding a TheseusLayer involves solving an optimization problem which is a sequential operation that can be time consuming. Nevertheless, as mentioned above, we are actively working on reducing time for many operations, so you should expect to see improvements over time.

[fantaosha]

Hi @zinuok, we have refactored AutoDiffCostFunction with functorch and there is a significant speedup. More details can be found in #296.

[zinuok]

Thank you for your hard work! I'll try it