Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how run camm on one RTX 3090 GPU? #2

Closed
limaolin2017 opened this issue Jun 13, 2024 · 2 comments
Closed

how run camm on one RTX 3090 GPU? #2

limaolin2017 opened this issue Jun 13, 2024 · 2 comments

Comments

@limaolin2017
Copy link

Hi, I encounter an error:

Traceback (most recent call last):
File "/gpfs/home/mli/camm/main.py", line 40, in
app.run(main)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "/gpfs/home/mli/camm/main.py", line 37, in main
trainer.train()
File "/gpfs/home/mli/camm/nnutils/train_utils.py", line 904, in train
self.train_one_epoch(epoch, log)
File "/gpfs/home/mli/camm/nnutils/train_utils.py", line 1129, in train_one_epoch
total_loss,aux_out = self.model(batch)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/nn/parallel/distributed.py", line 886, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/gpfs/home/mli/camm/nnutils/camm.py", line 686, in forward_default
mesh_rest = pytorch3d.structures.meshes.Meshes(
File "/gpfs/home/mli/camm/third_party/pytorch3d/pytorch3d/structures/meshes.py", line 406, in init
if len(self._num_faces_per_mesh.unique()) == 1:
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/_tensor.py", line 530, in unique
return torch.unique(self, sorted=sorted, return_inverse=return_inverse, return_counts=return_counts, dim=dim)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/_jit_internal.py", line 422, in fn
return if_false(*args, **kwargs)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/_jit_internal.py", line 422, in fn
return if_false(*args, **kwargs)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/functional.py", line 821, in _return_output
output, _, _ = _unique_impl(input, sorted, return_inverse, return_counts, dim)
File "/home/mli/.conda/envs/camm/lib/python3.9/site-packages/torch/functional.py", line 735, in _unique_impl
output, inverse_indices, counts = torch._unique2(
RuntimeError: std::bad_alloc

hardware device: one RTX 3090 GPU

@kts707
Copy link
Owner

kts707 commented Jun 13, 2024

You can try to see if the steps mentioned here would resolve the issue.

@limaolin2017
Copy link
Author

Thank you for your reply! Its fixed by pip install all dependencies again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants