Unable to log into R Studio server

[yaaminiv]

I'm running a pretty intensive randomization test on the R Studio server that involves interacting calculateDiffMeth 1000 times:

for (i in 1:1000) {
  pHRandTreat <- sample(sampleMetadataFem$pHTreatment,
                        size = length(sampleMetadataFem$pHTreatment),
                        replace = FALSE)
  pHRandDML <- methylKit::reorganize(methylationInformationFilteredCov5Fem,
                                     sample.ids = sampleMetadataFem$sampleID,
                                     treatment = pHRandTreat) %>%
    methylKit::calculateDiffMeth(., covariates = covariateMetadataFem,
                                 overdispersion = "MN",
                                 test = "Chisq", mc.cores = 40) %>%
    methylKit::getMethylDiff(., difference = 25, qvalue = 0.01) %>%
    nrow() -> pHRandTest25Fem[i]
}

My session timed out after 60 minutes and when I tried to log back in, I can't even access http://localhost:8787/ ! The log in page is not loading, so I can't check on the progress of my code. I killed my mox tunnel and opened a new one to see if I could load http://localhost:8787/ and that didn't work.

Is there anything I can do? My R Studio server SLURM script is still running, but I guess I could end that too and just interrupt whatever is running.

[kubu4]

Is your terminal throwing an error?

Regardless, I suspect that the port 8787 didn't close properly after the session timeout. So, you need to kill the SSH session that's still occupying that port:

lsof -ti:8787 | xargs kill -9

Then, you'll need to re-establish your tunnel and then the web browser URL should work (refresh the page).

[kubu4]

Basically, I read that the session load time is highly dependent upon the amount of information in the R environment. So, if you have large data objects, it will lead to lengthy session load times. This could be the issue.

[yaaminiv]

What's the longest you've waited without closing the browser (tab) and the SSH tunnel?

Maybe 15 min? I can let this sit for a few hours since I have a lot of data in the R environment!

[kubu4]

Any luck with waiting on the web browser?

Another option is to kill the SLURM job entirely and edit the SLURM script. If you add the following to your script (just add it immediately after the line which loads the singularity module), it should prevent session timeouts from occurring.

# Do not suspend idle sessions.
# Alternative to setting session-timeout-minutes=0 in /etc/rstudio/rsession.conf
# https://github.com/rstudio/rstudio/blob/v1.4.1106/src/cpp/server/ServerSessionManager.cpp#L126
export SINGULARITYENV_RSTUDIO_SESSION_TIMEOUT=0

This was taken from this: https://github.com/rocker-org/website/blob/master/content/use/singularity.md

[yaaminiv]

Still loading, so I just killed my script and will add that code. Thanks!

[kubu4]

Dang. Sorry. Hope this helps!!

[yaaminiv]

Update: I started a randomization test and left for a few hours, came back and I had been signed out after 60 min of inactivity! I put in my user name and pw, but I can't sign in! When I click "Sign In," nothing happens and I am not directed to my R Studio server session.

(ignore the "connection refused" error in the Terminal window, I was trying to tunnel in this morning using information from a previous SLURM script. I set up the tunnel correctly around 12 and haven't had any Terminal errors since)

edited to include a description of what happens when I try to sign in

[kubu4]

Don't have any real suggestions. When I tested things, I definitely signed in/out of a single session multiple times and didn't have any issues. I'd close the browser (quit it entirely) and quit Terminal entirely. Then, re-establish the tunnel. Then, open web browser and add the appropriate URL for the RStudio Server. Copy/paste username and password found in your SLURM output (make sure you're copying from the correct SLURM output; just in case you have more than one).

Alternatively, kill your SLURM job and modify your SBATCH script so that the part which executes the Singularity container looks like this:

# This example bind mounts the /gscratch/scrubbed/${USER} directory on the host into the Singularity container.
# By default the only host file systems mounted within the container are $HOME, /tmp, /proc, /sys, and /dev.
singularity exec \
--bind="$TMPDIR/var/lib:/var/lib/rstudio-server" \
--bind="$TMPDIR/var/run:/var/run/rstudio-server" \
--bind="$TMPDIR/tmp:/tmp" \
--bind=/gscratch/scrubbed/${USER} \
${container_path}/${container} \
rserver --www-port ${PORT} --auth-none=0 --auth-pam-helper-path=pam-helper --auth-stay-signed-in-days=30 --auth-timeout-minutes=0

There are two changes there:

1. Removes automatic sign out
2. Keeps your session signed in for 30 days.

[yaaminiv]

I put in my username/pw and let it load over night to see what happened, and I got this menu:

Not entirely sure what "Safe Mode" is, but I clicked that to see what would happen. Still loading.

I wonder if this is related to the fact that I'm now using mc.cores in my calculateDiffMeth code? I had issues with that previously.

[yaaminiv]

Alternatively, kill your SLURM job and modify your SBATCH script so that the part which executes the Singularity container looks like this

I created a modified script (found here). I get an error in my slurm.out:

I'm also unable to tunnel into mox:

Am I missing any code for the singularity container?

[yaaminiv]

Was missing {container} variable in my script, added container path and it worked!