Permission denied! "cp: cannot create regular file"

[zhiweifang]

Hi,

I follow the instructions to build a dst-server.

When I start the docker:

docker-compose up -d   # without sudo!!

The output is:

Starting dst_master ... done
Starting dst_caves  ... done

Then I attach the docker, I found the following logs:

Logged in OK
Waiting for user info...OK
 Update state (0x61) downloading, progress: 34.77 (68694186 / 197594637)
 Update state (0x61) downloading, progress: 44.68 (88275816 / 197594637)
 Update state (0x61) downloading, progress: 56.41 (111460953 / 197594637)
 Update state (0x61) downloading, progress: 74.52 (147245728 / 197594637)
 Update state (0x61) downloading, progress: 88.71 (175295215 / 197594637)
 Update state (0x101) committing, progress: 0.00 (0 / 197594637)
Success! App '343050' fully installed.
cp: cannot create regular file '/home/dst/server_dst/mods/dedicated_server_mods_setup.lua': Permission denied
cp: cannot create regular file '/home/dst/.klei/DoNotStarveTogether/DSTWhalesCluster/Master/modoverrides.lua': Permission denied
cp: cannot create regular file '/home/dst/.klei/DoNotStarveTogether/DSTWhalesCluster/Caves/modoverrides.lua': Permission denied

Any suggestions for this?

[mathielo]

Hey @zhiweifang, that's usually related to permissions in the files you cloned from git. Is it possible that you either ran sudo git clone (...) OR you cloned it with another user? Run ls -l and see if the ownership of the files matches the user who's running the docker-compose up command.

That happens because Docker uses volumes to persist some files, such as downloaded mods and the generated world and its saves. If the user who's executing the Docker command doesn't have write permission on the project folders, you'll have that permission error.

[zhiweifang]

I changed the access permissions of DSTClusterConfig and volumes and then it works.

chmod -R 777 DSTClusterConfig
chmod -R 777 volumes

Thanks a lot.

[X-lem]

@mathielo Relevant to this question. Recently my server was having issues. I was able to extract the DSTClusterConfig folder. I rebuild the server, went through the documentation again. And pushed my DSTClusterConfig folder into the dst-dedicated-server folder.

Launched the server and everything works fine (as far as I can tell), except for the mods. For some reason I keep getting errors such as:

dst_caves     | [00:00:24]: [CRITICAL] Failed to save file: /home/dst/.klei//DoNotStarveTogether/DSTWhalesCluster/Caves/save/boot_modindex

dst_caves     | [00:00:20]: could not load modsettings: ../mods/modsettings.lua
dst_caves     | [00:00:20]: Warning: You may want to try reinstalling the game if you need access to forcing mods on.

cp: cannot create regular file '/home/dst/server_dst/mods/dedicated_server_mods_setup.lua': Permission denied

I also get this error when shutting down which isn't mod related.

[00:09:55]: [CRITICAL] Failed to save file: /home/dst/.klei//DoNotStarveTogether/DSTWhalesCluster/Master/save/saveindex

I used chown -R user DSTClusterConfig to change the user that I used to install everything and when I run docker-compose up. Does it matter which group the files belong to. I'm not sure why It can successfully launch, play, and save the game, but not use the mods.

[mathielo]

Hey @X-lem! All the issues seems to be the same, permissions in the /DSTClusterConfig folder. The most common cause for it is wrong ownership as the default permission levels (0755) should work if the user or group is right.

Launched the server and everything works fine (as far as I can tell), except for the mods.

Nope, you probably won't have any saves as well. You'll be able to run the server and play, but it won't save the progress (as it's getting permission denied).

I used chown -R user DSTClusterConfig to change the user that I used to install everything and when I run docker-compose up

☝️ That's werid, because this command shuold fix the ownership problem. What do you see when you go into DSTClusterConfig and run the commands below?

pwd && echo $USER && ls -lha

Can you confirm that your user/group is actually listed there as the owner of DSTClusterConfig and all of its subfolders and files? If yes, what about the folder above, the project root? Is it owned by your user?

If still unsolved, what OS are you in?

[X-lem]

Thank you @mathielo for taking the time! I appreciate it.

I ran the command as instructed. Here is my output on the two folders. I'm currently running Ubuntu, 18.04. The project root is also owned by my user.

[zhiweifang]

In my case, I think it is the problem of docker configuration.

I notice that the docker container is not owned by my user although it is started in my user account by docker-compose up -d.

Using htop to show more details:

 PID   USER      PRI NI VIRT   RES   SHR  S CPU% MEM%  TIME+    Command
 1219 zwfang     20   0  816M 39816 17212 S  0.0  0.2  0:00.89 docker attach dst_master
 1220 zwfang     20   0  816M 39816 17212 S  0.0  0.2  0:00.00 docker attach dst_master
 1221 zwfang     20   0  816M 39816 17212 S  0.0  0.2  0:00.01 docker attach dst_master
 1318 zwfang     20   0  816M 39816 17212 S  0.0  0.2  0:00.00 docker attach dst_master
 1319 zwfang     20   0  816M 39816 17212 S  0.0  0.2  0:00.78 docker attach dst_master
 1320 zwfang     20   0  816M 39816 17212 S  0.0  0.2  0:00.77 docker attach dst_master
 1352 zwfang     20   0  816M 39816 17212 S  0.0  0.2  0:00.77 docker attach dst_master
 1233 ltguo      20   0  728M  587M 14576 S 10.6  3.7 10h32:50 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1376 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:11.51 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1377 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:11.25 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1251 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:00.00 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1252 ltguo      20   0  728M  587M 14576 S  0.0  3.7 16:57.74 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1258 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:08.29 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1259 ltguo      20   0  728M  587M 14576 S  0.0  3.7  4:33.72 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1260 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:00.73 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1261 ltguo      20   0  728M  587M 14576 S  2.7  3.7  2h22:07 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1263 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:00.00 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1267 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:00.00 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1268 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:00.00 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1269 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:12.19 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1270 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:35.37 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1271 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:14.55 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1272 ltguo      20   0  728M  587M 14576 S  0.0  3.7  1:03.28 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1274 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:11.57 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1234 ltguo      20   0  728M  587M 14576 S  0.0  3.7  1:14.88 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 1235 ltguo      20   0  728M  587M 14576 S  0.0  3.7  0:00.00 ./dontstarve_dedicated_server_nullrenderer -cluster DSTWhalesCluster -shard Master
 ...
 ...

in which zwfang is my user account, but my dst docker container is owned by ltguo (not a super user). I test other containers and get the same results.

I don't know much about docker but I guess it was not configured properly.

[mathielo]

Damn that's really weird. I never saw different users running the docker processes before 🤔 😕

There could be something really off with the docker installation. @X-lem can you check with htop if you're having the same issue as @zhiweifang? Different users running the processes?

I see you have granted 775 permissions to the folders, but if you're having the same issue and the (other) user doesn't belong to the ubuntu group it still won't have write permissions.

---

I know this might be a bit too much, but could you please describe the steps taken from the beginning? Is this your PC with Ubuntu, or did you get a virtual/hosted machine for it? How was docker installed?

I'm actually quite curious to understand how this can happen. I've run this image several different installations of Debian, Ubuntu and MacOS and never faced such problem 😔 If I can find a way to reproduce it then I can look into a "fix" for it, be it changing something in the docker images/setup or in the documentation.

[mathielo]

By the way, if you change permissions to 777 as @zhiweifang did the server should be able to start, save and apply mods properly. I know this is not a proper solution but it's a way for you to be able to play while we figure it out 🤓

[X-lem]

@mathielo Took a snapshot of htob. I appears that the ubuntu user is launching the DST server. I don't understand why as I'm running the docker-compose up -d command with the super user regis. Ubuntu is the user I believe automatically created by AWS when you launch an instance (I'm assuming the Ubuntu group is auto created too as I didn't create it). I'm not sure if it's a super user or not. Should I assign regis to the Ubuntu group as well then?

HTOP results

---

Order of operation

I essentially followed the docs when I set the server up. I'll post below exactly what I did. I'm running a EC2 AWS instance for this. Specifically ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20180912 (ami-0bbe6b35405ecebdb)

sudo apt-get install git

I created my own superuser.

adduser regis
usermod -aG sudo regis

Then I switched to the user

su regis

Followed the Ubuntu guide for installing Docker.

sudo apt-get update

    sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Verified that I had the key with fingerprint. The expected results were printed out.
apt-key fingerprint 0EBFCD88

   sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

created docker group

sudo groupadd docker
sudo usermod -aG docker regis

logged out and back in.

Ran docker run hello-world

I think at this point it said I needed to install docker compose. I believe it suggested to run

sudo install docker-compose

Reran the docker run hello-world and it printed out the expected results.

Then cloned the git repository

cd ~ && git clone https://github.com/mathielo/dst-dedicated-server.git

I deleted the DSTClusterConfig folder and replaced it with my own.

Ran sudo chown regis dst-dedicated-server/ just to make sure the user had ownership over everything.

That's about it for the process. I obviously added my cluster token as well.

[mathielo]

Hey @X-lem thanks for the detailed response, sorry I couldn't get back at you sooner. I can't see any flaw in the steps, they look correct and should work just fine. Not sure if the issue is because stuff was moved (it shoudn't as you did chown'd it) or if it's an AWS thing.

Should I assign regis to the Ubuntu group as well then?

I guess, then. That should solve the permission issues.

---

On another note, check these tips on dedicated server performance.

When choosing a VPS solution, make sure to get one that does not work based on CPU credits concept. Once the server is running the shards idle at about ~30% CPU usage; when players connect they keep at 90~100% constantly. Therefore pick a service that allows you to use the resources in its full otherwise you'll run into trouble.

It's not advised to use AWS EC2 instances that rely on the "CPU credits" model. This model only works to run applications that have low CPU usage with eventual spikes. The dedicated server uses plenty CPU (specially when there's at least one player connected), so shortly you would exceed your credits and AWS would cap the processing power of your instance.

[X-lem]

@mathielo No worries 😄 I appreciate the help. I'll assign my user to the Ubuntu group as well. If that doesn't work either than 🤷‍♂️ chmod 777 it is I suppose.

I don't believe I'm using a credit model in AWS. I do pay a couple bucks a month for it depending upon usage. The model I'm using isn't a free tier so I don't think they cap it (except for when it reaches the t3.small limit which I have not had a problem with yet).

Edit: Actually, it doesn't look like there is a Ubuntu group... odd.

[thatjames]

@X-lem it's because you created a user.

id regis should return something like uid=1001(regis) gid=1001(regis)

Docker has the user dst mapped to 1000. This is why you are getting permission denied errors: You need to remap the UID and GID bits for the envvars

You need the following in your compose file:

      environment:
      - SHARD_NAME=Master
    # Replace this to run as another user
    # Find a user's gid and uid with id _username_
      - PUID=1000
      - PGID=1000

Do this for both dst_master and dst_caves

@mathielo submitting a pull request to update the compose file

[DavHau]

Hey @zhiweifang, that's usually related to permissions in the files you cloned from git. Is it possible that you either ran sudo git clone (...) OR you cloned it with another user? Run ls -l and see if the ownership of the files matches the user who's running the docker-compose up command.

That happens because Docker uses volumes to persist some files, such as downloaded mods and the generated world and its saves. If the user who's executing the Docker command doesn't have write permission on the project folders, you'll have that permission error.

That's not correct. Which user is used to execute any docker / docker-compose commands is completely irrelevant. Docker uses a root daemon process which does everything for you in the end. The user interacting with the docker daemon has no effect on anything.

The user which is used inside the container is defined inside the Dockerfile or the process running inside the container. With your docker image this user appears to have UID=1000 and GID=1000.
That means the only possible configuration this docker compose project will work is if people checkout the git project on their host system with a user/group matching exactly ID 1000. Anything else will fail.

A possible fix could be to add chown -R 1000.1000 /home/dst/.klei somewhere in the startup script before switching the user to dst
This would also solve #19

[KishanMaurya]

cp: cannot stat '.env.example': No such file or directory

[peanutchun]

I just got the same problem, and I try to add the information about PUID and PGID in the docker-compoe.yml after check the output of id -u & id -g. But the problem was not sovled. Here is the log:

Update state (0x101) committing, progress: 100.00 (1854967154 / 1854967980)
Error! App '343050' state is 0x626 after update job.
cp: cannot create regular file '/home/dst/server_dst/mods/dedicated_server_mods_setup.lua': Permission denied
cp: cannot create regular file '/home/dst/.klei/DoNotStarveTogether/DSTWhalesCluster/Master/modoverrides.lua': Permission denied
cp: cannot create regular file '/home/dst/.klei/DoNotStarveTogether/DSTWhalesCluster/Caves/modoverrides.lua': Permission denied
[00:00:00]: PersistRootStorage is now /home/dst/.klei//DoNotStarveTogether/DSTWhalesCluster/Master/ 
[00:00:00]: Starting Up
[00:00:00]: Version: 305472
[00:00:00]: Current time: Wed Jan  5 18:23:20 2022