Hi there,
I’ve made a clean install using base ISO and everything worked fine. So nothing was supposed to be broken but today I just tried rebooting my laptop and got the following:
S6-log: warning: unable to rename /var/log/udevd/current to /var/log/udevd/previous: no such file or directory
The message above was continuously repeated line by line and prevented the system from rebooting. I had to shut down the laptop manually. Could you please give me a hint of what the issue might be? Could anything be wrong with syslog-ng service?
This is udevd not syslog-ng. Unfortunately, I have never seen this error before and have no idea what it means. To my knowledge, s6-log doesn't rename the current file to previous. I've never heard of this behavior. All I can say is make sure that your /var/log/udevd/ directory exists and that the s6log user has appropriate permissions to it (it should though). You can use tty7 if need be for debugging.
There’s no /var/log/udevd/ directory and I’m not sure if it should exist.
It is absolutely supposed to exist.
I seemed to have forgotten to add a user to S6log group. Maybe that omission caused such a problem.
The s6log user is a part of the s6log group. You shouldn't have needed to add a user to anything. This is supposed to happen when you install s6.
I can’t realise what caused the problem then. Maybe some kind of a bug.
Just chiming in that I had this exact issue also but I rectified it by making the directory /var/log/udevd/ and assigning ownership of it to the s6log user and group
This is bizarre. The script is supposed to make this directory. The only reason it would not is if /var/log isn't writeable for some reason (but remount-root is supposed to do that).
I didn’t have the directory /var/log/udevd/ created upon installation. Could it be due to separate subvolume for /var/log being created?
The only directory I have is /run/service/udevd-log. I found there a file “run.user” containing the following:
“#! /bin/execlineb -p
envfile /etc/s6/config/udevd. conf
importas -sCUD "n3 s2000000 T' DIRECTIVES DIRECTIVES
ifelse { test -w /var/log } {
foreground { install -d -o slog -g s6log /var/log/udevd }
s6-setuidgid s6log exec -c s6-log -d3 -b -- ${DIRECTIVES} /var/log/udevd
}
foreground { install -d -o s6log -g s6log /run/log/udevd }
s6-setuidgid s6log exec -c s6-log -d3 -b ${DIRECTIVES} / run/log/udevd”
It looks like the directory should be created automatically, but it never has been in my case.
Could you please advise me what kind of permissions I should have and how to check it? I’m stuck with this matter.
Upd: tried booting into live session from Plasma-ISO and made sure that “var/log/udevd” directory existed with appropriate permissions assigned to s6log (when running live), but after installation using Calamares (I normally do a base installation via terminal) it would not be created.
The actual service files should be in /etc/s6/sv.
The only reason I can think of for it failing would be that the "test -w /var/log" test always fails (meaning you never have read/write access to /var/log). I have no idea why this would happen. Do you have this problem with any other *-log service? udev isn't the only one.
Yep, I’ve got the udevd-srv and udevd-log in /etc/s6/sv. For example, lvm2-monitor with ownership of s6log does exist.
Maybe it will give you a clue:
I’ve noticed strange system’s behaviour: if ext4 is chosen upon fresh installing, all the necessary directories will be created, including “var/log/udevd” and “var/log/NetworkManager”, etc. Unfortunately, the same is not true for BTRFS being used.
The following are BTRFS mount options set:
mount -o subvol=/@,rw,noatime,compress=zstd,ssd,space_cache=v2 /dev/nvme0n1p2 /mnt
mount -o subvol=/@home,rw,noatime,compress=zstd,ssd,space_cache=v2 /dev/nvme0n1p2 /mnt/home
mount -o subvol=/@cache,rw,noatime,compress=zstd,ssd,space_cache=v2 /dev/nvme0n1p2 /mnt/var/cache
mount -o subvol=/@log,rw,noatime,compress=zstd,ssd,space_cache=v2 /dev/nvme0n1p2 /mnt/var/log
I reckon that the separate subvolume for “var/log” could cause the problem, but this is the way Calamares does.
In theory, /var/log being on a separate partition should already be handled. What is supposed to happen is that udevd will initially log itself in /run instead (which will always be there because of s6-linux-init). After all the filesystems mount and root is remounted, all the logging services get a restart, if needed, to transition the logging into /var/log instead.
Sounds good, but something went wrong in my case :) I suppose S6 is well-designed and forward-looking system, but I get frustrated when such errors occur. Anyway, if you say that s6-linux-init should be working as intended, then I need to look into my issue.
Thank you!
Hi,
Just tried installing on BTRFS without a separate subvolume for “var/log” being created, and everything worked perfectly. It seems that something wrong is going to happen when additional subvolumes are added.
I am having the exact same issue. First time on Artix, coming from Endeavour and Arch. Not really up to par with linux commands and nkow how, as most of the people here, I'm sure. I was wondering if there was a way to fix this issue or if there are commands I can use to help everyone figure this out. My install was plasma-s6 on btrfs system, I believe with subvolumes (is that LVM?). I have rebooted a few times since install but this last time was the only time I saw this message. Thanks for making a non systemd linux an actual choice for people! Much appreciated!
I guess that means there must be some sort of race condition but that only happens on BTRFS and only if you have /var/log on a separate subvolume. I'm not sure why this would occur though. I'm not a BTRFS user, but I assume the existing mount commands/services are sufficient for mounting everything? The startup scripts do wait on the mount to finish.
Edit: if either of you want to try some hacks, try changing your /etc/s6/sv/remount-root/up file to this:
#!/bin/execlineb -P
foreground { mount -o remount,rw / }
foreground { sleep 1 }
foreground { sh /etc/s6/log-service-reload.sh }
Then reload the database with s6-db-reload as root. It might possibly make a difference? Another possibility is that /var/log is always mounted read-only for you guys I suppose although you should have problems with other services not just udevd.
I just reboot again this morning while configuring CUPs (another absolute joy, not). I did not have the same errors, it just reboot very quickly with very little output.
EDIT. Just tried a reboot again. Total reboot form multiple applications and windows open, ~20s to get back to the login screen on Plasma. No errors. Not sure what that was all about before.
Just experimented. If I reboot using the apllication menu with GUI, there's no errors. If i reboot using cli command sudo reboot, then I see the error, and it's just a loop, it will never fully reboot. This happens on linux kernel, lts kernel and zen kernel. Same thing.
You should be able to switch to tty7 when it fails and do what you need at least. In any case, it sounds very bizarre that there's a difference between the cli and whatever the GUI menu does under the hood (loginctl reboot maybe?)
Sorry to dig this back up. I think it is related, but I'm also, apparently randomly, having the same issue.
I'm however on a zfs installation.
The /var/log is a separate zfs filesystem.
Everything is working, just the random warning: unable to rename /var/log/udevd/current appearing in a loop, preventing me to shutdown via a poweroff.
To mount all filesystems I've put /bin/zfs mount -a in /etc/s6/rc.local.
`/boot` partition gets mounted via fstab.
You're getting this on shutdown? Are you unmounting /var/log or something yourself with a down script?
If it happens, it happens as soon as I get on tty1 on power-on.
It still lets me log in; if, however, the s6-log udevd error message appeared there it will prevent me from powering down.
A poweroff kills the current sway instance and remains on tty1 waiting indefinitely while the s6-log messages keep getting printed.
I'm not unmounting /var/log myself, should I?
Definitely no. I'm struggling to understand how it could affect anything during shutdown. Does the /var/log/udevd/current message just simply always print itself and never stop?
That's correct. I can attach a video, somehow, if you want.
Wait sudden idea. What if you simply add rc-local as a dependency to remount-root? Does that fix your issue?
touch /etc/s6/sv/remount-root/dependencies.d/rc-local
s6-db reload
This appeared to have solved the issue. Thanks
edit: it happened again, I don't seem to understand the pattern
Hi there,
I have recently encountered the same race condition with a separate BTRFS subvolume fstab mounted to /var/log.
I found out unexpectedly my grub mounts the BTRFS rootfs in rw mode instead of ro. So at the early boot stages /var/log appeared to be writable also, it's just an empty not mounted directory. S6 happily initializes log services there, bypassing /run/log. /var/log is mounted to BTRFS subvol after all (at mount-filesystems/up), burying all logs. As a result, I have neither the /var/log/udev directory, nor /run/log/udev, same for all other sv logs. This makes s6-log crazy, endless looping on reboot.
It is clear that several conditions have come together here, but not so rare imho. s6 still needs to be improved. In the meantime, just check your bootloader configuration to ensure classical ro / mounting.
Aha, that explanation makes sense. If you have /var available at that point in time, then /var/log will be writeable. Unfortunately, I'm not really sure how one would distinguish between between /var/log really being available then vs needing to wait on another partition to mount. Maybe I need to give in and just make this a configurable option somehow.
Sorry for the delay on this one but it just occurred to me, what if you change /etc/s6/sv/udevd-log/run to the following and then recompile the database? Does that solve it for people that have this problem?
#!/bin/execlineb -P
envfile /etc/s6/config/udevd.conf
importas -sCuD "n3 s2000000 T" DIRECTIVES DIRECTIVES
ifelse { test -w /var/log/old } {
foreground { install -d -o s6log -g s6log /var/log/udevd }
s6-setuidgid s6log exec -c s6-log -d3 -b -- ${DIRECTIVES} /var/log/udevd
}
foreground { install -d -o s6log -g s6log /run/log/udevd }
s6-setuidgid s6log exec -c s6-log -d3 -b -- ${DIRECTIVES} /run/log/udevd
Hello!
error
10 minutes after turning on and loading the system, an endless loop begins on the first console ctrl+alt+F1:
s6-log: warning: enabled to rename /var/log/syslog-ng/current to /var/log/syslog-ng/previous: No such file or directory
Shutting down and rebooting is not possible. Only reset
Solution. It turned out like this:
#mkdir /var/log/syslog-ng
#touch /var/log/syslog-ng/current
#touch /var/log/syslog-ng/previous
#chown -R s6log:s6log /var/log/syslog-ng
Next, the /var/log/syslog-ng/previous file is changed by the system itself to /var/log/syslog-ng/
4000000065216bde27fa108a.s or something similar.
Reboot normally after a minute.
This helped on the following builds:
artix-lxqt-s6-20230814-x86_64.iso
artix-plasma-s6-20230814-x86_64.iso
artix-plasma-s6-20231002-x86_64.iso
The following system works fine initially, no error:
artix-mate-s6-20231002-x86_64.iso
Question: is this a good solution? Could something else be broken?
p.s. sorry for the translation...)))
Hi there,
S6 and S6-rc packages appear to be updated, but unfortunately Skarnet hasn’t provided a changelog of work done. I’m keen that such annoying bug should be fixed, so has anybody tested it?
Would be grateful for your feedback.
Just tried myself. Everything works as intended.
Thank you!
Just had the same issue with /var/log on separate btrfs subvolume(Another disk even) on a fresh install. For me the solution seems to be ro mount at boot time.
Would it be possible for you to test what I posted here (https://forum.artixlinux.org/index.php/topic,5193.msg36321.html#msg36321)? I don't have a system I can test so it would be helpful to know if that actually does the trick.