Skip to main content ↑ ↓
Topic: early-fs-fstab.target and fsck failed (Read 1283 times) previous topic - next topic
0 Members and 4 Guests are viewing this topic.

Re: early-fs-fstab.target and fsck failed

Reply #15 –
Quote from: [email protected] – on
I noticed that the random-seed service is also failed; I started it (and it remains active).
After that I manually started the early-fs-fstab.target service; it did not bail out, but it did not finish starting either :

The command
Code: [Select]
dinitctl start early-fs-fstab.target
hangs indefinitely.


It is waiting on the fsck service, which is configured to run on the console (and the console is busy as the login service is running on it). In general you cannot stop and start critical system services like this; the Artix configuration (perhaps incorrectly) allows you to stop early-fs-fstab.target without bringing the whole system down, but you won't be able to start it again. Reboot the system to do so. (That doesn't help if the service is failing to start, of course, but I can't help with that issue I'm afraid).

Re: early-fs-fstab.target and fsck failed

Reply #16 –
Quote from: davmac – on

It is waiting on the fsck service, which is configured to run on the console (and the console is busy as the login service is running on it). In general you cannot stop and start critical system services like this; the Artix configuration (perhaps incorrectly) allows you to stop early-fs-fstab.target without bringing the whole system down, but you won't be able to start it again. Reboot the system to do so. (That doesn't help if the service is failing to start, of course, but I can't help with that issue I'm afraid).


Thanks for clearing that up.

The curious thing is that if I run the command corresponding to the early-fs-fstab.target service, from the root console, it executes just fine.

Could it be that some rights and/or environment variables are not yet as they are expected when the service is started, which leads to the failure ?

It would be of tremendous help to be able to "dump" all the contents of the context in which the service is executed (that includes the environment variables, the rights, and the available services).
I suppose there is no way to make dinit to tell us all the detail, right ? Something like a special flag that lists in a trace file everything happening in dinit ...

Re: early-fs-fstab.target and fsck failed

Reply #17 –
Quote from: [email protected] – on
It would be of tremendous help to be able to "dump" all the contents of the context in which the service is executed (that includes the environment variables, the rights, and the available services).
I suppose there is no way to make dinit to tell us all the detail, right ? Something like a special flag that lists in a trace file everything happening in dinit ...


Well you can change the executable ("command = ") to something that dumps whatever information you want.

It seems to me like a better place to start would be just to configure some kind of logging for the "early-fs-fstab.target" service. The service description doesn't currently specify a "log-type" or "logfile" at all.

Re: early-fs-fstab.target and fsck failed

Reply #18 –
Quote from: davmac – on
Well you can change the executable ("command = ") to something that dumps whatever information you want.

It seems to me like a better place to start would be just to configure some kind of logging for the "early-fs-fstab.target" service. The service description doesn't currently specify a "log-type" or "logfile" at all.

Well, it did crap out immediately - because the file-system is not yet ready and the log file cannot be written.
Code: [Select]
type       = scripted
command    = /usr/bin/mount -a -t nonfs,nonfs4,nosmbfs,nocifs,nocodafs,noncpfs,noshfs,nofuse,nofuseblk,noglusterfs,nodavfs,nofuse.glusterfs -O no_netdev
restart    = false
depends-on = early-fs-pre.target
depends-ms = fsck
waits-for  = early-root-rw.target
# added log file specification
logfile    = /var/log/dinit/early-fs-fstab.target.log

My /var is mounted on another partition - which was not yet mounted when dinit tried to open the log file, because early-fs-fstab.target was not yet processed.

Code: [Select]
[  OK  ] early-root-rw.target
dinit: early-fs-fstab.target: execution failed - opening log file: No such file or directory
[FAILED] early-fs-fstab.target

I will try with
Code: [Select]
logfile    = /root/early-fs-fstab.target.log
to see what it gives ...
 

Re: early-fs-fstab.target and fsck failed

Reply #19 –
Well, the plot thickens: now that the log has been created (/root/early-fs-fstab.target.log), it says:
Code: [Select]
mount: /boot/efi: mount point does not exist.
       dmesg(1) may have more information after failed mount system call.

The path /boot/efi exists but is is not mounted - even once the boot process is finished; /boot/efi is mounted from a FAT32 partition (512Mb in size). It can be mounted manually (as root #> mount /boot/efi )

/boot itself is mounted on a ext4 partition (768Mb in size), and it appears mounted once the boot process is finished.

Since I am not the only one encountering this issue, there are some questions:
  • Is there some wrong assumption in the way the targets have been defined and dinit cannot resolve all of them correctly during boot time ?
  • Is there a limitation in the way dinit uses system resources when booting ? I'm thinking "it misses some information or tries too early to obtain some information" kind of thing ...

So far, the file early-fs-fstab.target was pristine; Artix was installed not too long ago from the latest ISO available.
Have we reached a limitation / bug in :
a. the way Artix was configured
or
b. the way dinit handles the targets at boot time

What can I do to provide additional useful information / help ?

Re: early-fs-fstab.target and fsck failed

Reply #20 –
Quote from: [email protected] – on
Well, the plot thickens: now that the log has been created (/root/early-fs-fstab.target.log), it says:
Code: [Select]
mount: /boot/efi: mount point does not exist.
       dmesg(1) may have more information after failed mount system call.

The path /boot/efi exists but is is not mounted - even once the boot process is finished; /boot/efi is mounted from a FAT32 partition (512Mb in size). It can be mounted manually (as root #> mount /boot/efi )

/boot itself is mounted on a ext4 partition (768Mb in size), and it appears mounted once the boot process is finished.

At this point I'd guess that the attempt to mount /boot/efi is made before /boot itself is mounted.

Quote
Is there some wrong assumption in the way the targets have been defined and dinit cannot resolve all of them correctly during boot time ?

It's just the "mount" command that is failing, so I'm guessing that's the culprit. With a quick experiment I found that listing multiple mount points in /etc/fstab where an earlier mount point requires a later one to be mounted first, does indeed cause "mount -a" to fail. If I reverse the order then it's fine.

Check your /etc/fstab and see if you have /boot/efi listed before /boot itself.

Quote
Is there a limitation in the way dinit uses system resources when booting ? I'm thinking "it misses some information or tries too early to obtain some information" kind of thing ...

Dinit's just a service manager; it just does what it is told. It's really not trying to have any higher level control of how the system works or what happens where. That's by design; it's supposed to be comprehensible. You configure it by writing service descriptions and it runs the commands in those according to the order required by the dependencies; there's no additional magic.

Quote
Have we reached a limitation / bug in :
a. the way Artix was configured
or
b. the way dinit handles the targets at boot time

I'm inclined to say it's a bug in the "mount" utility which should be doing some basic analysis to check whether one mount point requires another to be processed first. But that's arguable, I guess. However, I don't think dinit is doing anything wrong here.

Re: early-fs-fstab.target and fsck failed

Reply #21 –
Quote from: davmac – on
At this point I'd guess that the attempt to mount /boot/efi is made before /boot itself is mounted.

It's just the "mount" command that is failing, so I'm guessing that's the culprit. With a quick experiment I found that listing multiple mount points in /etc/fstab where an earlier mount point requires a later one to be mounted first, does indeed cause "mount -a" to fail. If I reverse the order then it's fine.

Check your /etc/fstab and see if you have /boot/efi listed before /boot itself.

You are right !

The fstab /boot/efi entry was indeed before the /boot entry (as they were put by the install procedure).

Having swapped the two lines, on reboot there is no longer an error for the early-fs-fstab.target service, which now is being displayed as it should:
Code: [Select]
...
[{+}     ] early-root-rw.target
[{+}     ] fsck-root
[{+}     ] early-fs-fstab.target
[{+}     ] fsck
[{+}     ] early-console.target
[{+}     ] early-keyboard.target
...

Phew, there were some head scratches with this one !

So far, dinit does indeed do its job; if only it could talk a little bit more ...  ;)

Thank you very much for the analysis and the help provided !

Re: early-fs-fstab.target and fsck failed

Reply #22 –
Related to the problem discovered: this article gives us a potential solution: sorting fstab by mountpoint.
Perhaps it is worth considering to include a similar command in the target command line ?

The following command will alphabetically sort all fstab entries:
Code: [Select]
grep "[[:space:]]/" /etc/fstab | sort -k2,2

Re: early-fs-fstab.target and fsck failed

Reply #23 –
Quote
The fstab /boot/efi entry was indeed before the /boot entry (as they were put by the install procedure).

Having swapped the two lines, on reboot there is no longer an error for the early-fs-fstab.target service, which now is being displayed as it should
Same here. Thank you.