I've addressed the problem...not fun. I'm posting as this info may be of some use for others.
Background
I did a pacman update today. This included an update to the kernel (4.12.10-1 -> 4.12.12-1). I rebooted and right after grub loading I was presented with this on my screen (https://i.imgur.com/NijCPXh.jpg). System had failed to start due to a kernel panic saying it couldn't find the init system.
Steps to Resolve
Google search of the error message brought me to Arch Linux forum pages. This Link (https://bbs.archlinux.org/viewtopic.php?pid=1353997#p1353997) and this one (https://bbs.archlinux.org/viewtopic.php?id=192296) both make mention of moving /usr/lib64 out, regenerating init images and then grub-mkconfig. These steps didn't work for me.
I then went looking for the latest reports of problems and found this item (https://bbs.archlinux.org/viewtopic.php?id=229874) in the Arch forums. I checked and indeed I did have the latest glibc (ver 2.26-3) installed. I downgraded glibc to 2.25-7(after removing valgrind dependency), fixed up the steps I did previously(move lib64 link), re-installed filesystem mkinitcpio linux, and then mkinitcpio/grub-mkconfig. All is working again.
I'm getting the suggestion that a bug should be filed in Arch against glibc, but I doubt that would be useful. The minute I tell them I'm using openrc/artix I'll get shut down and/or ignored. But as the OP of the latest post noted, there's a problem in the 2.26-3 patch of glibc that brought my system to a screeching halt on reboot.
Update:
A bug report (https://bugs.archlinux.org/task/55620) has been filed by the same person who is the OP of the forum post which lead to solution for me. This person notes the problem started with the update of glibc from version 2.26-1 to 2.26.2.
Thanks for the report, the suspicious glibc is in [system-testing] right now. Are you booting the arch latest linux kernel or our LTS?
I'm using the latest kernel 4.12.12-1.
I want to believe the observations from Lasse100 (OP of bug and forum post). They were most observant of when things went south. The suggestion to downgrading glibc to 2.25-7 (below his observed point of failure at 2.26-2) did work for me. So there seems to be some truth to this.
Why it happened...that sounds like a release management problem - another form of dependency hell
I am using 4.12 too, from Artix-testing, and it lists as Kernel: 4.12.10-2-zen and there is no newer update.
So, where did you get your 4.12.12-1 from? Is it linux-ck 4.12.12-1 in AUR?
In most cases where I got a confused and panicked kernel in an Arch related installation, it was an incorrectly created grub command. Either grub written by another system or a uuid mixup, or some incorrect syntax of the grub menuentry.
Are you on a multisystem boot environment? Is it possible your kernel was updated and the grub is handled by another system and the entry doesn't match the initramfs?
Just for your info glibc has just been updated to 2.26-4 in the system-testing repo.
No idea if this fixes things as I did not have a problem.
This was an Arch system converted to Artix. Kernel comes from Arch core. grub and mkinitcpio commands were standard. No, this is not a multiboot environment. You're reading too much into this. Please see this (https://bbs.archlinux.org/viewtopic.php?id=229874) link as I posted earlier.
I'm a little confused by the mess in how this happened.
Was glibc updated but not libraries that called it (things like mkinitcpio or similar boot generating software)?
Was there an condition in glibc that boot generating software tripped over/triggered?
shoulder shrugThanks for the heads up. I'm still using system (no testing). I keep an eye on things when this propagates down.
Is the conversion from Arch the same as Arch-OpenRC? If you have an arch or arch based systemd system, can you convert to Artix as easily following the applicable procedure moves?
As
@robin0800 noted above, 2.26-4 hit the repos recently.
A look at the git log (https://git.archlinux.org/svntogit/packages.git/log/trunk?h=packages/glibc) for glibc shows the latest change from two days ago as:
2.26-4: backport patch restoring x86_64 to RPATH search paths
The description in the code commentary makes sense.
Lasse100 (OP on arch thread and bug reporter) has updated to this version and reports things as working. He has marked his thread post as [solved]. I want to believe 2.26-4 fixes the problem. I'll report back if this doesn't work but let's assume for the moment things will go well.
https://artixlinux.org/migrate.php
This is for migrating from systemd. The guide at https://systemd-free.org/news.php is for migrating from arch and manjaro openrc.
welp.. I just spent a couple of hours getting my system back after updating something, and since this topic shows the kernel panic I was getting, I post the problem here:
glibc-2.26-? ( I have -4 )
after trying all steps above and a few others I found, i saw a post somewhere (cant access now since it was on a temporary login but search that error), they suggested downgrading glibc. did that, and I was finally able to boot back into the system and fix things.
so FYI, if you get OPs error after upgrading glibc/linux-lts try downgrading glibc back
Honestly FML
Just got this issue after building my own kernel, spent hours switching between old and new kernels, redoing grub from scratch, manually configuring mkinitcpio and grub, and reinstalling everything kernel and bootloader related. Finally gave up after so many hours with searches only leading me to people who had broken their /lib64 (which I had not)
Decided to do a full rebuild instead from the live CD as I was actually on arch migrated to artix, and now I find this. Great, thanks google, not.
After the 3rd attempt to get Manja-Openrc to Artix which worked, it took about a week to realize how to really clean it up.
With pacman you would get a list of all packages that said that were newer versions of those in the system (artix).
I used either yaourt-gui or pamac because you can see what each of those pkgs's origin was and which was an arix version. You just choose the one from the artix repositories which throws away the later non-artix version. It took a little time but it is only then you know you have a real artix installation.
I had wondered whether Manjaro-Settings gui would work at all in Artix, but the important parts of it didn't. That was the last little bit left of the old into the new. At a friend's pc we had an old Manjaro 17.01 since May-June-update we tried to convert, and it still listed pkgs newer than Artix. After a couple of failures just for the learning experience, we dumped it, installed clean artix and all the non-manjaro pkgs, the old fstab and the old home, and it was running like an ace. Unless you have spent years in customizing conf. files that may or may not work in Artix I doubt very much if the conversion is of any value.
after last nights upgrade I get kernel panic in all three of my installations.
The first error message says - error in line 28 (or is it 20 I have a bad monitor which is hard to read in console) of init
Anyone else?
Sorry, no issue here whatsoever after the update (With all
[testing] repo enabled)
Did you updated AUR packages alongside with it?
I did an upgrade on Friday or Saturday (not sure) and after rebooting I am getting the same kernel panic as the OP ("no working init found"). Not sure how to downgrade glibc as I was under the impression I should use artix-chroot to fix things (based on the post here https://artixlinux.org/forum/index.php?topic=100.msg682#msg682) but that does not seem to b present in the live i3 iso. Is this something that should be there or do I have to use the traditional chroot?
on one I may have but the rest I did with pacman -Syyu
I remember mkinitcpio run during upgrade
system-testing/glibc 2.26-4 (base) [installed] GNU C Library
system-testing/glibc-openrc 20170904-1 (openrc-system) [installed] OpenRC nscd init script
system/glibc 2.25-7 (base) [installed: 2.26-4] GNU C Library
system/glibc-openrc 20170712-1 (openrc-system) [installed: 20170904-1] OpenRC nscd init script
system-testing/elogind 234.4-1 [installed] The systemd project's logind, extracted to a standalone package
system-testing/elogind-openrc 20171001-1 (openrc-system base) [installed] OpenRC elogind init script
system-testing/libelogind 234.4-1 (base-devel) [installed] elogind client libraries
system/elogind 234.2-2 [installed: 234.4-1] The systemd project's logind, extracted to a standalone package
system/elogind-openrc 20170717-1 (openrc-system base) [installed: 20171001-1] OpenRC elogind init script
system/libelogind 234.2-2 (base-devel) [installed: 234.4-1] elogind client libraries
Should I try pacman -U on those?
People on the list told me to revert glibc but I don't understand the reason when this one I have came from system-testing and nobody else seems to be having a problem with it. So what if I did revert, then what? Do I hold back upgrades?
I have no idea how to trouble shoot it. The error comes from a line 28 of init which I am unable to locate.
With my one installation I managed after chroot to it to remove glibc-openrc andleft glibc (2.26-4)
I rebooted into recovery and it seems to have fixed itself.
I tried the same with my main installation and it is not working.
I have no network with chroot and unable to run mkinitcpio or grub-update
The error again says:
/init error line 28 syntax ...
Hmm, I have no idea what the problem is since glibc works quite fine for me. However there is no reason why your network and etc. would stop working with chroot. Are you chrooting properly? I'm just copying these from the Arch Wiki (https://wiki.archlinux.org/index.php/Chroot#Enter_a_chroot) page.
# cd /location/of/new/root
# mount -t proc proc proc/
# mount --rbind /sys sys/
# mount --rbind /dev dev/
# cp /etc/resolv.conf etc/resolv.conf
# chroot /location/of/new/root /bin/bash
Of course you may need to alter that a bit depending on your setup. If you have a separate boot partition, you would need to mount that too since you're messing with grub and all that.
Thank you,
It seems as if when I upgraded not all upgradeable packages had made it to the repository as the one I am on now didn't need these.
Packages (11) glib2-2.54.1-1 glib2-docs-2.54.1-1 libpng-1.6.34-2 meson-0.42.1-3 mkinitcpio-24-1.3
openrc-0.32-1 p11-kit-0.23.9-1 python-gitdb-2.0.3-1 python-gitpython-2.1.7-1
python2-pillow-4.3.0-1 smplayer-17.10.0-1
Is it possible that mkinitcpio run in its previous version with incompatible upgraded stuff and messed everything up?
It worked!
Now the question is that I removed glibc-openrc from both installations wondering if this was the problem and the system works without them? Are they necessary and if not why were they there? Both systems were Artix installations, now I am going to fix the Manjaro conversion which also broke 36hrs ago (same time).
Sort long story .....
In the rare occasion that I would update all three installations at about the same time, which defeats the purpose of having them, and having blind faith on artix-testing, I was caught in the middle of repositories being half-synced. All I did to fix things was continue the half done upgrade process which two days earlier seemed as complete.
The question that remains is the role and reason for glibc-openrc. Is it an initial set of default scripts for OpenRC during installation and after that it is useless? Nothing was affected (AFAIK) but I am willing to investigate further. It did not seem as being a dependency of anything. Maybe it is transferred from the live image and remains as an orphan.
Ramdisk created with mkinitcpio 24-1* cannot mount artix root btrfs subvolume (neither automatically, nor with manual running modprobe btrfs+mount ...). Does someone else have same problem?
I didn't even know that package existed until you pointed it out. I don't have it installed on any of my machines. I suppose it's orphaned. Not sure what the purpose is either.
glibc-openrc is the name service cache daemon (nscd) it can be added to auto start "sudo rc-update add nscd default" but what it does I don't know. "sudo nscd -g" shows what is set and what its doing. Not sure its needed though?
It starts and shutsdown the Name Service Cache Daemon as robin says. Same here, I can''t figure out the difference with or without it.
#!/usr/bin/openrc-run
# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
depend() {
use dns ldap net slapd
}
checkconfig() {
if [ ! -d /run/nscd ] ; then
mkdir -p /run/nscd
chmod 755 /run/nscd
fi
if [ -z "${NSCD_PERMS_OK}" ] && [ "$(stat -c %a /run/nscd)" != "755" ] ; then
echo ""
ewarn "nscd run dir is not world readable, you should reset the perms:"
ewarn "chmod 755 /run/nscd"
ewarn "chmod a+rw /run/nscd/socket"
echo ""
ewarn "To disable this warning, set 'NSCD_PERMS_OK' in /etc/conf.d/nscd"
echo ""
fi
}
start() {
checkconfig
ebegin "Starting Name Service Cache Daemon"
local secure=`while read curline ; do
table=${curline%:*}
entries=${curline##$table:}
table=${table%%[^a-z]*}
case $table in
passwd*|group*|hosts)
for entry in $entries ; do
case $entry in
nisplus*)
/usr/bin/nscd_nischeck $table || \
/echo "-S $table,yes"
;;
esac
done
;;
esac
done < /etc/nsswitch.conf`
local pidfile="$(strings /usr/bin/nscd | grep nscd.pid)"
mkdir -p "$(dirname ${pidfile})"
save_options pidfile "${pidfile}"
start-stop-daemon --start --quiet \
--exec /usr/bin/nscd --pidfile "${pidfile}" \
-- $secure
eend $?
}
stop() {
local pidfile="$(get_options pidfile)"
[ -n "${pidfile}" ] && pidfile="--pidfile ${pidfile}"
ebegin "Shutting down Name Service Cache Daemon"
start-stop-daemon --stop --quiet --exec /usr/bin/nscd ${pidfile}
eend $?
}
# vim:ts=4
# Begin /etc/nsswitch.conf
passwd: compat mymachines
group: compat mymachines
shadow: compat
publickey: files
hosts: files mymachines resolve [!UNAVAIL=return] dns myhostname
networks: files
protocols: files
services: files
ethers: files
rpc: files
netgroup: files
# End /etc/nsswitch.conf
After you add it check to see what services OpenRC runs
$ sudo rc-update show -a