Artix Linux Forum

Init systems => S6 => Topic started by: dnkl on 08 December 2019, 19:56:14

Title: dnsmasq dependency on dbus?
Post by: dnkl on 08 December 2019, 19:56:14
With the default dnsmasq s6 scripts, some boots result in an endless stream of

> Dec  8 19:27:22 [daemon|warning] dnsmasq[1129]: DBus error: Connection ":1.0" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file

This doesn't stop until I restart dnsmasq. I'm thinking this has to do with dnsmasq being started with --enable-dbus?

> exec dnsmasq -k --enable-dbus -u dnsmasq -g dnsmasq

Doesn't that imply a dbus dependency? I did try to add a 'dependencies' (with dbus in it...) file under /etc/s6/sv/dnsmasq (and rebuilt the database), but it didn't seem to help. Not sure if the problem isn't that dnsmasq is being started before dbus, or if I didn't add the dependency correctly...
Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 08 December 2019, 23:35:27
Someone noted this same behavior as well in this thread (https://forum.artixlinux.org/index.php/topic,1201.msg8591/topicseen.html#msg8591). A workaround for now is:

Quote
2. If running 'dnsmasq', you'll find hundreds of Dbus error messages in the console log. These can be eliminated by editing /usr/share/dbus-1/system.d/dnsmasq.conf to have an extra 'allow' policy for user 'dnsmasq' (don't delete the 'root' one).

Now as to whether or not dbus should be an explicit dependency on the dnsmasq server. If I'm interpreting the man page (http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html) correctly, I think the answer is "no" here. Like Arch, Artix builds dnsmasq with dbus support and launches it with --enable-dbus by default, but that option doesn't require the use of dbus. It just uses dbus if the daemon can be found on the system.

The default dnsmasq conf should be changed though. I believe running the service as the dnsmasq user/group is correct and makes sense. Therefore, the default configuration should reflect this to avoid spamming a bunch of errors. I need to double check how the other inits handle this. They might use some conf settings within the init script itself or something.
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 09 December 2019, 18:49:22
Ah, I somehow missed that while reading through the other threads here. Sorry about that.

I checked an OpenRC-based artix installation, and dnsmasq runs as the dnsmasq user/group, so that shouldn't be changed. I didn't see a direct, explicit dependency on dbus, but perhaps there's an implicit one. I didn't see any other command line options either.

Perhaps dnsmasq is the wrong place to look? Maybe dbus is where the problem is?
Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 09 December 2019, 20:14:44
I checked runit and it doesn't give me that error when I enable the service (it connects nicely to dbus). This means that something in s6 is misconfigured/not-configured the same way. The " /usr/share/dbus-1/system.d/dnsmasq.conf" file doesn't set any policy users other than root by default, so it must be something else that grants permissions (not sure what).
Title: Re: dnsmasq dependency on dbus?
Post by: nix-user on 10 December 2019, 15:10:07
so it must be something else that grants permissions (not sure what).
Hi, Dudemanguy and everybody,  I think any text editor is a great solution to this problem.
Code: [Select]
[tester@vm-test ~]$ cat /etc/passwd | grep dns
dnsmasq:x:979:979:dnsmasq daemon:/:/sbin/nologin
[tester@vm-test ~]$ cat /etc/group | grep dns
dnsmasq:x:979:
Look at my file:
Code: [Select]
[tester@vm-test ~]$ cat /usr/share/dbus-1/system.d/dnsmasq.conf
It's just work on my runit mashine:
Code: [Select]
<!DOCTYPE busconfig PUBLIC
 "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
<policy user="root">
<allow own="uk.org.thekelleys.dnsmasq"/>
<allow send_destination="uk.org.thekelleys.dnsmasq"/>
</policy>
        <policy user="dnsmasq">
<allow own="uk.org.thekelleys.dnsmasq"/>
<allow send_destination="uk.org.thekelleys.dnsmasq"/>
</policy>
<policy context="default">
                <deny own="uk.org.thekelleys.dnsmasq"/>
                <deny send_destination="uk.org.thekelleys.dnsmasq"/>
        </policy>
</busconfig>

Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 10 December 2019, 19:03:52
Yes, editing that file does work as noted above. However on runit, you don't need to edit that file in order to get a working dbus connection. It works with the default below:
Code: [Select]
<!DOCTYPE busconfig PUBLIC
 "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
<policy user="root">
<allow own="uk.org.thekelleys.dnsmasq"/>
<allow send_destination="uk.org.thekelleys.dnsmasq"/>
</policy>
<policy context="default">
                <deny own="uk.org.thekelleys.dnsmasq"/>
                <deny send_destination="uk.org.thekelleys.dnsmasq"/>
        </policy>
</busconfig>

I haven't figured out why you don't need to edit the file just yet. Of course, I haven't tried openrc yet. Maybe that one has the same error message.
Title: Re: dnsmasq dependency on dbus?
Post by: nous on 10 December 2019, 19:44:00
I remember having seen the dbus error on openrc some time in the past, although I don't remember what exactly I had edited to fix it. A quick google search reveals same reports from other distributions too, including Redhat.
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 10 December 2019, 20:39:13
Hmm, interesting. I've been running three OpenRC-based installation (two of which now are s6...) and don't remember ever seeing it. The fact that it doesn't happen every time, with s6, is also weird.

In a way, it's strange that it works at all, given that the policy only allows "root", while we run dnsmasq with the dnsmasq user/group...
Title: Re: dnsmasq dependency on dbus?
Post by: nix-user on 10 December 2019, 20:41:09
Dudemanguy
OK, here is log on my runit mashine after reboot with default /usr/share/dbus-1/system.d/dnsmasq.conf:
Code: [Select]
[tester@vm-test ~]$ tail -f /var/log/socklog/daemon/current
daemon.warn: Dec 10 22:20:09 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:09 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:09 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:10 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:10 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:10 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:10 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:11 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
daemon.warn: Dec 10 22:20:11 dnsmasq[1065]: DBus error: Connection ":1.1" is not allowed to own the service "uk.org.thekelleys.dnsmasq" due to security policies in the configuration file
and so on ....
I don't understand, why we talking about runit, s6, OpenRC and what else? systemd? (oh, no). It's not a dependency problem at all. User should change the default config /usr/share/dbus-1/system.d/dnsmasq.conf
sorry, English is not my native lang
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 10 December 2019, 20:45:32
A debian bug report says the policy should allow both root and dnsmasq:
Quote
You should have a file /etc/dbus-1/system.d/dnsmasq.conf which
configures the system dbus-daemon to allow either root or dnsmasq to own
the name uk.org.thekelleys.dnsmasq; it's part of dnsmasq-base.
Please reinstall dnsmasq-base if necessary.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=773001#15
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 10 December 2019, 20:50:07
Finally, if I read the default systemd script from the dnsmasq package correctly, it is run as root:root there.
Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 10 December 2019, 22:03:19
Dudemanguy
OK, here is log on my runit mashine after reboot with default /usr/share/dbus-1/system.d/dnsmasq.conf:

Well that settles it in my mind then. I don't know why I'm not able to see those errors, but maybe my logging is not configured correctly in runit (wouldn't be the first time).

Quote
I don't understand, why we talking about runit, s6, OpenRC and what else? systemd? (oh, no).

Apparently this error isn't always seen (only sometimes) which made me wonder if I just was doing something wrong in s6. It's probably safe to assume that all init systems have this error even if logs don't always catch it for some reason.

Finally, if I read the default systemd script from the dnsmasq package correctly, it is run as root:root there.

Arch runs it as dnsmasq:dnsmasq. I'll go with that. It seems like a dnsmasq 2.80-5 package needs to be made here.
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 11 December 2019, 11:11:37
I looked at the dnsmasq sources to try to find out why it works sometimes at boot, and/or with other init systems, and always when I manually restart it on an already booted system.

And it soon became obvious. When initially started, it runs as root:root. As part of it's initialization, it drops privileges to the dnsmasq user.

But before that, it does its first dbus connection attempt. At this point we're still root and the default policy works. If this works, we're all good. No log spamming.

If it fails (because dbus either hasn't started, or just started but isn't accepting connections yet), it will retry later. But, when it does, it will have dropped the root privileges. And now the default policy prevents the connection. This is where it starts spamming the log, because it will keep retrying over and over again.

So in a way, this is a dependency problem. Or a configuration issue. It depends on where you're coming from I guess. It can also be seen as a bug in dnsmasq.

S6 has no way of knowing when dbus is ready; as soon as the process has been started, it considers the service ready. This explains why adding a dbus dependency to dnsmasq didn't work.
Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 11 December 2019, 15:43:54
So wait, you're telling me that dbus doesn't accept connections immediately upon starting the daemon? Yikes, this sounds like bad dbus design to me actually. :P

Thanks for investigating this. I think just having dnsmasq be a part of the configuration by default is the best and cleanest way to solve this. I don't think anybody wants to add "sleep 1" hacks into the init scripts. That brings back all the crappy things about sysvinit. I've pushed dnsmasq-2.80-5 to testing right now (which comes with dnsmasq as a valid user in the default configuration). I'll move it to the world repo later.
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 11 December 2019, 16:44:18
Quote
So wait, you're telling me that dbus doesn't accept connections immediately upon starting the daemon?
From a human perspective it does. But it has to execute X number of instructions to get there. This creates a small window where the process is running but the service hasn't fully started up. More or less all services have this problem.

So, it's a race; sometimes dnsmasq is started and tries to connect to dbus before it is accepting connections. Other times dbus wins the race.

PID-file based service managers are actually better off since they effectively let the services decide when they've started. Dbus, for example, can hold off with writing its PID-file until after it has started accepting connections.

Quote
I don't think anybody wants to add "sleep 1" hacks into the init scripts.
God no!

Quote
I think just having dnsmasq be a part of the configuration by default is the best and cleanest way to solve this
Agreed.
Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 11 December 2019, 17:12:44
From a human perspective it does. But it has to execute X number of instructions to get there. This creates a small window where the process is running but the service hasn't fully started up. More or less all services have this problem.

So, it's a race; sometimes dnsmasq is started and tries to connect to dbus before it is accepting connections. Other times dbus wins the race.

PID-file based service managers are actually better off since they effectively let the services decide when they've started. Dbus, for example, can hold off with writing its PID-file until after it has started accepting connections.

Hmm, I thought s6's dependency management is supposed to magically solve this problem in a race-free way. Admittedly, I don't know how it really works
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 11 December 2019, 19:03:05
It might be possible to do something with s6's variant of socket activation, but I'm not familiar enough with s6 to say how, it if it even can be done.
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 12 December 2019, 17:42:11
When I get some free time I'll play around with this: https://skarnet.org/software/s6/notifywhenup.html
Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 12 December 2019, 19:26:42
Quote
s6 supports that mechanism natively: when the service directory for the daemon contains a valid notification-fd file, the daemon's supervisor, i.e. the s6-supervise program, will properly catch the daemon's message, update the status file (supervise/status), then notify all the subscribers with a 'U' event, meaning that the service is now up and ready.

Interesting. So it sounds like a patch doing that for dbus would completely solve the problem. Not sure how easy/difficult that would be of course.
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 13 December 2019, 15:28:54
Preliminary result: this looks like it was a lot easier than I had anticipated.

I'm piggybacking on dbus' --print-pid=fd option. I figured that it writes the PID to this fd at the same time it creates the PID file, which should be after it is done initializing.

For a "real" pipe (not stdout or stderr) it closes the fd right after writing to it (verified by strace:ing). Thereby fulfilling s6's requirements for a notification-fd.

So, here's what I did:

* Revert the dbus policy (i.e. only allow root)
* Added /etc/s6/sv/dbus/notification-fd, with the content '3'
* Added --print-pid=3 to /etc/s6/sv/dbus/run
* Added dbus as a dependency to dnsmasq

5-6 reboots later I haven't seen a single dbus connection failure.
Title: Re: dnsmasq dependency on dbus?
Post by: dnkl on 13 December 2019, 15:52:35
Quote
* Revert the dbus policy (i.e. only allow root)
Note that I still thinks it is a good idea to have the dnsmasq user allowed in the policy since we are running as that user. I reverted it just to verify the notification-fd worked.
Title: Re: dnsmasq dependency on dbus?
Post by: Dudemanguy on 13 December 2019, 16:20:02
Preliminary result: this looks like it was a lot easier than I had anticipated.

I'm piggybacking on dbus' --print-pid=fd option. I figured that it writes the PID to this fd at the same time it creates the PID file, which should be after it is done initializing.

For a "real" pipe (not stdout or stderr) it closes the fd right after writing to it (verified by strace:ing). Thereby fulfilling s6's requirements for a notification-fd.

So, here's what I did:

* Revert the dbus policy (i.e. only allow root)
* Added /etc/s6/sv/dbus/notification-fd, with the content '3'
* Added --print-pid=3 to /etc/s6/sv/dbus/run
* Added dbus as a dependency to dnsmasq

5-6 reboots later I haven't seen a single dbus connection failure.

This is great! I didn't know dbus had such a handy option. Not only would dnsmasq benefit from this, but so does every other service that depends on dbus (which is quite a few of course). Doing this would make dnsmasq-s6 have a hard dependency on dbus-s6, but I think at this point, it's probably OK.