Hello guys want to let you know that connman sends tons of queries (1 query/2minutes) toward a domain "ipv4.connman.net" . Tried to block it with hosts file but it doesn't work that way. Restarted the pc and still those queries reach DNS server. I blocked it with firewall and black holed it with nextDNS but still tries to send syn packets to connect. I also edited connman.conf to not check the connectivity but still that doesn't do anything, connman no matter what tries to make it's circus thing. Is there any way to get rid of this or i'm doomed forever? I don't want this because it's unnecessary and second it's privacy non respecting. Anyone down the wire can see i use connman as my network manager. Any options left? Thank you and keep on rolling ;)
PS: when i tried to block that domain i used something like this in hosts
0.0.0.0 ipv4.connman.net
Interesting, I use connman 1.40, and it doesn't make such connections (I check with watch netstat).
Yes check it out. From where i am it tries to resolve with this IP addresses 82.165.8.211 over port 80. I will try to use hosts file again this time pointing that domain toward loopback hope it works that way.
Yeah it worked sending to loopback
127.0.0.1 ipv4.connman.net
Now when i ping that domain it's like pinging 127.0.0.1 my localhost so that query doesn't land anymore on DNS servers but it bounces inside my machine XD. Still think this is not a proper fix as that "connmand -n" to be more accurate keep trying failing at host level but when i check with netstat -atupln still shows me it tries to syn ack that ip
After discussing it on Telegram, seems it's a known thing, just connman detecting captive portals:
https://wiki.archlinux.org/title/ConnMan#Unknown_route_on_connection
Don't think it'll make much difference, since none of Artix's inits have an equivalent of systemd's network-online.target.
So, that's why connman on my machine doesn't make such connections. Also:
# Range of intervals between two online check requests.
# When an online check request fails, another one is triggered after a
# longer interval. The intervals follow the power of two series of numbers
# between OnlineCheckInitialInterval and OnlineCheckMaxInterval.
# Default range is [1, 12], corresponding to the following intervals, in
# seconds: 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121 and 144.
# OnlineCheckInitialInterval = 1
# OnlineCheckMaxInterval = 12
Well mine still did that until i send it to talk to my localhost. Anyway if i will try to use encrypted dns probably the hosts file approach will fail too cos it would totally bypass it. So in conclusion on runit that "EnableOnlineCheck=false" doesn't have any results, it's like you're doing nothing
There's also this:
# WARNING: Experimental feature!!!
# In addition to EnableOnlineCheck setting, enable or disable use of HTTP GET
#| to detect the loss of end-to-end connectivity.
# If this setting is false, when the default service transitions to ONLINE
# state, the HTTP GET request is no more called until next cycle, initiated
# by a transition of the default service to DISCONNECT state.
# If this setting is true, the HTTP GET request keeps beeing called to guarantee
# that end-to-end connectivity is still successful. If not, the default service
# will transition to READY state, enabling another service to become the
# default one, in replacement.
EnableOnlineToReadyTransition = false
The entire source code can be seen on kernel.org (https://git.kernel.org/pub/scm/network/connman/connman.git/tree/).
Something is not right. According to connman doc (https://git.kernel.org/pub/scm/network/connman/connman.git/tree/README#n388), this connection is used to determine internet connection status. If the server replies right, connman should transit from Ready to Online state, and no more connection attempt is needed. Either the connection is not made or the server doesn't reply right. If you wanna dig in, I would suggest you put a tcpdump/wireshark on to trace what's going on.
These are sample msg exchanges between your system and ipv{4|6}.connman.net
ConnMan sends this very minimal information in http header when doing
the online check request (example):
Host: ipv4.connman.net
User-Agent: ConnMan/1.23 wispr
Connection: close
Currently following information is returned from connman.net if
the connection is successful (200 OK http response code is returned):
Server: nginx
Date: Mon, 09 Jun 2014 09:25:42 GMT
Content-Type: text/html
Connection: close
X-ConnMan-Status: online
Why would any one with a common sense would want such a feature? I mean that domain/IP would know when you connect/disconnect to the internet in my book that's kinda spyware. Thank you guys for the replies i will try that solution suggested by
@strajder "EnableOnlineToReadyTransition = false" thanks once again. Cheers ;)
PS: and i will check it also with wireshark 8)
Deleted the other stuff as was not enough accurate
EDIT2
So i've checked and double checked. The thing about connman is as follow.
1) If you try to block at the ip level connman will send 1 query/2 minutes (this was detected with nextdns log)
2) If you try to blackhol it meaning blocking "ipv4.connman.net" at dns level connman will send several ~7hundreds queries/in just few seconds
3) If you let it doing it's thing meaning not blocking it in any way it will call home 82.165.8.211 and after that will remain silent (like any other spyware would do)
4) If you try to block it with hosts file it will ignore that and make it's queries without giving a f
So if you ask me i opted for first choice and gave that ip the middle finger. Strongly think it's not influenced by the init system or other settings is just the way connman was coded. Would be cool if anyone can verify but i doubt anyone would give so much time i gave. Connman 1 - 0 Me
Again, no connections to connman.net are being made by connman on my system. I don't think it is related, but I use suite66. I'd say it is probably your setup. Have you perhaps migrated from another distribution or are you using Arch version of connman?
Here's what grep finds on the current git source tree of connman:
$ git clone git://git.kernel.org/pub/scm/network/connman/connman.git
$ cd connman
$ grep -r 'connman\.net'
README:from ipv4.connman.net (for IPv4 connectivity) and ipv6.connman.net
README:http://ipv{4|6}.connman.net/online/status.html
README:a host route to both the ipv4.connman.net and ipv6.connman.net so that
README: Host: ipv4.connman.net
README:Currently following information is returned from connman.net if
src/6to4.c:#define STATUS_URL "http://ipv6.connman.net/online/status.html"
src/wispr.c:#define STATUS_URL_IPV4 "http://ipv4.connman.net/online/status.html"
src/wispr.c:#define STATUS_URL_IPV6 "http://ipv6.connman.net/online/status.html"
src/dnsproxy.c: * Example: caching www.connman.net uses 97 bytes memory.
tools/wispr.c:#define DEFAULT_URL "http://www.connman.net/online/status.html"
tools/private-network-test.c: * IP packet: src: 192.168.219.2 dst www.connman.net
$ grep -r STATUS_URL
src/6to4.c:#define STATUS_URL "http://ipv6.connman.net/online/status.html"
src/6to4.c: web_request_id = g_web_request_get(web, STATUS_URL,
src/wispr.c:#define STATUS_URL_IPV4 "http://ipv4.connman.net/online/status.html"
src/wispr.c:#define STATUS_URL_IPV6 "http://ipv6.connman.net/online/status.html"
src/wispr.c: wp_context->status_url = STATUS_URL_IPV4;
src/wispr.c: wp_context->status_url = STATUS_URL_IPV6;
Here (
src/service.c) we can see that if the
EnableOnlineCheck is not set to 1, the function
start_online_check returns:
static void start_online_check(struct connman_service *service,
enum connman_ipconfig_type type)
{
if (!connman_setting_get_bool("EnableOnlineCheck")) {
connman_info("Online check disabled. "
"Default service remains in READY state.");
return;
}
enable_online_to_ready_transition =
connman_setting_get_bool("EnableOnlineToReadyTransition");
online_check_initial_interval =
connman_setting_get_uint("OnlineCheckInitialInterval");
online_check_max_interval =
connman_setting_get_uint("OnlineCheckMaxInterval");
if (type != CONNMAN_IPCONFIG_TYPE_IPV4 || check_proxy_setup(service)) {
cancel_online_check(service);
__connman_service_wispr_start(service, type);
}
}
I've further traced a line of execution which makes a connection to connman.net, regardless of the above settings, but it depends on the
Enable6to4 setting being set to true, when its default value is false:
# Automatically enable Anycast 6to4 if possible. This is not recommended, as
# the use of 6to4 will generally lead to a severe degradation of connection
# quality. See RFC6343. Default value is false (as recommended by RFC6343
# section 4.1).
# Enable6to4 = false
it's in the file
src/6to4.c:
static void tun_newlink(unsigned flags, unsigned change, void *user_data)
{
int index = GPOINTER_TO_INT(user_data);
if ((newlink_flags & IFF_UP) == (flags & IFF_UP)) {
newlink_flags = flags;
return;
}
if (flags & IFF_UP) {
/*
* We try to verify that connectivity through tunnel works ok.
*/
if (newlink_timeout_id > 0) {
g_source_remove(newlink_timeout_id);
newlink_timeout_id = 0;
}
web = g_web_new(index);
if (!web) {
tunnel_destroy();
return;
}
g_web_set_accept(web, NULL);
g_web_set_user_agent(web, "ConnMan/%s", VERSION);
g_web_set_close_connection(web, TRUE);
if (getenv("CONNMAN_WEB_DEBUG"))
g_web_set_debug(web, web_debug, "6to4");
web_request_id = g_web_request_get(web, STATUS_URL,
web_result, NULL, NULL);
newlink_timeout(NULL);
}
newlink_flags = flags;
}
This is still problematic, but at least not enabled by default. This should check the
EnableOnlineCheck, but it doesn't. Here:
static bool apply_lease_available_on_network(GDHCPClient *dhcp_client,
struct connman_dhcp *dhcp)
{
char **nameservers, **timeservers, *pac = NULL;
struct connman_service *service;
GList *list, *option = NULL;
int ns_entries;
int i;
if (!dhcp->network)
return true;
service = connman_service_lookup_from_network(dhcp->network);
if (!service) {
connman_error("Can not lookup service");
return false;
}
/* ... and so on ... */
if (connman_setting_get_bool("Enable6to4"))
__connman_6to4_probe(service); /* <-- this calls the above function which */
/* makes the connection, unconditionally */
return true;
}
Yeah for the guys with big brains 8) for sure this is a child play modifying the code but for me is gibberish. I won't dive deeper than that because when they will make an update everything could go back or even get worse. Wonder if network manager is any better or it has same spyware capabilities. I mean they seem to have made a spyware who gather info about every single device that uses connman, unless you block it at the ip level (best solution i think) everybody is on dady "ipv4.connman.net" "facebook"
PS: wonder what the heck the guys maintaining nexdns said when they saw 1000 queries in 10 seconds lol
Anyway thank you
@strajder for trying out to find a solution. Perhaps we need to fork connman an make it easier to disable that online check without so much hassle . Cheers :)
When you start your machine and you're in the middle of the boot connman already is sending it's packets and after that remains silent. But if you block that ip you'll see how active is in reality.
I usually connect manually via very simple short scripts based on these principles, and some of the underlying actions in the Debian ifupdown package which I used to use, this makes it simpler to know what is happening:
https://wiki.archlinux.org/title/Network_configuration/Wireless
But if you do this then you find you need to throw in short arbitrary "sleep" pauses, whether with ethernet or wireless, because otherwise subsequent actions may fail, it takes a while for the network and interface to initialize. Also a "proper" network manager usually provides a service that other things rely on, for example netmount - parts of the OS filesystem might be on different computers, even in different countries. So the (only?) correct and reliable way to do this is to make a positive check that the network connection is working somehow, by pinging some random address, or apparently in the case of Connman, checking if it can reach a presumably purpose made web address. It would send the request rapidly at first so as not to delay boot, then probably gives up hope of an immediate connection and sends at a lower frequency afterwards.
That would mean that
watch netstat (from a terminal emulator) wouldn't show connections to ipv4.connman.net.
In any case, I've checked the source code, and like I said, while there is one execution flow which would lead to unconditional connections to connman.net, it is not done by default as it relies on the variable which is not set by default,
Enable6to4. The only other way (aside from
EnableOnlineCheck=true and
EnableOnlineToReadyTransition = true) for anyone to see connections to connman.net being made is to set
Enable6to4 = true
in /etc/connman/main.conf.
Tried to set those like you said again and with no effect. Those settings seems to be just for fun they don't have any functionality in connman behaviour. It's sending like a spam bot queries after queries. Told you if you wanna see this behavior you have to block it in some way if you let it connect to that domain since is doing what he wants after that he's not sending other requests unless you disconnect and reconnect from connman. Don't know how you managed to review the code so fast but for sure something is missing or you looked in a hurry. Don't think it's related to init system either. Would be useful to have your DNS somehow logged otherwise will be hard for you to know when or what King Connman is really doing.
Program cannot do anything outside of what is written in its source code. Unless you are running a modified version of connman (or have in place some convoluted way of aliasing connman to 'connman -c /some/other/config/file', just to bypass /etc/connman/main.conf), I don't believe it.
Man if you are so sure then yeah maybe is something different on my end even i didn't change anything related to connman. I changed though default kernel from Artix with hardened kernel (linux-hardened), i use apparmor in enforce mode only for my firefox browser, xfce desktop environment installed very few packages. I have only 791 packages installed and also i use F2FS instead of EXT4 file system. Given the fact that i'm ok with blocking that ip with my firewall and you say that is everything ok if you change those "knobs" in main.conf. I will mark this topic as solved. Cos for sure anyone would choose what works for him from this long list of options. Thanks once again
@strajder you really rock. I will mark this as solved at the end of the day if nothing "groundbreaking" new information intervenes. Cheers ;)
I stand corrected. There is a piece of code in
src/wispr.c:
static int wispr_portal_detect(struct connman_wispr_portal_context *wp_context)
{
/* ... */
if (wp_context->type == CONNMAN_IPCONFIG_TYPE_IPV4) {
g_web_set_address_family(wp_context->web, AF_INET);
wp_context->status_url = STATUS_URL_IPV4;
} else {
g_web_set_address_family(wp_context->web, AF_INET6);
wp_context->status_url = STATUS_URL_IPV6;
}
/* ... */
}
This function is in turn called by
__connman_wispr_start in two places: 1) in function
redo_wispr and 2) in
__connman_service_wispr_start.
1.
redo_wispr is called by
redo_wispr_ipv4 and
redo_wispr_ipv6, which are both called only by
__connman_service_online_check. This last function is called by: 1.1)
portal_manage_status, but only if
enable_online_to_ready_transition (set by configuration option) is non-zero, and by 1.2)
wispr_portal_web_result, which is called by itself and
wispr_portal_request_portal. This function is called by: 1.2.1)
wispr_manage_message, which is called by
wispr_portal_web_result and 1.2.2)
proxy_callback, which is called by
no_proxy_callback and
wispr_portal_detect. So, a dead end.
2.
__connman_service_wispr_start is called by: 2.1)
wispr_portal_browser_reply_cb, 2.2)
start_online_check, which is controlled by the configuration option, 2.3)
set_property and 2.4)
default_changed.
The functions set_property and default_changed are indeed called in multiple places.Conclusion: this patch should introduce the absolute respecting of the configuration setting
EnableOnlineCheck.
https://paste.artixlinux.org/18859960
Disclaimer: There could be interactions with regards to IPv4/IPv6 querying of localhost, or not.
Maybe i forgot something important, i don't have ipv6 networking even my ISP provides that cos i'm using some oldish router which doesn't provide ipv6 functionality. So I see if i get this straight you made some sort of a patch that presumably should enforce that connman respect it's "knobs" from main.conf. So if we want to apply that patch where should we place it? Also i will leave this topic open till we make a conclusion given the fact things still seems to be in a dynamic. ;)
I'd wait for Artix developers to first give their opinion on this and possibly make a separate package, as that would be the recommended route. If you want to know how to manually apply it, here's how it would be done:
The patch should be applied to the source tree of a cloned git repository of connman, available at https://git.kernel.org/pub/scm/network/connman/connman.git/tree/. One can do that using the commands:
$ git clone git://git.kernel.org/pub/scm/network/connman/connman.git
$ cd connman
Download paste from
https://paste.artixlinux.org/view/18859960 (
Edit: I made a mistake in the test in this one, should be negated, so use the fixed one - https://paste.artixlinux.org/view/c18d165e) to the newly created directory connman and apply the patch. If you download from paste.artixlinux.org by using the "Download paste", you'll need to convert line endings from CR+LF (DOS) to LF (Unix). If you downloaded the file to respect-enable_online_check.patch:
$ sed -e's/\r//g' < respect-enable_online_check.patch > respect-enable_online_check-lf.patch
Then apply the patch using git apply and build:
$ git apply respect-enable_online_check-lf.patch
$ ./bootstrap-configure # gives error, just ignore
$ ./configure && make
$ sudo make install
In the meantime, setnet (https://katolaz.net/setnet/) is a lighter alternative made as a shell script (which I didn't try yet).
I don't see a separate package happening, unless someone from the team is actively using connman (and is equally concerned about its pinging home). In the good, old Arch Way, we follow upstream - so I suggest you file a bug report / feature request with them.
I can do that, but if this bug is "a feature", which seems likely, I don't have any hope for an answer, much less for them accepting this patch.
By the way, Artix package connman has:
$ pacman -Qi connman | grep ^URL | sed -e's/^URL\s\+: //g'
https://01.org/connman
and that website gives "Access denied". I think that might need an update?
Update: I asked in IRC #connman on irc.oftc.net.
Update 2: After no reply for more than an hour, I sent a patch to
[email protected] via email.
Update 3: Modified the test (should use negation :P). Fixed one is at: https://paste.artixlinux.org/view/c18d165e
Good job
@strajder 8) you are such an inspiration for many, myself included ;) . What i discovered was totally by chance cos if i didn't have those dns logs probably we would never know about that spooky "feature", at least they can make that main.conf option really work not just for misleading people (or maybe is just a bug let's not jump in the conspiracy just yet). Many thanks for all the folks out there who gave it a look for this small but sneaky bug or whatever it is.
I think you have a good point there thinking about the implications, even presuming the connman test website is entirely well intentioned, it could still be compromised, and the data used by unknown dubious parties, and if you configure something not to happen because you are concerned about it and it still does happen, then that's surely a bug.
Any HTTP/HTTPS request can be logged, and along with it requester's IP (containing geolocation), user agent (not applicable here though) and time of the request.
If the authors did "phone home" on purpose (I didn't get a response to my patch yet), they are not to be trusted.
I plan to test the script I mentioned above, setnet (https://katolaz.net/setnet/), as it is a lighter alternative to NetworkManager/connman anyway.
@strajder yeah if it's on purpose it's kinda evil but i would let them more time since it's weekend so probably they didn't had enough time to react yet. Hope they will patch that soon. In the mean time the good old school firewall (ufw) can solve that partially or network manager
ufw insert 1 deny out from any to 82.165.8.211
Let's wait and see what's next :)
EDIT
Another solution would be to uninstall connman remove it's services and issue something like this for the ethernet connection
# dhclient eth0
Hope i remembered correctly the command
EDIT 2
Tried it for myself, removed connman + it's "services" and internet just works after that small command
There's also
dhcpcd +
dhcpcd-<your_init> (I have
dhcpcd-suite66).
Meanwhile, I got a response from a connman dev, he proposed a slightly different solution, but with the same effect. Still discussing over email, will post an update here if/when this gets patched.
Yhaaaa :D you made it
@strajder yupii :P . It means it was just a bug or someone tested our vigilance 8) . Congrats once again!
This patch adds configuration options OnlineCheckIPv4URL and OnlineCheckIPv6URL to connman, so users can
configure their own URLs for the online check:
https://git.kernel.org/pub/scm/network/connman/connman.git/commit/?id=9d7e55f40e807ea7d28a34b20e50af1b06ea2fda
This patch organizes the online check code into a single function:
https://git.kernel.org/pub/scm/network/connman/connman.git/commit/?id=a84f03f6f6c5d69cbb2e2c32326f9af603b0f8a7
This patch makes the setting EnableOnlineCheck globally respected. If it is set to false, no server pinging will be done by connman:
https://git.kernel.org/pub/scm/network/connman/connman.git/commit/?id=85d04c03dd0f9f35394d84ba2fcc6ccfc17af293
Note: For anyone who values simplicity in software, running connman (as a daemon) is an overkill, so even in the scenario when user is on a laptop, frequently changing networks (for example, using wifi in cafes or hotels), using setnet as a quick and easy one-shot configuration generator for wpa_supplicant should be preferable. Otherwise, one can use wpa_cli, which is simpler, but requires some learning.
the correct "arch way" would be to package a default config with the config knobs set so the "feature" is disabled by default, you then have to enable it if you want it..
if the code upstream is broken so the configuration is not applied correctly, then you found a bug, that is to be fixed upstream...
until its fixed upstream a patch can be applied if maintainers think its worth the work..
so now that the bug is fixed upstream, but not yet in a tagged release, it should be generated a patch via git to cherry-pick the needed commits against the used tag so artix-testing can actually be the backporting bleeding edge its supposed to be :)
there will most likely be atleast a few months before we get a tagged connman release with the commits added.
great job in fixing this strajder and the others that have been involved
This is up to the package maintainers. I'm sure they already have something like this in mind.