Artix Linux Forum

Artix Linux => Applications & Software => Topic started by: dreieck on 09 January 2023, 21:22:15

Title: libtiff: Library dependency problems in at least xsane and digikam.
Post by: dreieck on 09 January 2023, 21:22:15
digikam and xsane do not start anymore after an update due to broken library dependencies.

First, update and explicit installation of the Artix' versions of the softwares:

Code: [Select]
pacman -Syu world/libtiff world/digikam omniverse/xsane

Then, trying to start:

digikam:
Code: [Select]
digikam: error while loading shared libraries: libtiff.so.5: cannot open shared object file: No such file or directory


xsane:
Code: [Select]
xsane: error while loading shared libraries: libtiff.so.5: cannot open shared object file: No such file or directory

I have to install Arch's extra/xsane to make xsane work, and Arch's extra/digikam together with Arch's extra/opencv to make digikam work.

Versions in the repos:

pacman -Ss '^(libtiff|xsane|digikam)$' | grep -v '^[[:space:]]':
Code: [Select]
world/digikam 7.9.0-2 [installed: 7.9.0-4]
world/libtiff 4.5.0-1 [installed]
omniverse/xsane 0.999-5 [installed: 0.999-6]
extra/digikam 7.9.0-4 [installed]
extra/libtiff 4.5.0-1 [installed]
extra/xsane 0.999-6 [installed]

My Artix mirrors are:


xsane is in the omniverse repository, there I use the following mirror:

Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 10 January 2023, 01:25:39
Yeah sorry about that, the kde update is still in process (in the testing repos). Those should be built against the new library so once the move happens that should resolve this.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Artist on 10 January 2023, 11:20:25
xsane in the omniverse repo should work, pls test

artist
Title: Update as atomic operation?
Post by: dreieck on 10 January 2023, 12:01:55
Ahoj,

thanks for the explanation!

A remark:

Yeah sorry about that, the kde update is still in process (in the testing repos). Those should be built against the new library so once the move happens that should resolve this.

Would it be possible to have the move all at once as an (almost) atomic operation, so that there is either a complete consistent set of working packages of the old versions available in the "production" repos, or a complete consistent set of packages of the new versions available in the "production" repos, but not a mixture?

For an everyday-to-use-operating-system it is not really reliable if it on the one hand suggest frequent full upgrades, on the other hand there is always reason to fear that some software needed for work breaks.

Those problems seem to appear every now and then (I "feel" more often than on Arch, and contrary to Arch which, if such things happen, publishes guides for fixups timely on the main page http://archlinux.org/, for Artix there is usually no such information at all), this is not the only issue of this kind.

Regards!
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: nous on 10 January 2023, 14:23:21
Would it be possible to have the move all at once as an (almost) atomic operation, so that there is either a complete consistent set of working packages of the old versions available in the "production" repos, or a complete consistent set of packages of the new versions available in the "production" repos, but not a mixture?
The inter-repo move is practically atomic. Packages known to break things go first into goblins/staging until rebuilds/updates have finished and then moved into production. However, slip-ups do happen from time to time...
Title: Re: Update as atomic operation?
Post by: lq on 10 January 2023, 14:27:25
For an everyday-to-use-operating-system it is not really reliable if it on the one hand suggest frequent full upgrades, on the other hand there is always reason to fear that some software needed for work breaks.

Strange.
I always thought that the admins and not the developers or package maintainers were responsible for system maintenance.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: gripped on 10 January 2023, 15:30:56
Strange.
I always thought that the admins and not the developers or package maintainers were responsible for system maintenance.
And you'd be correct.

But mismatched libraries, within just the Artix repo's, doesn't fall under normal system maintenance. That's package maintenance.

I know how to fix such issues and I'm sure you do too. But not all Artix users do. And ideally they shouldn't have to. In this case digikam was broken for around a week. Why xsane is not in the normal repos I have no idea?
I'm not complaining. On a scale of 1 to 10 I care about 0.3 but @dreieck makes a reasonable point.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 10 January 2023, 15:37:16
Would it be possible to have the move all at once as an (almost) atomic operation, so that there is either a complete consistent set of working packages of the old versions available in the "production" repos, or a complete consistent set of packages of the new versions available in the "production" repos, but not a mixture?
It's not really an infrastructure problem (well I say this but our buildserver is down for some reason as I type this which delays things :P) but kind of just bad luck. Arch recently did a massive amount of rebuilds on us (I can't recall this many in such a short time period happening before) which took a while to catch up. I guess we could have held back the libtiff updates, but a ton of packages were affected by that and people were noticing soname mismatches in arch packages so it's probably better that we moved them.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: lq on 10 January 2023, 17:34:25
But not all Artix users do. And ideally they shouldn't have to.

Imho, everyone who sets up an OS is an admin and as such he MUST know what he is dealing with, this applies not only to the software but also to the hardware.

The trick is: you don't have to learn how to solve problems, you should learn how to prevent problems.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: dreieck on 11 January 2023, 11:57:13
xsane in the omniverse repo should work, pls test

No, I won't test now, because I am afraid that things will break again when I install Artix' libtiff. Now I have xsane and digikam working with Arch's libtiff, and as long as not all Artix shipped packages are built against Artix' libraries I will not change that.

I know that I can test and revert as long as I have the old packages still in my pacman cache, but I trust you that xsane is now linked agains the correct libtiff and I will for now stick with the packages at the current version.

I will wait a week or so, then try a full update.

Imho, everyone who sets up an OS is an admin and as such he MUST know what he is dealing with, this applies not only to the software but also to the hardware.

The trick is: you don't have to learn how to solve problems, you should learn how to prevent problems.

This sounds like a double bind dilemma here: On the one hand, I am supposed to do frequent full updates. On the other hand, only after the update the problems that the packages provided by the distribution are improperly linked.

Preventing here would be to have some system snapshotting which takes a system snapshot before each pacman run, than a revert is easy. But that takes extra storage locally and on some laptops local storage is scarce.

I know how I can fix this (as a last resource: Grab the PKGBUILD and build locally, resulting in linkage to the locally installed libraries), but from my perspective the distribution is responsible for taking care that the packages they ship are correctly linked.

Mistakes can happen, and that is OK.

The inter-repo move is practically atomic. Packages known to break things go first into goblins/staging until rebuilds/updates have finished and then moved into production. However, slip-ups do happen from time to time...

With "atomic" I mean, when there is some new package version, it will only be relased to the production repositories if all packages that depend on that package that is new have been rebuilt (if necessary, recursively if there are recursive dependencies), and only after all rebuilds habe been done all those packages are moved at the same time to the production repositories.

Arch recently did a massive amount of rebuilds on us[...]. I guess we could have held back the libtiff updates, but a ton of packages were affected by that and people were noticing soname mismatches in arch packages so it's probably better that we moved them.

You mean, users who were using packages not from Artix' repositories but from Arch's repositories (e.g. from "community" or "extra"), because those packages are not available in Artix' repositories?

Well, I see the point, given that this really creates some tension with my thought of "atomic" and I see that there can be no overall clean solution then as long as Arch's and Artix' packages are mixed, and I see that this mixing can be practical.


Regards!,
and thanks for all your maintanance work!
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 11 January 2023, 15:12:34
digikam should be updated in our repos now. btw, I'm not actually sure why we don't just import xsane into the repos.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: gripped on 11 January 2023, 16:39:36
and people were noticing soname mismatches in arch packages so it's probably better that we moved them.
I've had another think about this. It's a strange logic imho. Adding the Arch repos is 'unsupported'. Yet if there was truly a decision to update libtiff to prevent breakage of packages installed from the Arch repos, knowingly (or not) breaking libtiff dependant packages in the Artix repo, it comes across as - in this instance - Arch repo users were more supported than pure Artix users.

Just a thought. Again not complaining, discussing.

Praise be Artix, and hallowed be the devs who give up their free time to maintain it.
Title: digikam now complains about missing 'libZXing.so.1'.
Post by: dreieck on 13 January 2023, 15:05:46
digikam should be updated in our repos now.

But there still is a library problem. Can you please do a test before you make a fix via an update to something that is reported to have problems?

The problem now is:

Code: [Select]
digikam: error while loading shared libraries: libZXing.so.1: cannot open shared object file: No such file or directory

/usr/lib/libZXing.so* belongs to the package zxing-cpp, and both Arch and Artix have version 2.0.0.

xsane in the omniverse repo should work, pls test

It works now.

Rergards!
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 13 January 2023, 20:41:41
But there still is a library problem. Can you please do a test before you make a fix via an update to something that is reported to have problems?
Sorry, I didn't realize that digikam had to be moved from testing. That should solve it this time.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: dreieck on 13 January 2023, 22:04:36
Sorry, I didn't realize that digikam had to be moved from testing. That should solve it this time.

And I still had opencv from the Arch "extra" repository (version 4.7.0-1) on my system, from my previus workaround to the issues reported in this thread. Now, when I downgraded to Artix' world/opencv (version 4.6.0-6), digikam first complains about libopencv_dnn.so.407:

Code: [Select]
digikam: error while loading shared libraries: libopencv_dnn.so.407: cannot open shared object file: No such file or directory

A new version of digikam has not yet arrived at my mirror. 
As general Artix mirrors I use:
Code: [Select]
Server = http://mirror1.artixlinux.org/repos/$repo/os/$arch
Server = https://eu-mirror.artixlinux.org/repos/$repo/os/$arch



I am now resorting to build digikam-git from the AUR (https://aur.archlinux.org/packages/digikam-git), because I am fed up with this right now.

Regards and thanks again for your work and efforts!
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 13 January 2023, 23:06:46
Sorry, my last message had a bad error that might have confused you. What had to be moved from testing was the prison package. Digikam actually indirectly depends on that. That's where the linking error was coming from. It make take a few hours for the mirrors to sync.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: dreieck on 13 January 2023, 23:22:09
Sorry, my last message had a bad error that might have confused you. What had to be moved from testing was the prison package. Digikam actually indirectly depends on that. That's where the linking error was coming from. It make take a few hours for the mirrors to sync.

I have installed prison version 5.101.0-2.1, build date Fr 13 Jan 2023 20:21:22 CET. So that should be the new one, but I get library error.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 14 January 2023, 03:25:58
In that case, I'm not sure what it is. I'm running prison 5.101.0-2.1 and digikam 7.9.0-4. Unless there's some other secret dependency I'm missing that I happen to have installed and you don't?

Edit: okay hopefully prison-5.101.0-2.3 does the trick.
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: dreieck on 15 January 2023, 00:47:41
In that case, I'm not sure what it is. I'm running prison 5.101.0-2.1 and digikam 7.9.0-4. Unless there's some other secret dependency I'm missing that I happen to have installed and you don't?

Edit: okay hopefully prison-5.101.0-2.3 does the trick.

I just did a full pacman -Syu and the zxing-cpp dependency problem  is fixed, but the libtiff problem is still there.

I see that digikam now is at version 7.9.0-4.1.

I also did re-install all first-level dependencies of digikam, as well all packages that depend on libtiff.

When I look at digikam via 
ldd /usr/bin/digikam | grep tiff
I see that it wants libtiff.so.6 and libtiff.so.5:
Code: [Select]
	libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007f858702f000)
libtiff.so.5 => not found

What is going on here?

After I installed libtiff5 from the AUR (https://aur.archlinux.org/packages/libtiff5), digikam starts.


I've had another think about this. It's a strange logic imho. Adding the Arch repos is 'unsupported'. Yet if there was truly a decision to update libtiff to prevent breakage of packages installed from the Arch repos, knowingly (or not) breaking libtiff dependant packages in the Artix repo, it comes across as - in this instance - Arch repo users were more supported than pure Artix users.

When adding Arch repositories like "community" or "extra" is explicitly unsupported, then I see it as a "bug in Artix governance" when packages are pushed to new versions to make Arch packages not break but thereby risking breaking of Artix packages. 
In this case I come back that upgrades should be "atomic" in the sense that newly built packages should only be moved to the production repositories when all the packages that are connected via dependencies also got a clean rebuild, and then the move should be done at the same time for all packages within one dependency graph. 
What can be done that package breaking due to partial update in the repositories is much less likely to happen in the future?

Regards!
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 15 January 2023, 03:25:31
When I look at digikam via 
ldd /usr/bin/digikam | grep tiff
I see that it wants libtiff.so.6 and libtiff.so.5:
Code: [Select]
	libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007f858702f000)
libtiff.so.5 => not found

What is going on here?

Regards!

Something is definitely weird here. Mine only links to version 6.

Code: [Select]
$ ldd /usr/bin/digikam | grep tiff
libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007f2c7cd9b000)

Does anybody else have @dreieck's issue?
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: jjg1965 on 15 January 2023, 08:54:42
Me. On a new installation made yesterday.

Code: [Select]
$ ldd /usr/bin/digikam | grep tiff
        libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007f51fd627000)
        libtiff.so.5 => not found
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: gripped on 15 January 2023, 11:21:41
Can confirm:
Directly installed from https://mirrors.dotsrc.org/artix-linux/repos/world/os/x86_64/digikam-7.9.0-4.1-x86_64.pkg.tar.zst
Code: [Select]
❯ ldd /usr/bin/digikam | grep tiff
        libtiff.so.5 => not found
        libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007f05059a2000)

Locally built from https://gitea.artixlinux.org/packagesD/digikam.git the same, and this is in the cmake output
Code: [Select]
[ 95%] Building CXX object core/dplugins/bqm/convert/converttodng/CMakeFiles/Bqm_ConvertToDNG_Plugin.dir/converttodng.cpp.o
/usr/bin/ld: warning: libtiff.so.5, needed by /usr/lib/libopencv_imgcodecs.so.406, not found (try using -rpath or -rpath-link)
[ 95%] Building CXX object core/dplugins/bqm/decorate/border/CMakeFiles/Bqm_Border_Plugin.dir/borderplugin.cpp.o
I had a thought to rebuild whatever provides libopencv_imgcodecs.so
Code: [Select]
❯ pkgfile /usr/lib/libopencv_imgcodecs.so
world/opencv
world/opencv-cuda
extra/opencv
extra/opencv-cuda

opencv  is what I have installed but building that failed
Code: [Select]
/usr/bin/ld: CMakeFiles/opencv_rgbd.dir/src/dynafu.cpp.o: in function `cv::dynafu::DynaFuImpl<cv::Mat>::DynaFuImpl(cv::kinfu::Params const&)':
dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x3c8): undefined reference to `glGenRenderbuffersEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x3d6): undefined reference to `glBindRenderbufferEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x3ec): undefined reference to `glRenderbufferStorageEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x3fb): undefined reference to `glGenFramebuffersEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x409): undefined reference to `glBindFramebufferEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x421): undefined reference to `glFramebufferRenderbufferEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x430): undefined reference to `glGenRenderbuffersEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x43e): undefined reference to `glBindRenderbufferEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x454): undefined reference to `glRenderbufferStorageEXT'
/usr/bin/ld: dynafu.cpp:(.text._ZN2cv6dynafu10DynaFuImplINS_3MatEEC2ERKNS_5kinfu6ParamsE[_ZN2cv6dynafu10DynaFuImplINS_3MatEEC5ERKNS_5kinfu6ParamsE]+0x46c): undefined reference to `glFramebufferRenderbufferEXT'
collect2: error: ld returned 1 exit status
make[2]: *** [modules/rgbd/CMakeFiles/opencv_rgbd.dir/build.make:504: lib/libopencv_rgbd.so.4.7.0] Error 1
make[1]: *** [CMakeFiles/Makefile2:9507: modules/rgbd/CMakeFiles/opencv_rgbd.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
I'm going to try it in a chroot......
Title: opencv version in Arch and Artix.
Post by: dreieck on 15 January 2023, 12:24:57
Locally built from https://gitea.artixlinux.org/packagesD/digikam.git the same, and this is in the cmake output
Code: [Select]
[ 95%] Building CXX object core/dplugins/bqm/convert/converttodng/CMakeFiles/Bqm_ConvertToDNG_Plugin.dir/converttodng.cpp.o
/usr/bin/ld: warning: libtiff.so.5, needed by /usr/lib/libopencv_imgcodecs.so.406, not found (try using -rpath or -rpath-link)
[ 95%] Building CXX object core/dplugins/bqm/decorate/border/CMakeFiles/Bqm_Border_Plugin.dir/borderplugin.cpp.o

OK, opencv.

Just for information, if it might be helpful: Arch's extra/opencv is at version 4.7.0-1, Artix' world/opencv at version 4.6.0-6.

Regards!
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: gripped on 15 January 2023, 13:01:42
The newest version (4.7.0-2) at https://gitea.artixlinux.org/packagesO/opencv built successfully in a chroot
Installed the resulting package and built and installed digikam. It was now correctly linked
Code: [Select]
❯ ldd /usr/bin/digikam | grep tiff
        libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007fa00a9ae000)
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 15 January 2023, 18:11:33
Sorry for all the issues. There was an opencv in staging, testing, and world at the same time I totally didn't see the testing one (and was using it on my machine because I usually run testing repos). I have moved that to world now hopefully for real this time.
Title: digikam and gst-plugin-opencv linked against wrong opencv version.
Post by: dreieck on 15 January 2023, 18:12:07
Now the updated opencv is available in the Artix repos, thanks for that!

But digikam is still build against the old opencv, so again failing, now because of wrong opencv dependency:

ldd /usr/bin/digikam | grep -E '(opencv|tiff)'
Code: [Select]
	libopencv_dnn.so.406 => not found
libopencv_imgproc.so.406 => not found
libopencv_ml.so.406 => not found
libopencv_core.so.406 => not found
libtiff.so.6 => /usr/lib/libtiff.so.6 (0x00007fc51000e000)
libopencv_imgcodecs.so.406 => not found
libopencv_dnn.so.406 => not found
libopencv_imgproc.so.406 => not found
libopencv_core.so.406 => not found

I have done a full upgrade.

world/opencv is version 4.7.0-1, world/digikam is version 7.9.0-4.1.

Since the system update did not update much, I suspect also other packages are affected. I could identify at least world/gst-plugin-opencv (version 1.20.5-5).

Can you please push new versions to official (non-testing/ non-staging) repositories only when all packages of a dependency graph have been rebuilt, and then push all packages belonging to the new dependency graph at once?

Or can anyone enlighten me why this is not feasible? Bzw. what can be helped that things can work that or another way with a comparable result?

Regards!
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: Dudemanguy on 15 January 2023, 19:40:03
Can you please push new versions to official (non-testing/ non-staging) repositories only when all packages of a dependency graph have been rebuilt, and then push all packages belonging to the new dependency graph at once?

That's what normally happens but this one has been an absolute linking clusterfucker of multiple library version updates, things I didn't even know it depended on, and packages in repos I didn't even know were there. Rebuilt again and hopefully solved?
Title: Re: libtiff: Library dependency problems in at least xsane and digikam.
Post by: dreieck on 19 January 2023, 12:16:27
That's what normally happens but this one has been an absolute linking clusterfucker of multiple library version updates, things I didn't even know it depended on, and packages in repos I didn't even know were there.

Are there no tools to automatically check for it and by default deny any update, only when the check passes allow update?


Yes, thanks!