Skip to main content
Topic: No more Librewolf? (Read 3694 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: No more Librewolf?

Reply #1
https://artixlinux.org/news.php#Repository_structure_changes_%28updates%29
Quote
...
Also, librewolf had to be dropped due to critical build issues that occur since the last two versions.

Is it possible to have it back someday?

If someone sponsors an 128GB memory build server; with that it does build.  ;)
Or until librewolf gets back to normal as in build requirements.

artist

Re: No more Librewolf?

Reply #2
Sorry to see this package go, but it's understandable considering the constraints of consistently compiling and packaging a  browser. For anyone with a similar issue. I simply uninstalled librewolf and reinstalled the librewolf-bin from the AUR using paru. All my configs, bookmarks, extensions, settings, etc. remained unchanged.

Re: No more Librewolf?

Reply #3
Code: [Select]
 trizen librewolf
 2 galaxy/librewolf 115.0.2-2
     Community-maintained fork of Firefox, focused on privacy, security and freedom.
 3 chaotic-aur/firedragon 117.0-1
     Librewolf fork build using custom branding, settings & KDE patches by OpenSUSE
 4 chaotic-aur/librewolf 117.0-1
     Community-maintained fork of Firefox, focused on privacy, security and freedom.
/code]

Re: No more Librewolf?

Reply #4
Not sure what's wrong lately with Librewolf/maybe something in mozilla upstream, cause it also was on the cachyos-v3 repo and they probably hit the same issues and removed it from there too.

Interesting how chaotic-aur also has it's firedragon fork-ception up to date, that must've built somehow or they messed with the patches. I still won't use that repo for my own reasons :-) For the time being i replaced with librewolf-bin too.

Re: No more Librewolf?

Reply #5
Today I tried 2 rebuilds of librewolf to get confirmation if the build problems are related to librewolf itself or to the build process. One build was with a heavily stripped and modified pkgbuild, and one with a pkgbuild created almost from scratch. Both failed due to an abnormally high requirement for memory. This confirms one report I received earlier; librewolf only builds on servers with 128Gb or more.
Because firefox still builds normally, the problems must be caused by librewolf itself.
And while librewolf earlier - as in the past 1.5 years - built successfully in maximum 1 attempt out of 2, firedragon built in 1 out of 5 on average...

Therefore the sad conclusion is that the deterioration of quality of both privacy enhanced firefox derivatives has made them unusable.

artist

Re: No more Librewolf?

Reply #6
I built it on my laptop with 8GB RAM and 16GB swap in this thread:
https://forum.artixlinux.org/index.php/topic,5926
I'd thought it might be helpful for you to try and do this, and it came up in that discussion too so I gave it a go and it worked on the second attempt with the aid of some helpful hints and a couple of small changes to the basic AUR PKGBUILD.
For a distro repo build you probably wouldn't want to use swap so would need a bit more than 16GB RAM restricting threads to 4, if you could upgrade to 32GB it should be ample and you could probably increase the thread count with that too.

Re: No more Librewolf?

Reply #7
The pkgbuild used can build firefox on a system with 16 GB. But it cannot build librewolf on a 64GB server; linking requires at least 128GB. Also the AUR pkgbuild for versions 116 and 117 has the same problem during linking. The problem is not related to rust; with 4 or 6 jobs all compiling works without errors, but the linking is the part that requires 128GB.

artist

Re: No more Librewolf?

Reply #8
It could be this as mentioned in the other thread:
This changed in the librewolf PKGBUILD when 116 was introduced:
-ac_add_options --enable-lto=cross
+ac_add_options --enable-lto=cross,full
https://aur.archlinux.org/cgit/aur.git/commit/?h=librewolf&id=3a2f93dbb9f8628b651edb2665690cd5918405fa
That's the only significant change during that period, and lto = LINK time optimisation which is going to affect linking. If you look at the AUR page then on the right there is a link View Changes which will show you this - there has actually been another change now since I built it.

If that doesn't work, if you post more details people might be able to help more, ie your PKGBUILD, system spec, build failure messages, how many threads you are using - this could be configured in various places, /etc/makepkg.conf, the PKGBUILD, or a custom .mozconfig. You could even be experiencing a bug from something else related to the build process,  no 128GB was required here.

Re: No more Librewolf?

Reply #9
Builds were tried with lto disabled, lto=cross and lto=cross,full
The nr of threads is not relevant here - used between 4 and 12 in mozconfig, as it's only the linking that fails to complete.
This with ld and with ld.gold.
I also removed LDFLAGS+=" -Wl,--no-keep-memory"
One machine  machine had 16gb mem and 12gb swap, the other one has 32gb mem and 80gb swap.
The linker will take up between 21 and 42GB of memory, run forever and not procude any executable, nor an error message.
Are you building on a fully updated system, and in a clean chroot?
Can you provide a PKGBUILD that worked for your build?

artist

Re: No more Librewolf?

Reply #10
but the linking is the part that requires 128GB.
There must be some other variable at play here ?
For science I just built the AUR librewolf with 32GB of ram and 32GB swap. While the swap was needed the highest swap usage ever got was 24GB during linking. No modification to the PKGBUILD

From my /etc/makepkg.conf
Code: [Select]
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=znver2 -mtune=znver2 -O2 -pipe -fno-plt"
CXXFLAGS="-march=znver2 -mtune=znver2 -O2 -pipe -fno-plt"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"


Re: No more Librewolf?

Reply #11
This was the PKGBUILD I used, and I updated just before building, because it needed to install clang and llvm. It built using clang, I think it is possible to build with gcc as an alternative but that is what it did by default. I was just  building on my regular OpenRC install on my Dell E7470, not in a clean chroot, which has a 4th generation I7 CPU, a low power consumption type so only 4 threads available as this is a lightweight portable laptop. I built on the HDD not in /tmp and RAM, after building I only have about 16GB free space on the partition, there was less than 64GB even adding free HDD space when I started, swap and RAM together.
I forgot I'd done this, but before the second attempt I downgraded rust to rust-1 1.71.0-1-x86_64.pkg.tar.zst from the Artix archive, as this predates the problem versions and my first try failed during a rustc bit. My /etc/makepkg.conf is unmodified.
The other slightly non-standard thing was just making clean copies for comparison and further attempts without redownloading.
Code: [Select]
$ git clone https://aur.archlinux.org/librewolf.git
$ cp -a librewolf/ vanillalibrewolf/
$ cd librewolf
$ makepkg -so
$ cd ..
$ sudo cp -a librewolf/ makepkg-o-librewolf/
$ cd librewolf/
$ makepkg -e

Code: [Select]
# Maintainer: ohfp/lsf <lsf at pfho dot net>

pkgname=librewolf
_pkgname=LibreWolf
pkgver=117.0
pkgrel=1
pkgdesc="Community-maintained fork of Firefox, focused on privacy, security and freedom."
url="https://librewolf.net/"
arch=(x86_64 aarch64)
license=(
  GPL
  LGPL
  MPL
)
depends=(
  dbus-glib
  ffmpeg
  gtk3
  libpulse
  libxss
  libxt
  mime-types
  nss
  ttf-font
)
makedepends=(
  binutils
  cbindgen
  clang
  diffutils
  dump_syms
  git
  imake
  inetutils
  jack
  lld
  llvm
  mesa
  nasm
  nodejs
  pciutils
  python
  rust
  unzip
  'wasi-compiler-rt>15'
  'wasi-libc++>15'
  'wasi-libc++abi>15'
  'wasi-libc>=1:0+314+a1c7c2c'
  xorg-server-xvfb
  yasm
  zip
) # pciutils: only to avoid some PGO warning
optdepends=(
  'hunspell-en_US: Spell checking, American English'
  'libnotify: Notification integration'
  'networkmanager: Location detection via available WiFi networks'
  'pulseaudio: Audio support'
  'speech-dispatcher: Text-to-Speech'
  'xdg-desktop-portal: Screensharing with Wayland'
)
backup=('usr/lib/librewolf/librewolf.cfg'
        'usr/lib/librewolf/distribution/policies.json')
options=(
  !debug
  !emptydirs
  !lto
  !makeflags
  !strip
)
_arch_git=https://raw.githubusercontent.com/archlinux/svntogit-packages/packages/firefox/trunk
_arch_git_blob=https://raw.githubusercontent.com/archlinux/svntogit-packages

install='librewolf.install'
source=(
  https://gitlab.com/api/v4/projects/32320088/packages/generic/librewolf-source/${pkgver}-${pkgrel}/librewolf-${pkgver}-${pkgrel}.source.tar.gz # {,.sig} sig files are currently broken, it seems
  $pkgname.desktop
  "default192x192.png"
  "0018-bmo-1516081-Disable-watchdog-during-PGO-builds.patch"
)

sha256sums=('446e9479547dceda58cbcdc2f9f503539086ddb193c8b1206b7c736cd56e44c3'
            '7d01d317b7db7416783febc18ee1237ade2ec86c1567e2c2dd628a94cbf2f25d'
            '959c94c68cab8d5a8cff185ddf4dca92e84c18dccc6dc7c8fe11c78549cdc2f1'
            '1d713370fe5a8788aa1723ca291ae2f96635b92bc3cb80aea85d21847c59ed6d')

validpgpkeys=('034F7776EF5E0C613D2F7934D29FBD5F93C0CFC3') # maltej(?)

# change this to false if you do not want to run a PGO build for aarch64 or x86_64
_build_profiled_aarch64=false
_build_profiled_x86_64=false

prepare() {
  mkdir -p mozbuild
  cd librewolf-$pkgver-$pkgrel

  mv mozconfig ../mozconfig

  cat >>../mozconfig <<END

# TODO: check things here one after another if (still) required
ac_add_options --enable-linker=lld

ac_add_options --prefix=/usr

ac_add_options --disable-bootstrap

export CC='clang'
export CXX='clang++'

# Branding
ac_add_options --with-app-name=${pkgname}
# is this one required? upstream lw doesn't use it
ac_add_options --enable-update-channel=release
# unlear?
# ac_add_options --with-app-basename=${_pkgname}

# needed? yep.
export MOZ_APP_REMOTINGNAME=${_pkgname}

# System libraries
ac_add_options --with-system-nspr
ac_add_options --with-system-nss

# Features
# keep alsa option in here until merged upstream
ac_add_options --enable-alsa
ac_add_options --enable-jack

# options for ci / weaker build systems
# mk_add_options MOZ_MAKE_FLAGS="-j4"
# ac_add_options --enable-linker=gold

# wasi
ac_add_options --with-wasi-sysroot=/usr/share/wasi-sysroot
END

if [[ $CARCH == 'aarch64' ]]; then
  cat >>../mozconfig <<END
# taken from manjaro build:
ac_add_options --enable-optimize="-g0 -O2"
END

  export MOZ_DEBUG_FLAGS=" "
  export CFLAGS+=" -g0"
  export CXXFLAGS+=" -g0"
  export RUSTFLAGS="-Cdebuginfo=0"

  # we should have more than enough RAM on the CI spot instances.
  # ...or maybe not?
  export LDFLAGS+=" -Wl,--no-keep-memory"
else

  cat >>../mozconfig <<END
# Arch upstream has it in their PKGBUILD, ALARM does not for aarch64:
ac_add_options --disable-elf-hack

# might help with failing x86_64 builds?
export LDFLAGS+=" -Wl,--no-keep-memory"
END
fi

  # upstream Arch fixes

  # pgo improvements
  # TODO: test if still required
  # patch -Np1 -i ../0018-bmo-1516081-Disable-watchdog-during-PGO-builds.patch
}


build() {
  cd librewolf-$pkgver-$pkgrel

  export MOZ_NOSPAM=1
  export MOZBUILD_STATE_PATH="$srcdir/mozbuild"
  export MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE=pip
  # export PIP_NETWORK_INSTALL_RESTRICTED_VIRTUALENVS=mach # let us hope this is a working _new_ workaround for the pip env issues?

  # LTO needs more open files
  ulimit -n 4096

  # Do 3-tier PGO
  echo "Building instrumented browser..."

  if [[ $CARCH == 'aarch64' && $_build_profiled_aarch64 == true ]]; then

    cat >.mozconfig ../mozconfig - <<END
ac_add_options --enable-profile-generate
END

  elif [[ $CARCH == 'x86_64' && $_build_profiled_x86_64 == true ]]; then

    cat >.mozconfig ../mozconfig - <<END
ac_add_options --enable-profile-generate=cross
END

  fi

  if [[ $CARCH == 'aarch64' && $_build_profiled_aarch64 == true || $CARCH == 'x86_64' && $_build_profiled_x86_64 == true ]]; then

    ./mach build

    echo "Profiling instrumented browser..."

    ./mach package

    LLVM_PROFDATA=llvm-profdata \
      JARLOG_FILE="$PWD/jarlog" \
      xvfb-run -s "-screen 0 1920x1080x24 -nolisten local" \
      ./mach python build/pgo/profileserver.py

    stat -c "Profile data found (%s bytes)" merged.profdata
    test -s merged.profdata

    stat -c "Jar log found (%s bytes)" jarlog
    test -s jarlog

    echo "Removing instrumented browser..."
    ./mach clobber

    echo "Building optimized browser..."

    if [[ $CARCH == 'aarch64' ]]; then

      cat >.mozconfig ../mozconfig - <<END
ac_add_options --enable-lto
ac_add_options --enable-profile-use
ac_add_options --with-pgo-profile-path=${PWD@Q}/merged.profdata
ac_add_options --with-pgo-jarlog=${PWD@Q}/jarlog
END

    else

      cat >.mozconfig ../mozconfig - <<END
ac_add_options --enable-lto=cross
ac_add_options --enable-profile-use=cross
ac_add_options --with-pgo-profile-path=${PWD@Q}/merged.profdata
ac_add_options --with-pgo-jarlog=${PWD@Q}/jarlog
END

    fi
  fi

  if [[ $CARCH == 'aarch64' && $_build_profiled_aarch64 == false || $CARCH == 'x86_64' && $_build_profiled_x86_64 == false ]]; then
    cat >.mozconfig ../mozconfig
  fi

  ./mach build

  echo "Building symbol archive..."
  ./mach buildsymbols
}

package() {
  cd librewolf-$pkgver-$pkgrel
  DESTDIR="$pkgdir" ./mach install

  # mv ${pkgdir}/usr/local/lib ${pkgdir}/usr/lib/
  # mv ${pkgdir}/usr/local/bin ${pkgdir}/usr/bin/
  # rm -r ${pkgdir}/usr/local

  local vendorjs="$pkgdir/usr/lib/$pkgname/browser/defaults/preferences/vendor.js"

  install -Dvm644 /dev/stdin "$vendorjs" <<END
// Use system-provided dictionaries
pref("spellchecker.dictionary_path", "/usr/share/hunspell");

// Don't disable extensions in the application directory
// done in librewolf.cfg
// pref("extensions.autoDisableScopes", 11);
END

  local distini="$pkgdir/usr/lib/$pkgname/distribution/distribution.ini"
  install -Dvm644 /dev/stdin "$distini" <<END

[Global]
id=io.gitlab.${pkgname}-community
version=1.0
about=LibreWolf

[Preferences]
app.distributor="LibreWolf Community"
app.distributor.channel=$pkgname
app.partner.librewolf=$pkgname
END

  for i in 16 32 48 64 128; do
    install -Dvm644 browser/branding/${pkgname}/default$i.png \
      "$pkgdir/usr/share/icons/hicolor/${i}x${i}/apps/$pkgname.png"
  done
  # install -Dvm644 browser/branding/librewolf/content/about-logo.png \
    # "$pkgdir/usr/share/icons/hicolor/192x192/apps/$pkgname.png"
  install -Dvm644 ${srcdir}/default192x192.png \
    "$pkgdir/usr/share/icons/hicolor/192x192/apps/$pkgname.png"

  # arch upstream provides a separate svg for this. we don't have that, so let's re-use 16.png
  install -Dvm644 browser/branding/${pkgname}/default16.png \
    "$pkgdir/usr/share/icons/hicolor/symbolic/apps/$pkgname-symbolic.png"

  install -Dvm644 ../$pkgname.desktop \
    "$pkgdir/usr/share/applications/$pkgname.desktop"

  # Install a wrapper to avoid confusion about binary path
  install -Dvm755 /dev/stdin "$pkgdir/usr/bin/$pkgname" <<END
#!/bin/sh
exec /usr/lib/$pkgname/librewolf "\$@"
END

  # Replace duplicate binary with wrapper
  # https://bugzilla.mozilla.org/show_bug.cgi?id=658850
  ln -srfv "$pkgdir/usr/bin/$pkgname" "$pkgdir/usr/lib/$pkgname/librewolf-bin"
  # Use system certificates
  local nssckbi="$pkgdir/usr/lib/$pkgname/libnssckbi.so"
  if [[ -e $nssckbi ]]; then
    ln -srfv "$pkgdir/usr/lib/libnssckbi.so" "$nssckbi"
  fi
}

Re: No more Librewolf?

Reply #12
I updated and tried again, with the latest Rust and using the standard --enable-lto=cross,full this time, although still not the PGO bit and using the existing src and PKGBUILD I'd downloaded before to save a bit of time, and it built fine again. At various points in the build, including near the start, it has a message like this which could be worth checking if it gives the right result for your system:
Code: [Select]
  Parallelism determined by memory: using 4 jobs for 4 cores based on 7.7 GiB RAM and estimated job size of 1.0 GiB
 0:27.40 /usr/bin/make -f client.mk -j4 -s
The build clearly detects and adapts itself to the hardware, so that is something to consider, whether that is going wrong. Possibly it could be some complication from chroots, containers, vm's or something? Or some hardware specific memory management issue, perhaps try different kernel versions? I used the linux-zen kernel, 6.5.2-zen1-1-zen.

Re: No more Librewolf?

Reply #13
Building librewolf seems jinxed somehow; now the error is:

Code: [Select]
Site not up-to-date reason: "/build/librewolf/src/mozbuild/srcdirs/librewolf-117.0.1-1-cfe13ca7d490/_virtualenvs/common" does not exist
AFTER MACH PACKAGE
Firefox exited with code -4 during profile initialization

The mentioned directory does exist prior to the time of the error, and is owned and accessible by the build user.

Tried with 1 job and with 12 jobs, tried version 117.0 and version 117.0.1, also tried with gcc.
Even replaced the complete build code block with that from firefox.

There are many bugs and reports related to this error, but alas no clear fix.

artist

Re: No more Librewolf?

Reply #14
There doesn't seem to be much noise over on Librewolf's Arch Issues Page regarding failures to build:
https://codeberg.org/librewolf/issues/issues

I did find one issue made regarding a build attempt for Endeavor OS:
https://codeberg.org/librewolf/issues/issues/1597

But am unsure if that even remotely helps as the issue poster  just dumped the compile logs...

I mean, I'm happy to just grab it from the AUR, and worse case scenario I can simply heavily configure  Vanilla Firefox, I just liked having some of the work done for me out of the box.

My sincere thanks go out to the Artix devs for the extensive work and explanations though. Always appreciate the insights and hard work.