Skip to main content ↑ ↓
Topic: Is there any new change in gitea repos structure? (Read 547 times) previous topic - next topic
0 Members and 3 Guests are viewing this topic.

Is there any new change in gitea repos structure?

Hi all!
It seems, another gitea repos update for packages was recently without any announcement at all. As i can see, new repos structure contains PKGBUILD in the root of repo itself. For me it poses some questions for ARMtix builds:
1) how do i determine which repo package belongs to?
2) how to make sure current PKGBUILD is for stable repos, not gremlins/goblins?
3) how to spot package migration between repos?
ARMtix

Re: Is there any new change in gitea repos structure?

Reply #1 –
News to me too, some quick testing shows the repo head  PKGBUILD's aren't at stable versions:
Code: [Select]
linux PKGBUILD:
_ver=6.4.4
_rel=1
_tag=arch${_rel}
_artix=artix${_rel}

pkgbase=linux

pacman -Syi linux
Repository      : system
Name            : linux
Version         : 6.4.3.artix1-2
I suppose it's possible to checkout the correct version after getting the current value with Pacman.

Re: Is there any new change in gitea repos structure?

Reply #2 –
@artoo: has been working hard on redoing our tools to move to structured data instead. This finally possible since Arch now uses git per package. Anyways if you check inside the .artixlinux linux directory, there is now a pkgbase.yaml file that shows the exact state of the package and all the repos it is in.

Re: Is there any new change in gitea repos structure?

Reply #3 –
Quote from: Dudemanguy – on
there is now a pkgbase.yaml file that shows the exact state of the package and all the repos it is in.
I just figured that out then saw your post that would have saved me bothering  :D
I'm sure this has been done for good reasons to make your (devs) workflow better.

But it's not very user friendly for the users. Especially newer users who want to tinker and dabble. They may not realise they have acquired the 'staging' build files.
Users should have a simple method of downloading the build files for the current stable version of a given package.

They did. Now they don't.

In my perfect Barbie & Ken world they'd be another repo on Gitea, Github, Gitlab or whatever with all the build files for each package and repo separated by repo.
reponame/packagename/

Re: Is there any new change in gitea repos structure?

Reply #4 –
Quote from: Dudemanguy – on
@artoo: has been working hard on redoing our tools to move to structured data instead. This finally possible since Arch now uses git per package. Anyways if you check inside the .artixlinux linux directory, there is now a pkgbase.yaml file that shows the exact state of the package and all the repos it is in.
Compared to the old structure, the new one feels like a violation of Unix philosophy (which to me looks like on of the principles behind Artix existence). Based on file system hierarchy i could get the same info - which repo package belongs to, which version does it have, and could easily distinguish between testing and stable PKGBUILDs. Moreover, when i want to know if a stable version in git is updated, i can iterate over git commits from the last one and only check for files changed.
The new structure lacks that inner simplicity: now i have to check for pkgbase.yaml changes and only trigger stable update... when? Only when there are no entries for testing repos in pkgbase.yaml? And is there any guarantee both system and world entries will never be in this file at the same time?
I totally agree with @gripped, the structure based on explicit mentions of repos in file system hierarchy of the repo is much more clear, and, honestly, i don't see a way how CI could benefit from parsing yaml file over it.
ARMtix

Re: Is there any new change in gitea repos structure?

Reply #5 –
Quote
The new structure lacks that inner simplicity: now i have to check for pkgbase.yaml changes and only trigger stable update...

Any thing that uses yaml becomes a nightmare in the end  :(
 

Re: Is there any new change in gitea repos structure?

Reply #6 –
I don't get the complaining.
We have now a structure that produces way less history, because no useless copies in folders.
It is very easy to query yaml with yq/go-yq even from shell, it is the counterpart to jq for json.
All devops etc return usually json or yaml data, so yaml just intergrates nicely into any CI, which are usually yaml based.
The folder structure was a glue to make it easy to have arch compat, but thats gone now, thankfully, because it had lot of downsides.
Now we have simply arch remotes for upstream set. Much better.

Re: Is there any new change in gitea repos structure?

Reply #7 –
Quote from: artoo – on
I don't get the complaining.
More questioning than complaining. An announcement would have prevented any confusion.

What would be helpful imho would be to tag the commits, with new versions added, with the version. Then switching versions in scripts would be simple. Unless I'm missing something and it already is ?

Re: Is there any new change in gitea repos structure?

Reply #8 –
Quote from: gripped – on

What would be helpful imho would be to tag the commits, with new versions added, with the version. Then switching versions in scripts would be simple. Unless I'm missing something and it already is ?


Well, we just switched to something I always had in mind, but upstream arch prevented it.
This is just initial stage.
We probably gonna have a centralized git repo that has yaml tracking, ie our CI would push a tag on successful build, and then write a yaml data file in the global yaml repo db, easy with the yaml already present in the pkgbase git repo.
That's roughly somewhat the next phases of this thing started now.

I should add, I chose go-yq over the yq python implementation for the tools

Re: Is there any new change in gitea repos structure?

Reply #9 –
Quote from: artoo – on
It is very easy to query yaml with yq/go-yq even from shell, it is the counterpart to jq for json.
But in order to get stable PKGBUILD it is required to checkout several commits until i find the one in which pkgbase.yaml doesn't contain testing/staging entries. Also, there is no protection from human mistake when repo migration occurs: at some point pkgbase.yaml may contain two entries for system+world, for example. It could be mitigated by parsing diff for each commit but this task is much harder compared to parsing yaml.
For example, if repo for package consists only of directories corresponding to repos to which package belongs, there is no need to checkout commits in search for stable version and a mistake i described can be mitigated by parsing list of files in diff which is rather trivial
ARMtix

Re: Is there any new change in gitea repos structure?

Reply #10 –
We won't go back to folders, kind of pointless to discuss really.
Tags is something that will be implemented at some point though.

Re: Is there any new change in gitea repos structure?

Reply #11 –
As such, a message for ARMtix users: i don't know when i could implement the nessesary changes to adapt build system to this new world, so till then builds will be (again) freezed.
In case someone is brave enough to dig into shitcode, here is the source of build system itself
ARMtix

Re: Is there any new change in gitea repos structure?

Reply #12 –
I also thought the old folder structure was strange, actually, and I don't think it was especially well documented anywhere either. For finding the current goblins / gremlins / stable versions, how about having the yaml file contain a field for each of these that points to the relevant commit where it can be found, or null if it doesn't exist because stable or testing is the newest version available?  If the yaml file had an extra 3 informational data fields something like this:
Code: [Select]
current_commit:
  goblins: null
  gremlins : abc123commithash
  stable: xyz456commithash
then someone could parse the file in git head with whatever was convenient (ie it wouldn't need to be yaml) and directly git checkout the version you wanted. Perhaps the tag idea is better or easier to implement, it's just a suggestion as an alternative.