Bug 6239 - An "everything" directory for X.org releases/individual would be useful for packagers
Summary: An "everything" directory for X.org releases/individual would be useful for p...
Status: RESOLVED NOTOURBUG
Alias: None
Product: freedesktop.org
Classification: Unclassified
Component: Administration (show other bugs)
Version: unspecified
Hardware: All All
: high normal
Assignee: fd.o Admin Massive
QA Contact:
URL: http://xorg.freedesktop.org/releases/...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-03-13 08:48 UTC by David Coulthart
Modified: 2006-03-12 15:25 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Attachments

Description David Coulthart 2006-03-13 08:48:07 UTC
I wasn't sure whether this should be directed at sitewranglers or the xorg-team,
but hopefully it will reach the right people.

For http://xorg.freedesktop.org/releases/X11R7.0/src/ and other "releases" there
is a subdirectory "everything" that contains all of the packages for that release.

However, http://xorg.freedesktop.org/releases/individual/ does not have an
everything directory.

Having an everything directory would make it much easier for distributions to
easily embed URLs in packaging files.  In particular, I am interested in this
for easing packaging of modular X.org for rPath Linux.

In mentioning this on IRC, it seemed one of the concerns was making sure the
links were up-to-date.  One possible solution for this is to have a simple cron
job that runs finding all files under /releases/individual not in everything and
symlinks them into everything  For exampele,

cd /path/to/releases/individual/everything
for i in `find .. -type f -a ! -path '../everything/*'`
do
ln -sf $i
done
Comment 1 Daniel Stone 2006-03-13 09:10:01 UTC
how would it help packagers?  tarball releases are currently announced to xorg@
(although we could fix that and create xorg-announce@ for release announcements,
or announce them to release-wranglers@ as well) already.  the cron script just
means that there's a lag in between tarballs being uploaded and being available
at the second URL.

i did the modular packaging for ubuntu/debian, and I don't see how an
'everything' directory would've helped in the slightest.  a watch script can
just as easily contain xserver/driver/proto/lib/app/font as everything, and ...
yeah.

(noting that, over time, an 'everything' directory would probably threaten the
maximum-inodes-in-directory limit, over time; noting that a) we'd be expected to
keep 'everything' around basically forever, and b) every release has a .tar.gz
and a .tar.bz2.  other formats, such as rzip, will probably also be added when
they get some element of popularity.)
Comment 2 David Coulthart 2006-03-13 09:35:14 UTC
For rPath recipes, you use a line like the following to reference the source for
a package:

r.addArchive('http://foo.bar.com/baz-1.0.tar.bz2')

rPath recipes support inheritance (they're really just specially crafted Python
classes), so I created a base recipe for all modular X.org packages that does
the appropriate r.addArchive() call:

r.addArchive('http://xorg.freedesktop.org/releases/%(xrelease)s/src/everything/%(name)s-%(xrelease)s-%(version)s.tar.bz2')

This allows most recipes to be very simple and just define the name and version
& everything else related to building the package just comes from the inheritance.

But since there is no everything directory for individual releases, this
inheritance can't be used when packages are updated between full X.org releases.
 I'm forced to either specify a full URL for each updated package or do some
work to tell what type of package it is (i.e., driver, lib, proto, ...) just to
get the URL even though the rest of the building of the package does not need to
distinguish this.

I understand the concern over the maximum-inodes-in-directory limit.  The only
real solution I have to that is using directories with each package name.  I
know they are there within the subdirectories already but their use has been
discouraged, though I'm not certain why.  It is also unclear to me why there is
concern over hitting the maximum inodes limit in an everything directory as
opposed to say the drivers directory which I imagine will likely have the most
releases in it over time.  While it would happen sooner in the everything
directory I can't imagine the drivers directory would be very far behind it.
Comment 3 Daniel Stone 2006-03-13 09:55:19 UTC
for the drivers directory to hit maximum inodes, every single module would need
to make 219 releases.  assuming 6 tarballs for every module but ati, nv, sis and
i810, those four would need to make 1896 releases each.  that's ... a lot.

the reason we moved away from module-specific subdirs was, funnily enough,
packagers.  the gentoo people wanted to move closer to a single subdirectory as
well, so it would be easier to see what's going on.  in theory, an rss feed
isn't even out of the question.

this seems to be entirely a conary-specific issue.  solving it doesn't seem
entirely difficult (certainly no more difficult than writing competent
descriptions for each, or a basic copyright file, or ...)?  if you wanted to be
portable across URLs in future, why not something like this?
class BaseXorgPackage:
    def addXorgArchive(component, name, version):
       
r.addArchive("http://xorg.freedesktop.org/releases/individual/%s/%s-%s.tar.bz2"
% (component, name, version))
Comment 4 David Coulthart 2006-03-13 10:10:45 UTC
Ok, I'm willing to accept this one and try to come up with something in conary.
 I guess my point was that the component is almost completely useless for
packaging purposes except for the URL, so having to maintain that data in the
recipe seems wasteful.  As a comparison point, GNOME's directory structure makes
it very easy to package their software with conary:

r.macros.majversion = '.'.join(r.version.split('.')[0:2])
r.addArchive('ftp://ftp.gnome.org/pub/GNOME/sources/%(name)s/%(majversion)s/%(name)s-%(version)s.tar.bz2')

But I will accept how X.org has decided to do it & figure something out for conary.

Can you offer a bit more info about Debian watch files?  I found:

http://dehs.alioth.debian.org/uscan.html

but skimming that, it's still unclear to me where these watch files are
maintained.  Are they part of the debian source package?  Sorry if I have some
terminology wrong, I'm not familiar with Debian packaging.
Comment 5 Daniel Stone 2006-03-13 10:25:49 UTC
no worries.  obviously our structure could be a bit better, but that's already
how it's been set up, so it's a bit hard to change now.

the watch files are in our source packages, yeah (in the debian/ directory). 
they're a brutally simple regex (e.g.
http://xorg.freedesktop.org/releases/lib/libxkbfile-(.*).tar.bz2), and it just
tells you when new upstream versions are around.

if you're fixing it for conary, i'm going to resolve this as notourbug.  cheers.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.