Bug 94737

Summary: Git history is broken; prevents github upload
Product: Mesa Reporter: Ulrich Sibiller <uli42>
Component: OtherAssignee: mesa-dev
Status: RESOLVED FIXED QA Contact: mesa-dev
Severity: normal    
Priority: medium CC: brianp, daniel
Version: git   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Ulrich Sibiller 2016-03-29 10:09:27 UTC
As some already posted in https://lists.freedesktop.org/archives/mesa-dev/2016-February/106268.html the project's git history is broken. While this might look like a minor cosmetical problem it has severe impact on people who want to clone the repo to github. Those commits prevent the upload:

$ git clone git://anongit.freedesktop.org/mesa/mesa
Cloning into 'mesa'...
remote: Counting objects: 790257, done.
remote: Compressing objects: 100% (117141/117141), done.
remote: Total 790257 (delta 673546), reused 785057 (delta 669000)
Receiving objects: 100% (790257/790257), 146.18 MiB | 3.67 MiB/s, done.
Resolving deltas: 100% (673546/673546), done.
Checking connectivity... done.
Checking out files: 100% (4880/4880), done.
$ cd mesa
$ git remote rename origin upstream
$ git remote add origin git@github.com:<user>/<repo>.git
$ git push -u origin --tags
Counting objects: 740005, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (113457/113457), done.
remote: error: object feb356caff82e996ba0b898c02383fdfa3effc5f:
missingSpaceBeforeDate: invalid author/committer line - missing space
before date
remote: fatal: Error in object
error: pack-objects died of signal 13
error: failed to push some refs to 'git@github.com:<user>/<repo>.git'
Comment 1 Daniel Stone 2016-03-29 11:55:43 UTC
Looks like the R300_DRIVER_0 tag just needs to be deleted; it's one of the tags that doesn't really add any value anyway. Just deleting that would let you push.

Brian, we already mirror Wayland to GitHub; would you like us to set up a Mesa mirror as well?
Comment 2 Ulrich Sibiller 2016-03-29 12:02:12 UTC
Unfortunately deleting the single tag does not help. There are 30 or more commits that are having that problem, see the list in the mentioned posting: https://lists.freedesktop.org/archives/mesa-dev/2016-February/106268.html

BTW: xorg/lib/libX11 has the same problem, there are probably more repos with this error.
Comment 3 Daniel Stone 2016-03-29 12:23:16 UTC
All of these are tags, specifically:
before_upgrade_03_01_05
blended_fountain
embedded-1-20030120
embedded-1-20030224
embedded-1-20030305
embedded-1-20030324
embedded-1-20030417
embedded-1-20030502
gliding_penguin
jump_and_click
kw-mesa-1
mesa-6_5-20060712
mesa_20050504
mesa_20050526
mesa_20050610
mesa_20050715
mesa_20060201
mesa_20060325
mesa_3_1
mesa_3_1_beta_3
mesa_3_2
mesa_3_2_1
mesa_3_2_beta_1
mesa_3_3
mesa_3_4
mesa_3_4_1
mesa_3_4_2
mesa_3_5
mesa_4_0
mesa_4_0_1
mesa_4_0_2
mesa_4_0_3
mesa_4_0_4
mesa_4_1
mesa_5_0
mesa_5_0_1
mesa_6_3
mesa_6_3_1
mesa_6_3_1_1
mesa_6_3_2
mesa_6_4
mesa_6_4_1
mesa_6_4_2
mesa_6_5
mesa_6_5_1
mesa_6_5_2
mesa_texman_20060210
noisy_cube
red_tinted_cube
rotating_gears
shimmering_gears
start
texman_0_1_20060325
texman_0_1_20060330
texman_0_1_20060621
texman_0_1_20060705
texman_0_1_pre_ttm
texmem_0_2_20060912
texmem_0_3_20060923
texmem_0_3_20060929
the_perfect_frag
trunk_20040329
unichrome-last-xinerama
vtx-0-2-21112003-freeze
vtx-0-2-24112003


I would suggest that it's only the version tags (mesa_[3456]_*) which need to be preserved, and those can just be retagged to the same commit. The rest seem entirely deletable.
Comment 4 Emil Velikov 2016-03-29 12:54:14 UTC
There is a bit more info (including a link to possible git fix) [1]. Fwiw I'm all for creating a mirror of mesa, libdrm and/or others on github, although the tags will need to be manually resolved.

Afaics most/all of them are signed, thus removing them from fd.o will be impossible without rewriting git history ?

[1] https://lists.freedesktop.org/archives/mesa-dev/2015-June/087904.html
Comment 5 Brian Paul 2016-03-29 18:07:46 UTC
(In reply to Daniel Stone from comment #1)
> Brian, we already mirror Wayland to GitHub; would you like us to set up a
> Mesa mirror as well?

Just to serve as a backup?  Sounds OK to me <shrug>.
Comment 6 Daniel Stone 2016-03-29 18:10:28 UTC
(In reply to Brian Paul from comment #5)
> (In reply to Daniel Stone from comment #1)
> > Brian, we already mirror Wayland to GitHub; would you like us to set up a
> > Mesa mirror as well?
> 
> Just to serve as a backup?  Sounds OK to me <shrug>.

Both as a backup, and also to make it easier for people to fork/branch Mesa and submit back. (We disable most of the usual GH functionality, e.g. issues.
Comment 7 Ernst Sjöstrand 2016-03-30 11:20:28 UTC
Some alternatives: A have a hardcoded filter when mirroring to Github? 
Can you move them to a non-default namespace like refs/broken/ ?
Comment 8 Daniel Stone 2016-03-30 11:35:37 UTC
(In reply to Emil Velikov from comment #4)
> There is a bit more info (including a link to possible git fix) [1]. Fwiw
> I'm all for creating a mirror of mesa, libdrm and/or others on github,
> although the tags will need to be manually resolved.

Indeed.

> Afaics most/all of them are signed, thus removing them from fd.o will be
> impossible without rewriting git history ?

Not that I'm aware of ... ? Tags are individual standalone Git objects, with no references to them other than requesting the list of tags from the server, so removing these objects has no effect on any other objects (the usual sense of rewriting history).

It's true that the ones to be preserved would need to be recreated; from what I understand, this would require a manual force-fetch from users.

> [1] https://lists.freedesktop.org/archives/mesa-dev/2015-June/087904.html

Oh, interesting. I guess we could mostly wait it out: keep the tags on fd.o, and just not bother pushing them to GitHub. Deleting most of the r300 tags (red_tinted_cube) is probably sensible anyway, as I don't imagine anyone would want them ...
Comment 9 Daniel Stone 2016-03-30 12:07:37 UTC
I've created https://github.com/mesa3d/mesa now with all the version tags/branches; happy to set up other repositories on request.
Comment 10 Ulrich Sibiller 2016-03-30 12:21:09 UTC
Well, thanks for that. However, that mirror's oldest release tag is mesa_6_5_3_rc2. I need mesa_6_4_1 for the stuff I am currently working on...
Comment 11 Daniel Stone 2016-03-30 12:27:09 UTC
(In reply to Ulrich Sibiller from comment #10)
> Well, thanks for that. However, that mirror's oldest release tag is
> mesa_6_5_3_rc2. I need mesa_6_4_1 for the stuff I am currently working on...

Those tags are missing whilst we figure out what to do with them. In the meantime, those commits are there (e.g. https://github.com/mesa3d/mesa/commit/53e595017e7f34a52117b1b7f731003c55a395c1), so you should be able to work with it.
Comment 12 Ulrich Sibiller 2016-03-30 12:29:07 UTC
Ok, thank you.
Comment 13 Ulrich Sibiller 2016-03-30 15:41:13 UTC
See https://sunweavers.net/blog/node/36 for information how we fixed it for libX11.git. Should work for Mesa, too, but has not been tried yet.
Comment 14 Daniel Stone 2016-03-30 15:48:37 UTC
(In reply to Ulrich Sibiller from comment #13)
> See https://sunweavers.net/blog/node/36 for information how we fixed it for
> libX11.git. Should work for Mesa, too, but has not been tried yet.

This would give people differing tags (and if signed, signed by different people) between GitHub and fd.o, which could well be enough to make people worry. We can't easily push these into fd.o either, because tags don't auto-update when you pull, and it would again look odd.

I've raised a support request with GitHub to ask they relax this restriction, but there's not a lot we can do in the meantime that won't make people unnecessarily worried about the integrity of their repository.
Comment 15 Daniel Stone 2016-04-01 11:32:27 UTC
All the old tags have now been pushed.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.