Bug 1650 - make distcheck fails because of non-empty MIME files
Summary: make distcheck fails because of non-empty MIME files
Status: RESOLVED WONTFIX
Alias: None
Product: shared-mime-info
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: x86 (IA32) Linux (All)
: high major
Assignee: Shared Mime Info group
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-10-16 08:12 UTC by Gustavo J. A. M. Carneiro
Modified: 2011-05-25 07:07 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:


Attachments
patch (9.30 KB, patch)
2007-01-05 06:53 UTC, Yevgen Muntyan
Details | Splinter Review

Description Gustavo J. A. M. Carneiro 2004-10-16 08:12:39 UTC
My packages 'make distcheck' fails with this error:

ERROR: files left after uninstall:
./share/mime/XMLnamespaces
./share/mime/globs
./share/mime/magic

The Makefile.am contains:
install-data-hook:
	update-mime-database $(datadir)/mime
uninstall-hook:
	update-mime-database $(datadir)/mime

./share/mime/globs contains:
# This file was automatically generated by the
# update-mime-database command. DO NOT EDIT!

./share/mime/magic contains:
MIME-Magic

Looking at the code, "make distcheck" ignores empty files.  The problem is that
those two files are not empty, and they should be.
Comment 1 Christophe Fergeau 2004-10-16 09:52:31 UTC
Why do you think those files should be empty? If you are installing new mime
types, the whole point of update-mime-database is to generate non empty files,
if you don't want to install new mime types, you don't need to run
update-mime-database.
However, there probably is an issue with those autogenerated files during make
install which can't be removed using make clean. Maybe there's a way to tell
automake to ignore them?
Comment 2 Gustavo J. A. M. Carneiro 2004-10-16 10:03:42 UTC
You don't understand.  The error is detected after "make uninstall".  Automake
sees that some files were created during "make install", but were not removed
during "make uninstall".
Comment 3 Christophe Fergeau 2004-10-16 10:16:46 UTC
Yep, I perfectly understood. What I was saying is that the files automake is
complaining about are created by update-mime-database, that running
update-mime-database isn't necessary if your app doesn't have to add its own
mime types and that there is indeed an issue with update-mime-database which is
not really able to cleanly remove the files it created.

Maybe what you are trying to make me understand is that if share/mime/packages
is empty, then update-mime-database shouldn't create any file (which would fix
your problem)?
Comment 4 Gustavo J. A. M. Carneiro 2004-10-16 10:47:21 UTC
Yes, but s/shouldn't create any file/should remove files with no information/
Comment 5 Christophe Fergeau 2004-10-16 11:02:03 UTC
yeah, more or less the same actually, update-mime-database reads xml files from
share/mime/packages and creates new globs and magic files from their content. So
when there are no files, it should remove the existing files (as it does even if
there are files), but it shouldn't create new ones.
Comment 6 Gustavo J. A. M. Carneiro 2005-02-27 10:18:29 UTC
No progress on this?  I think update-mime-database should remove (or truncate to
zero bytes) the files XMLnamespaces, globs, and magic, after updating them, if
they contain no MIME information.  It's the only way to ensure "make distcheck"
passes for an application that installs mime types.

  You only don't get more reports because shared-mime-info already provides most
mime types.  But applications with new mime types have to install custom mime
type files and suffer the distcheck warnings :|
Comment 7 Bastien Nocera 2006-07-09 03:19:35 UTC
(In reply to comment #6)
> No progress on this?  I think update-mime-database should remove (or truncate to
> zero bytes) the files XMLnamespaces, globs, and magic, after updating them, if
> they contain no MIME information.  It's the only way to ensure "make distcheck"
> passes for an application that installs mime types.

Why does your application package those files? update-mime-database should be
called in your post installation section, or post uninstallation. You shouldn't
be packaging it.
Comment 8 Gustavo J. A. M. Carneiro 2006-07-09 03:42:36 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > No progress on this?  I think update-mime-database should remove (or truncate to
> > zero bytes) the files XMLnamespaces, globs, and magic, after updating them, if
> > they contain no MIME information.  It's the only way to ensure "make distcheck"
> > passes for an application that installs mime types.
> 
> Why does your application package those files? update-mime-database should be
> called in your post installation section, or post uninstallation. You shouldn't
> be packaging it.
> 

I am _not_ packaging them.  Here's what happens

during make install:
 1. appname.xml is installed
 2. update-mime-database is executed
   2a. XMLnamespaces, globs, and magic are created (no shipped) by
update-mime-database; these files did _not_ exist before.

during make uninstall:
 1. appname.xml is removed
 2. update-mime-database is executed
   2a. XMLnamespaces, globs, and magic are modified to reflect the now missing
appname.xml file

The problem is that in the end XMLnamespaces, globs, and magic are not removed
and they have non-zero length.  Hence, "make distcheck" complains that 'make
uninstall' did not remove all the files that 'make install' had created before.
Comment 9 Yevgen Muntyan 2007-01-05 04:02:05 UTC
To fix distcheck put this into toplevel Makefile.am:

distuninstallcheck_listfiles = find . -type f -print | grep -v share/mime/
Comment 10 Yevgen Muntyan 2007-01-05 06:53:23 UTC
Created attachment 8302 [details] [review]
patch

This patch (tries to) removes all generated files and directories if it didn't
read any data from packages. In fact, I was more annoyed by
update-mime-database dumping core when it can't write, this patch fixes it too.
Should I open seaprate bug report for this?
Comment 11 Bastien Nocera 2007-01-07 16:13:11 UTC
(In reply to comment #10)
> Created an attachment (id=8302) [edit]
> patch
> 
> This patch (tries to) removes all generated files and directories if it didn't
> read any data from packages. In fact, I was more annoyed by
> update-mime-database dumping core when it can't write, this patch fixes it too.
> Should I open seaprate bug report for this?

I'd rather the core dump issue was filed as a different bug. As for this
particular problem, adding those generated files to the distclean target is the
thing to do, rather than rely on update-mime-database not generating empty files.
Comment 12 Yevgen Muntyan 2007-01-09 07:11:42 UTC
(In reply to comment #11)
> (In reply to comment #10)
> > Created an attachment (id=8302) [edit] [edit]
> > patch
> > 
> > This patch (tries to) removes all generated files and directories if it didn't
> > read any data from packages. In fact, I was more annoyed by
> > update-mime-database dumping core when it can't write, this patch fixes it too.
> > Should I open seaprate bug report for this?
> 
> I'd rather the core dump issue was filed as a different bug. 

OK.

> As for this
> particular problem, adding those generated files to the distclean target is the
> thing to do, rather than rely on update-mime-database not generating empty files.

Let's look at it like this: should update-mime-database generate garbage on user
computer if there's nothing to generate or not? Say, it generates mime.cache
with nothing inside, but xdgmime still reads this file and watches it, and looks
into it every time it's asked about something. Or even worse, if cache is not
used, then xdgmime will restat and reread five files. Not the worst thing a
package could do; still, let's clean after ourselves where we can. I can't do rm
-fr $prefix/share/mime in my uninstall target obviously, but it's *my* package
that leaves garbage there, and I don't want it to leave garbage.

Note, it has nothing to do with distclean.
Comment 13 Gustavo J. A. M. Carneiro 2007-01-09 07:27:10 UTC
(In reply to comment #11)
> As for this particular problem, adding those generated files to the distclean
target is the thing to do, rather than rely on update-mime-database not
generating empty files.

make distcheck doesn't do 'make distclean' to test for leftover files; it does
'make uninstall' and checks that every file that was installed is also
uninstalled.  What you propose makes no difference, it will simply not work.
Comment 14 Bastien Nocera 2007-01-09 07:45:26 UTC
(In reply to comment #13)
> (In reply to comment #11)
> > As for this particular problem, adding those generated files to the distclean
> target is the thing to do, rather than rely on update-mime-database not
> generating empty files.
> 
> make distcheck doesn't do 'make distclean' to test for leftover files; it does
> 'make uninstall' and checks that every file that was installed is also
> uninstalled.  What you propose makes no difference, it will simply not work.

You're clutching at straws. update-mimedatabase creates files, it's the
packager's job to remove the created files.
Comment 15 Yevgen Muntyan 2007-01-09 07:53:49 UTC
(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #11)
> > > As for this particular problem, adding those generated files to the distclean
> > target is the thing to do, rather than rely on update-mime-database not
> > generating empty files.
> > 
> > make distcheck doesn't do 'make distclean' to test for leftover files; it does
> > 'make uninstall' and checks that every file that was installed is also
> > uninstalled.  What you propose makes no difference, it will simply not work.
> 
> You're clutching at straws. update-mimedatabase creates files, it's the
> packager's job to remove the created files.

Automake developers do not think so for some reason. They didn't succeed
completely, but they do a good job in "make uninstall". Lot of other people do
not think so either. "make uninstall" should remove all the files, including
generated files like mime database or compiled python modules or whatever, and
if it doesn't it's a bug that needs to be fixed.
Comment 16 Gustavo J. A. M. Carneiro 2007-01-09 08:00:56 UTC
(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #11)
> > > As for this particular problem, adding those generated files to the distclean
> > target is the thing to do, rather than rely on update-mime-database not
> > generating empty files.
> > 
> > make distcheck doesn't do 'make distclean' to test for leftover files; it does
> > 'make uninstall' and checks that every file that was installed is also
> > uninstalled.  What you propose makes no difference, it will simply not work.
> 
> You're clutching at straws. update-mimedatabase creates files, it's the
> packager's job to remove the created files.

It's not safe for a packager to remove files MIME cache files that it didn't
create in the first place.  It requires complicated and brittle heuristics to
determine whether or not the file can be removed (because it is not empty but
contains no MIME information) or not.
Comment 17 Daniel Leidert 2008-02-05 11:16:43 UTC
Seems the problem has already been fixed by an appropriate uninstall-hook target.

There are some alternatives one can use

(1) Pass --disable-update-mimedb to DISTCHECK_CONFIGURE_FLAGS in Makefile.am. This will turn off the update-mime-database run during `make distcheck'. However, if the distcheck target should check the work of update-mime-database too, then this is not an option.

(2) https://bugs.freedesktop.org/show_bug.cgi?id=1650#c9 lists a second alternative. You can also define an empty distuninstallcheck target in Makefile.am. This simply turns off the detection of files left after deinstallation.

However, probably the current solution is the correct one.

But one issue occured: `./autogen.sh && make distcheck' does not work, because the top-level Makefile.am lists po/shared-mime-info.pot in EXTRA_DISTS but misses a rule to create it. A simple

po/shared-mime-info.pot:
        $(MAKE) -C $(top_srcdir)/po shared-mime-info.pot

solves the problem.

The originally repoted issue seems to be fixed.
Comment 18 Yevgen Muntyan 2008-02-05 11:45:25 UTC
(In reply to comment #17)
> Seems the problem has already been fixed by an appropriate uninstall-hook
> target.
> 
> There are some alternatives one can use
> 
> (1) Pass --disable-update-mimedb to DISTCHECK_CONFIGURE_FLAGS in Makefile.am.
> This will turn off the update-mime-database run during `make distcheck'.
> However, if the distcheck target should check the work of update-mime-database
> too, then this is not an option.

This bug is not about the shared-mime-info package, it's 
specifically about update-mime-database not removing
generated files when the mime database is empty.
Or do you mean that every package out there should have 
--disable-update-mimedb option, and have it in 
DISTCHECK_CONFIGURE_FLAGS.

> 
> (2) https://bugs.freedesktop.org/show_bug.cgi?id=1650#c9 lists a second
> alternative. 

Yes, that does work, but is ugly and not too extensible,
say, I have this:

distuninstallcheck_listfiles = find . -type f -print | grep -v share/mime/ | grep -v share/icons/hicolor

That's two tools - update-mime-database and gtk-update-icon-cache.

> You can also define an empty distuninstallcheck target in
> Makefile.am. This simply turns off the detection of files left after
> deinstallation.

It's not a solution at all.
Comment 19 Daniel Leidert 2008-02-05 13:43:59 UTC
Well, the original report was about `make distcheck' failing because of the failing distuninstallcheck target. I just spoke about this one.

However, the current uninstall target empties the directory for the known/hardcoded media types. This still leaves the problem, that faked (e.g. KDEs all/all) or unofficial (e.g. chemical/*) media types will lead to files left in the directory.

Now one could add these paths to the uninstall target too. However, then the uninstall target of shared-mime-info has to take care about media types it never created nor shipped. IMO this is impossible.

I also think, that packages shipping e.g. faked MIME types cannot uninstall files created by update-mime-database. For me (chemical-mime-data) this would mean to add 40-50 files to the uninstall target, that I did not create nor ship. I mean, I would have to walk through my XML file and get the MIME type names and add them somehow to the uninstall target. I don't think, this is reasonable.

IMHO the only acceptable way is, that update-mime-database removes the files, before it is deinstalled. Taking a quick look into your patch you seem to follow this approach. However, if I understand your patch corretcly, it just removes some files, if mime/packages is empty. But you cannot assume, that this directory is empty! Packages installing files into mime/packages do not depend on the existence of shared-mime-info. I mean, why should the removal of shared-mime-info lead to a comlete removal of all packages  that installed files to mime/packages? So your patch would still leave e.g. all/all.xml and other files.

Maybe a solution is add an explicit --uninstall option, that parses mime/packages, but instead to create files, it removes files, that would have been created, leading to removing all files ever created, but mime/packages can still exist.
Comment 20 Yevgen Muntyan 2008-02-05 13:49:06 UTC
(In reply to comment #19)
> Well, the original report was about `make distcheck' failing because of the
> failing distuninstallcheck target. I just spoke about this one.

The original bug report is about failing 'make distcheck' in packages
which do update-mime-database in their install hooks. Not 'make distcheck'
in shared-mime-info package.
Comment 21 Daniel Leidert 2008-02-05 14:46:20 UTC
Oh, ok. Most (to be honest: all) packages I know simply pass an option similar to --disable-update-(mime)database (which is almost always necessary for packagers to exist) to DISTCHECK_CONFIGURE_FLAGS. There is IMO no need to run update-mime-database during a distcheck (except maybe for shared-mime-info itself).

Ok, seems I understood the patch but not your intention.
Comment 22 Yevgen Muntyan 2008-02-05 15:27:50 UTC
(In reply to comment #21)
> Oh, ok. Most (to be honest: all)

You missed some ;)

> packages I know simply pass an option similar
> to --disable-update-(mime)database (which is almost always necessary for
> packagers to exist)

It is unnecessary (since programs do get packaged somehow). Is it 
'--enable-update-mimedatabase' or '--enable-update-mime-database'
anyway?

> to DISTCHECK_CONFIGURE_FLAGS. There is IMO no need to run
> update-mime-database during a distcheck (except maybe for shared-mime-info
> itself).

It is wrong to use such an option in distcheck, because
you may want to make sure your rule is still working
(the non-standard one, which you copied from another
project, like you do with most autoconf and automake code). 
Not a very reliable check, but still a check, better 
than nothing.

Then, --enable-update-mime-database for update-mime-database, 
--enable-update-icon-cache for gtk-update-icon-cache. Good we 
don't have update-desktop-icon-database.

I mean, of course there are workarounds, and of course you can write
much code (three lines of an autoconf macro and five lines in
Makefile.am is a *LOT* of code). But it'd be better to have
the solution in one central place. First comes automake,
good luck with that; then come some copy-pastable auto* macros,
impossible; then come the tools in question. This bug is about
fixing the tool in question to make life easier for many people around.
Not a show stopper, absolutely. But a fix isn't "fix the packages".
Comment 23 Bastien Nocera 2011-05-25 07:07:13 UTC
As mentioned in the bug, I'm not interested in adding such work-arounds in the shared-mime-info code. For GNOME and a number of other platforms, there are common macros split out, and available to all programs. Use those, and extend them to include --disable-update-mime-database, or something of that ilk.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.