Bug 94889

Summary: provide support for the audible audio file type
Product: xdgmime Reporter: kendell clark <coffeekingms>
Component: xdgmimeAssignee: Hans Petter Jansson <hpj>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: medium CC: bugzilla, hpj
Version: unspecified   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments: xml file providing basic support for the audible audio format
0001-Add-Audible.Com-mime-type.patch
0001-Add-Audible.Com-mime-type.patch
0001-Add-Audible.Com-mime-type.patch
Add Audible.com mime type

Description kendell clark 2016-04-11 07:07:02 UTC
Created attachment 122855 [details]
xml file providing basic support for the audible audio format

audible audio files are containers with other audio data (usually mpeg layor 3) inside of them. I'll attach an xml file which, when incorporated into the mime database provides support for this filetype. It matches purely on file extension for now, but I'll be happy to provide unique identifiers if I can find one. I've had no luck finding one, and I only know how to use mime editor, which can't seem to add offset and integer or string identifiers yet.
Comment 1 kendell clark 2016-04-11 07:18:26 UTC
I've got some additional information. I've found a magic number for audible format files. The magic number is
1469084982
 Taken from the ffmpeg source code, in libavformat/aadec.c. I'm looking for a hex offset. If I get that, I'll post that hear.
Comment 2 Bastien Nocera 2016-04-11 10:15:35 UTC
Please follow the instructions in the HACKING file to add support for a new mime-type. Make sure to also include a test file, as mentioned in that file.
Comment 4 kendell clark 2016-04-11 15:47:26 UTC
hi
Hacking file? I haven't found one but I'll look on the website and
follow the instructions I see. Did I make a mistake by including the xml
file update-mime-database created?
Thanks
Kendell Clark


bugzilla-daemon@freedesktop.org wrote:
> Bastien Nocera <mailto:bugzilla@hadess.net> changed bug 94889
> <https://bugs.freedesktop.org/show_bug.cgi?id=94889>
> What 	Removed 	Added
> Status 	NEW 	NEEDINFO
>
> *Comment # 2 <https://bugs.freedesktop.org/show_bug.cgi?id=94889#c2>
> on bug 94889 <https://bugs.freedesktop.org/show_bug.cgi?id=94889> from
> Bastien Nocera <mailto:bugzilla@hadess.net> *
> Please follow the instructions in the HACKING file to add support for a new
> mime-type. Make sure to also include a test file, as mentioned in that file.
> ------------------------------------------------------------------------
> You are receiving this mail because:
>
>   * You reported the bug.
>
Comment 5 kendell clark 2016-04-11 15:50:55 UTC
hi
I'll be happy to provide a test file. I have two audible audio files.
The problem is, I can't make audible files hear, only convert them. So
I'll have to copy an entire book to the site for testing. This is legal,
I believe, but my main concern is how to upload such a large file (I
think it's around 15mb) without  bugzilla saying my file is too large.
Please be patiente with me, I'll get the hang of this eventually (grin)
Thanks
Kendell Clark


bugzilla-daemon@freedesktop.org wrote:
> Bastien Nocera <mailto:bugzilla@hadess.net> changed bug 94889
> <https://bugs.freedesktop.org/show_bug.cgi?id=94889>
> What 	Removed 	Added
> Status 	NEW 	NEEDINFO
>
> *Comment # 2 <https://bugs.freedesktop.org/show_bug.cgi?id=94889#c2>
> on bug 94889 <https://bugs.freedesktop.org/show_bug.cgi?id=94889> from
> Bastien Nocera <mailto:bugzilla@hadess.net> *
> Please follow the instructions in the HACKING file to add support for a new
> mime-type. Make sure to also include a test file, as mentioned in that file.
> ------------------------------------------------------------------------
> You are receiving this mail because:
>
>   * You reported the bug.
>
Comment 6 kendell clark 2016-04-11 17:25:20 UTC
hi
Okay, I've found the hacking file, and have checked out the shared mime
info database via git. Where should I upload an audible audio file to
run the test against? I might need some help with this as I can't write
c code or understand xml very well, but I'm willing to do my part. Sorry
for the spam, I usually respond to all my email at once, and I sometimes
miss comments in the flood.
Thanks
Kendell Clark


bugzilla-daemon@freedesktop.org wrote:
>
> *Comment # 3 <https://bugs.freedesktop.org/show_bug.cgi?id=94889#c3>
> on bug 94889 <https://bugs.freedesktop.org/show_bug.cgi?id=94889> from
> Bastien Nocera <mailto:bugzilla@hadess.net> *
> https://www.reddit.com/r/audiobooks/comments/3ek5af/ffmpeg_supports_audible_aax_files_now/
> ------------------------------------------------------------------------
> You are receiving this mail because:
>
>   * You reported the bug.
>
Comment 7 kendell clark 2016-04-20 05:51:30 UTC
I've gotten a little farther. I can't find the magic number anywhere in audible files, either opening them up in a text editor or using a hex editor on them, so I've switched to using a string, Muser_alias. The problem is, I can't get the offset right. Right now, any file with the extension .aa with any text in it at all is shown to be an audible file, which isn't right. I'm stuck.
Comment 8 Hans Petter Jansson 2016-07-02 00:01:51 UTC
The magic seems to be a four-byte hex sequence 0x57, 0x90, 0x75, 0x36 starting at four bytes into the file.

The magic seems to be different for aax: There's a hex sequence
0x61, 0x61, 0x78, 0x20 starting at offset 8. It corresponds to the ASCII string
"aax" followed by a space.
Comment 9 Hans Petter Jansson 2016-07-03 19:17:37 UTC
Created attachment 124874 [details] [review]
0001-Add-Audible.Com-mime-type.patch

Patch that adds Audible.Com audio support corresponding to audio/x-pn-audibleaudio, audio/vnd.audible, audio/vnd.audible.aax.

Those were the MIME types I found referenced in a search, except for audio/audible, which I left out since it's not IANA registered.

make check passes with current xdgmime.
Comment 10 Hans Petter Jansson 2016-07-03 19:20:17 UTC
The sample files I have are kind of big at 3.1MB and 24MB. Ideally they would be a couple of kilobytes, but I have no way of producing aa/aax files here. It would be possible to just truncate the files, but then they technically wouldn't be valid samples anymore.
Comment 11 kendell clark 2016-07-05 04:40:52 UTC
hi
thank you very much for your help. I've just applied this patch to my own copy of the shared mime info database and it works like a charm. I've changed my audible samples file extension and it doesn't fool the system. I can't wait for the new shared mime info update to come out. This is fantastic work!
Comment 12 kendell clark 2016-07-05 05:47:52 UTC
hi
I've got another sample from the inaudible project that might do, it's a 32 kb enhanced audible audio file. Would that do as a test file? I can also find a normal .aa file if 3 mb is too large to work
Comment 13 kendell clark 2016-07-05 05:49:36 UTC
or not, turns out it's an html file I saved. This brings up one point. Are you sure the magic works? It misidentified an html file as an audible file because of the extension. I'm not criticizing
Comment 14 kendell clark 2016-07-05 05:51:29 UTC
hi
I'm so sorry for the spam. I've just removed the file extension from my audible files and they still say they're audible, so the magic seems to work. On older audible format files at least. I'll find an enhanced audible file and see if it works there. This should be the last comment I post hear tonight to reduce the spam
Comment 15 Hans Petter Jansson 2016-08-01 19:24:42 UTC
(In reply to kendell clark from comment #14)
> hi
> I'm so sorry for the spam. I've just removed the file extension from my
> audible files and they still say they're audible, so the magic seems to
> work. On older audible format files at least. I'll find an enhanced audible
> file and see if it works there. This should be the last comment I post hear
> tonight to reduce the spam

No worries. Were you able to find shorter samples?
Comment 16 kendell clark 2016-08-03 04:46:52 UTC
alas, I wish I could find shorter samples. Would the people who run the mime database accept such a large sample? If not, I could try to create one that's small enough to still contain the magic so that it can go into the mime database as a test but not large enough so that it's a burden on anyone maintaining or checking out the git repo. Maybe ... oh, say, 4 mb? It wouldn't be a technically correct audible file no, but the only way you'd get that would be to have someone generate you an audible file without any audio, and I don't know how they're made. Amazon probably knows but they're not going to help us because we won't proprietize and eula the resulting data. We're open source, and to some companies that's a red flag.
Comment 17 Hans Petter Jansson 2016-08-24 16:19:44 UTC
Bastien, do you have time to look at the patch? I doubt it's controversial, but I'd like a go-ahead to commit. I'll add truncated audio samples in a separate commit.
Comment 18 Bastien Nocera 2017-04-26 11:50:01 UTC
(In reply to Hans Petter Jansson from comment #17)
> Bastien, do you have time to look at the patch? I doubt it's controversial,
> but I'd like a go-ahead to commit. I'll add truncated audio samples in a
> separate commit.

Please add them in the same commit. I can't ack this patch if I can't test it. Please add the test files inside the git-formatted patch.

+      <match type="big32" value="1469084982" offset="4"/>

This doesn't look super readable, could you please add a comment above as to the source of the magic?
Comment 19 Hans Petter Jansson 2017-04-26 16:33:07 UTC
Created attachment 131055 [details] [review]
0001-Add-Audible.Com-mime-type.patch

Updated patch including test files and comment on magic.
Comment 20 Hans Petter Jansson 2017-04-26 16:36:48 UTC
Created attachment 131056 [details] [review]
0001-Add-Audible.Com-mime-type.patch

Make XML comment formatting conform too.
Comment 21 Bastien Nocera 2017-04-26 16:39:15 UTC
Created attachment 131057 [details] [review]
Add Audible.com mime type
Comment 22 Bastien Nocera 2017-04-26 16:40:27 UTC
commit 107ca505d667f3e11facea5fe31d564e91007185
Author: Hans Petter Jansson <hpj@cl.no>
Date:   Sun Jul 3 21:12:51 2016 +0200

    Add Audible.com mime type
    
    https://bugs.freedesktop.org/show_bug.cgi?id=94889

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.