Bug 814814

I opened a bug report for taglib at gibhub. But it has already been closed beacause they said, it's an openSuse packaging issue. The rusxmms patch causes the problems. Bug is not fixed.

https://github.com/taglib/taglib/issues/129
https://bugzilla.novell.com/show_bug.cgi?id=788567
https://bugzilla.novell.com/show_bug.cgi?id=780256

See also bug 815157.

I'm busy working on bug 815157 and I've just opened kid3-qt, pasted both the offending text from the above bug and the text from c1 in this bug into both id3 headers in kid3-qt. They display correctly. Next I opened clementine and played the track and the display was correct. Easytag which is a gnome app was claimed as working in bug 815157 and a possible clue to this is the fact that I'm currently using Xfce and not Kde.

@Dave Plater
I installed Xfce 4.10 under openSuse 12.3 and Kid3-qt v2.3 from OBS multimedia:apps repo.
taglib and libtag* v1.8-3.2.1 (OSS Repo)
Also tested with:
taglib and libtag* v1.8.55-7 (multimedia:lib Repo)

I made a little screencast, using Kid3-qt under Xfce. Please take a look:

http://youtu.be/ITpkGeb_4tk
(adjust quality level to see details)

So it's still the same behaviour. It looks like not being a KDE problem.

Kai

*** Bug 815157 has been marked as a duplicate of this bug. ***

I'm still using openSUSE 12.1 btw but because I'm maintainer of multimedia apps and libs, I have the latest packages from those repos. We can work through the kid3-qt dependencies to see if anything is different.
I'm attaching the mp3 file that I created using the reproduction steps.

Created attachment 536200 [details]
Created with touch and edited with kid3-qt

I've built taglib without the taglib-1.8-ds-rusxmms-r2.patch and you can grab it from :
http://download.opensuse.org/repositories/home:/plater/openSUSE_12.3

if this resolves the issue perhaps you can save me a bit of time by letting me know the purpose of the rusxmms patch, it was made for a feature request and seems to have caused a few bugs.
https://features.opensuse.org/313273

Hi Dave, I just added your repo and updated my taglib-files. Thx a lot, it resolves the issue. All taglib based apps like Kid3, Kid3-qt, Amarok, ..., now work without any problems like they did before with extended German character set. I can also convert the tags between Id3 v2.3 and v2.4 without any issues.
So for German locale the bug is fixed. 

The purpose of the rusxmms-patch?
All I know about it is from http://rusxmms.sourceforge.net/index.php?page=about.php 
Their forum is only in russian language. Sorry, but I can't help on this.
But I think without the rusxmms patch, the there will be probably some other regressions with Russian locale character set.
Kai

I'd like to fix this problem once and for all, if you have the time to help.
I'm adding the latest librcc to my home repo and am going to try to get the problem to surface on 12.1. I've had the time to do a bit of research and I suspect that the problem stems from the patch and your version of librcc.
I've also reopened bug 780256. I maintain both kid3 and clementine so I've a big interest in taglib.

I'll let you know when I've finished playing.

Your taglib bulit from yesterday without rusxmms patch worked for me. I accidentally updated to your current built of taglib (taglib-1.8-63.1) from today and grabbed it from your home:plater repo. In contrary to yesterday it's broken again for German extended characters. rusxmms-patch applied again?

I'm trying to get the bug to appear on my 12.1 machine, I've added the command line tools that build from the libtag examples still wip. You can try installing the librcc0 that I have in my home repo and check to see if the taglib rpm contains the cli binaries to read and write tags. I'm at work atm so I only have a phone to work with.

(In reply to comment #1)
> Tested on openSuse 12.3 (x86-64), Kernel 3.7.10-1.1-Desktop, KDE 4.10.2.
> taglib and libtag* v1.8-3.2.1 (OSS Repo)
> Also tested with:
> taglib and libtag* v1.8.55-7 (multimedia:lib Repo)
> 
> This setup leads into corrupted tag-fields when dealing with special
> characters. When using id3lib (ID3v2.3) based applications, everything is fine,
> even with German special characters.

Cannot reproduce with the same configuration. What locale do you use by default?

Here is an updated patch fixing the issue. And, please, let me know if there
are still problems. 
http://dside.dyndns.org/darklin/portage/media-libs/taglib/files/taglib-1.8-ds-rusxmms-r3.patch

The RusXMMS patches helps to handle music with broken non-Unicode ID3 tags.
There is still a lot of such files and I will describe why. The problem that
ID3 v.1 had defined ISO8859-1 as only supported encoding. This obviously was
not working for non-Latin languages, Russian being one of the examples. So,
people was just ignoring specification and put CP1251 (sometimes KOI8-R)
encoded strings in and this became a standard de-facto, especially on Windows.
RusXMMS patches are trying to detect which encoding is actually used and recode
the text to Unicode to avoid problems. It is used mainly by Russian and
Ukrainian Linux communities, but there is support for other European and Far
East countries as well.

Now, with ID3 v.2 and Unicode support, there should be no problems.
Unfortunately, there is still a lot of old broken MP3 around. Many windows
applications still produce MP3 using CP1251 tags labelled as ISO8859-1. And
there is hardware MP3 players which are only understand CP1251 tags. So,
disabling RusXMMS will cause a lot of problems for Russian community.

Thanks for your help Suren, I'll put the patch into my home:plater taglib for testing. If it works I'll push a 12.3 update.

I've built taglib using the new patch, you need to make sure the version-release string is at least libtag1-1.8-66.1 or greater.
Please install and test.

Please make sure that your  libtag_c0 matches libtag1

taglib is built and published in my home repo. The taglib package contains binaries which read and write tags from the command line. Of particular interest is tagreader and tagreader_c which list the tags in a track, tagreader lists the german characters but tagreader_c gives me garbage from the same mp3.

Afaics, you updated your 12.1/i586 and 12.2/i586 repo. But for 12.3/x86-64 I can't find any new taglib stuff. So I can't test it.

Hm, strange... I can confirm that tagreader_c produces invalid results. However, if I remove rusxmms patch, both tagreader and tagreader_c produce unreadable data whatever locale I set. Either I'm doing something completely wrong or there are problems upstream. 

Anyway, the tagreader_c may be easily fixed by commenting out 'taglib_set_strings_unicode(FALSE);' in the start of file. I'll add this to patch, but I'm quite curious if taglib_c from upstream is running properly for somebody and, if yes, which locales do they use.

(In reply to comment #19)
I thought you were on i586, the 12.3 x86_64 build was slower than the rest but it's completed now :
> osc ls -v --binaries home:plater/taglib openSUSE_12.3  x86_64
      181 Apr 23 10:19 _statistics                             
  7470545 Apr 23 10:19 libtag-devel-1.8-67.1.x86_64.rpm        
   252397 Apr 23 10:19 libtag1-1.8-67.1.x86_64.rpm             
  1062309 Apr 23 10:19 libtag1-debuginfo-1.8-67.1.x86_64.rpm   
    16822 Apr 23 10:19 libtag_c0-1.8-67.1.x86_64.rpm           
    30131 Apr 23 10:19 libtag_c0-debuginfo-1.8-67.1.x86_64.rpm 
      468 Apr 23 10:19 rpmlint.log                             
   624078 Apr 23 10:19 taglib-1.8-67.1.src.rpm                 
    25015 Apr 23 10:19 taglib-1.8-67.1.x86_64.rpm              
    68414 Apr 23 10:19 taglib-debuginfo-1.8-67.1.x86_64.rpm    
   179607 Apr 23 10:19 taglib-debugsource-1.8-67.1.x86_64.rpm

The factory i586 build is only publishing now.

(In reply to comment #20)
I'm assuming that tagreader_c uses libtag_c0 and tagreader uses libtag1 but ldd says otherwise, ldd -u /usr/bin/tagreader_c
says :
Unused direct dependencies:                                                                                                 
        /usr/lib/libtag_c.so.0
so I may be wrong.
easytag which M00n Raker claimed as the only tag editor that worked only depends on libtag_c0 and doesn't have a direct dependency on libtag1. It will only pull it in because libtag_c0 depends on libtag1.
I'm still trying to work out why this bug now only exists in 12.3 and it seems to be with characters with Umlauts.
Was this problem as a result of the upgraded librcc?

Further easytag from packman doesn't depend on any taglib libraries.

Cornel can you also see if this new patch resolves your problem?

(In reply to comment #22)
> (In reply to comment #20)
> easytag which M00n Raker claimed as the only tag editor that worked only
> depends on libtag_c0 and doesn't have a direct dependency on libtag1. It will
> only pull it in because libtag_c0 depends on libtag1.
No, easytag (from packman repo) doesn't depend on any taglib lib. It depends on id3lib. That's why it worked with German extended character set (no rusxmms patch in it). id3lib is completely different from taglib, afaik.

I updated to your version release *-.67.1 a few minutes ago.
It seems, that it fixes the issue, at least with German characters. Let's see what our Rus friends will report :-)

Sorry, I was a bit too hasty. 
It worked with the ID3 V2 tags only, but Id3 v1.1 tags are still broken.

Can you tell me which locale do you exactly use? Output of 'locale' would be nice.

Sure:
openSuse 12.3(x86-64)/Kernel 3.7.10-1.1-desktop/KDE 4.10.2 Release 556

locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

Thanks, I see a problem. It is a bit more complicated to fix. For ID3 v.1 tags the kid3 is trying to make recoding on its own by overloading some of the taglib classes. I.e. there is a conflict between RusXMMS patch and KID3 both trying to fix encodings. I need a bit of time to think how to fix it. I'll post updated version of the patch next 2 days.

The easytag branched from GNOME:Apps in my home project does use libtag1.
I'm resetting NEEDINFO until Suren has finished working on the patch.

Erick Osorio, I've added you to the cc list of this bug where we are trying to resolve this problem.

Dave with taglib-1.8-67.1 (with rusxmms-r3 i think) from your repo the problem still happening in m4a.

locale
LANG=es_MX.UTF-8
LC_CTYPE="es_MX.UTF-8"
LC_NUMERIC="es_MX.UTF-8"
LC_TIME="es_MX.UTF-8"
LC_COLLATE="es_MX.UTF-8"
LC_MONETARY="es_MX.UTF-8"
LC_MESSAGES="es_MX.UTF-8"
LC_PAPER="es_MX.UTF-8"
LC_NAME="es_MX.UTF-8"
LC_ADDRESS="es_MX.UTF-8"
LC_TELEPHONE="es_MX.UTF-8"
LC_MEASUREMENT="es_MX.UTF-8"
LC_IDENTIFICATION="es_MX.UTF-8"
LC_ALL=

I don't need use special characters.

Some screen shots of the problem:

Original tag:
https://files.myopera.com/evgom/files/original.png

Modified tag:
https://files.myopera.com/evgom/files/modificado.png

Result:
https://files.myopera.com/evgom/files/resultado.png

I don't know if are the same problem that here... But its for rusxmms patch (i think), in the vainilla taglib works properly.

Ohh right, the result can be seen by opening the file again.

I also encountered this problem with programs using Taglib (kid3 and soundKonverter).

Not all tags containing "Umlaut characters" are, however, treated faultily, only tags containing the so-called Latin 1 (Western European) characters (á, ä, é, í, ó, ö, ü, ú, ß). Tags containing characters other than Latin-1, (I tested Latin 2 characters - Hungarian ő and ű - and Russian Cyrillic characters) are treated correctly.

E.g.:
When I save a tag containing the character "á" in Kid3, it gets encoded as "á" and when I click to edit the tag, no text encoding is assigned to the tag.
So when I save it again, I get "ÃNBH¡" (NBH is an upper index), so it seems that the two characters of "á" are further encoded into Unicode, and no text encoding is assigned to the tag.

When the tag contains Latin-2 (ő) or Cyrillic characters, the tag is marked as Text encoding UTF-8 and it is displayed properly, and it does not change with any further saves made to the tags.

So it seems that whenever at least one non-Latin-1 character is present, the tag is marked as UTF-8, and they are encoded and displayed properly, when, however, only Latin-1 characters are present, no text encoding is assigned to the tag.

This happens on the level of the individual tags. If the Artist is e.g. "á" and the Title is "ő", then only the Artist tag gets corrupted, the Title tag (marked as text encoding UTF-8) is allright.

Addition: I use openSuse 12.3, taglib version is: 1.8-3.2.1.

Kubuntu does not seem to have this bug, nor does Fedora. Indeed, if I delete taglib, libtag1, libtag_c0, and install the taglib rpm from Fedora 18, this problem disappears.

Another addition:

When using soundKonverter, id3v1 tags are also corrupted. When they contain, however, a non Latin-1 character, the tag remains empty.

Dave,

Your version of taglib 1.8-67.1 solves most issues, though not all of them.

It solves the id3v2 issue of text tags. However, when the filename of the embedded picture contains non ASCII characters, conversion with soundKonverter corrupts the name of the embedded picture, Kid3, however, treats them okay.

id3v1 tags show the corruption.

Using the rpm of Fedora 18 (i.e. no rusxmms patch) solves all the problems. Not just the problem M00nraker reported, but also the issue Erick Osorio has, (Artist, Title, Album, Comment tags disappear from m4a files).

Only one small problem remains, i.e. this issue is in taglib 1.8 itself: when converting from Flac files (which have UTF-8 encoded Vorbis tags), no conversion of the name of the embedded picture is done. So if I have a picture "Bartók.jpg" embedded in a flac file, after conversion with soundKonverter, the embedded file is shown in Kid3 as "Bartók.jpg"
With mainstream taglib 1.8, however, tags which have at least one non-ISO-8859-1 character, remain empty in id3v1. 

It seems, the rusxmms patch is not rife enough, has not been tested, creates a lot of additional issues not present in taglib, and should be removed till these issues are solved.

(In reply to comment #39)
Suren, whom I assume is the patch author, is working flat out to resolve this issue. I'm sure within a few days we will have a taglib that satisfies everybody that uses non  Utf-8 us characters in their tags. 
The taglib rpm now contains some executables to read and write tags directly with libtag to test with.

(In reply to comment #40)
Well, let's see what he comes up with. 
But as I understand, this patch (rusxmms) is just a hack, to solve shortcomings of the mp3 format, and the chaos created by non-Unicode encodings and regards only mp3s having only id3v1 tags. Offering taglib with serious bugs just for these users does not seem to me right. I think the priority of openSuse should be the whole community, that is to offer a taglib which works as it should (i.e. not breaking m4a tagging, or encoding of extended Latin-1 mp3s.) We do not know whether the patch works with other formats (e.g. APEv2 tags in ape, wavpack files), or also creates havoc in them as well.
I could imagine to have a separate repository with the patched taglib offered only for those who need it, but the default should be the mainstream taglib until the rusxmms patch is well tested with every format and proves to be okay.

I also think so like Imre does. Patch should be removed as soon as possible until it is really well tested with all file formats and encodings.

If we can't satisfy all the users with non english locales then I will create a separate taglib in multimedia:libs with the patch for the cyrillic users and remove the patch from factory but in the spirit of the openSUSE community with your help we can create a taglib which works for all users.
I can understand the frustration of having your music collection tags messed up, I'm sorry I've only become aware of this problem now although I've been bug owner for both kid3 and clementine for a few years.
The information that's been provided now should be sufficient to fix this issue and the command line tools which will be a permanent part of the taglib package give a simple and clean way of testing whether libtag is doing what it should.
If you read Comment 29 you will see that kid3 for instance, complicates the issue by trying to correct errors but I'm confident that Suren will come up with a fix.

One last thing, if libtag doesn't handle cyrillic properly it's an upstream bug and whatever the outcome of this bug Suren or myself will take the issue upstream.

The next version seems to fix both remaining issues: id3 v.1 editing and disappearance of m4a tags. I'll do some more testing, but at least this issues seems to be gone.

http://dside.dyndns.org/darklin/portage/media-libs/taglib/files/taglib-1.8-ds-rusxmms-r4.patch

I and few other people have tried to talk to taglib team few years ago. Unfortunately, their view was that non-Latin tags in ID3 v.1 tags are breaking specification and they were against supporting it. Though, if the point is raised by the packager of major distribution, the result, hopefully, may be different.

I'll try, am I wrong in assuming that the entire Eastern Europe and Greece use cyrillic. What about Asia? This is new territory for me.
I'll incorporate the patch tonight and the patched packages should be ready for testing tomorrow.

The enconding problem had existed for all countries using characters outside of Latin1. Rusxmms has support for Asian languages (Japanese, Korean, Chinese) which was confirmed by some of Russian users speaking Japanese. Tough, I dont know if it is still a problem in these countries or their are fine with v2 Unicode tags. As I wrote earlier, it is certainly still a major problem in the countries belonging to former Soviet Union.

PS. Probably half of East Europe using Lating language but with different set of special characters (latin2). Greece using it is own alphabet.

(In reply to comment #46)

Well, I suspected this may have been the attitude of Taglib developers. Since your patch is a hack which breaks specifications, I'm not surprised they refused to incorporate it.

I do not know what kind of hardware players you are talking about which cannot read id3v2 tags. I have now checked, my mp3 player is perfectly capable of displaying Cyrillic characters in id3v2.4 encoded mp3 file.

Wouldn't it be easier to write a script which takes CP1251 encoded tags from id3v1 and converts them to UTF and inserts them into id3v2 tags?

Or even better, not creating mp3 files with only id3v1 tags in the first place. Or am I guessing right, that there are many CP1251 encoded id3v1 tagged mp3 files on Russian file-sharing sites? Other than this, I cannot understand why you are not trying to tackle the problem of the not properly encoded mp3 files and tag them properly.

The problem is that the specification was faulty and had left most of the world without any option to store their tags. Obviously, some chaos and, then, standard de facto had emerged. You cant blame people for it, there was no way to be complient to the specification. Now we cant change the reality and behavior of Windows users. We only can deny reality and make life of Linux-users complicated or accept it and make it easy. 

I mean there is no problem for me. I can recode everything or to just patch the taglib myself. But there is a horde of people who has no clue about encodings. They just see that the music they downloaded from Internet got from a friend is not displayed properly and they have no clue why. On the friends PC running Windows there is no problems...

And of course there is uses when it is helpful even if you are know about encodings and Unicode. I believe that modern players from respectable brands are all supporting ID3 v.2. But I personally have a simple USB flash/MP3 player combo and it is not working with it. There is still lots of such devices. And there are lots of cases when online recoding is helpful. Your frined who is using Windows and dont care about specification is comming with his hard drive with a lots of music and you just want to quickly see what he has without doing prior recoding whatever. The file sharing is also quite often in ID3 v.1 only. 

Even for recoding, the RusXMMS patches are helpful. Quite often people are bringing the stuff in some horible mix of encodings. Some files are using CP1251, others - KOI8-R. I have seen UTF8 in ID3 v.1 tags. With encoding autodetection, you will recode everything automatically. Im just running tagwriter over all files in the collection.

(In reply to comment #47)
> I'll incorporate the patch tonight and the patched packages should be ready for
> testing tomorrow.
You mean the 1.8-68.1 release in your repo?
Well, on the first view the issue seems to be fixed with that. But I only tested  with German locale and mp3-files. ID3v1 is also back on track with the new patch. Let's see what the others say. Thx a lot, good work guys.

These are the packages to look for :
 osc ls --binaries home:plater/taglib openSUSE_12.3  i586
_statistics
libtag-devel-1.8-68.1.i586.rpm
libtag1-1.8-68.1.i586.rpm
libtag1-32bit-1.8-68.1.x86_64.rpm
libtag1-debuginfo-1.8-68.1.i586.rpm
libtag1-debuginfo-32bit-1.8-68.1.x86_64.rpm
libtag_c0-1.8-68.1.i586.rpm
libtag_c0-32bit-1.8-68.1.x86_64.rpm
libtag_c0-debuginfo-1.8-68.1.i586.rpm
libtag_c0-debuginfo-32bit-1.8-68.1.x86_64.rpm
rpmlint.log
taglib-1.8-68.1.i586.rpm
taglib-1.8-68.1.src.rpm
taglib-debuginfo-1.8-68.1.i586.rpm
taglib-debugsource-1.8-68.1.i586.rpm
and :
 osc ls --binaries home:plater/taglib openSUSE_12.3  x86_64
_statistics
libtag-devel-1.8-68.1.x86_64.rpm
libtag1-1.8-68.1.x86_64.rpm
libtag1-debuginfo-1.8-68.1.x86_64.rpm
libtag_c0-1.8-68.1.x86_64.rpm
libtag_c0-debuginfo1.8-68.1.x86_64.rpm
rpmlint.log
taglib-1.8-68.1.src.rpm
taglib-1.8-68.1.x86_64.rpm
taglib-debuginfo-1.8-68.1.x86_64.rpm
taglib-debugsource-1.8-68.1.x86_64.rpm

I would appreciate if everyone can test.

I once made a hardware coin operated mp3 juke box (most probably one of the first in the world) and am familiar with Id3v1 tags as I parsed them for both the juke box display, a simple lcd and to print out a playlist for the front panel. Id3v1 fields are a simple string of 8 bit wide characters so Utf-8 characters should be able to occupy those fields. A quick glance (I have to get to work) at :
http://www.joelonsoftware.com/articles/Unicode.html a link which I found at http://www.utf-8.com leads me to believe that Utf-8 displays Asian characters let alone Cyrillic. I see these characters on my "en_US.UTF-8" system.

I haven't read up on any Id3 standards yet, I simply viewed the headers of mp3s and worked it out for myself so I'm not an expert but I would like to know how this mess started. Reading a little more of the Utf-8 website, I can understand where the problem is. This is most probably why clementine likes Id3v2 tags. Not sure where an Id3v2 tag sits when there is also an Id3v1 tag but I hope it's always after and not before. 
I think the solution for Russian community is a converter to create a unicode Id3v2 tag from the Id3v1 tag. The older players will parse the Id3v1 tags so they need to be preserved.

Hopefully the next time some upgrade breaks the rusxmms patch, it will be in Factory and the problem won't find it's way into a release.

Does anyone in this bug test Factory?

(In reply to comment #51)
> (In reply to comment #47)
> > I'll incorporate the patch tonight and the patched packages should be ready for
> > testing tomorrow.
> You mean the 1.8-68.1 release in your repo?
> Well, on the first view the issue seems to be fixed with that. But I only
> tested  with German locale and mp3-files. ID3v1 is also back on track with the
> new patch. Let's see what the others say. Thx a lot, good work guys.


I upgraded my taglib packages from David's repository as well (release 1.8-68.1 as you), but on my system I still get the corrupted tag's :(
I'm missing something, maybe ?

My locale are still en_US.UTF8
(results from "locale" command are these:
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
)

To not make any confusions, now I have these packages on my system:
taglib-1.8-68.1.i586
libtag1-1.8-68.1.i586
libtag_c0-1.8-68.1.i586
all from David's repo.

and tried to edit some tag to insert character ä (or in another test case, the character ü ) inside my clementine player.

Ahh, on the other hand, when editing the same files with kid3-qt, I get the tags written correctly, now ....  which is good enough (I still miss the ability to edit with clementine, but since these cases are not that often, this is good enough for me.)

(In reply to comment #54)
> I upgraded my taglib packages from David's repository as well (release 1.8-68.1
> as you), but on my system I still get the corrupted tag's :(
> I'm missing something, maybe ?
> ...
> and tried to edit some tag to insert character ä (or in another test case, the
> character ü ) inside my clementine player.

So we should not praise the day before nightfall:

I picked up a normal mp3-file, tagged with Kid3 and Dave's taglib libs (*.68.1). Kid3 shows that the ID3v1 and V2 tags are in a good shape. They also showed up fine in Amarok, tagreader and even in easytag. Then I change the title tag of this mp3 with Amarok to a simple charakter ä. After saving changes I load that mp3 file into Kid3. There I can see, that ID3v2 title has changed to ä, but the ID3v1 title tag is corrupted. There is still an issue.

Many of us don't know what the rusxmms patch really does, how it works and how it affects the whole thing. The patch should be removed till all issues are solved. For the people who depend on the patch, Dave could make a separate version of taglib with the rusxmms patch. Whenn all issues are fixed and everything is well testet, Dave could integrate the patch again. Testing needs some time. At at the moment it's not in a good shape, because it causes kind of dataloss and everything seems to be unsave.

If I update 12.3 with a libtag with no patch the russian users don't have to update then the patched version can remain in multimedia:libs and factory. I also want to make sure the patch is properly fixed by the next release. I wish I could duplicate this bug on my own system. I'll start the update process this evening meanwhile could you try using tagreader from the taglib package on your tracks and see if the tags look right and post a few results.

(In reply to comment #56)
> I'll start the update process this evening meanwhile could you try using 
> tagreader from the taglib package on your tracks and see if the tags look right > and post a few results.

I made another screencast for it. That's easier for me :-)

http://youtu.be/WWb1RtGHxeg
(please adjust quality level to at least 720p to see details).

(In reply to comment #57)
I'm a cash disadvantaged person and I use an old nokia 9300i as a modem so I prefer text but I'm watching it anyway.

Ok, next time a know better :-)

As stated in Comment 56 I'm updating 12.3's taglib package with the russxmms patch removed and then I'll reincorporate it for factory where it can be tested and will hopefully not cause problems in the next release.

*** Bug 780256 has been marked as a duplicate of this bug. ***

(In reply to comment #59)
I was actually unable to view due to my slow connection, set it to view later
which may work but I would appreciate a text output from tagreader from
someone.
It will make my quest to duplicate this bug on my own system easier and
guarantee that this problem will never surface again.

One last question, does anyone have libtag-extras1 installed?

The update destined for 12.3 is in :
http://download.opensuse.org/repositories/home:/plater:/branches:/openSUSE:/12.3:/Update/standard

Please test.

Interesting fact, the submit request to openSUSE:factory displays this for the special characters :
äÃöÃüÃÃ

(In reply to comment #65)
> Interesting fact, the submit request to openSUSE:factory displays this for the
> special characters :
> äÃöÃüÃÃ

I updated to 1.8-69.1.x86_64.

Now we run into other problems. Ok, no screencast but text:

1) Create new file: touch test.mp3 (no tags, no audio stream)
2) tagreader test.mp3
   title   - ""
3) tagwriter -t äöü test.mp3
4) tagreader test.mp3
   title   - "äöü"
That seems to be correct, but when I load test.mp3 into Amarok, Amarok shows up a corrupted title string: äöü
Kid3 shows the same corrupted title string after loading the file. 

Now another test:
1) Create new file: touch test2.mp3 (no tags, no audio stream)
2) tagreader test2.mp3: 
   title   - ""
3) This time I first edit ID3v2 title tag of test2.mp3 with Kid3 to "äöü"
4) tagreader test2.mp3 shows up a corrupted title tag now:
   title   - "���"
5) Loading the same test2.mp3 into Amarok, it shows the correct title tag: "äöü". So tagreader seems to produce a corrupted output or whatever.

A last test also without tagwriter modifying the file:
1) Create new file: touch test3.mp3 (no tags, no audio stream)
2) tagreader test3.mp3: 
   title   - ""
3) I load test3.mp3 into Kid3 and modify the ID3v1 title tag to "äöü".
4) tagreader test3.mp3:
   title   - "���"
   Ok, tagreader produces shit.
5) I then load test3.mp3 into Amarok. Amarok shows up this tag: "äöü". Correct. But now i modify this title tag within Amarok itself to this title tag: "äöüß". After saving the changes in Amarok, the title tag gets corrupted and Amarok shows up this "äöüÃ" instead of "äöüß". Kid3 shows up the same corrupted string for Id3v1 and v2 title tag.

From my first two tests, you could think, that it is an tagreader/tagwriter issue now. But the last test shows, that it isn't. Very confusing.

Did you change s.th else except removing the rusxmms patch?

This is an autogenerated message for OBS integration:
This bug (814814) was mentioned in
https://build.opensuse.org/request/show/173418 Factory / taglib

With libtag 1.8-70.1 from Daves' repo now work ok with m4a.

M00n Raker, thanks for detailed reports :) 

I'll take a look for problem with Amarok. However, the tagreader and tagwriter are certainly broken in vanilla. Unless you are on ISO8859-1 locale, you'll get corrupted output from applications. Take a look, on http://suren.me/ss/tagreader.png

This is using the latest build from Dave, no rusxmms. On UTF-8 terminal, you get ISO8859-1 output. The same problem is also with tagwriter.

(In reply to comment #63)
> One last question, does anyone have libtag-extras1 installed?

I'm.

I can't reproduce the problem with Amarok described by M00n Raker on freshly installed 12.3 desktop using 69.1 build. Do you have any encoding-specific options enabled in Amarok? By the chance do you have any LibRCC configurations present (either ~/.rcc/xmms.xml or /etc/rcc.xml)? Can you provide me original good file before Amarok have corrupted it and file corrupted with Amarok.

PS. I have reproduced the problem with tagwriter, but as I said this is a vanilla issue. Anyway I'll fix it.

Okay, my experience is a little bit different that M00nRaker's :

I have now taglib, libtag1 and libtag_c0 version 1.8-70.1 from Dave's repository.

When I try to edit a test file inside amarok, or in clementine, I can write properly the characters like ä ! (they are saving them alright now: actually I wrote the character with amarok, saved it, and then opened the file in clementine, modified the file and saved it again, and ... no corruption anymore).

On the other hand, when I try tagwriter/tagread on the same file, with the same character, I get a corrupted tag in the file ("ü" instead of "ä")

Confusing, isn't it ?
BTW, I use i386 install of Opensuse 12.3 ...

P.S. I have also taglib-extras1 (version 1.0.1-17.1.1) installed in my system.

(In reply to comment #72)
> Okay, my experience is a little bit different that M00nRaker's :
> 
> I have now taglib, libtag1 and libtag_c0 version 1.8-70.1 from Dave's
> repository.
> 
> When I try to edit a test file inside amarok, or in clementine, I can write
> properly the characters like ä ! (they are saving them alright now: actually I
> wrote the character with amarok, saved it, and then opened the file in
> clementine, modified the file and saved it again, and ... no corruption
> anymore).

Ok, but you have to test and edit the same file with different programs, like Amarok and Kid3, e.g. editing title tag with Amarok and then loading it into Kid3, edit again, and the load it back into Amarok, and so on. This shows the issues.

(In reply to comment #71)
> I can't reproduce the problem with Amarok described by M00n Raker on freshly
> installed 12.3 desktop using 69.1 build. Do you have any encoding-specific
> options enabled in Amarok? By the chance do you have any LibRCC configurations
> present (either ~/.rcc/xmms.xml or /etc/rcc.xml)? 

First to your questions:
I haven't xmms installed on my system. No ~/.rcc/xmms, and no file /etc/rcc.xml. To the Amarok settings: In "Configure Amarok/Metadata" you can configure the metadata handling. There is a checkbox "Enable character set detection in ID3 tags". It was allways enabled in my settings. But now I did all my tests even with this option set to disabled. I haven't found any differences between setting it to enabled or disabled. So for my tests this setting doesn't seem to have any influences.

> Can you provide me original good file before Amarok have corrupted it and file > corrupted with Amarok.
Sure, but for the moment I have to leave. I come back later this day and upload everything you want.

Now I updated to the ltest version from Dave's home repo (taglib-1.8-70.1.x86_64
) and also run into several issues:

1) Create new file: touch test.mp3 (no tags, no audio stream)
2) tagreader test.mp3
   title   - ""
3) tagwriter -t äöü test.mp3
4) tagreader test.mp3
   title   - "äöü"
   Tagreader shows up a corrupted title string.
5) Now I load this file into Kid3 and Amarok. Both are also showing a corrupted title string like tagreader: "äöü" (for v1 and v2 tags).

Now another test:
1) Create new file: touch test2.mp3 (no tags, no audio stream)
2) tagreader test2.mp3: 
   title   - ""
3) This time I first edit only ID3v2 title tag of test2.mp3 with Kid3 to "äöü"
4) tagreader test2.mp3 shows up the correct title tag now:
   title   - "äöü"
5) Loading the same test2.mp3 into Amarok, it shows the correct title tag:
"äöü". Then I modify title tag within Amarok to "äöüß".
6) tagreader test2.mp3:
   title   - "äöüß"
   Output Ok now.
7) But loading test2.mp3 it into Kid3 again, Kid3 shows a correct v2 title-tag "äöüß", but a corrupted v1 title-tag: "äöüÃ"

BTW, I have installed Amarok from packman repo, v2.7.0-14.52-x86_64

I must explain the current repository state :
The taglib package with no rusxmms patch is in this repository :
http://download.opensuse.org/repositories/home:/plater:/branches:/openSUSE:/12.3:/Update/standard
This is destined for an openSUSE:12.3 update and contained the following this morning :
osc ls --binaries home:plater:branches:openSUSE:12.3:Update taglib standard x86_64
_statistics
libtag-devel-1.8-3.5.1.x86_64.rpm
libtag1-1.8-3.5.1.x86_64.rpm
libtag1-debuginfo-1.8-3.5.1.x86_64.rpm
libtag_c0-1.8-3.5.1.x86_64.rpm
libtag_c0-debuginfo-1.8-3.5.1.x86_64.rpm
rpmlint.log
taglib-1.8-3.5.1.src.rpm
taglib-1.8-3.5.1.x86_64.rpm
taglib-debuginfo-1.8-3.5.1.x86_64.rpm
taglib-debugsource-1.8-3.5.1.x86_64.rpm


My home repository and multimedia:libs contain a taglib with the taglib-1.8-ds-rusxmms-r4.patch.
This has been submitted to factory.

So if you want to help Suren to fix the patch, please use my home repo.
Your help will be appreciated.

taglib-extras1 is an add on for Amarok and may complicate things with Amarok.
For any specific amarok problems please test without taglib-extras1 as well.

New version with taglib and tagwriter problems fixed. Also I think it should fix the problem M00n had with Amarok. Here:
http://dside.dyndns.org/darklin/portage/media-libs/taglib/files/taglib-1.8-ds-rusxmms-r8.patch

The binaries are also available here:
https://build.opensuse.org/package/show?package=taglib&project=home%3Acsa7fff%3Arusxmms

(In reply to comment #76)
> taglib-extras1 is an add on for Amarok and may complicate things with Amarok.
> For any specific amarok problems please test without taglib-extras1 as well.

I've installed Amarok from the Packman repo. taglib-extras1 is one of Amaroks dependencies, so it can't be removed normally or it comes to a dependency error.

(In reply to comment #77)
> New version with taglib and tagwriter problems fixed. Also I think it should
> fix the problem M00n had with Amarok. Here:
> http://dside.dyndns.org/darklin/portage/media-libs/taglib/files/taglib-1.8-ds-rusxmms-r8.patch
> 
> The binaries are also available here:
> https://build.opensuse.org/package/show?package=taglib&project=home%3Acsa7fff%3Arusxmms

I'm a bit confused. You wrote about a new version of taglib, but with updated rusxmms patch (r8) and about a fixed tagwriter. So far so good. But please read my Comment #74 again. For the both tests there, I used taglib-1.8-70.1.x86_64 from Dave's home repo. AFAIK, that was a version WITHOUT any rusxmms patch. And in the second test, I even didn't use tagwriter. But there is still an issue. So for this test, a new version of rusxmms patch doesn't matter, because 1.8-70 does't use anything of rusxmms patch. Right? So perhaps, as what Dave says in Comment #76, the issue with Amarok comes from taglib-extras1. But Amarok has a dependency to it. So removing taglib-extras1 will break dependencies.

Anyway, when Dave updated his home repo with a new version of taglib, I'll do my best to find out any issues again. When there aren't any issues and taglib works again like in the old days, I will also test taglib with applied rusxmms patch. But for now, there are issues with or without rusxmms patch. May be, there is also a problem with metadata handling in Amarok. But I don't know it.

Sorry, I made a mistake. I haven't seen Comment #75 with Dave's repo layout :-)
So taglib-1.8-70.1 from his home repo of course has the rusxmms (r4) patch incorporated, as I also see in the changelog. So now Suren's Comment #77 makes sense to me. Please forget my Comment #79, I was wrong there.

Dave's link http://download.opensuse.org/repositories/home:/plater:/branches:/openSUSE:/12.3:/Update/standard in Comment #75 to the unpatched version of Taglib seems to be wrong. Nothing with Taglib there.

Yeh I find nothing there either and I copied the link from the right place. Must be a build service glitch. I'll post the correct link.
Meanwhile I've just incorporated taglib-1.8-ds-rusxmms-r8.patch in my home project. Give it a while to build.

The publish flag wasn't enabled by default. It is now so the packages should appear when they're published. The build service machine now has to create the repository and rsync the rpms. Should be there between now and morning.

All published in the update branch repo

(In reply to comment #66)
At last I've got the bug to show on my system by repeating your steps with the r4 patched libtag1, identical results.
Take note when updating taglib libraries they are used by desktops so you may have to CTRL ALT BACKSPACE to restart X.

Also I updated to the r8 patched libtag1 and it didn't pull in the new taglib rpm which has the all important tagwriter and tagreader binaries, will fix in next update to my home repo.
Using libtag-1.8-76.1 with the earlier tagreader/writer brought up the same problem but after updating taglib to 1.8-76.1 the problem was fixed. I didn't have to restart X.

This could have important repercussions on kid3, clementine and amarok if they have built against an earlier libtag.

To simplify matters lets get everything working on the command line first and I've clementine and kid3-qt building in my home repo against whichever taglib is the latest there.
 Amarok is a bit of a problem if you have anything but standard kde4 installed but I'll branch it and I suppose I'll need to branch taglib-extras too.
This I'll only be able to do by tomorrow.

The next taglib release in my home repo will pull in the same release of the taglib rpm.

It looks to me as if the r8 version of rusxmms patch works properly, pity we can't find a french user to test as well.
I'm inclined to push this as an update to 12.3. Please follow the guidelines in Comment 84 when testing.
I've branched 12.3's amarok and taglib-extras as well so please test with clementine, kid3-qt, amarok and libtag-extras1 from my home repo once you're satisfied that tagreader and tagwriter work correctly with the special characters.

To reinforce my feeling that this bug is fixed tagreader_c reads the umlauts in my original bug814814.mp3 correctly as well as tagreader.

(In reply to comment #86)
> It looks to me as if the r8 version of rusxmms patch works properly, pity we
> can't find a french user to test as well.
> I'm inclined to push this as an update to 12.3. Please follow the guidelines in
> Comment 84 when testing.
> I've branched 12.3's amarok and taglib-extras as well so please test with
> clementine, kid3-qt, amarok and libtag-extras1 from my home repo once you're
> satisfied that tagreader and tagwriter work correctly with the special
> characters.

I updated to the versions in your home repo (with added rusxmms r8 patch):
taglib-1.8-77.1.x86_64
libtag1-1.8-77.1.x86_64
libtag_c0-1.8-77.1.x86_64
libtag-extras1-1.0.1-17.2.1.x86_64
But I didn't installed your branched Amarok and Kid3-qt version. For playing around, I used the packman version of Amarok and Kid3 from the KDE:Release:410 repo.

I followed Dave's guideline in Comment #84. All my previous tests proved successful and went without a hitch. Even the issues with Amarok disappeared. Tagreader and tagwriter now work fine. Modifying tags with Amarok does't produce any issues in tagreader now. I can do what I want, I don't find anywhing wrong at the moment.

> I've branched 12.3's amarok and taglib-extras as well so please test with....

For playing around with Amarok I used libtag-extras1-1.0.1-17.2.1 from your home. As I allready wrote, that worked well. But now I replaced libtag-extras1-1.0.1-17.2.1 with libtag-extras1-1.0.1-36.1 from the KDE:Release:410 repo. I also couldn't find any issues with that. Amarok also works nice without any tagging problems. So it seems that it doesn't make any difference if I use libtag-extras1 from your repo or from the KDE repo.

Finally it seems that all my tagging issues are gone with taglib, libtag1 and libtag_c0 with version string 1.8-77.1 from Dave's home repo. For the moment a big thank you to Dave and Suren, who made that possible. Many thanks for your support.

With taglib 1.8.77.1 work property tag in m4a.

It would be a big update if clemntine, kid3, kid3-qt, amarok and everything else that uses libtag needed to be updated.
I'm pushing the patched version into 12.3 Update and based on the testing that's been done in this bug and the patch authors willingness to help, I'm sur upstream will accept it too.

Mr Maintenance, when can I submit to 12.3 update?

This is an autogenerated message for OBS integration:
This bug (814814) was mentioned in
https://build.opensuse.org/request/show/173627 Factory / taglib

Dave, sorry for the late reply. Feel free to open a maintenancerequest with the updated package. Thank you very much for your efforts and the testing!

Created request 173881

This is an autogenerated message for OBS integration:
This bug (814814) was mentioned in
https://build.opensuse.org/request/show/173881 Maintenance /

Closing bug as fixed, please reopen if any problem.

Reopening. There is still a problem with data corruption for non-latin1 characters (i.e., characters that are both outside US-ASCII and outside ISO-8859-1). This problem is *caused* by the RusXMMS patches, as it is *not* present in unpatched taglib.

See detailed description of the bug and how to reproduce it in bug #780658. (I guess one of these bug reports should be marked as a duplicate.)

The discussion goes on here: #780658.
This bug report should be closed again.

This bug is against 12.3 but exists in 12.2 as well so I'll close it as a duplicate of bnc#780658 and use that to push the 12.2 update.

*** This bug has been marked as a duplicate of bug 780658 ***

I've reopened this bug for the submission of taglib-1.8-ds-rusxmms-r9.patch to factory and as a 12.3 update. See bnc#780658

Taglib with taglib-1.8-ds-rusxmms-r9.patch is included in taglib >= 1.8-62.1 in :
http://download.opensuse.org/repositories/home:/plater/openSUSE_12.3
for testing.
If ok I'll submit to factory and 12.3 update.

This is an autogenerated message for OBS integration:
This bug (814814) was mentioned in
https://build.opensuse.org/request/show/176426 Factory / taglib

taglibs 12.3 update is ready to submit.

Dave, could you open a maintenancerequest with the updated package, please. Thanks!

created mr#176904

This is an autogenerated message for OBS integration:
This bug (814814) was mentioned in
https://build.opensuse.org/request/show/176904 Maintenance /

Update released for openSUSE 12.3. Resolved fixed. Thanks all of you.

openSUSE-RU-2013:0940-1: An update that has one recommended fix can now be installed.

Category: recommended (low)
Bug References: 814814
CVE References: 
Sources used:
openSUSE 12.3 (src):    taglib-1.8-3.5.3

openSUSE-RU-2013:0974-1: An update that has one recommended fix can now be installed.

Category: recommended (low)
Bug References: 814814
CVE References: 
Sources used:
openSUSE 12.3 (src):    taglib-1.8-3.9.1