[FM Discuss] Fwd: About http://translate.flossmanuals.net encoding

Mon Dec 13 04:46:04 PST 2010

when adam said "flat files" he wanted to say it is twiki platform.
twiki platform is using flat files to save data (with utf8 encoding).
there is no sqlite or postgres here.

now, this is not the first book with non ascii characters in latin. so
it is not general problem with encodings. encodings in chapters seems
fine.
encodings in TOC seems to have problems. normaly it does not have
problems.... but this time it has. so this is isolated problem and i
will have a look at this. might be that someone was manualy editing
toc on the disk.. that would make a problem... or maybe it is
this silly character that makes problem. english is not my native
language, so i am familiar with this non ascii problem. i have tested
it
with many silly croatian characters and it worked fine. it worked for
farsi that is also RTL :) will check this problem after work.

On Mon, Dec 13, 2010 at 1:16 PM, Lachlan Musicman <datakid at gmail.com> wrote:
> If it's lat file, then it must be sqllite, I'm not up to speed on
> encoding in said db, but can study if you need. Now that you mention
> it, my postgres isn't so great either. But I can look into and get
> back to you if you need....but at the end of the day, it's an encoding
> issue despite your protestations ( :) sorry mate). It'll be fine,
> we'll be good by launch date ;P
>
> L.
>
> On Mon, Dec 13, 2010 at 23:07, adam <adam at xs4all.nl> wrote:
>> On Mon, 2010-12-13 at 13:05 +0100, adam wrote:
>>> its not the db,...we have no db...its all flat file...
>>
>> I should say the *old* fm is flat file. booki uses postgres but the
>> translation zone is still the old platform
>>
>> adam
>>
>>
>>> its a page
>>> encoding issue. i think its the encoding of the manuals page...i will
>>> look at it
>>>
>>>
>>> adam
>>>
>>>
>>> On Mon, 2010-12-13 at 23:01 +1100, Lachlan Musicman wrote:
>>> > On Mon, Dec 13, 2010 at 22:55, adam <adam at xs4all.nl> wrote:
>>> > > if u can look at the webpage and work out what the encoding should be i
>>> > > think i can hack this in
>>> >
>>> > You should have utf-8 as the default on all fields in all table in
>>> > every database (I presume mysql?). There's no other way to do it.
>>> >
>>> > It's not fool proof - you will then run into collation problems, but
>>> > that's a well known issue that's being worked on by the wonks.
>>> > Basically collations are "how do we sort UTF8?", and that's where the
>>> > problems start. For two, minor, examples:
>>> >
>>> > Language
>>> > Swedish:    z < ö
>>> > German:     ö < z
>>> >
>>> > Usage
>>> > German Dictionary:  of < öf
>>> > German Telephone:   öf < of
>>> >
>>> > I've always figured that this is a lesser issue than the actual
>>> > encoding itself. Just make sure the encoding is UTF8 and we can wait
>>> > for the unicode collation algorithm to get more mature:
>>> > http://www.unicode.org/reports/tr10/
>>> >
>>> >
>>> > cheers
>>> > L.
>>> >
>>> > > adam
>>> > >
>>> > >
>>> > > On Mon, 2010-12-13 at 22:50 +1100, Lachlan Musicman wrote:
>>> > >> Was there any fix found for this yet?
>>> > >>
>>> > >> On Fri, Dec 10, 2010 at 10:39, Lachlan Musicman <datakid at gmail.com> wrote:
>>> > >> > On Thu, Dec 9, 2010 at 23:57, James Simmons <nicestep at gmail.com> wrote:
>>> > >> >> Luis has noticed that the left hand menu of the Spanish version of
>>> > >> >> "Make Your Own Sugar Activities!" has some funny characters in it.
>>> > >> >> Does anyone know how to fix that?
>>> > >> >
>>> > >> > It stems from an encoding issue.
>>> > >> >
>>> > >> > For instance, you can fix those chars into  ¿ by changing the encoding
>>> > >> > of the page to Western ISO 8859-1, but then the centre text becomes
>>> > >> > problematic. It looks like the two columns have different encodings at
>>> > >> > some level - probably at the database table or row level in the left
>>> > >> > column (set to Latin, swedish or western). I would suggest that the
>>> > >> > main body of FM is fine, ie already set to UTF-8.
>>> > >> >
>>> > >> > In chromium you can see this by going to options->tools->encoding
>>> > >> > In firefox it's view->character encoding
>>> > >> >
>>> > >> > Depending on how they were encoded in the first place, the best bet is
>>> > >> > to confirm that all tables and rows in the DB are set to UTF-8
>>> > >> > encoding. If not, you can change the encoding and it wont affect the
>>> > >> > content. I can help at a more fine grained level if you want, but it
>>> > >> > can also be done in phpmyadmin (for mysql) or with perl/sed/grep on a
>>> > >> > dump that is re-imported.
>>> > >> >
>>> > >> > If there is still a problem after that, the actual content will need
>>> > >> > to be changed, but only on those chars that are now problematic -
>>> > >> > future chars should be fine.
>>> > >> >
>>> > >> > cheers
>>> > >> > L.
>>> > >> >
>>> > >> >>
>>> > >> >> Thanks,
>>> > >> >>
>>> > >> >> James Simmons
>>> > >> >>
>>> > >> >>
>>> > >> >> ---------- Forwarded message ----------
>>> > >> >> From: Luis Miguel Morillas <morillas at gmail.com>
>>> > >> >> Date: Thu, Dec 9, 2010 at 1:52 AM
>>> > >> >> Subject: About http://translate.flossmanuals.net encoding
>>> > >> >> To: nicestep at gmail.com
>>> > >> >>
>>> > >> >>
>>> > >> >> Hello, Jim
>>> > >> >>
>>> > >> >> I'm preparing an activity for XO. Walter Bender suggested me this link
>>> > >> >> [1]. There is a problem with encoding. I think you're using two
>>> > >> >> different encodigs (the left hand menu and the content). Walter said
>>> > >> >> that you were the responsible of that document ;) .
>>> > >> >>
>>> > >> >> [1] http://translate.flossmanuals.net/ActivitiesGuideSugar_es/Introduction
>>> > >> >>
>>> > >> >> cheers,
>>> > >> >>
>>> > >> >> -- lm
>>> > >> >> _______________________________________________
>>> > >> >> Discuss mailing list
>>> > >> >> Discuss at lists.flossmanuals.net
>>> > >> >> http://lists.flossmanuals.net/listinfo.cgi/discuss-flossmanuals.net
>>> > >> >>
>>> > >> >
>>> > >> >
>>> > >> >
>>> > >> > --
>>> > >> > "... imagine a puddle waking up one morning and thinking, 'This is an
>>> > >> > interesting world I find myself in - an interesting hole I find myself
>>> > >> > in - fits me rather neatly, doesn't it? In fact it fits me
>>> > >> > staggeringly well, must have been made to have me in it!' This is such
>>> > >> > a powerful idea that as the sun rises in the sky and the air heats up
>>> > >> > and as, gradually, the puddle gets smaller and smaller, it's still
>>> > >> > frantically hanging on to the notion that everything's going to be
>>> > >> > alright, because this world was meant to have him in it, was built to
>>> > >> > have him in it; so the moment he disappears catches him rather by
>>> > >> > surprise."
>>> > >> > Douglas Adams
>>> > >> >
>>> > >>
>>> > >>
>>> > >>
>>> > >
>>> > >
>>> > > _______________________________________________
>>> > > Discuss mailing list
>>> > > Discuss at lists.flossmanuals.net
>>> > > http://lists.flossmanuals.net/listinfo.cgi/discuss-flossmanuals.net
>>> > >
>>> >
>>> >
>>> >
>>>
>>>
>>> _______________________________________________
>>> Discuss mailing list
>>> Discuss at lists.flossmanuals.net
>>> http://lists.flossmanuals.net/listinfo.cgi/discuss-flossmanuals.net
>>>
>>
>>
>> _______________________________________________
>> Discuss mailing list
>> Discuss at lists.flossmanuals.net
>> http://lists.flossmanuals.net/listinfo.cgi/discuss-flossmanuals.net
>>
>
>
>
> --
> "... imagine a puddle waking up one morning and thinking, 'This is an
> interesting world I find myself in - an interesting hole I find myself
> in - fits me rather neatly, doesn't it? In fact it fits me
> staggeringly well, must have been made to have me in it!' This is such
> a powerful idea that as the sun rises in the sky and the air heats up
> and as, gradually, the puddle gets smaller and smaller, it's still
> frantically hanging on to the notion that everything's going to be
> alright, because this world was meant to have him in it, was built to
> have him in it; so the moment he disappears catches him rather by
> surprise."
> Douglas Adams
> _______________________________________________
> Discuss mailing list
> Discuss at lists.flossmanuals.net
> http://lists.flossmanuals.net/listinfo.cgi/discuss-flossmanuals.net
>

-- 
 http://www.binarni.net/