OESF Portables Forum

Model Specific Forums => Sharp Zaurus => Zaurus - pdaXrom => Topic started by: ShiroiKuma on November 21, 2005, 09:37:24 am

Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 21, 2005, 09:37:24 am
Hi:

Has anyone been successful in displaying any unicode text in the terminal, i.e. rxvt or so?

I've fiddled with it for the past couple of days, can't do it somehow.

I can change the font that rxvt uses to display stuff by for instance:
Code: [Select]
rxvt -fn "xft:Sazanami Gothic-16"and rxvt comes up using this font, but then when doing cat on a UTF-8 file it displays garbage.

Can anyone do it, not necessarily Japanese, any unicode characters will do, because the process is the same.

How can I do this?
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ashikase on November 21, 2005, 08:10:48 pm
Quote
Has anyone been successful in displaying any unicode text in the terminal, i.e. rxvt or so?
You might try using rxvt-unicode (a.k.a. urxvt); I've recently started using it on my notebook. One of it's key points (besides supporting unicode) is that it is very flexible with fonts - it even allows simultaneous usage of true/freetype fonts with X11 fonts.

I haven't tried it on the zaurus, as I don't use pdaXrom, but I saw a report from someone stating that it worked with OZ, so it shouldn't be difficult to build.

The source is available at: http://software.schmorp.de/#rxvt-unicode (http://software.schmorp.de/#rxvt-unicode)

- ashikase
- anpachi, gifu, japan
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 22, 2005, 02:50:19 am
Quote
Quote
Has anyone been successful in displaying any unicode text in the terminal, i.e. rxvt or so?
You might try using rxvt-unicode (a.k.a. urxvt); I've recently started using it on my notebook. One of it's key points (besides supporting unicode) is that it is very flexible with fonts - it even allows simultaneous usage of true/freetype fonts with X11 fonts.

I haven't tried it on the zaurus, as I don't use pdaXrom, but I saw a report from someone stating that it worked with OZ, so it shouldn't be difficult to build.

The source is available at: http://software.schmorp.de/#rxvt-unicode (http://software.schmorp.de/#rxvt-unicode)

- ashikase
- anpachi, gifu, japan
[div align=\"right\"][{POST_SNAPBACK}][/a][/div]
 (http://index.php?act=findpost&pid=104486\")
I am using rxvt-unicode. And I can change the font the console uses for display to a UTF-8 font. But then it still displays garbage...

It might be something to do with locales, I installed the UTF-8 locale originally for XQt from [a href=\"http://xqt.sourceforge.jp/feed-testing/]http://xqt.sourceforge.jp/feed-testing/[/url] and all other programs work in Japanese, so I'm assuming it works.

But maybe there is some tweaking for the rxvt to be done. But can't get it to work...
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 25, 2005, 09:23:04 am
I found a solution for this issue, so if anyone is interested:

1. Install vte_0.11.10_armv5tel.ipk (http://www.shiroikuma.com/peiji/hoka/peiji/linazau/peiji/sofuto/vte_0.11.10_armv5tel.ipk).
2. Install gtkterm2_0.2.3_armv5tel.ipk (http://www.shiroikuma.com/peiji/hoka/peiji/linazau/peiji/sofuto/gtkterm2_0.2.3_armv5tel.ipk)

Then if you have Japanese locale installed and configured you'll see text in Japanese in gtkterm2 no problem.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: cs_jacky on November 26, 2005, 05:08:29 am
Hi, can you input CJK? I got "Error (Invalid or incomplete multibyte or wide character) converting data for child, dropping" when I input Chinese character. How can you set the font of gtkterm2? It is too small for me.

Moreover, do you know how to get the Chinese locale? I see Chinese filenames as "boxes".  Thanks.

Quote
I found a solution for this issue, so if anyone is interested:

1. Install vte_0.11.10_armv5tel.ipk (http://www.shiroikuma.com/peiji/hoka/peiji/linazau/peiji/sofuto/vte_0.11.10_armv5tel.ipk).
2. Install gtkterm2_0.2.3_armv5tel.ipk (http://www.shiroikuma.com/peiji/hoka/peiji/linazau/peiji/sofuto/gtkterm2_0.2.3_armv5tel.ipk)

Then if you have Japanese locale installed and configured you'll see text in Japanese in gtkterm2 no problem.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=104947\"][{POST_SNAPBACK}][/a][/div]
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 26, 2005, 06:07:47 am
Quote
Hi, can you input CJK? I got "Error (Invalid or incomplete multibyte or wide character) converting data for child, dropping" when I input Chinese character. How can you set the font of gtkterm2? It is too small for me.
Edit ~/gtkterm2rc.
Change the line starting with terminalFont to
Code: [Select]
terminalFont=Terminal 16
Quote
Moreover, do you know how to get the Chinese locale? I see Chinese filenames as "boxes".  Thanks.
Don't know about Chinese locale. You can install Japanese locale from the xqt.sourceforge.jp/feed-testing/

It works for pdaXrom, but for Chinese don't know.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 26, 2005, 08:21:54 am
Thanks for the tips and packages!!
Actually I've also been looking for the Chinese solution for quite a long time.  It will surely be a big news for the Chinese language users.  
... Just so eager to use the console for most of the actions, commands, notepad, outliner, web browser, ebook reader, etc)  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: cs_jacky on November 26, 2005, 05:15:36 pm
Thanks ShiroiKuma! The font size works for me.

I found that we need to install any UTF-8 locale and set the LANG environment variable to that to ready CJK filenames. For example, if you installed locale zh_TW.utf8, you need to set LANG to zh_TW.UTF-8. The side effect is all applications to display Chinese menu, etc.

If you want to be able to read CJK only and leaving all other things English. Try en_US.utf8 locale and LANG=en_US.UTF-8.

I attached the locale data ipks for you.

 [ You are not allowed to view attachments ]  [ You are not allowed to view attachments ]
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 26, 2005, 05:26:11 pm
Quote
If you want to be able to read CJK only and leaving all other things English. Try en_US.utf8 locale and LANG=en_US.UTF-8.

I attached the locale data ipks for you.

 [ You are not allowed to view attachments ]  [ You are not allowed to view attachments ]
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105062\"][{POST_SNAPBACK}][/a][/div]
Great, thanks for these. We've made some progress here  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 26, 2005, 06:27:28 pm
My heartfelt thanks for your packages, cs_jacky!  Have been waiting for this for ages.  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: cs_jacky on November 27, 2005, 01:14:21 am
I compiled gnome-terminal. The gnome-terminal is more user friendly and more option (such as backgrounds). However, it requires more libraries and slower (compare with gtkterm2).

EDIT: ipk updated. Please download it from later post.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 27, 2005, 06:35:46 am
Quote
I compiled gnome-terminal. The gnome-terminal is more user friendly and more option (such as backgrounds). However, it requires more libraries and slower (compare with gtkterm2).

 [ You are not allowed to view attachments ]
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105086\"][{POST_SNAPBACK}][/a][/div]
Can you open more than one session in gnome-terminal? How do you specify which font it uses for display?
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 27, 2005, 07:33:48 am
Thanks again, cs_jacky.  I wonder if it's also possible to use big5 locale as well?  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: cs_jacky on November 27, 2005, 04:16:44 pm
Yes, you can open new windows or new tabs. You can specify font by edit current profile.

Quote
Quote
I compiled gnome-terminal. The gnome-terminal is more user friendly and more option (such as backgrounds). However, it requires more libraries and slower (compare with gtkterm2).

 [ You are not allowed to view attachments ]
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105086\"][{POST_SNAPBACK}][/a][/div]
Can you open more than one session in gnome-terminal? How do you specify which font it uses for display?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105099\"][{POST_SNAPBACK}][/a][/div]
Title: Displaying Japanese Or Any Other Unicode Character
Post by: cs_jacky on November 27, 2005, 05:35:18 pm
I updated gnome-terminal ipk. The changes are as follows:

1. schema are installed. You have Default profile and shortcut key.
2. schema files are installed to /etc/...

If you have changed gnome-terminal profile, you need to delete gnome-terminal directories (there are 2) under ~/.gconf.

 [ You are not allowed to view attachments ]

P.S. You need to install vte as well.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 28, 2005, 02:06:58 am
Thank you so much again for your efforts on making the CJK solutions available, cs_jacky!  Please keep up the great work.
Sorry for bothering you again.  After trying your new UTF8 locales packages on pdaXrom, I would like to ask you a few things:  (Or someone who could enlighten me)
1) It seems the Z still cannot read big5 encoded files, such as in various terminal apps, lynx and Jpilot.  Does it mean an extra big5 locale package is a must?  If so how can I get it?
2) I've also read about a newer glibc 2.3.2 which is said to support Chinese encoding ( https://www.oesf.org/forums/index.php?showtopic=14089 (https://www.oesf.org/forums/index.php?showtopic=14089) ), but the link to the complete file is broken.  Is this also necessary for reading big5 stuff?
3) Also it seems to me using the locales will take up quite a chunk of the system memory.  Does this depend on which unicode ttf font I'm using?  
4) Can (some) unicode fonts also display big5 characters?
5) Is it also necessary to make changes in /etc/fonts/local.conf to fix the font choice and finetune the display?

Thanks!  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: cs_jacky on November 28, 2005, 05:57:18 am
Quote
Thank you so much again for your efforts on making the CJK solutions available, cs_jacky!  Please keep up the great work.
Sorry for bothering you again.  After trying your new UTF8 locales packages on pdaXrom, I would like to ask you a few things:  (Or someone who could enlighten me)
1) It seems the Z still cannot read big5 encoded files, such as in various terminal apps, lynx and Jpilot.  Does it mean an extra big5 locale package is a must?  If so how can I get it?
2) I've also read about a newer glibc 2.3.2 which is said to support Chinese encoding ( https://www.oesf.org/forums/index.php?showtopic=14089 (https://www.oesf.org/forums/index.php?showtopic=14089) ), but the link to the complete file is broken.  Is this also necessary for reading big5 stuff?
3) Also it seems to me using the locales will take up quite a chunk of the system memory.  Does this depend on which unicode ttf font I'm using?   
4) Can (some) unicode fonts also display big5 characters?
5) Is it also necessary to make changes in /etc/fonts/local.conf to fix the font choice and finetune the display?

Thanks! 
[div align=\"right\"][{POST_SNAPBACK}][/a][/div] (http://index.php?act=findpost&pid=105180\")

I think your terminal based app failed to read big5 encoded files because the app don't support big5 encoding. I think you can read big5 files with vim (but you need to specify file encoding). Since I can display Chinese filenames and input Chinese with gnome-terminal, I think the current glibc works fine with big5.

I use a unicode TTF (ArialUni from Microsoft) to display CJK. You can also get a free unicode font from [a href=\"http://www.info.gov.hk/digital21/eng/hkscs/terms/terms34.html]Hong Kong Government Website[/url]. I don't notice any huge memory usage with these unicode TTF.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: sashz on November 28, 2005, 07:06:10 am
i added in builder system creating next locale packages: en_US.UTF-8 ru_RU.UTF-8 zh_TW.UTF-8 ja_JP.UTF-8 ko_KR.UTF-8 ru_RU.KOI8-R ru_RU.CP1251

look if there need another
and it will in next pdaXrom release
Title: Displaying Japanese Or Any Other Unicode Character
Post by: badog on November 28, 2005, 08:25:27 am
why not zh_CN.UTF-8?
Title: Displaying Japanese Or Any Other Unicode Character
Post by: sashz on November 28, 2005, 09:20:56 am
Quote
why not zh_CN.UTF-8?
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105222\"][{POST_SNAPBACK}][/a][/div]

will add it now)
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 28, 2005, 09:57:43 am
Thanks, cs_jacky! I will try again later today and see how it goes.  
sashz, how about zh_HK.UTF-8 as well (which seems to support even more char than zh_TW)?   Many thanks!
Title: Displaying Japanese Or Any Other Unicode Character
Post by: sashz on November 28, 2005, 10:57:48 am
Quote
Thanks, cs_jacky! I will try again later today and see how it goes. 
sashz, how about zh_HK.UTF-8 as well (which seems to support even more char than zh_TW)?   Many thanks!
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105232\"][{POST_SNAPBACK}][/a][/div]

ok, its there:)
Title: Displaying Japanese Or Any Other Unicode Character
Post by: iamphet on November 30, 2005, 01:46:24 am
gtkterm can show russian characters with en_US.UTF-8 locale! Although rxvt-unicode still can't. I'm afraid I need to debug it to solve this problem  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: cs_jacky on November 30, 2005, 02:57:46 am
Quote
gtkterm can show russian characters with en_US.UTF-8 locale! Although rxvt-unicode still can't. I'm afraid I need to debug it to solve this problem 
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105514\"][{POST_SNAPBACK}][/a][/div]

I think any UTF-8 locale can display any characters in any language.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: iamphet on November 30, 2005, 03:48:52 am
Quote
I think any UTF-8 locale can display any characters in any language.
Yeah, this is the reason why I complained about rxvt-unicode.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 30, 2005, 04:00:57 am
Quote
Quote
gtkterm can show russian characters with en_US.UTF-8 locale! Although rxvt-unicode still can't. I'm afraid I need to debug it to solve this problem 
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105514\"][{POST_SNAPBACK}][/a][/div]

I think any UTF-8 locale can display any characters in any language.
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105519\"][{POST_SNAPBACK}][/a][/div]
The problems with rxvt and other consoles though that I've identified are somehow associated with the fact that these are not GTK applications. They use xfonts, not ttf, and though for some you can specify that they use ttf fonts for display, somehow the coding process or something for processing UTF-8 characters must depend on something, because the coding is wrong.

Thus I can run rxvt-unicode and use a ttf font for it, but it still messes up Japanese characters and doesn't display them properly on the Z.

That's why the solution via using gtkterm2 or gnome-terminal is needed.
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 30, 2005, 05:07:58 am
Oh I see. Thanks for educating me.  I've been trying mrxvt, rxvt-unicode and mlterm (got it from a Korean Zaurus group), but so far I haven't got unicodes TTF displayed properly in any of these.  Time to head on to the gtkterm2...
Title: Displaying Japanese Or Any Other Unicode Character
Post by: iamphet on November 30, 2005, 09:40:58 am
I've straced rxvt-unicode and noticed it doesn't try to use glibc locale. It does something with X11 locale though. AFAIK gtk apps don't use X11 locale and I think it might be the reason why they work
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 30, 2005, 05:08:38 pm
Oops.  I was trying gtkterm2 but it got this when trying to initiate
Code: [Select]
gtkterm2: error while loading shared libraries: libvte.so.4: cannot open shared object file: No such file or directory
I went through the filelist in the feed but didn't find this libvte.  Is it an individual issue of my Z or something needs to be installed on top of gtkterm2?  Thanks!  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ShiroiKuma on November 30, 2005, 05:17:15 pm
Quote
Oops.  I was trying gtkterm2 but it got this when trying to initiate
Code: [Select]
gtkterm2: error while loading shared libraries: libvte.so.4: cannot open shared object file: No such file or directory
I went through the filelist in the feed but didn't find this libvte.  Is it an individual issue of my Z or something needs to be installed on top of gtkterm2?  Thanks! 
[div align=\"right\"][a href=\"index.php?act=findpost&pid=105610\"][{POST_SNAPBACK}][/a][/div]
That's what the second package is for - vte_0.11.10_armv5tel.ipk

Look above...  
Title: Displaying Japanese Or Any Other Unicode Character
Post by: ZDevil on November 30, 2005, 05:49:02 pm
Oh yes, thanks!  Sorry I must be too tired now after trying out so many new packages (windowmake, jpilot, locales, gtkterms...) at the same time.