Help - Search - Members - Calendar
Full Version: Displaying Japanese Or Any Other Unicode Character
OESF Forums > Distros, Development, and Model Specific Forums > Distro Support and Discussion > pdaXrom
ShiroiKuma
Hi:

Has anyone been successful in displaying any unicode text in the terminal, i.e. rxvt or so?

I've fiddled with it for the past couple of days, can't do it somehow.

I can change the font that rxvt uses to display stuff by for instance:
CODE
rxvt -fn "xft:Sazanami Gothic-16"

and rxvt comes up using this font, but then when doing cat on a UTF-8 file it displays garbage.

Can anyone do it, not necessarily Japanese, any unicode characters will do, because the process is the same.

How can I do this?
ashikase
QUOTE(ShiroiKuma @ Nov 21 2005, 11:37 PM)
Has anyone been successful in displaying any unicode text in the terminal, i.e. rxvt or so?

You might try using rxvt-unicode (a.k.a. urxvt); I've recently started using it on my notebook. One of it's key points (besides supporting unicode) is that it is very flexible with fonts - it even allows simultaneous usage of true/freetype fonts with X11 fonts.

I haven't tried it on the zaurus, as I don't use pdaXrom, but I saw a report from someone stating that it worked with OZ, so it shouldn't be difficult to build.

The source is available at: http://software.schmorp.de/#rxvt-unicode

- ashikase
- anpachi, gifu, japan
ShiroiKuma
QUOTE(ashikase @ Nov 22 2005, 02:10 AM)
QUOTE(ShiroiKuma @ Nov 21 2005, 11:37 PM)
Has anyone been successful in displaying any unicode text in the terminal, i.e. rxvt or so?

You might try using rxvt-unicode (a.k.a. urxvt); I've recently started using it on my notebook. One of it's key points (besides supporting unicode) is that it is very flexible with fonts - it even allows simultaneous usage of true/freetype fonts with X11 fonts.

I haven't tried it on the zaurus, as I don't use pdaXrom, but I saw a report from someone stating that it worked with OZ, so it shouldn't be difficult to build.

The source is available at: http://software.schmorp.de/#rxvt-unicode

- ashikase
- anpachi, gifu, japan
*


I am using rxvt-unicode. And I can change the font the console uses for display to a UTF-8 font. But then it still displays garbage...

It might be something to do with locales, I installed the UTF-8 locale originally for XQt from http://xqt.sourceforge.jp/feed-testing/ and all other programs work in Japanese, so I'm assuming it works.

But maybe there is some tweaking for the rxvt to be done. But can't get it to work...
ShiroiKuma
I found a solution for this issue, so if anyone is interested:

1. Install vte_0.11.10_armv5tel.ipk.
2. Install gtkterm2_0.2.3_armv5tel.ipk

Then if you have Japanese locale installed and configured you'll see text in Japanese in gtkterm2 no problem.
cs_jacky
Hi, can you input CJK? I got "Error (Invalid or incomplete multibyte or wide character) converting data for child, dropping" when I input Chinese character. How can you set the font of gtkterm2? It is too small for me.

Moreover, do you know how to get the Chinese locale? I see Chinese filenames as "boxes". Thanks.

QUOTE(ShiroiKuma @ Nov 26 2005, 01:23 AM)
I found a solution for this issue, so if anyone is interested:

1. Install vte_0.11.10_armv5tel.ipk.
2. Install gtkterm2_0.2.3_armv5tel.ipk

Then if you have Japanese locale installed and configured you'll see text in Japanese in gtkterm2 no problem.
*
ShiroiKuma
QUOTE(cs_jacky @ Nov 26 2005, 11:08 AM)
Hi, can you input CJK? I got "Error (Invalid or incomplete multibyte or wide character) converting data for child, dropping" when I input Chinese character. How can you set the font of gtkterm2? It is too small for me.

Edit ~/gtkterm2rc.
Change the line starting with terminalFont to
CODE
terminalFont=Terminal 16


QUOTE
Moreover, do you know how to get the Chinese locale? I see Chinese filenames as "boxes".  Thanks.

Don't know about Chinese locale. You can install Japanese locale from the xqt.sourceforge.jp/feed-testing/

It works for pdaXrom, but for Chinese don't know.
ZDevil
Thanks for the tips and packages!!
Actually I've also been looking for the Chinese solution for quite a long time. It will surely be a big news for the Chinese language users. smile.gif
... Just so eager to use the console for most of the actions, commands, notepad, outliner, web browser, ebook reader, etc) rolleyes.gif
cs_jacky
Thanks ShiroiKuma! The font size works for me.

I found that we need to install any UTF-8 locale and set the LANG environment variable to that to ready CJK filenames. For example, if you installed locale zh_TW.utf8, you need to set LANG to zh_TW.UTF-8. The side effect is all applications to display Chinese menu, etc.

If you want to be able to read CJK only and leaving all other things English. Try en_US.utf8 locale and LANG=en_US.UTF-8.

I attached the locale data ipks for you.

Click to view attachmentClick to view attachment
ShiroiKuma
QUOTE(cs_jacky @ Nov 26 2005, 11:15 PM)
If you want to be able to read CJK only and leaving all other things English. Try en_US.utf8 locale and LANG=en_US.UTF-8.

I attached the locale data ipks for you.

Click to view attachmentClick to view attachment
*

Great, thanks for these. We've made some progress here smile.gif
ZDevil
My heartfelt thanks for your packages, cs_jacky! Have been waiting for this for ages. smile.gif
cs_jacky
I compiled gnome-terminal. The gnome-terminal is more user friendly and more option (such as backgrounds). However, it requires more libraries and slower (compare with gtkterm2).

EDIT: ipk updated. Please download it from later post.
ShiroiKuma
QUOTE(cs_jacky @ Nov 27 2005, 07:14 AM)
I compiled gnome-terminal. The gnome-terminal is more user friendly and more option (such as backgrounds). However, it requires more libraries and slower (compare with gtkterm2).

Click to view attachment
*

Can you open more than one session in gnome-terminal? How do you specify which font it uses for display?
ZDevil
Thanks again, cs_jacky. I wonder if it's also possible to use big5 locale as well? smile.gif
cs_jacky
Yes, you can open new windows or new tabs. You can specify font by edit current profile.

QUOTE(ShiroiKuma @ Nov 27 2005, 10:35 PM)
QUOTE(cs_jacky @ Nov 27 2005, 07:14 AM)
I compiled gnome-terminal. The gnome-terminal is more user friendly and more option (such as backgrounds). However, it requires more libraries and slower (compare with gtkterm2).

Click to view attachment
*

Can you open more than one session in gnome-terminal? How do you specify which font it uses for display?
*

cs_jacky
I updated gnome-terminal ipk. The changes are as follows:

1. schema are installed. You have Default profile and shortcut key.
2. schema files are installed to /etc/...

If you have changed gnome-terminal profile, you need to delete gnome-terminal directories (there are 2) under ~/.gconf.

Click to view attachment

P.S. You need to install vte as well.
ZDevil
Thank you so much again for your efforts on making the CJK solutions available, cs_jacky! Please keep up the great work.
Sorry for bothering you again. After trying your new UTF8 locales packages on pdaXrom, I would like to ask you a few things: (Or someone who could enlighten me)
1) It seems the Z still cannot read big5 encoded files, such as in various terminal apps, lynx and Jpilot. Does it mean an extra big5 locale package is a must? If so how can I get it?
2) I've also read about a newer glibc 2.3.2 which is said to support Chinese encoding ( http://www.oesf.org/forums/index.php?showtopic=14089 ), but the link to the complete file is broken. Is this also necessary for reading big5 stuff?
3) Also it seems to me using the locales will take up quite a chunk of the system memory. Does this depend on which unicode ttf font I'm using?
4) Can (some) unicode fonts also display big5 characters?
5) Is it also necessary to make changes in /etc/fonts/local.conf to fix the font choice and finetune the display?

Thanks! smile.gif
cs_jacky
QUOTE(ZDevil @ Nov 28 2005, 06:06 PM)
Thank you so much again for your efforts on making the CJK solutions available, cs_jacky!  Please keep up the great work.
Sorry for bothering you again.  After trying your new UTF8 locales packages on pdaXrom, I would like to ask you a few things:  (Or someone who could enlighten me)
1) It seems the Z still cannot read big5 encoded files, such as in various terminal apps, lynx and Jpilot.  Does it mean an extra big5 locale package is a must?  If so how can I get it?
2) I've also read about a newer glibc 2.3.2 which is said to support Chinese encoding ( http://www.oesf.org/forums/index.php?showtopic=14089 ), but the link to the complete file is broken.  Is this also necessary for reading big5 stuff?
3) Also it seems to me using the locales will take up quite a chunk of the system memory.  Does this depend on which unicode ttf font I'm using? 
4) Can (some) unicode fonts also display big5 characters?
5) Is it also necessary to make changes in /etc/fonts/local.conf to fix the font choice and finetune the display?

Thanks!  smile.gif
*


I think your terminal based app failed to read big5 encoded files because the app don't support big5 encoding. I think you can read big5 files with vim (but you need to specify file encoding). Since I can display Chinese filenames and input Chinese with gnome-terminal, I think the current glibc works fine with big5.

I use a unicode TTF (ArialUni from Microsoft) to display CJK. You can also get a free unicode font from Hong Kong Government Website. I don't notice any huge memory usage with these unicode TTF.
sashz
i added in builder system creating next locale packages: en_US.UTF-8 ru_RU.UTF-8 zh_TW.UTF-8 ja_JP.UTF-8 ko_KR.UTF-8 ru_RU.KOI8-R ru_RU.CP1251

look if there need another
and it will in next pdaXrom release
badog
why not zh_CN.UTF-8?
sashz
QUOTE(badog @ Nov 28 2005, 05:25 AM)
why not zh_CN.UTF-8?
*


will add it now)
ZDevil
Thanks, cs_jacky! I will try again later today and see how it goes. smile.gif
sashz, how about zh_HK.UTF-8 as well (which seems to support even more char than zh_TW)? Many thanks! smile.gif
sashz
QUOTE(ZDevil @ Nov 28 2005, 06:57 AM)
Thanks, cs_jacky! I will try again later today and see how it goes.  smile.gif
sashz, how about zh_HK.UTF-8 as well (which seems to support even more char than zh_TW)?  Many thanks! smile.gif
*


ok, its there:)
iamphet
gtkterm can show russian characters with en_US.UTF-8 locale! Although rxvt-unicode still can't. I'm afraid I need to debug it to solve this problem sad.gif
cs_jacky
QUOTE(iamphet @ Nov 30 2005, 05:46 PM)
gtkterm can show russian characters with en_US.UTF-8 locale! Although rxvt-unicode still can't. I'm afraid I need to debug it to solve this problem  sad.gif
*


I think any UTF-8 locale can display any characters in any language.
iamphet
QUOTE(cs_jacky @ Nov 30 2005, 11:57 AM)
I think any UTF-8 locale can display any characters in any language.

Yeah, this is the reason why I complained about rxvt-unicode.
ShiroiKuma
QUOTE(cs_jacky @ Nov 30 2005, 08:57 AM)
QUOTE(iamphet @ Nov 30 2005, 05:46 PM)
gtkterm can show russian characters with en_US.UTF-8 locale! Although rxvt-unicode still can't. I'm afraid I need to debug it to solve this problem  sad.gif
*


I think any UTF-8 locale can display any characters in any language.
*


The problems with rxvt and other consoles though that I've identified are somehow associated with the fact that these are not GTK applications. They use xfonts, not ttf, and though for some you can specify that they use ttf fonts for display, somehow the coding process or something for processing UTF-8 characters must depend on something, because the coding is wrong.

Thus I can run rxvt-unicode and use a ttf font for it, but it still messes up Japanese characters and doesn't display them properly on the Z.

That's why the solution via using gtkterm2 or gnome-terminal is needed.
ZDevil
Oh I see. Thanks for educating me. I've been trying mrxvt, rxvt-unicode and mlterm (got it from a Korean Zaurus group), but so far I haven't got unicodes TTF displayed properly in any of these. Time to head on to the gtkterm2... rolleyes.gif
iamphet
I've straced rxvt-unicode and noticed it doesn't try to use glibc locale. It does something with X11 locale though. AFAIK gtk apps don't use X11 locale and I think it might be the reason why they work smile.gif
ZDevil
Oops. I was trying gtkterm2 but it got this when trying to initiate
CODE
gtkterm2: error while loading shared libraries: libvte.so.4: cannot open shared object file: No such file or directory


I went through the filelist in the feed but didn't find this libvte. Is it an individual issue of my Z or something needs to be installed on top of gtkterm2? Thanks! smile.gif
ShiroiKuma
QUOTE(ZDevil @ Nov 30 2005, 11:08 PM)
Oops.  I was trying gtkterm2 but it got this when trying to initiate
CODE
gtkterm2: error while loading shared libraries: libvte.so.4: cannot open shared object file: No such file or directory


I went through the filelist in the feed but didn't find this libvte.  Is it an individual issue of my Z or something needs to be installed on top of gtkterm2?  Thanks!  smile.gif
*

That's what the second package is for - vte_0.11.10_armv5tel.ipk

Look above... smile.gif
ZDevil
Oh yes, thanks! Sorry I must be too tired now after trying out so many new packages (windowmake, jpilot, locales, gtkterms...) at the same time. biggrin.gif
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2014 Invision Power Services, Inc.