[Pkg-fonts-devel] Fwd: Draining the font swamp

[Denis Jacquerye]

On 5/29/07, "Arne Götje (高盛華)" <arne at linux.org.tw> wrote:
>
> Matt Zimmerman wrote:
> >> * GTK2 apps use fontconfig
> >> * QT3 apps do not use fontconfig, at least not for the alias fonts
> >> serif, sans and monotype
> >> * Legacy X apps and GTK1 also cannot use fontconfig. They only use xft
> >> to select the fonts. For these ones defoma is still neded.
> >
> > What does QT3 use?
>
> Ok, it seems QT3 can use fontconfig to see the available fonts, but the
> alias fonts sans, serif and monospace don't work. At least selecting
> glyphs from multiple fonts doesn't work.
> QT3 comes with qtconfig-qt3, where the user can define substitute fonts.
> Those have to be setup manually. That one is supposed to work, but I
> didn't try it yet. (I'm a GTK2 user, because of certain QT3 limitations)
>
> > I am not very concerned for legacy X apps or GTK1; I think we are
> > approaching a time when such packages can be expected to declare an explicit
> > dependency relationship to obtain their fonts if necessary, rather than
> > assuming that they are available.
>
> hmm... I don't know the details how X handles fonts... need to ask
> someone else...
>
> >>> - Which fonts are any good, and for which languages (no easy answer here)
> >> IMHO the following needs to be done:
> >>
> >> 1. classify the available fonts into "Decorative" and "Desktop" fonts.
> >> With "Decorative" I mean fonts, which are nice for *printing*, "Desktop"
> >> refers to fonts which are suitable for *screen* display.
> >> Example for Decorative fonts: AR PL ZenKai family, this is a brushstroke
> >> CJK font, which is nice for printing documents, but horrible for screen
> >> display.
> >> Example for Desktop fonts: DejaVu sans. It's a smart font and a very
> >> simple stroked font, makes it perfect for screen display... no issues
> >> with hinting and so on...
> >
> > Given that this needs input from users of many character sets, perhaps
> > creating a wiki page would be a good start.
>
> yes... first a list of all available fonts in Debian/Ubuntu, then
> classify them whether or not they are suitable for screen display and if
> yes, which Unicode region (some are fine for other regions than Latin ;) ).
>
> >> 2. A list of default fonts should be made for certain languages (this is
> >> only interesting for screen display):
> >>  * For Latin script DejaVu Sans and the SIL fonts for sans and serif
> >> respectively should be on the top of the list. Both are smart fonts
> >> which can compose almost any diacritical combination, as for Vietnamese,
> >> European languages and African languages based on Latin script. Also
> >> both fonts include the full list of IPA characters...
> >
> > Which one of the SIL fonts do you mean?  Is it packaged and available in
> > Debian and Ubuntu?
>
> ttf-sil-doulos
> ttf-sil-charis
>
> both are smart serif fonts. Doulos is the more popular one.

Doulos SIL only come in Regular weight. Bold and Slanted are faux
(generated by freetype/fontconfig).
Frankly Charis SIL is much better, it has the 4 common variants
(Regular, Bold, Bold Oblique and Oblique) but isn't a Times New Roman
look-a-like like Doulos SIL.
Both fonts are great for printing but not so much for screen, their
hinting was not done manually and is bad at some sizes (some straight
lines are blurry sometimes).

> > In Ubuntu, we currently use DejaVu for both sans and serif.
>
> the DejaVu Serif fonts are not smart enough... :( DejaVu Sans is better
> though, but not optimal.

What do you mean by smart? DejaVu Serif is smart (character
positioning, ligatures, language specific glyphs), it just has less
coverage than Sans and has less hinting.

> >>  * For CJK this is difficult. Here we have several issues to take care of:
> >>   a) embedded bitmap glyphs are needed to render acceptable glyphs in
> >> small fontsizes (12 pt and below)
> >
> > Is this an inherent limitation, or one which is specific to the free
> > rendering engines currently available?  Is it possible to overcome this
> > problem without creating so many bitmap glyphs?  Who creates them?
>
> the problem is stroke hinting in small fontsizes (< 16pt). The
> autohinting feature in freetype does not produce good enough glyphs,
> especially for the more complex glyphs with many strokes. It would be
> possible to use binary hinting, but a) binary hinting is patented by
> Adobe and b) this is a whole lot of work for the font maintainer, you
> can spend years until you have hinted all the CJK glyphs. This is
> Sisyphus work...
> So, CJK font maintainers include bitmaps to bypass the rendering engine
> to produce glyphs on the screen. Many glyphs have much more strokes than
> fit into a 16x16 bitmap grid (12 pt). So, the art of embedding bitmaps
> is, to leave information away, but keep it distinguishable for the user.
> (It doesn't always work, though...)
>
> >>   b) there is currently no font avaliable which covers all CJK glyphs in
> >> Unicode
> >>   c) we don't have any acceptable sans-serif font for CJK
> >>   d) currently we can only use the ming/mincho style for screen display,
> >> the Kochi Mincho and AR PL ShanHeiSun Uni fonts contain embedded bitmaps
> >> already.
> >>   e) CJK glyphs in China, Hong Kong, Taiwan, Japan and Korea have
> >> different shapes which share the same Unicode codepoints. The available
> >> fonts (ttf-arphic-uming, ttf-kochi-mincho and ttf-unfonts) overlap each
> >> other in the CJK range, which confuses fontconfig! Chinese users usually
> >> prefer the ttf-arphic-uming package, while Japanese users might prefer
> >> the ttf-kochi-mincho or ttf-sazanami-mincho fonts and Korean users stick
> >> with the ttf-unfonts package. What makes it worse is, that fontconfig
> >> comes with a predefined list of fonts which should be preferred. This
> >> does not suit all CJK users, as they have different preferences.
> >
> > This seems like a real mess.  In Ubuntu, we try to work around this by
> > changing font preferences depending on which supported languages are
> > selected by the user, but it is not ideal.  What if the system needs to
> > support more than one of these?
>
> That's exactly the problem. It is a real mess.
> See below for the explanation of what we need to change in fontconfig
> (or better, upstream should change it!)
>
> >> For all these CJK issues I'm working on a solution. But it takes time
> >> until it's ready.
> >
> > Where can we learn more about your work?
>
> I didn't put any web page up for that. But I can tell you now. :)
> I'm the font maintainer of the ttf-arphic-{ukai|uming} packages, my
> project is CJK-Unifonts and it aims to provide a free set of fonts
> covering all CJK glyphs currently in Unicode for all CJK regions (China,
> Hong Kong/Macao, Taiwan, Japan, Korea).
> Currently it's still a mess, but I'm getting somewhere... slowly...
> For now, the fonts support Big5, GB2312 and HKSCS (Hong Kong
> supplemental charset), but the glyph shapes are those which came with
> the font and do not follow any standard... this is a problem many users
> have with the fonts.
> Currently the fonts come in two styles, Unicode and MBE (MBE is only of
> interest for Taiwanese users and even then optional).
> For the next release I plan to distribute the fonts as ttc (truetype
> collection), which allows me to shrink the font size dramatically.
> For now, each font is about 20MB in size, but the difference between the
> Unicode and MBE styles are only 12 glyphs. So, it's actually  a waste of
> space to have two full size fonts around. A single TTC file would save
> about 50% of space in this case.
> For the future I plan to include glyphs for the different CJK regions
> (if they differ). Compared to providing seperate fonts for each region
> (at current size of 20MB, that would be at least 6 times (China, Hong
> Kong/Macao, Taiwan, Japan, Korea, Taiwan MBE) as much, while with a
> single TTC it would be only around 22 MB total or so...
> The user however still sees 6 different fonts in his system and would
> have to choose which style he wants to use as default for CJK glyphs.
> That one would have to be configured in fontconfig.
>
> I'm also working on a sans-serif CJK font (DejaVu style) for screen display.

We've had some serious request for adding CJK to DejaVu but cannot do
it since we don't have the required knowledge. Let us know if we can
help you.

> >>> - Which criteria are important for selecting which font to use in which
> >>>   context (language, character set, ...)
> >> 1. locale setting is probably best to determine the default preferred
> >> fonts, but the user should have a possibility to change it.
> >
> > As I understand it, this is something that fontconfig does not consider.  I
> > think Michael (CCed) has some background from discussions about this.
>
> yes, fontconfig currently does not support this.

This feature is important. When some people don't like some script
that was added to DejaVu they _always_ say that it was better before
because another font was the fallback for that script _but_ that its
Latin sucks compared to DejaVu. For them it is necessary to be able to
set a locale dependent priority list in fontconfig.

> >> 2. documents which use the ODF format can have paragraphs or even single
> >> letters marked with a language tag. Those can override the default
> >> system font settings.
> >> 3. fontconfig should take care of selecting the proper font for each
> >> script. But it needs some tuning IMHO... see below!
> >>
> >>> - Whether fontconfig requires adjustments in order to respect those criteria
> >> YES! IMHO fontconfig needs some big improvement.
> >> Currently fontconfig's decision on which font to use for a specific
> >> glyph is influenced by the following:
> >>  * which charsets the font package has registered in defoma. If the font
> >> has registered itself for ISO8859-1, fontconfig will consider the font
> >> suitable for that codepoint range. Same is true for all other charsets.
> >>  * how many glyphs are covered by the font and in which codepoint
> >> regions (determined by fontconfig)
> >>  * the config files provided by fontconfig contain a default font
> >> preference ordering. This might not suit all users... and usually does not.
> >>  * config files provided by font packages, which can label their own
> >> fonts as preferred ones and override the system wide setting by this.
> >
> > Right, there doesn't seem to be a way to tell fontconfig which language is
> > in use in a particular context, or even good defaults for the system as a
> > whole.
> >
> >> The following needs to be done:
> >>  * the default config files which specify any font preference should be
> >> removed!
> >
> > Why?  Surely we need to have a globally consistent view of which fonts are
> > preferred under which circumstances.  The user should of course be able to
> > override this, but we need to have a common starting point.
> >
> > If it is because the existing preferences are not flexible enough, then we
> > should try to fix that instead.
>
> They are not flexible enough.
> The default installation comes with the following config files enabled:
[...]
> These defaults do more harm than good.
> For Latin script a good default would be:
>  * serif: Doulos SIL, Charis SIL, DejaVu Serif, Bitstream Vera Serif
>  * sans: DejaVu Sans, Bitstream Vera Sans
>  * monospace: DejaVu Sans Mono, Bitstram Vera Sans Mono

The current config puts Bitstream Vera in before DejaVu, this is a
mistake. DejaVu has fixed some bugs that were in Vera, the order
ignores that.
Proposing Doulos SIL and Charis SIL as first serif fonts is good for
printing, but would not be for screen. They are not properly hinted
for small sizes.

Cheers,

Denis Moyogo Jacquerye