diff options
Diffstat (limited to 'src/gxvalid/README')
| -rw-r--r-- | src/gxvalid/README | 532 |
1 files changed, 0 insertions, 532 deletions
diff --git a/src/gxvalid/README b/src/gxvalid/README deleted file mode 100644 index 28e535b..0000000 --- a/src/gxvalid/README +++ /dev/null @@ -1,532 +0,0 @@ -gxvalid: TrueType GX validator -============================== - - -1. What is this ---------------- - - `gxvalid' is a module to validate TrueType GX tables: a collection of - additional tables in TrueType font which are used by `QuickDraw GX - Text', Apple Advanced Typography (AAT). In addition, gxvalid can - validates `kern' tables which have been extended for AAT. Like the - otvalid module, gxvalid uses Freetype 2's validator framework - (ftvalid). - - You can link gxvalid with your program; before running your own layout - engine, gxvalid validates a font file. As the result, you can remove - error-checking code from the layout engine. It is also possible to - use gxvalid as a stand-alone font validator; the `ftvalid' test - program included in the ft2demo bundle calls gxvalid internally. - A stand-alone font validator may be useful for font developers. - - This documents documents the following issues. - - - supported TrueType GX tables - - fundamental validation limitations - - permissive error handling of broken GX tables - - `kern' table issue. - - -2. Supported tables -------------------- - - The following GX tables are currently supported. - - bsln - feat - just - kern(*) - lcar - mort - morx - opbd - prop - trak - - The following GX tables are currently unsupported. - - cvar - fdsc - fmtx - fvar - gvar - Zapf - - The following GX tables won't be supported. - - acnt(**) - hsty(***) - - The following undocumented tables in TrueType fonts designed for Apple - platform aren't handled either. - - addg - CVTM - TPNM - umif - - - *) The `kern' validator handles both the classic and the new kern - formats; the former is supported on both Microsoft and Apple - platforms, while the latter is supported on Apple platforms. - - **) `acnt' tables are not supported by currently available Apple font - tools. - - ***) There is one more Apple extension, `hsty', but it is for - Newton-OS, not GX (Newton-OS is a platform by Apple, but it can - use sfnt- housed bitmap fonts only). Therefore, it should be - excluded from `Apple platform' in the context of TrueType. - gxvalid ignores it as Apple font tools do so. - - - We have checked 183 fonts bundled with MacOS 9.1, MacOS 9.2, MacOS - 10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. In addition, - we have checked 67 Dynalab fonts (designed for MacOS) and 189 Ricoh - fonts (designed for Windows and MacOS dual platforms). The number of - fonts including TrueType GX tables are as follows. - - bsln: 76 - feat: 191 - just: 84 - kern: 59 - lcar: 4 - mort: 326 - morx: 19 - opbd: 4 - prop: 114 - trak: 16 - - Dynalab and Ricoh fonts don't have GX tables except of `feat' and - `mort'. - - -3. Fundamental validation limitations -------------------------------------- - - TrueType GX provides layout information to libraries for font - rasterizers and text layout. gxvalid can check whether the layout - data in a font is conformant to the TrueType GX format specified by - Apple. But gxvalid cannot check a how QuickDraw GX/AAT renderer uses - the stored information. - - 3-1. Validation of State Machine activity - ----------------------------------------- - - QuickDraw GX/AAT uses a `State Machine' to provide `stateful' layout - features, and TrueType GX stores the state transition diagram of - this `State Machine' in a `StateTable' data structure. While the - State Machine receives a series of glyph IDs, the State Machine - starts with `start of text' state, walks around various states and - generates various layout information to the renderer, and finally - reaches the `end of text' state. - - gxvalid can check essential errors like: - - - possibility of state transitions to undefined states - - existence of glyph IDs that the State Machine doesn't know how - to handle - - the State Machine cannot compute the layout information from - given diagram - - These errors can be checked within finite steps, and without the - State Machine itself, because these are `expression' errors of state - transition diagram. - - There is no limitation about how long the State Machine walks - around, so validation of the algorithm in the state transition - diagram requires infinite steps, even if we had a State Machine in - gxvalid. Therefore, the following errors and problems cannot be - checked. - - - existence of states which the State Machine never transits to - - the possibility that the State Machine never reaches `end of - text' - - the possibility of stack underflow/overflow in the State Machine - (in ligature and contextual glyph substitutions, the State - Machine can store 16 glyphs onto its stack) - - In addition, gxvalid doesn't check `temporary glyph IDs' used in the - chained State Machines (in `mort' and `morx' tables). If a layout - feature is implemented by a single State Machine, a glyph ID - converted by the State Machine is passed to the glyph renderer, thus - it should not point to an undefined glyph ID. But if a layout - feature is implemented by chained State Machines, a component State - Machine (if it is not the final one) is permitted to generate - undefined glyph IDs for temporary use, because it is handled by next - component State Machine and not by the glyph renderer. To validate - such temporary glyph IDs, gxvalid must stack all undefined glyph IDs - which can occur in the output of the previous State Machine and - search them in the `ClassTable' structure of the current State - Machine. It is too complex to list all possible glyph IDs from the - StateTable, especially from a ligature substitution table. - - 3-2. Validation of relationship between multiple layout features - ---------------------------------------------------------------- - - gxvalid does not validate the relationship between multiple layout - features at all. - - If multiple layout features are defined in TrueType GX tables, - possible interactions, overrides, and conflicts between layout - features are implicitly given in the font too. For example, there - are several predefined spacing control features: - - - Text Spacing (Proportional/Monospace/Half-width/Normal) - - Number Spacing (Monospaced-numbers/Proportional-numbers) - - Kana Spacing (Full-width/Proportional) - - Ideographic Spacing (Full-width/Proportional) - - CJK Roman Spacing (Half-width/Proportional/Default-roman - /Full-width-roman/Proportional) - - If all layout features are independently managed, we can activate - inconsistent typographic rules like `Text Spacing=Monospace' and - `Ideographic Spacing=Proportional' at the same time. - - The combinations of layout features is managed by a 32bit integer - (one bit each for selector setting), so we can define relationships - between up to 32 features, theoretically. But if one feature - setting affects another feature setting, we need typographic - priority rules to validate the relationship. Unfortunately, the - TrueType GX format specification does not give such information even - for predefined features. - - -4. Permissive error handling of broken GX tables ------------------------------------------------- - - When Apple's font rendering system finds an inconsistency, like a - specification violation or an unspecified value in a TrueType GX - table, it does not always return error. In most cases, the rendering - engine silently ignores such wrong values or even whole tables. In - fact, MacOS is shipped with fonts including broken GX/AAT tables, but - no harmful effects due to `officially broken' fonts are observed by - end-users. - - gxvalid is designed to continue the validation process as long as - possible. When gxvalid find wrong values, gxvalid warns it at least, - and takes a fallback procedure if possible. The fallback procedure - depends on the debug level. - - We used the following three tools to investigate Apple's error handling. - - - FontValidator (for MacOS 8.5 - 9.2) resource fork font - - ftxvalidator (for MacOS X 10.1 -) dfont or naked-sfnt - - ftxdumperfuser (for MacOS X 10.1 -) dfont or naked-sfnt - - However, all tests were done on a PowerPC based Macintosh; at present, - we have not checked those tools on a m68k-based Macintosh. - - In total, we checked 183 fonts bundled to MacOS 9.1, MacOS 9.2, MacOS - 10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. These fonts - are distributed officially, but many broken GX/AAT tables were found - by Apple's font tools. In the following, we list typical violation of - the GX specification, in fonts officially distributed with those Apple - systems. - - 4-1. broken BinSrchHeader (19/183) - ---------------------------------- - - `BinSrchHeader' is a header of a data array for m68k platforms to - access memory efficiently. Although there are only two independent - parameters for real (`unitSize' and `nUnits'), BinSrchHeader has - three additional parameters which can be calculated from `unitSize' - and `nUnits', for fast setup. Apple font tools ignore them - silently, so gxvalid warns if it finds and inconsistency, and always - continues validation. The additional parameters are ignored - regardless of the consistency. - - 19 fonts include such inconsistencies; all breaks are in the - BinSrchHeader structure of the `kern' table. - - 4-2. too-short LookupTable (5/183) - ---------------------------------- - - LookupTable format 0 is a simple array to get a value from a given - GID (glyph ID); the index of this array is a GID too. Therefore, - the length of the array is expected to be same as the maximum GID - value defined in the `maxp' table, but there are some fonts whose - LookupTable format 0 is too short to cover all GIDs. FontValidator - ignores this error silently, ftxvalidator and ftxdumperfuser both - warn and continue. Similar problems are found in format 3 subtables - of `kern'. gxvalid warns always and abort if the validation level - is set to FT_VALIDATE_PARANOID. - - 5 fonts include too-short kern format 0 subtables. - 1 font includes too-short kern format 3 subtable. - - 4-3. broken LookupTable format 2 (1/183) - ---------------------------------------- - - LookupTable format 2, subformat 4 covers the GID space by a - collection of segments which are specified by `firstGlyph' and - `lastGlyph'. Some fonts store `firstGlyph' and `lastGlyph' in - reverse order, so the segment specification is broken. Apple font - tools ignore this error silently; a broken segment is ignored as if - it did not exist. gxvalid warns and normalize the segment at - FT_VALIDATE_DEFAULT, or ignore the segment at FT_VALIDATE_TIGHT, or - abort at FT_VALIDATE_PARANOID. - - 1 font includes broken LookupTable format 2, in the `just' table. - - *) It seems that all fonts manufactured by ITC for AppleWorks have - this error. - - 4-4. bad bracketing in glyph property (14/183) - ---------------------------------------------- - - GX/AAT defines a `bracketing' property of the glyphs in the `prop' - table, to control layout features of strings enclosed inside and - outside of brackets. Some fonts give inappropriate bracket - properties to glyphs. Apple font tools warn about this error; - gxvalid warns too and aborts at FT_VALIDATE_PARANOID. - - 14 fonts include wrong bracket properties. - - - 4-5. invalid feature number (117/183) - ------------------------------------- - - The GX/AAT extension can include 255 different layout features, but - popular layout features are predefined (see - http://developer.apple.com/fonts/Registry/index.html). Some fonts - include feature numbers which are incompatible with the predefined - feature registry. - - In our survey, there are 140 fonts including `feat' table. - - a) 67 fonts use a feature number which should not be used. - b) 117 fonts set the wrong feature range (nSetting). This is mostly - found in the `mort' and `morx' tables. - - Apple font tools give no warning, although they cannot recognize - what the feature is. At FT_VALIDATE_DEFAULT, gxvalid warns but - continues in both cases (a, b). At FT_VALIDATE_TIGHT, gxvalid warns - and aborts for (a), but continues for (b). At FT_VALIDATE_PARANOID, - gxvalid warns and aborts in both cases (a, b). - - 4-6. invalid prop version (10/183) - ---------------------------------- - - As most TrueType GX tables, the `prop' table must start with a 32bit - version identifier: 0x00010000, 0x00020000 or 0x00030000. But some - fonts store nonsense binary data instead. When Apple font tools - find them, they abort the processing immediately, and the data which - follows is unhandled. gxvalid does the same. - - 10 fonts include broken `prop' version. - - All of these fonts are classic TrueType fonts for the Japanese - script, manufactured by Apple. - - 4-7. unknown resource name (2/183) - ------------------------------------ - - NOTE: THIS IS NOT A TRUETYPE GX ERROR. - - If a TrueType font is stored in the resource fork or in dfont - format, the data must be tagged as `sfnt' in the resource fork index - to invoke TrueType font handler for the data. But the TrueType font - data in `Keyboard.dfont' is tagged as `kbd', and that in - `LastResort.dfont' is tagged as `lst'. Apple font tools can detect - that the data is in TrueType format and successfully validate them. - Maybe this is possible because they are known to be dfont. The - current implementation of the resource fork driver of FreeType - cannot do that, thus gxvalid cannot validate them. - - 2 fonts use an unknown tag for the TrueType font resource. - -5. `kern' table issues ----------------------- - - In common terminology of TrueType, `kern' is classified as a basic and - platform-independent table. But there are Apple extensions of `kern', - and there is an extension which requires a GX state machine for - contextual kerning. Therefore, gxvalid includes a special validator - for `kern' tables. Unfortunately, there is no exact algorithm to - check Apple's extension, so gxvalid includes a heuristic algorithm to - find the proper validation routines for all possible data formats, - including the data format for Microsoft. By calling - classic_kern_validate() instead of gxv_validate(), you can specify the - `kern' format explicitly. However, current FreeType2 uses Microsoft - `kern' format only, others are ignored (and should be handled in a - library one level higher than FreeType). - - 5-1. History - ------------ - - The original 16bit version of `kern' was designed by Apple in the - pre-GX era, and it was also approved by Microsoft. Afterwards, - Apple designed a new 32bit version of the `kern' table. According - to the documentation, the difference between the 16bit and 32bit - version is only the size of variables in the `kern' header. In the - following, we call the original 16bit version as `classic', and - 32bit version as `new'. - - 5-2. Versions and dialects which should be differentiated - --------------------------------------------------------- - - The `kern' table consists of a table header and several subtables. - The version number which identifies a `classic' or a `new' version - is explicitly written in the table header, but there are - undocumented differences between Microsoft's and Apple's formats. - It is called a `dialect' in the following. There are three cases - which should be handled: the new Apple-dialect, the classic - Apple-dialect, and the classic Microsoft-dialect. An analysis of - the formats and the auto detection algorithm of gxvalid is described - in the following. - - 5-2-1. Version detection: classic and new kern - ---------------------------------------------- - - According to Apple TrueType specification, there are only two - differences between the classic and the new: - - - The `kern' table header starts with the version number. - The classic version starts with 0x0000 (16bit), - the new version starts with 0x00010000 (32bit). - - - In the `kern' table header, the number of subtables follows - the version number. - In the classic version, it is stored as a 16bit value. - In the new version, it is stored as a 32bit value. - - From Apple font tool's output (DumpKERN is also tested in addition - to the three Apple font tools in above), there is another - undocumented difference. In the new version, the subtable header - includes a 16bit variable named `tupleIndex' which does not exist - in the classic version. - - The new version can store all subtable formats (0, 1, 2, and 3), - but the Apple TrueType specification does not mention the subtable - formats available in the classic version. - - 5-2-2. Available subtable formats in classic version - ---------------------------------------------------- - - Although the Apple TrueType specification recommends to use the - classic version in the case if the font is designed for both the - Apple and Microsoft platforms, it does not document the available - subtable formats in the classic version. - - According to the Microsoft TrueType specification, the subtable - format assured for Windows and OS/2 support is only subtable - format 0. The Microsoft TrueType specification also describes - subtable format 2, but does not mention which platforms support - it. Aubtable formats 1, 3, and higher are documented as reserved - for future use. Therefore, the classic version can store subtable - formats 0 and 2, at least. `ttfdump.exe', a font tool provided by - Microsoft, ignores the subtable format written in the subtable - header, and parses the table as if all subtables are in format 0. - - `kern' subtable format 1 uses a StateTable, so it cannot be - utilized without a GX State Machine. Therefore, it is reasonable - to assume that format 1 (and 3) were introduced after Apple had - introduced GX and moved to the new 32bit version. - - 5-2-3. Apple and Microsoft dialects - ----------------------------------- - - The `kern' subtable has a 16bit `coverage' field to describe - kerning attributes, but bit interpretations by Apple and Microsoft - are different: For example, Apple uses bits 0-7 to identify the - subtable, while Microsoft uses bits 8-15. - - In addition, due to the output of DumpKERN and FontValidator, - Apple's bit interpretations of coverage in classic and new version - are incompatible also. In summary, there are three dialects: - classic Apple dialect, classic Microsoft dialect, and new Apple - dialect. The classic Microsoft dialect and the new Apple dialect - are documented by each vendors' TrueType font specification, but - the documentation for classic Apple dialect is not available. - - For example, in the new Apple dialect, bit 15 is documented as - `set to 1 if the kerning is vertical'. On the other hand, in - classic Microsoft dialect, bit 1 is documented as `set to 1 if the - kerning is horizontal'. From the outputs of DumpKERN and - FontValidator, classic Apple dialect recognizes 15 as `set to 1 - when the kerning is horizontal'. From the results of similar - experiments, classic Apple dialect seems to be the Endian reverse - of the classic Microsoft dialect. - - As a conclusion it must be noted that no font tool can identify - classic Apple dialect or classic Microsoft dialect automatically. - - 5-2-4. gxvalid auto dialect detection algorithm - ----------------------------------------------- - - The first 16 bits of the `kern' table are enough to identify the - version: - - - if the first 16 bits are 0x0000, the `kern' table is in - classic Apple dialect or classic Microsoft dialect - - if the first 16 bits are 0x0001, and next 16 bits are 0x0000, - the kern table is in new Apple dialect. - - If the `kern' table is a classic one, the 16bit `coverage' field - is checked next. Firstly, the coverage bits are decoded for the - classic Apple dialect using the following bit masks (this is based - on DumpKERN output): - - 0x8000: 1=horizontal, 0=vertical - 0x4000: not used - 0x2000: 1=cross-stream, 0=normal - 0x1FF0: reserved - 0x000F: subtable format - - If any of reserved bits are set or the subtable bits is - interpreted as format 1 or 3, we take it as `impossible in classic - Apple dialect' and retry, using the classic Microsoft dialect. - - The most popular coverage in new Apple-dialect: 0x8000, - The most popular coverage in classic Apple-dialect: 0x0000, - The most popular coverage in classic Microsoft dialect: 0x0001. - - 5-3. Tested fonts - ----------------- - - We checked 59 fonts bundled with MacOS and 38 fonts bundled with - Windows, where all font include a `kern' table. - - - fonts bundled with MacOS - * new Apple dialect - format 0: 18 - format 2: 1 - format 3: 1 - * classic Apple dialect - format 0: 14 - * classic Microsoft dialect - format 0: 15 - - - fonts bundled with Windows - * classic Microsoft dialect - format 0: 38 - - It looks strange that classic Microsoft-dialect fonts are bundled to - MacOS: they come from MSIE for MacOS, except of MarkerFelt.dfont. - - - ACKNOWLEDGEMENT - --------------- - - Some parts of gxvalid are derived from both the `gxlayout' module and - the `otvalid' module. Development of gxlayout was supported by the - Information-technology Promotion Agency(IPA), Japan. - - The detailed analysis of undefined glyph ID utilization in `mort' and - `morx' tables is provided by George Williams. - ------------------------------------------------------------------------- - -Copyright 2004, 2005, 2007 by -suzuki toshiya, Masatake YAMATO, Red hat K.K., -David Turner, Robert Wilhelm, and Werner Lemberg. - -This file is part of the FreeType project, and may only be used, -modified, and distributed under the terms of the FreeType project -license, LICENSE.TXT. By continuing to use, modify, or distribute this -file you indicate that you have read the license and understand and -accept it fully. - - ---- end of README --- |