The Unicode Technical Committee and the World Wide Web Consortium's (W3C) Internationalization Working Group jointly issued a technical report Friday that clarifies areas of conflict between the two standards.
The different approaches to adding relevance and functionality to documents -- character encoding in Unicode and markup in XML (Extensible Markup Language) -- were beginning to overlap in some areas, and the two organizations have stated their keenness to iron out any areas of conflict.
Unicode defines a 65,536-character set which holds all the letters used in alphabets and syllabaries worldwide, radical characters used in logographic (pictorial) languages such as Chinese and diacritical markers used in many scripts to mark vowels or voice tones.
But it also includes many characters which define the direction which text runs in, such as from right to left as in Arabic scripts or from top to bottom as in Japanese, paragraph separation codes and ways to deal with odd items such as fractions and superscripts. It is mainly in these areas where Unicode and XML have begun to grate against one another.
The two organizations have decided that markup, as used in XML, is generally more robust and functional than Unicode's character encoding for matters not strictly related to producing exotic characters. They cited:
-- character encoding works only with a range of integers used as character codes, while XML, as a user-defined tool, can have a much wider vocabulary and is inherently more extensible
-- character markup can handle necessary control characters more efficiently between transmitter and receiver, who may be using different operating systems which handle control characters in different ways.
-- the hierarchical structure of markup is better an maintaining the context of a complex document than the linear structure of character encoding.
-- markup often coincides with the need for styling, such as if a paragraph needs to be displayed differently because it is a citation in a foreign language
For these reasons, most of the skirmishes between Unicode character encoding and XML markup have been settled in favor of XML, the two organizations said in the technical report.
Among the decisions made are:
-- Unicode's line and paragraph separator will be discouraged in favor of the
and
markup tags
-- language tag codepoints will be replaced by tags
-- musical notation will be replaced by a customized XML language
-- the fraction slash is maintained, but may be handled better by the MathML markup language
-- superscripts and subscripts are retained, but could be replaced by markup tags such as and or and
The full technical report can be viewed at http://www.unicode.org/reports/tr20/.
