A report on Character encoding and UTF-8

Punched tape with the word "Wikipedia" encoded in ASCII. Presence and absence of a hole represents 1 and 0, respectively; for example, "W" is encoded as "1010111".
Declared character set for 10million most popular websites since 2010
Hollerith 80-column punch card with EBCDIC character set
Use of the main encodings on the web from 2001 to 2012 as recorded by Google, with UTF-8 overtaking all others in 2008 and over 60% of the web in 2012 (since then approaching 100%). The ASCII-only figure includes all web pages that only contain ASCII characters, regardless of the declared header. Other encodings of Unicode such as GB2312 are added to "others".
365x365px

UTF-8 is a variable-width character encoding used for electronic communication.

- UTF-8

A code unit in UTF-8, EBCDIC and GB 18030 consists of 8 bits;

- Character encoding
Punched tape with the word "Wikipedia" encoded in ASCII. Presence and absence of a hole represents 1 and 0, respectively; for example, "W" is encoded as "1010111".

12 related topics with Alpha

Overall

Logo of the Unicode Consortium

Unicode

8 links

Logo of the Unicode Consortium
Many modern applications can render a substantial subset of the many scripts in Unicode, as demonstrated by this screenshot from the OpenOffice.org application.
15px
Various Cyrillic characters shown with upright, oblique and italic alternate forms

Unicode, formally The Unicode Standard is an information technology standard for the consistent encoding, representation, and handling of text expressed in most of the world's writing systems.

The most common encodings are the ASCII-compatible UTF-8, the obsolete UCS-2, the UCS-2-compatible UTF-16, and GB18030 which is not an official Unicode standard but is used in China and implements Unicode fully.

The first 216 Unicode code points. The stripe of solid gray near the bottom are the surrogate halves used by UTF-16 (the white region below the stripe is the Private Use Area)

UTF-16

6 links

The first 216 Unicode code points. The stripe of solid gray near the bottom are the surrogate halves used by UTF-16 (the white region below the stripe is the Private Use Area)

UTF-16 (16-bit Unicode Transformation Format) is a character encoding capable of encoding all 1,112,064 valid character code points of Unicode (in fact this number of code points is dictated by the design of UTF-16).

Since May 2019, Microsoft has begun supporting UTF-8 (as well as UTF-16) and encouraging its use.

ASCII chart from a pre-1972 printer manual

ASCII

5 links

ASCII chart from a pre-1972 printer manual
ASCII (1963). Control pictures of equivalent controls are shown where they exist, or a grey dot otherwise.

ASCII, abbreviated from American Standard Code for Information Interchange, is a character encoding standard for electronic communication.

ASCII was the most common character encoding on the World Wide Web until December 2007, when UTF-8 encoding surpassed it; UTF-8 is backward compatible with ASCII.

IBM code page numbers (CPGIDs and CCSIDs) used for CJK encodings. Microsoft use of code page numbers for CJK encodings differs, and is noted in brackets where applicable.

Code page

4 links

IBM code page numbers (CPGIDs and CCSIDs) used for CJK encodings. Microsoft use of code page numbers for CJK encodings differs, and is noted in brackets where applicable.

In computing, a code page is a character encoding and as such it is a specific association of a set of printable characters and control characters with unique numbers.

Vendors that use a code page system allocate their own code page number to a character encoding, even if it is better known by another name; for example, UTF-8 has been assigned page numbers 1208 at IBM, 65001 at Microsoft, and 4110 at SAP.

Windows-1252

5 links

Windows-1252 or CP-1252 (code page 1252) is a single-byte character encoding of the Latin alphabet, used by default in the legacy components of Microsoft Windows for English and many European languages including Spanish, French, and German.

An unknown (but probably large) subset of other pages use only the ASCII portion of UTF-8, or only the codes matching Windows-1252 from their declared character set, and could also be counted.

ISO/IEC 8859-1 code page layout

ISO/IEC 8859-1

3 links

ISO/IEC 8859-1 code page layout

ISO/IEC 8859-1:1998, Information technology — 8-bit single-byte coded graphic character sets — Part 1: Latin alphabet No. 1, is part of the ISO/IEC 8859 series of ASCII-based standard character encodings, first edition published in 1987.

This is sometimes assumed to be the encoding of text on Microsoft Windows (and Unix) if there is no byte order mark (BOM); this is only gradually being changed to UTF-8.

Code point

2 links

In character encoding terminology, a code point, codepoint or code position is a numerical value that maps to a specific character.

For Unicode, the particular sequence of bits is called a code unit – for the UCS-4 encoding, any code point is encoded as 4-byte (octet) binary numbers, while in the UTF-8 encoding, different code points are encoded as sequences from one to four bytes long, forming a self-synchronizing code.

Windows code page

2 links

Windows code pages are sets of characters or code pages (known as character encodings in other operating systems) used in Microsoft Windows from the 1980s and 1990s.

This method encodes uniquely all Unicode characters in the Basic Multilingual Plane and a 32-bit (four byte) code for others – but the rest of the industry (Unix-like systems and the web) chose UTF-8 (which uses one byte for the 7-bit ASCII character set, two or three bytes for other characters in the BMP, and four bytes for the remainder).

Universal Coded Character Set

2 links

The Universal Coded Character Set (UCS, Unicode) is a standard set of characters defined by the international standard ISO/IEC 10646, Information technology — Universal Coded Character Set (UCS) (plus amendments to that standard), which is the basis of many character encodings, improving as characters from previously unrepresented typing systems are added.

Rob Pike and Ken Thompson, the designers of the Plan 9 operating system, devised a new, fast and well-designed mixed-width encoding, which came to be called UTF-8,

Variable-width encoding

1 links

A variable-width encoding is a type of character encoding scheme in which codes of differing lengths are used to encode a character set (a repertoire of symbols) for representation, usually in a computer.

For example, the four character string "I♥NY" is encoded in UTF-8 like this (shown as hexadecimal byte values): 49 E2 99 A5 4E 59.