Skip to content
Dogu SangjaToolboxAll tools

Why character count and byte count differ

Published: · Last updated:

You paste the same text into two programs; one says 5 and the other says 15. Neither is wrong. They are counting different units. This guide covers the four units used to measure the length of text and shows you how to work out which one a given number refers to.

There are four units to count

  • Character: what looks like a single character on screen. The Unicode Standard calls it a grapheme cluster. It’s the unit people mean when they say “how many characters”.
  • Code point: the number Unicode assigns to each character. A is U+0041 and € is U+20AC. One character on screen can be made of several code points.
  • UTF-16 code unit: the 16-bit unit that JavaScript and many other environments use to hold strings internally. Most code points take 1 unit; those at U+10000 and above take 2.
  • Byte: the space the text actually takes when it’s saved to a file or sent over a network. It depends on the encoding.
Character Characters Code points UTF-16 code units UTF-8 bytes
A 1 1 1 1
é 1 1 1 2
€ 1 1 1 3
😀 1 1 2 4

UTF-8 bytes by type of character

UTF-8 uses 1–4 bytes, depending on the range the code point falls in.

Code point range Bytes What it covers Examples
U+0000–U+007F 1 English letters, digits, basic symbols (ASCII) A 7 !
U+0080–U+07FF 2 Accented Latin letters, Greek, Cyrillic and others é ß α Ж
U+0800–U+FFFF 3 Hangul, Chinese characters, kana, many symbols 한 字 あ €
U+10000–U+10FFFF 4 Most emoji, rarely used Chinese characters and others 😀

So two strings with the same number of characters can have different byte counts.

String Characters UTF-8 bytes Working
Hello 5 5 5 × 1
café 4 5 3 × 1 + 2
Größe 5 7 3 × 1 + 2 × 2
Привет 6 12 6 × 2
안녕하세요 5 15 5 × 3
こんにちは 5 15 5 × 3
Hi 😀 4 7 3 × 1 + 4

Why an emoji that looks like one character takes many bytes

Some emoji are several code points joined together and drawn as a single picture.

Emoji Characters Code points UTF-16 code units UTF-8 bytes Made of
👍🏽 1 2 4 8 Thumbs up U+1F44D + skin tone U+1F3FD
🇺🇳 1 2 4 8 Regional indicators U+1F1FA + U+1F1F3
👨‍👩‍👧 1 5 8 18 3 people + the 2 ZWJs joining them

The family emoji is three emoji, 👨 👩 👧, with a zero-width joiner (ZWJ, U+200D) between them. Each emoji is 4 bytes and each ZWJ is 3 bytes, so the total is 3 × 4 + 2 × 3 = 18 bytes. A program that counts characters shows 1, one that counts code points shows 5, and one that counts UTF-16 code units shows 8.

Emoji aren’t the only case. é can be written as a single code point (U+00E9, 2 bytes) or as e followed by a combining acute accent (U+0301), which is 2 code points (3 bytes). Both look identical on screen, but the byte counts differ.

Why the same character has different byte counts in different encodings

The byte count is set by the encoding, not by the character. Before UTF-8 became widespread, each language region used its own encoding, and those encodings stored their own language’s characters in 2 bytes or fewer.

Character UTF-8 UTF-16 Legacy encoding
A 1 2 1
é 2 2 1 (windows-1252)
한 3 2 2 (EUC-KR)
あ 3 2 2 (Shift_JIS)
字 3 2 2 (EUC-KR, Shift_JIS)
😀 4 4 Can’t be represented

This is why you’ll see the same character described as both “2 bytes” and “3 bytes”. The first refers to a legacy encoding such as EUC-KR or Shift_JIS and the second to UTF-8. 한글 abc is 10 bytes in UTF-8 and 8 bytes in EUC-KR; Tokyo 東京 is 12 bytes in UTF-8 and 10 bytes in Shift_JIS. Most systems built today use UTF-8, but older systems, and forms that inherited their rules, still count on the 2-byte basis.

What to count in practice

Every place that has a limit counts in its own unit. Before you trim text to fit a number, find out what is being counted.

  • Database columns: in some products the length is in characters, in others in bytes. For PostgreSQL’s varchar(n), the documentation states that n is a number of characters, not bytes. Check the documentation for the product and settings you use.
  • Web form fields: the HTML maxlength attribute and JavaScript’s length count UTF-16 code units. A field with a maxlength of 10 accepts only 5 of 😀.
  • URLs and Base64: both convert UTF-8 bytes, so the length of the result follows the byte count, not the character count. The single character € becomes %E2%82%AC (9 characters) when URL-encoded and 4oKs (4 characters) in Base64.
  • Text messages and forms with byte limits: how much fits depends on the encoding used internally. Check whether the unit in the instructions is characters or bytes.

If the unit isn’t stated, add characters one at a time in a field that shows a count, and watch how much the number goes up.

Increase when you add A / 字 / 😀 Unit being counted
1 / 1 / 1 Characters or code points
1 / 1 / 2 UTF-16 code units
1 / 3 / 4 UTF-8 bytes
1 / 2 / something else An older 2-byte scheme

Use 👍🏽 to tell characters from code points: an increase of 1 means characters, and 2 means code points.

How to count it yourself

Paste the code below into the console of your browser’s developer tools to see all four values. Change only the string on the first line.

const s = '👨‍👩‍👧';
[...new Intl.Segmenter().segment(s)].length; // 1: characters
[...s].length;                               // 5: code points
s.length;                                    // 8: UTF-16 code units
new TextEncoder().encode(s).length;          // 18: UTF-8 bytes

To check without code, paste your text into the related tools below. They show the character count, the UTF-8 byte count and the byte count on the 2-byte basis side by side.

Sources

Related tools