The silent page awaits the waking dawn.
@font-face {
font-family: 'Noto Sans SC';
font-style: normal;
font-weight: 400;
src: url(https://fonts.gstatic.com/s/notosanssc/v41/k3kCo84MPvpLmixcA63oeAL7Iqp5IZJF9bmaG9_FnYlNbPzT7HEL7j12XCOHJKg4RgZw3nFTvwZ8atTsBvwlvRUk7mYP2g.4.woff2) format('woff2');
unicode-range: U+1f1e9-1f1f5, U+1f1f7-1f1ff, U+1f21a, U+1f232, U+1f234-1f237, U+1f250-1f251, U+1f300, U+1f302-1f308, U+1f30a-1f311, U+1f315, U+1f319-1f320, U+1f324, U+1f327, U+1f32a, U+1f32c-1f32d, U+1f330-1f357, U+1f359-1f37e;
}
Sometimes we need to pull in a web font to make a page more pleasant to read.
纯英文页面通常只需要一套包含大小写字母、数字以及符号的字体,在经过 WOFF2 压缩之后,这类字体的体积往往非常小。但是对于中文为主的网页来说,一套包含完整中文的字体即使经过压缩后文件体积也可能达到 MB 级别。如果直接引用未经过优化的字体文件会给页面加载带来产生非常明显的网络开销。
There's already a whole set of mature optimization techniques for fonts, such as preloading, subsetting,unicode-range slicing,font-display: swap, and so on. Applying them brings a very clear improvement: a Chinese font can usually be brought down to one or two hundred KB or even less with no obvious impact on the page experience, which is more than enough for the vast majority of cases.
The straightforward, efficient base-game-plus-DLC approach
Recently, while developing, I ran into a rather interesting situation: the page had one fixed, unchanging set of characters, plus another set of dynamic characters that carry rare characters, change from time to time, and are reused often. The original solution was much like a game's base game plus its DLC: a script and a workflow on a schedule would pull all the dynamic characters from the server and subset the font, producing one font for the fixed characters and another for the dynamic ones. This approach was very direct and efficient, shrinking an original 5.7 MB font file down to a 110 KB font for the fixed characters plus an 8 KB font for the dynamic ones.
But after we changed the database architecture a while back, continuing to aggregate all dynamic characters the old way would have leaked certain details about private users. So there was a need for a solution that could load the missing characters from a font on demand, without collecting all the text up front.
The seemingly reasonable unicode-range approach
When you bring in a font with @font-face, CSS offers a unicode-range descriptor that lets you pass in Unicode code points, so the browser loads the corresponding font file based on the characters a page actually uses, instead of pulling in a massive font file outright.
Given the current requirements, the fixed character set can be split off into its own slice, and the remaining characters sliced using the same unicode-range slicing scheme that Google Fonts uses for Chinese fonts, with the fixed character set removed before slicing.
In practice, slicing schemes like Google Fonts' for Chinese fonts aren't a simple matter of dividing a font evenly by code point; they balance character frequency, the file size of each slice, the average number of requests per page, and character coverage.
This approach works excellently on ordinary sites, but as mentioned above, the dynamic characters on this page include rare characters — and plenty of them 😭. In practice, a large number of rare characters ended up scattered across various large Unicode slices, and a single rare character like "翳" alone was enough to pull down a slice tens of KB in size. Even with a cap on the maximum size of a single slice, the most extreme page still triggered 50 slices totaling over 400 KB of font downloads. The unicode-range, which was supposed to ease the font load, actually created a large amount of unnecessary overhead.
Data size on extreme pages
Requests on extreme pages
Average data size per page
Average requests per page
Total slices
Google Fonts slicing
705.6 KB
31
363.5 KB
13.9
98
Google Fonts slicing (max 8 KB per slice)
425.7 KB
50
239.2 KB
18.8
335
8 code points per slice
253.3 KB
77
174.1 KB
23.7
2854
32 code points per slice
431.6 KB
75
218.1 KB
23
765
64 code points per slice
643.9 KB
66
293.7 KB
22.4
406
IFT, and its promising future
As of the time of writing, no browser natively supports IFT yet; the tests below all use the WASM implementation from the official DEMO (https://garretrieger.github.io/ift-demo).
Incremental Font Transfer (IFT), currently a W3C Candidate Recommendation Draft, is incremental font transfer, or IFT, a brand-new option.
What is IFT?
Unlike Unicode slicing, which produces a series of complete font files, IFT is essentially one initial font file plus many binary font patches loaded afterward. The initial font file carries a patch mapping table. After loading it, the browser first checks whether its font subset already covers every glyph the page needs, then downloads the corresponding glyph patches according to the patch mapping table and merges them into the initial font file.
A TTF font file mainly holds two kinds of data: glyph data and font tables. Glyph data describes the shape of a character and how it should be drawn, while font tables carry information on how characters are located and laid out. Because IFT (IncrementalFont Transfer) lets a font file expand its character set by applying patches, IFT is mainly split into two mechanisms: glyph-keyed patches and table-keyed patches. Glyph patches are used to supply the missing glyph data, while table patches are used to incrementally add to or update font tables.
IFT may handle these two patch mechanisms — glyph-keyed and table-keyed — in two different ways:
Results
To use IFT, you need the w3c/ift-encoder library to encode the font file with IFT. Here we'll focus on a few key parameters:
Since the page is mostly Chinese, you can use --auto_config_primary_script=Script_chinese-simplified to specify Simplified Chinese as the primary writing system. But similar to the Google Fonts slicing method above, this encoding approach still performs poorly on pages with lots of rare characters. Fortunately, IFT lets you use FrequencyDataConfig to point to a Riegeli file encoded from our own character frequency data, so the result matches what the page actually uses. In the tests below, I'll build my own Riegeli file from the dynamic character data the page used previously.
As mentioned above, table patches can affect one another, so when encoding a font the encoder needs a patch graph to handle the relationships between different patches. As mentioned above, table patches can affect one another, so when encoding a font the encoder needs a patch graph to handle the relationships between different patches. The max_depth parameter to cap the maximum depth of the table patch graph, since otherwise the number and size of patches blow up.
An original TTF font may contain tens of thousands of characters, and when encoding, the encoder needs to pack the character combinations inside it into a number of patches. To find the optimal packing scheme, the encoder has to run extremely precise but also extremely slow compression — known as main merging — on every candidate combination. Doing main merging outright would make encoding unusably slow, so a rough preprocessing merge needs to happen first. The preprocessing merge workflow is mainly governed by two parameters, preprocess_merging_group_size and preprocess_merging_probability_threshold. It works as follows:
This keeps the quality while greatly cutting the time and work involved in the encoding process.
After encoding the font with IFT, the page test results are as follows:
Approach
Data size on extreme pages
Requests on extreme pages
Average data size per page
Average requests per page
Total slices
IFT (Script_chinese-simplified)
455.9 KB
428
436.3 KB
403
9845
unicode-range approach with 8 code points per slice
253.3 KB
77
174.1 KB
23.7
2854
IFT (custom frequency data used for all subsequent runs)
226.2 KB
49
182 KB
10.8
13026
IFT (depth=3 + preprocessing merge=12)
247.9 KB
49
169.3 KB
10.8
1944
IFT (depth=3 + preprocessing merge=8)
235.4 KB
49
169.9 KB
10.8
2648
IFT (depth=3 + preprocessing merge=6)
228.8 KB
49
170.8 KB
10.8
3354
IFT (depth=3)
224.2 KB
49
172.5 KB
10.8
4760
IFT (depth=4)
225.6 KB
49
172 KB
10.8
5362
IFT (depth=5)
223.7 KB
49
171.6 KiB
10.8
6762
So is the IFT approach recommended right now?
Given the constraint of 「load missing characters on demand without collecting all the text up front」 — even counting the dynamic on-the-fly subsetting approach I experimented with afterward — IFT is still the most satisfying approach I've tried so far.
On extreme pages, compared with the best 8-code-point slicing in the unicode-range approach, IFT cuts the number of requests by more than a third and transfers less data. And because its incremental transfer mechanism always works off one and the same font file, IFT has native advantages over unicode-range splitting the font into many different fragments — it's less likely to disrupt kerning and ligatures, or to break layout rules.
That said, there are still two obvious obstacles right now: first, IFT currently has to be bundled with a WASM implementation, needlessly adding two hundred-odd KB of overhead; second, it can't take full advantage of the browser's native font optimizations like preloading. So at this stage I don't recommend using this approach.
One day, when browsers natively support tech(incremental) and the WASM can be dropped entirely, IFT will probably go from 「promising」 to a genuine top-tier choice.