Text handling in the C and C++ Standards is a tale of legacy encodings and a demonstration of decisions made that work at the moment don't scale up to the needs of tomorrow. With Unicode on the horizon, C++20 prepared fundamental changes such as char8_t and polishing a things to make it easier to catch bad conversions and logical program errors when working with encoded text. Still, the landscape has poor support for transcoding from one encoding to the other, let alone talking about higher level algorithms without pulling in heavyweight dependencies whose age shows in terms of design and usability.

And so, we need to go deeper than C++.

This talk - part 4.5 of a a 5 part series - explains a way to achieve validation, counting, encode, decode, and transcode in a flexible C API. We will mirror our API off of iconv, and display a way to extend that API to allow for its usage on embedded devices by allowing the end-user to control allocation. This talk will also demonstrate how to turn the iconv model into an open, extensible model.