Support Forum
https://forum.hyperion-entertainment.com/
https://forum.hyperion-entertainment.com/viewtopic.php?t=2284
On the contrary, that's exactly what needs to happen. Anything else is a kludge that will also cripple the filesystems efficiency,salass00 wrote: Personally I would rather just use UTF-8 and get rid of all this character set conversion garbage in filesysbox but it just isn't going to happen.
Well FWIW I will change filesysbox to use UTF-8 strings exclusively but I don't expect anyone to be spurred to implement UTF-8 support in the rest of the OS because this.colinw wrote: On the contrary, that's exactly what needs to happen. Anything else is a kludge that will also cripple the filesystems efficiency,
so you might as well take the first step towards UTF-8 compatibility.
Actually that's exactly what I expect to happen to spur further development, when the "wobbly characters" from thesalass00 wrote: Well FWIW I will change filesysbox to use UTF-8 strings exclusively but I don't expect anyone to be spurred to implement
UTF-8 support in the rest of the OS because this.
You could use libunistring until locale.library gets UTF-8 support?salass00 wrote:To implement case insensitive string comparison and hash functions I need a toupper() function that supports unicode.
As I mentioned in a previous post, you must avoid single byte operations on a UTF-8 stream becausesalass00 wrote: To implement case insensitive string comparison and hash functions I need a toupper() function that supports unicode.
I wasn't talking about doing any single byte operations or even using toupper() itself. I already have code for decoding UTF-8 multibyte sequences into 32-bit unicode values. What I need is a toupper()-like function which takes this 32-bit unicode and converts it into it's equivalent upper case unicode if it has one.colinw wrote:As I mentioned in a previous post, you must avoid single byte operations on a UTF-8 stream becausesalass00 wrote: To implement case insensitive string comparison and hash functions I need a toupper() function that supports unicode.
one byte != one glyph anymore, at least for values >= 0x7F. ToUpper() / ToLower() can't work.
For example, take the Angstrom character in UTF-8, it's codepoint value is U+212B and is
represented by 3 bytes; 0xE2 0x84 0xAB, and using a function that performs single byte operations
on each of those 3 bytes within the UTF-8 stream, is simply not going to work.
I would rather not use GPL code in this case and this does much more than I need it to do, but thanks for the suggestion anyway.chris wrote:You could use libunistring until locale.library gets UTF-8 support?salass00 wrote:To implement case insensitive string comparison and hash functions I need a toupper() function that supports unicode.
Carefull, I think the wchar_t is defined as 16 bit in our includes, better check it.salass00 wrote: The newlib.library towupper() accepts a wchar_t which is a 32-bit integer which is why I mentioned it.
I know it's not (it was discussed on the developer mailing list a while back), but it probably doesn't matter because the UTF-8 support in the wide char functions has to be enabled first with setlocale() which means I probably won't be able to use it anyway.colinw wrote: Carefull, I think the wchar_t is defined as 16 bit in our includes, better check it.
Code: Select all
typedef int wchar_t;