This project has moved and is read-only. For the latest updates, please go here.
1

Closed

TIdMBCSEncoding GetByteCount() and GetCharCount() broken when using ICONV

description

The GetByteCount() and GetCharCount() method of TIdMBCSEncoding are broken when using the ICONV library.

Since iconv() does not support calculating a converted string's length without actually converting data, this logic is implemented manually in Indy. The goal was to convert the input data codepoint-by-codepoint, using a small byte buffer to save memory without having to allocate a buffer large enough to convert the entire input data in one go. So the code calls iconv() in a loop, passing it the remaining input data and letting it decode as much as it can into the local buffer. However, this logic appears to be flawed because of the following condition in the iconv() documentation (http://www.gnu.org/savannah-checkouts/gnu/libiconv/documentation/libiconv-1.13/iconv.3.html):

"
The conversion can stop for four reasons:
...
  1. The output buffer has no more room for the next converted character. In this case it sets errno to E2BIG and returns (size_t)(-1).
    "
If the remaining input data that is passed to iconv() on any given loop iteration would exceed the size of the local buffer when converted, I can see how this condition would being met.

The loop logic needs to be re-work to address this issue.
Closed Apr 6 at 1:49 AM by gambit47

comments