Repository navigation
The behavior of mb_strcut in mbstring has been changed in PHP8.1 #9535
Description
Description
When using the mb_strcut function to cut out ISO-2022-JP encoded strings, depending on the number of bytes specified, the behavior is different in PHP 8.1 than before.
The following code:
<?php
$input = 'あaいb';
$bytes_length = 10;
$encoding = "ISO-2022-JP";
$converted_str = mb_convert_encoding($input, $encoding, mb_internal_encoding());
$cut_str = mb_strcut($converted_str, 0, $bytes_length, $encoding);
$reconverted_str = mb_convert_encoding($cut_str, mb_internal_encoding(), $encoding);
var_dump($reconverted_str);Resulted in this output:
string(5) "あa?"
But I expected this output instead:
string(4) "あa"
This behavior has changed since PHP 8.1.
3v4l: https://3v4l.org/FaVWR
I believe the behavior up to PHP 8.0 is correct.
This is because PHP 8.1 and later output contains a "?", which is not supposed to be there.
PHP Version
PHP 8.1.10
Operating System
No response
Activity
cmb69 commentedon Sep 14, 2022
Although the behavior is not documented exactly, this looks like a bug. We likely want to truncate partial characters (code points) at the end. This apparently works nicely for UTF-8, see e.g. https://3v4l.org/oZgCU.
youkidearitai commentedon Sep 24, 2022
Hi, I have a question. mb_strcut is unaffected of mb_substitute_character that manual written by below.
However, there are other function affected by mb_substitute_character (e.g.: mb_convert_case https://3v4l.org/J7kVC )
For this issue, the 10th byte of
'あaいb'is 0x1b, meaning the escape sequence, then converted?. (3v4l: https://3v4l.org/Intbd )I'm curious why mb_substitute_character doesn't work.
If this other subject, then I open new feature request.
NathanFreeman commentedon Sep 26, 2022
@youkidearitai Hi.
I think then converted
?is a valid character now somb_substitute_characterdoesn't work.youkidearitai commentedon Sep 26, 2022
@NathanFreeman Thank you for reply.
My question was "Why mb_strcut is doesn't work of
mb_substitute_character". But I understand thatmb_substitute_characteroriginally (since PHP 4.3.0 https://3v4l.org/s0gbn#v430) doesn't work.Although the question remains why
mb_convert_caseis affected bymb_substitute_characterwhich isn't in the manual(3v4l: https://3v4l.org/2lqgg#v430), this is another issue, thank you for your time.alexdowad commentedon Oct 5, 2022
@youkidearitai Can you read C code? I would be happy to point you to the lines in the source code which would answer your question. Or do you just want a high-level answer which does not require delving into the details of the implementation?
@pakutoma Thanks for reporting this issue. It is nice to see that @NathanFreeman is working on a fix. I will try to guide him so he can complete this successfully.
There are some very tricky issues involved with the implementation of
mb_strcut, but I think we can work them out.Just for information, the unexpected change in behavior which you have kindly reported was (indirectly) caused by 4299e2d.
fix php#9535
Fix GH-9535 (unintended behavior change for mb_strcut in PHP 8.1)
Merge branch 'PHP-8.1' into PHP-8.2
Merge branch 'PHP-8.2'
Fix regression test for GH-9535 on PHP-8.2+
Merge branch 'PHP-8.2'
Merge branch 'master' into unserialize-notice-to-warning