The current behavior (\B not matching "") appears to be intentional:

Strictly speaking, re is working as designed here, even if that design is arguably "incorrect" with respect to other regex implementations.

Changing this behavior would technically be a feature request (not a bug), and would likely require a deprecation period. I'd recommend opening a thread on https://discuss.python.org/c/ideas/6 to see if there's community interest in changing this.

For reference, JavaScript treats the empty string as containing a non-word boundary:

>> /\b/.test("")
false
>> /\B/.test("")
true

Another Perl reference:

$ perl -E 'say "Boundary" if "" =~ /\b/; say "Non-boundary" if "" =~ /\B/;'
Non-boundary

Seems that it is taking Perl as a baseline to some extent, as indicated in test_re:

However the comments really confuse me since the behavior for \b atm is same as Perl's but it's still claiming different from the perlre behaviour. Or have I missed some changes on the behavior of \b later on? If that was the case, I think it's also reasonable to align the behavior of \B to Perl's.

However the comments really confuse me

I think both comments apply to the \B case, as that is the one different from Perl. Splitting the comment that way is a bit confusing if one does not know Perl, which most do not now. @serhiy-storchaka can comment, but I don't think we can really change this now.

This test was added in 5a045b9 (bpo-10713/gh-54922). It was not an assertion for the intended behavior, it was added to ensure that the current behavior would not change. Strictly speaking, the current behavior contradicts the documentation that says that \B is the opposite of \b.

Of course, it may be that the documentation is wrong. But taking into account that the current behavior differs from the behavior of many (if not all) other RE engines, that the code is most likely a copying error (it was not properly tested), that we already did several breaking changes related to zero-width matches in the past, and that it affects only very specific cases, I think that we can and should change this behavior.

It is preferable to emit a FutureWarning first, but I do not know how difficult to do this without producing false positives. If it is too difficult or impossible, we will have no other way as to change the behavior without warning.

We've been reluctant to make any changes to the re module behavior in the past as people's existing code depends on all of its behaviors per Hyrum's Law.

The best that we can do is to document the bug, if there is a bug, and explain that it's kept for backward compatibility.

The linked PR is now just a documentation note.

Documentation enhanced by d3e79d7.

I afraid that the new note is confusing, because \B (as a part of regular expression) matches an empty string (as a part of the input string).

I also think that this is a legitimate case for changing the behavior, for reasons mentioned above. I planned to work on this later.

There is also a lack of tests. There are three places in the code responsible for this behavior, and only one of them is covered by tests.