Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subprojects: update lua,pcre2,sdl2 #1812

Merged
merged 3 commits into from
Jun 18, 2024

Conversation

takase1121
Copy link
Member

Lua:

lua: Make library install by default

The lua library wasn't set to be installed which would result in the corresponding shared library being missing in the install directory. This would cause the linker being unable to find the library when using a dependent project unless the SO was manually copied out from the build directory.

PCRE2:

Version 10.45 xx-xxx-2024
-------------------------

1. Change 6 of 10.44 broke 32-bit compiles because pcre2test's reporting of 
memory size was changed to the entire compiled data block, instead of just the 
pattern and tables data, so as to align with the new length restriction. 
Because the block's header contains pointers, this meant the pcre2test output
was different in 32-bit mode. A patch by Carlo reverts to the preevious state
and makes sure that any limit set by pcre2_set_max_pattern_compiled_length()
also avoids the internal struct overhead.


Version 10.44 07-June-2024
--------------------------

1. If a pattern contained a variable-length lookbehind in which the first
branch was not the one with the shortest minimum length, and the lookbehind
contained a capturing group, and elsewhere in the pattern there was another
lookbehind that referenced that group, the pattern was incorrectly compiled,
leading to unpredictable results, including crashes in JIT compiling. An
example pattern is: /(((?<=123?456456|ABC)))(?<=\2)/

2. Further updates to the oss-fuzz support:

   (a) Limit quantifiers for groups and classes to be no more than 10. This
       avoids very long JIT compile times that happen in some cases when groups
       are replicated for quantification, and very long match times when
       classes contain a lot of non-ascii characters.

   (b) Added PCRE2_EXTENDED_MORE to the list of allowed options.

   (c) Arranged for text error messages to be shown in 16-bit and 32-bit modes.

   (d) Made the output in standalone mode more readable.

   (e) General code tidies.

   (f) Limit the size of compiled patterns to 10MB (see 6 below).

   (g) Do not run JIT on patterns whose compiled length is greater than 200K
       bytes because this takes a long time, causing oss-fuzz to time out.

   (h) Avoid compiling or matching twice with the same options (this could
       happen if the input didn't set any options).

3. Increase the maximum length of a name for a group from 32 to 128 because
there is a user for whom 32 is too small.

4. Cause pcre2test to output a message when pcre2_jit_compile() gives an error
return if either jitverify or info is specified.

5. Some auxiliary files for building under OpenVMS that were contributed by
Alexey Chupahin have been installed.

6. Added pcre2_set_max_pattern_compiled_length() to limit the size of compiled
patterns.

7. There was a bug in the implementation of \X caused by my (PH) misreading or
misunderstanding one of the grapheme sequence breaking rules in Unicode Annex
#29. A break should occur between two characters with the Extended Pictographic
break property unless a zero-width joiner intervenes. PCRE2 was not insisting
on the ZWJ, causing \X to match more than it should. See GitHub issue #410.

8. Avoid compilation issues with proprietary compilers in UNIX since 10.43.


Version 10.43 16-February-2024
------------------------------

1. The test program added by change 2 of 10.42 didn't work when the default
newline setting didn't include \n as a newline. One test needed (*LF) to ensure
that it worked.

2. Added the new freestanding POSIX test program to the ManyConfigTests script
in the maint directory (overlooked in 2 below). Also improved the selection
facilities in that script, and added a test with JIT in a non-source directory,
fixing an oversight that would have made such a test fail before.

3. Added pcre2_get_match_data_heapframes_size() and related pcre2test flags
to allow for finer control of the heap used when pcre2_match() without JIT is
used and the match_data might be reused. This began as PR #191, but has had
further refinement and documentation edits.

4. Applied PR #181, which tidies some casts in pcre2_valid_utf.c.

5. Applied PR #184, which avoids overflow issues with the heap limit
(introduced in 10.41/9).

6. Applied PR #192, which changes the timing units for pcre2test from
milliseconds to microseconds. This is more useful for modern CPUs.

7. Applied PR #193, which makes the requirement for C99 explicit in
configure.ac and CMakeLists.txt.

8. Fixed a bug in pcre2test when a ridiculously large string repeat required a
stupid amount of memory. It now gives a clean realloc() failure error.

9. Updates to restrict the interaction between ASCII and non-ASCII characters
for caseless matching and items like \d:

   (a) Added PCRE2_EXTRA_CASELESS_RESTRICT to lock out mixing of ASCII and
       non-ASCII when matching caselessly. This is also /r in pcre2test and
       (?r) within patterns.

   (b) Added PCRE2_EXTRA_ASCII_{BSD,BSS,BSW,POSIX} and corresponding (?aD) etc
       in patterns and /a in pcre2test.

   (c) Corresponding updates to pcre2test.

10. Unicode has been updated to 15.0.0.

11. The Python scripts and ucptest.c in maint have been updated (a) a minor
change needed for 9(a) above; (b) fix bugs in ucptest,

12. Integer overflow testing is now centralized in a new function.

13. Made PCRE2_UCP the default in UTF mode in pcre2grep, and added new options
--case-restrict and --no-ucp.

14. In the debugging printint module (which is normally only linked into
pcre2test), avoid the use of a variable called "not" because that's deprecated
in C and forbidden in C++. Also rewrite some code to avoid a goto into a block
that bypassed its initialization (though it didn't actually matter).

15. More minor code adjustments to avoid using reserved C++ words as variable
names ("new" and "typename") and another jump that bypassed an (irrelevant)
initialization.

16. Merged a pull request that removed pcre2_ucptables.c from the list of files
to compile in NON-AUTOTOOLS-BUILD because it is #included in pcre2_tables.c.
Also adjusted the BUILD.bazel and build.zig files, which had the same issue. At
the same time, fixed a typo in the Bazel file.

17. Add PCRE2_EXTRA_ASCII_DIGIT to allow [:digit:] to be kept on sync with \d
even in UCP mode.

18. Fix an invalid match of ascii word classes when invalid utf is enabled.

19. Add a --posix-digit to pcre2grep for compatibility with GNU grep, and
other tools that prefer the POSIX compatible unicode definition for \d.

20. Report the bit width of the library in use by pcre2test for usability.

21. A pathological pattern conversion test could result in a string longer than
the available input buffer. Cause such a test to fail.

22. Add a check that forces a compiler error if PCRE2_CODE_UNIT_WIDTH is not 8,
16, or 32 when compiling any of the library modules.

23. Update pcre2_compile() to treat a NULL pattern with zero length as an empty
string.

24. Add support for limited-length variable-length lookbehind assertions, with
default maximum length 255 characters (same as Perl) but with a function to
adjust the limit.

25. Applied pull request #262, which updates the zig configuration, and #278
which fixes a bug with out-of-source-tree CMake build testing.

26. Add support for LoongArch to JIT.

27. Fixed a bug in pcre2_match() in the code for handling the vector of
backtracking frames on the heap, which caused a heap overflow if *LIMIT_HEAP
restricted an attempt to extend to less than the frame size. Generally tidy up
the code for extending the heap frames vector. This fixes GitHub issue #275.

28. Update pcre2_fuzzsupport.c to avoid clang sanitize complaint about shifting
left by 16 when there are non-zeros in the top 16 bits.

29. Perl 5.34.0 changed the meaning of (for example) {,3} which did not used to
be treated as a quantifier. Now it is interpreted as {0,3} and PCRE2 has
changed to match. Note that {,} is still not a quantifier.

30. Perl allows spaces and/or horizontal tabs after { or before } in all items
that use braces, and also before or after the comma in quantifiers. PCRE2 now
does the same, except for \u{...}, which is recognized only when
PCRE2_EXTRA_ALT_BSUX is set. This an ECMAScript, non-Perl compatible,
extension, so PCRE2 follows ECMAScript rather than Perl.

31. Applied pull request #300 by Carlo, which fixes #261. The bug was that
pcre2_match() was not fully resetting all captures that had been set within a
(possibly recursive) subroutine call such as (?3).

32. Changed the meaning of \w (and its synonyms) in UCP mode to match Perl. It
now matches characters whose general categories are L or N or whose particular
categories are Mn (non-spacing mark) or Pc (combining punctuation). The latter
includes underscore.

33. Changed the meaning of [:xdigit:] in UCP mode to match Perl. It now also
matches the "fullwidth" versions of the hex digits. Just like it is done for
[:digit:], PCRE2_EXTRA_ASCII_DIGIT can be used to keep this class ASCII only
without affecting other POSIX classes.

34. GitHub PR305 fixes a potential integer overflow in pcre2_dfa_match().

35. Updated handling of \b and \B in UCP mode to match the changes to \w in 32
above because \b and \B are defined in terms of \w.

36. Within a pattern (?aT) and (?-aT) set and reset the PCRE2_EXTRA_ASCII_DIGIT
option, and (?aP) also sets (?aT) so that (?-aP) disables all ASCII
restrictions on POSIX classes.

37. If PCRE2_FIRSTLINE was set on an anchored pattern, pcre2_match() and
pcre2_dfa_match() misbehaved. PCRE2_FIRSTLINE is now ignored for anchored
patterns.

38. Add a test for ridiculous ovector offset values to the substring extraction
functions.

39. Make OP_REVERSE use IMM2_SIZE for its data instead of LINK_SIZE, for
consistency with OP_VREVERSE.

40. In some legacy environments with a pre C99 snprintf, pcre2_regerror could
return an incorrect value when the provided buffer was too small.

41. Applied pull request #342 which adds sanity checks for ctype functions and
locks out any accidental sign-extension.

42. In the 32-bit library, in non-UTF mode, a quantifier that followed a
literal character with a value greater than or equal to 0x80000000u caused
undefined behaviour.

43. \z was misbehaving when matching fragments inside invalid UTF strings.

44. Implement --group-separator and --no-group-separator for pcre2grep.

45. Fix \X matching in 32 bit mode without UTF in JIT.

46. Fix backref iterators when PCRE2_MATCH_UNSET_BACKREF is set in JIT.

47. Refactor the handling of whole-pattern recursion (?0) in pcre2_match() so
that its end is handled similarly to other recursions. This has altered the
behaviour of   /|(?0)./endanchored   which was previously not right.

48. Improved the test for looping recursion by checking the last referenced
character as well as the current character. This allows some patterns that
previously triggered the check to run to completion instead of giving the loop
error.

49. In 32-bit mode, the compiler looped for the pattern /[\x{ffffffff}]/ when
PCRE2_CASELESS and PCRE2_UCP (but not PCRE2_UTF) were set. Fixed by not trying
to look for other cases for characters above the Unicode range.

50. In caseless 32-bit mode with UCP (but not UTF) set, the character
0xffffffff incorrectly matched any character that has more than one other case,
in particular k and s.

51. Fix accept and endanchored interaction in JIT.

52. Fix backreferences with unset backref and non-greedy iterators in JIT.

53. Improve the logic that checks for a list of starting code units -- positive
lookahead assertions are now ignored if the immediately following item is one
that sets a mandatory starting character. For example, /a?(?=bc|)d/ used to set
all of a, b, and d as possible starting code units; now it sets only a and d.

54. Fix incorrect class character matches in JIT.

55. In pcre2test, ensure pcre2_jit_match() is used when jitfast is used with
substitution testing.

56. Insert omitted setting of subject length in match data at the end of
pcre2_jit_match().

57. Implemented PCRE2_DISABLE_RECURSELOOP_CHECK for pcre2_match() to enable
some apparently looping recursions to run to completion and therefore match the
JIT behaviour. With this set, real loops will eventually get caught by match or
heap limits or run out of resource.

58. AC did a lot of work on pcre2_fuzzsupport.c to extend it to 16-bit and
32-bit libraries and to compare JIT and non-JIT matching.

SDL2:
libsdl-org/SDL@release-2.30.3...release-2.28.1

and other wrap changes yada yada

meson.build Show resolved Hide resolved
Co-authored-by: Guldoman <giulio.lettieri@gmail.com>
@Guldoman Guldoman merged commit 05b388e into lite-xl:master Jun 18, 2024
10 checks passed
takase1121 added a commit to takase1121/lite-xl that referenced this pull request Jun 21, 2024
* subprojects: update lua,pcre2,sdl2

* meson.build: remove hidapi usage

* subprojects/sdl2: fix compilation with xinput

Co-authored-by: Guldoman <giulio.lettieri@gmail.com>

---------

Co-authored-by: Guldoman <giulio.lettieri@gmail.com>
takase1121 added a commit that referenced this pull request Jun 23, 2024
* subprojects: update lua,pcre2,sdl2

* meson.build: remove hidapi usage

* subprojects/sdl2: fix compilation with xinput

Co-authored-by: Guldoman <giulio.lettieri@gmail.com>

---------

Co-authored-by: Guldoman <giulio.lettieri@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants