organizing to release JQ 1.6.2 and JQ 1.7: Your suggestions/comments would be appreciated #2550
Description
This is an offshoot of #2305. Please read that for background.
cc: @stedolan @nicowilliams @wtlangford
I am posting this issue to organize a plan to create a new JQ fork with two primary objectives:
- Merge existing bug fixes into a new 1.6.2 release
- Merge existing new features into a new 1.7 release
The plan to create a new for is done out of necessity, not desire-- again, see #2305 for details. This is NOT an attempt to hijack or otherwise replace this repo, and the ultimate goal will be to either merge back into this repo, or to otherwise provide a successor with the blessing of this repo's owners/maintainers. If at any point in this process those maintainers would like to merge this effort back into this repo, that would be welcome
As a start, I would propose that we:
- agree on a decision-making framework
- identify a core set of roles, and folks who can fill said roles, to get this off the ground
- agree on a preliminary target timeline
- identify existing PRs for bug fixes and enhancements to target for 1.6.2 and 1.7, respectively
Request for comment
This entire issue post is a request for comment to see if we can get this off the ground. I will start with the below proposals. These are certainly incomplete and in need of further refinement / fleshing out, so please offer your suggestions / comments
Decision framework
I'm just going to throw out the simplest way to start: on any decisions that are not obvious, the core team votes (on the other hand, how do we decide the core team? kind of chicken & egg...). Maybe we start with the core team being, whoever are the first few volunteers who are willing commit meaningful time/effort/value to this?
As for decision guidelines, I would propose that a core value is impact-driven prioritization. Surely it is impossible to agree on exactly what outcomes have what impact, but we can also surely agree that a fatal bug fix is more important than a cosmetic one, and an enhancement with 100 likes and dozens of comments is probably more impactful than one with no likes and no comments.
Core set of necessary roles
- Administrator(s). owns the repo and handle stuff like collaborators, permissions etc.
- Merge management: approves merges; makes sure it is correct and agreed-on, and necessary parts are covered (for example, perhaps each merge includes someone who has reviewed code, some test that verifies the change and is incorporated into the CI/CD tests, and verification from original PR author)
- CI/CD testing: this will be critical
- C code reviewer: since for now we are only focused on existing PRs
- Cross compilation: for PRs that work in one platform but not others. Alternatively, the onus could be put on the PR author to fix any cross-compilation issues, and on CI/CD to verify across all platforms
Folks who can fill these roles?
Please volunteer. I'm happy to pitch in for any role, though surely others are more skilled than I at CI/CD or code review (and probably all the others too). @pkoppstein @wader @tst2005 @chickenandpork (apologies in advance for anyone, which there are sure to be, whom I should have mentioned and missed here-- pls do not take personally but rather blame my sleepiness): any takers?
Preliminary target timelines
Suggest May 1, 2023 for version 1.6.2. If there are more bug-fix PRs than can fit in that time, can always plan for 1.6.3 some time thereafter.
Suggest May 15, 2023 for version 1.7. Similarly, If there are more feature-enhancement PRs than can fit in that time, can always plan for 1.7.1 some time thereafter.
identify existing PRs for bug fixes and enhancements to target for 1.6.2 and 1.7, respectively
Note: I have not spent much time reviewing PRs, so I've just started by exporting the PRs to jq_prs-20230307.json.txt and spitting out in markdown below, and then mostly arbitrarily choosing a few to get started
| number | title | is_bug | version |
|---|---|---|---|
| #2548 | allow, and strip, comments (#, // or /*) | N | 1.7 |
| #2547 | add utf8tolower, utf8toupper | N | 1.7 |
| #2546 | Fix segmentation fault when using jq in threads | Y | 1.6.2 |
| #2538 | fix: tarball fails distcheck: add files missing from manifest | y | 1.6.2 |
| #2535 | Remove deprecated --nul-output documentation |
Y | 1.6.2 |
| #2523 | Added truncatestream builtin function |
N | 1.7? |
| #2519 | C99 compatibility enhancements for the configure script | 1.7? | |
| #2505 | Build and test using GitHub actions workflow | 1.6.2? | |
| #2504 | doc: Added information on --indent to --help | ||
| #2492 | Show missing backslash in manual | ||
| #2487 | Add !x as syntactic sugar for x | not | ||
| #2484 | Tidy up nomem_handler implementations to fix SEGVs and ensure more consistency between TLS and non-TLS implementations. | ||
| #2482 | Fix double-installed doc | ||
| #2480 | Use right headers and flags in configure | ||
| #2474 | Document how sort_by() can sort by multiple keys, including an example. | ||
| #2473 | Clarify in the manual how map_values() behaves differently from map(). | ||
| #2470 | Add examples for input & inputs to manual |
||
| #2468 | Use autotools standard ${docdir} | ||
| #2462 | docs: clarify split behavior | ||
| #2451 | Bump lxml from 4.6.3 to 4.9.1 in /docs | ||
| #2434 | Fixed typo | ||
| #2432 | [docs] Use yaml.safe_load() in build_*.py scripts | ||
| #2398 | Clarify use of --argjson option | ||
| #2391 | Add fixes for manual | ||
| #2357 | Validate module metadata to be an object | ||
| #2355 | Link to the Onigurama docs | ||
| #2336 | Fix typo | ||
| #2314 | Support binary strings, preserve UTF-8 and UTF-16 errors | ||
| #2311 | Replace bareword error | ||
| #2306 | Added usage under Powershell to the manual | ||
| #2296 | Use a regex to parse ISO8601 dates | ||
| #2292 | Disable some tests when building without regex support | ||
| #2285 | Add in-place modification (expression) shorthand in objects | ||
| #2256 | Fix --indent 0 implicitly enabling compact-output |
||
| #2255 | Added a first fuzzer for integration with OSS-Fuzz. | ||
| #2254 | Fix number lexer to avoid conflict with object indexing | ||
| #2253 | Make 0 divided by 0 results in NaN consistently | ||
| #2252 | Fold modulo operator on constant values and raise zero remainder error quickly | ||
| #2249 | Enhance @sh to output objects as bash associative array initialisers | ||
| #2241 | Added base/1 and unbase/1 | ||
| #2235 | fix handling of -0 | ||
| #2233 | Fix typo | ||
| #2216 | Add an example usage for the inputs builtin | ||
| #2212 | Manual error -- key should be "foo" without spaces | ||
| #2205 | add nix and guix to website and fix typo in manual | ||
| #2202 | Fix strflocaltime on daylight saving time values | ||
| #2188 | Add Windows installation via scoop | ||
| #2183 | Build out of tree | ||
| #2157 | Fix uri format to follow RFC 3986 | ||
| #2142 | Fix string multiplication with a value between 0.0 and 1.0 | Y | 1.6.2 |
| #2137 | allow MS Windows \r\n style line endings and it's (broken) "UTF-8 BOM" | Y | 1.6.2 |
| #2133 | Fix deletion using assigning empty against arrays | ||
| #2112 | Add debug(exp) builtin | ||
| #2108 | Remove undefined behavior caught by LLVM 10 UBSAN. | Y | 1.6.2 |
| #2102 | typos in manual.yml | ||
| #2099 | alloc: properly allocate memory handler per thread | ||
| #2098 | Adds "tobool" builtin function for converting strings to booleans | N | 1.7? |
| #2094 | manual: mention 0-based indexes in array slicing | ||
| #2093 | Fix documentation for default search paths | ||
| #2092 | spelling: text | ||
| #2081 | use homebrew addons | ||
| #2079 | Correct object construction with variables example | ||
| #2067 | Fedora link returns 404 | ||
| #2060 | Added lpad and rpad | ||
| #2059 | #618 alternative to dupn | ||
| #2010 | Fix to incorrect --stream paths | ||
| #2004 | Rewrite Dockerfile to use Alpine Linux, and produce a minimal image | ||
| #2000 | Fix assertion when using multiple data imports | ||
| #1997 | Use decnum for addition | ||
| #1993 | Better dumping | ||
| #1989 | Update manual.yml | ||
| #1977 | Override field color with JQ_COLORS | ||
| #1962 | [doc] replace find with select function as the example of jq-coded functions | ||
| #1961 | implement scan/2 function | ||
| #1907 | Unify/simplify the MultiByteToWideChar() code and add wrappers for open()/stat() | ||
| #1900 | Optimise size of docker file | ||
| #1869 | Make last(empty) empty | ||
| #1866 | Should work on the entire object instead of each key of it. | ||
| #1862 | Added jv_is_integer_large() to the library. | ||
| #1849 | add nested array indexing example | ||
| #1843 | dlopen(), random, I/O, eval, and co-expressions (co-routines) | ||
| #1798 | Add fromstream_with_dups() builtin (fix #1795) | ||
| #1791 | Set field color using JQ_COLORS | ||
| #1789 | Provide a jvp_dump_raw_string function to implement the combination o… | ||
| #1767 | Patches for AIX | Y | 1.6.2 |
| #1748 | Make reverse more DWIM-y | ||
| #1743 | Save literal value of the parsed number to preserve it for the output | ||
| #1738 | Add new builtin fetch |
||
| #1736 | Fix truncate_stream example | ||
| #1733 | fixed word 'outputting' in manual.yml v1.5 | ||
| #1726 | Rename gen_* functions to jq_gen_* | ||
| #1703 | wip: feat: terminate jq immediately after the outgoing pipe closed | ||
| #1701 | Added is{scalar,value,..}. Fixes leaf_paths bug. Leaf paths |
Y | 1.6.2 |
| #1673 | Add support for CRLF line endings in filters | N | 1.7 |
| #1672 | Add scoop installation instructions | ||
| #1671 | "writing output failed" with non ASCII input | ||
| #1654 | Make INDEX/2 more efficient | ||
| #1643 | Add mul builtin | ||
| #1631 | Fix Dockerfile | ||
| #1570 | Fixes issue #1569 | Y | |
| #1558 | Fix a try-catch example | Y | |
| #1549 | Use an unreachable commit ID instead of a tag | ||
| #1537 | Update manual.yml | ||
| #1527 | Update the man page to note that the sort functions are stable | ||
| #1523 | Update maruku and rake dependencies to latest versions | ||
| #1522 | Add ruby config dotfiles for docs | ||
| #1517 | Fix build with linux on travis-ci | Y | 1.6.2 |
| #1460 | Change tutorial to highlight filter part of query versus output part of query | ||
| #1458 | fix portability problems, found in NetBSD-7.1/amd64 | Y | 1.6.2 |
| #1454 | Example of object creation attribute shortcut | ||
| #1440 | Revisions to any/2, all/2, IN/1, IN/2 as per #1438 | ||
| #1327 | jv: Add some support for 64 bit ints in a very conservative way (ALTERNATIVE) | ||
| #1246 | jv: Add some support for 64 bit ints in a very conservative way | ||
| #1228 | Define format in jq code | ||
| #1215 | Add support for seccomp | ||
| #1201 | Add snap packaging support | ||
| #1183 | More info. Directly install the release binary. | ||
| #1169 | Fix some typos | ||
| #1159 | C++ compatibility, C and C++ examples | ||
| #1127 | Ignore jq.exe, fixed gcc compiler warnings for msys2 (windows). | ||
| #1103 | Enhance the "add" operator so that, when applied to objects, if two f… | ||
| #1102 | Allow tonumber to work on strings that contain commas as thousands separators | ||
| #1062 | project/1, query/1 and unify/1 added to builtin.jq, with tests and documentation | ||
| #1060 | Mention select() in if-then-else docs | ||
| #1058 | add debian/ directory | ||
| #1032 | Dump block | ||
| #673 | Working module/package system | ||
| #458 | Move tail call optimisations to compile.c. |