You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First I would love to profile q and try to understand where are the pain points, and I have no clue how to do that yet.
Assumption
The biggest operation is reading the file and parsing the JSON if the JSON is huge, this becomes more of a problem.
Exploration
I would expect OCaml multicore to help on the parallelization of parse/compile the user input while parse the JSON, that would speed up things, but I would love to parse the parts of the JSON that q needs, instead of parsing the entire file.
The other idea was improving the parsing of the JSON perse, there an exploration to that, which tries to make the JSON parsing a parallelizable operation, it's called "Stack Monoid" https://raphlinus.github.io/gpu/2020/09/05/stack-monoid.html.
Assumption
query-json's AST is recursive which makes the compilation to be recursive as well and it's tail-recursive, I heard that OCaml compiler optimizes the TL.
while jq, is a stack based described here and they are doing a lot of more work than query-json.
Exploration
See how the OCaml compiler optimizes our TL, and check that the recursion is TL.
Explain better how the jq Stack works.
Assumption
menhir it's a parser generator, which you write your rules in .mly files and OCaml creates the parser for you. It's obvious that having a parser written natively sounds that could be more optional in terms of performance.
Exploration
How much effort would be to turn the Parser.mly into a regular Reason file and how much it will improve performance.
The text was updated successfully, but these errors were encountered:
First I would love to profile q and try to understand where are the pain points, and I have no clue how to do that yet.
Assumption
The biggest operation is reading the file and parsing the JSON if the JSON is huge, this becomes more of a problem.
Exploration
I would expect OCaml multicore to help on the parallelization of parse/compile the user input while parse the JSON, that would speed up things, but I would love to parse the parts of the JSON that q needs, instead of parsing the entire file.
The other idea was improving the parsing of the JSON perse, there an exploration to that, which tries to make the JSON parsing a parallelizable operation, it's called "Stack Monoid" https://raphlinus.github.io/gpu/2020/09/05/stack-monoid.html.
Assumption
query-json's AST is recursive which makes the compilation to be recursive as well and it's tail-recursive, I heard that OCaml compiler optimizes the TL.
while jq, is a stack based described here and they are doing a lot of more work than query-json.
Exploration
See how the OCaml compiler optimizes our TL, and check that the recursion is TL.
Explain better how the jq Stack works.
Assumption
menhir it's a parser generator, which you write your rules in .mly files and OCaml creates the parser for you. It's obvious that having a parser written natively sounds that could be more optional in terms of performance.
Exploration
How much effort would be to turn the Parser.mly into a regular Reason file and how much it will improve performance.
The text was updated successfully, but these errors were encountered: