Scope & Purpose

Enterprise companion PR: arangodb/enterprise#647

Simplify the DistributeExecutor and avoid implicit modification of its input variable.

Previously the DistributeExecutor sometimes updated the input variable in-place, leading to unexpected results like in this example:

FOR i IN 1..3 LET x = { test: 1 } INSERT x INTO testi RETURN x

Execution plan:
 Id   NodeType            Site  Est.   Comment
  1   SingletonNode       COOR     1   * ROOT
  2   CalculationNode     COOR     1     - LET #3 = 1 .. 3   /* range */   /* simple expression */
  4   CalculationNode     COOR     1     - LET x = { "test" : 1 }   /* json expression */   /* const assignment */
  3   EnumerateListNode   COOR     3     - FOR i IN #3   /* list iteration */
  9   DistributeNode      COOR     3       - DISTRIBUTE  /* create keys: true, variable: x */
 10   RemoteNode          DBS      3       - REMOTE
  6   InsertNode          DBS      3       - INSERT x IN testi 
 11   RemoteNode          COOR     3       - REMOTE
 12   GatherNode          COOR     3       - GATHER   /* unsorted */
  8   ReturnNode          COOR     3       - RETURN x

This query should return {test: 1} three times. Instead it returns:

[ 
  { "test" : 1, "_key" : "2010078" }, 
  { "test" : 1, "_key" : "2010079" }, 
  { "test" : 1, "_key" : "2010080" } 
]

This PR moves the modification logic from the DistributeExecutor into three new internal AQL functions (MAKE_DISTRIBUTE_INPUT, MAKE_DISTRIBUTE_INPUT_WITH_KEY_CREATION, MAKE_DISTRIBUTE_GRAPH_INPUT). As a post-processing step after the optimization, we insert a new calculation node with the corresponding function call for each distribute node in the plan (if necessary). This is done in a post-processing step so that the calculation node does not interfere with any optimization rules operating on the distribute node and its variable,

This change not only simplifies the DistributeExecutor, but also makes any additional calculation (if necessary) explicit and avoids unexpected results like in the previous example.

• 💩 Bugfix (requires CHANGELOG entry)
• 🔨 Refactoring/simplification
• 📖 CHANGELOG entry made

Backports:

• No backports required

Related Information

• Main repository PR: arangodb/enterprise#647

Testing & Verification

• This change is already covered by existing tests, such as shell_server_aql.
• This PR adds tests that were used to verify all changes:
  • Added new integration tests shell_server_aql