Skip to content

Elasticsearch errors in indexing #2132

Closed
@Rahmon

Description

Describe the bug

I was trying to index documents that have a geo_location field type and when I tried sync from Dashboard, the indexing finish without any error but no documents were indexed.

Looking at the code, dashboard.php calls the method bulk_index (ElasticPress\Indexable::bulk_index) that can return a WP_Error or an array. The return of this call is checked to verify if is a WP_Error in dashboard.php (line 627):

$return = $indexable->bulk_index( array_keys( $queued_items ) );

if ( is_wp_error( $return ) ) {
   header( 'HTTP/1.1 500 Internal Server Error' );
   wp_send_json_error();
   exit;
}

However, the array can contain an Elasticsearch error that indicates that the document was rejected, that is, not indexed. Using WP-CLI to index with the option --show-errors, I can see that no documents were indexed, the type and the reason error but the caused_by field is not showed.

Steps to Reproduce

Using Elasticsearch v5.6 and WP Local Docker.

// Add a geo_point type field
add_filter(
	'ep_config_mapping',
	function( $mapping ) {
		$mapping['mappings']['post']['properties']['pin'] = [
			'properties' => [
				'location' => [
					'type'             => 'geo_point',
				],
			],
		];

		return $mapping;
	}
);

// Add empty string to lat and lon
add_filter( 
	'ep_post_sync_args',
	function( $post_args ) {
		$post_args['pin']['location'] = [ 'lat' => '', 'lon' => ''];

		return $post_args; 
	}
);

Running the index in the Dashboard, looks like everything works, and all documents were indexed. But trying to access elasticpressdevtest-post-1/post shows that nothing was indexed.

Running in WP-CLI, we got

...
- 6 (Post): 
[mapper_parsing_exception] failed to parse
- 5 (Post): 
[mapper_parsing_exception] failed to parse
- 4 (Post): 
[mapper_parsing_exception] failed to parse
- 3 (Post): 
[mapper_parsing_exception] failed to parse

Number of posts indexed: 0
Warning: Number of post index errors: 586

Expected behavior

In the Dashboard, the indexing stops when getting an error and shows the information about the error. For example, the error returned by Elasticsearch and the most useful information here is [lat] and [lon] must be valid double values.:

"took": 35,
"errors": true,
"items": [
  {
    "index": {
      "_index": "elasticpressdevtest-post-1",
      "_type": "post",
      "_id": "588",
      "status": 400,
      "error": {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse",
        "caused_by": {
          "type": "parse_exception",
          "reason": "[lat] and [lon] must be valid double values",
          "caused_by": {
            "type": "number_format_exception",
            "reason": "empty String"
          }
        }
      }
    }
  },
  {
    "index": {
      "_index": "elasticpressdevtest-post-1",
      "_type": "post",
      "_id": "587",
      "status": 400,
      "error": {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse",
        "caused_by": {
          "type": "parse_exception",
          "reason": "[lat] and [lon] must be valid double values",
          "caused_by": {
            "type": "number_format_exception",
            "reason": "empty String"
          }
        }
      }
    }
  },
  {
  	...
  }
]

In the WP-CLI, showing the caused_by field returned by Elasticsearch.

Screenshots

Additional context
Bulk API’s response body - https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-api-response-body

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions