Experienced Elasticsearch Engineer with 7+ years in designing, implementing, and maintaining large-scale Elasticsearch clusters. Proficient in the entire ELK Stack (Elasticsearch, Kibana, Logstash), ensuring seamless integration and optimization. Skilled in capacity planning, performance tuning, securing nodes, and managing Index lifecycle policies. Extensive experience with Logstash for data ingestion from diverse sources and creating comprehensive visualizations in Kibana. Adept in using Metricbeat, Prometheus, NGINX, and other tools for monitoring and optimization. Summary of Experience: Elasticsearch: - Designed, implemented, and maintained Elasticsearch clusters across multiple environments. - Conducted capacity planning and optimized performance with index tuning, sharding, replication. - Secured nodes with encryption and Basic Authentication. - Managed Snapshot and restore operations and Index lifecycle management (ILM) policies. - Developed complex queries using Query DSL and utilized Aggregations for data analysis. - Designed efficient Ingest pipelines and configured Tokenizers and token filters. - Implemented Geo queries and managed Index aliases, templates, and field mappings. - Handled Bulk indexing and integrated Elasticsearch into various systems. - Troubleshooted and resolved cluster issues. Logstash: - Imported and parsed data from diverse sources (CSV, logs, Cassandra, JSON, MongoDB, Kafka, RabbitMQ, SQL Server). - Collaborated with teams, providing technical support and guidance. - Implemented ETL processes using Elasticsearch and Kibana dashboards. Kibana: - Created charts and visualizations (line, bar, pie, area charts). - Designed complex aggregation-based visualizations using Kibana Lens. - Built real-time data dashboards and implemented time series visualizations. Beats and Other Tools: - Experienced in using Metricbeat (Filebeat, Metricbeat, Winlogbeat, Packetbeat) for log collection. - Integrated various monitoring tools such as Prometheus for monitoring Linux metrics. - Worked with various tools including AWS S3 for snapshots and restores, Prometheus for gathering server logs, NGINX for user management in Kibana(old versions), Apache NiFi, Graylog for creating clusters and pipelines, OpenSearch for cluster management, and Enterprise Search for integrating Confluence and creating search UI applications. Familiar with: - Python - Bash - Kubernetes - Docker - AWS - Digital Ocean