Elasticsearch consulting for 20-50 GB shards, ILM tiering, and low-latency search.
ELASTICSEARCH CLUSTER DIAGNOSTIC
es-master-01
ActiveCluster state updates: <15ms · Total shards: 1,180 (24 per data node) · Pending tasks: 0 · Snapshot repository s3://es-prod-snapshots verified.
es-data-hot-04
ActiveAvg shard size: 36.2 GB · Indexing rate: 92,000 docs/s · G1GC young pause: 19ms · Parent circuit breaker: 68%.
es-data-warm-02
ActiveILM policy logs-hot-warm-cold-frozen active · Rollover at max_primary_shard_size: 40gb · Searchable snapshots mounted on cold/frozen tiers.
es-vector-01
Active768-dim int8 quantized HNSW index · Page cache off-heap resident: 94% · Hybrid BM25 + kNN Reciprocal Rank Fusion (RRF) active.
Targets: 20-50 GB primary shards · <=31 GB JVM heap · Hot-Warm-Cold-Frozen ILM
Selected clients
Production Search Bottlenecks
Six failure modes that stall Elasticsearch clusters.
Most Elasticsearch incidents trace back to oversharding from daily index patterns, JVM heap saturation from unbounded aggregations, or missing Index Lifecycle Management (ILM) policies. Here is how our search engineers diagnose and fix them.
| Failure Mode | What You See in Production | JusDB Engineering Action |
|---|---|---|
| Oversharding & Small Shards | Thousands of sub-gigabyte shards from daily index patterns exhaust JVM heap, slow down master cluster-state publication, and increase query coordinator fan-out latency. | Consolidate indices into Data Streams with ILM rollover at max_primary_shard_size: 40gb, use the _shrink API on warm indices, and keep shard counts below 20 shards per GB of configured JVM heap. |
| JVM Heap & Circuit Breaker Trips | Data nodes hit 85-95% JVM heap utilization, trigger multi-second G1GC stop-the-world pauses, and reject requests with CircuitBreakingException: [parent] Data too large. | Cap JVM heap at 50% of host RAM (never exceeding 31 GB to preserve compressed OOPs), eliminate high-cardinality text fielddata, convert wide nested aggregations to composite paginations, and tune G1GC region sizing. |
| Hot-Warm-Cold-Frozen Tiering Cost | 30 to 365 days of observability logs or audit events sit on hot NVMe instances because ILM phase transitions or snapshot repositories are misconfigured. | Architect automated Hot-Warm-Cold-Frozen ILM policies: ingest on NVMe hot nodes, shrink and forcemerge on dense warm storage, and mount S3-backed searchable snapshots for cold and frozen retention. |
| Mapping Explosion & Refresh Overhead | Unbounded JSON payloads create 5,000+ dynamic fields per index, stalling master nodes on mapping updates while default 1-second refresh intervals saturate disk IOPS. | Enforce dynamic: strict or runtime fields for sparse attributes, use flattened or match_only_text types for log payloads, and tune refresh_interval to 30s for bulk ingestion. |
| Dense Vector (kNN / HNSW) Memory Sizing | Adding dense_vector HNSW indices for semantic search or RAG exhausts OS page cache, causing disk thrashing and 500ms+ vector query latencies. | Right-size off-heap OS page cache for Lucene HNSW graphs, enable int8_hnsw or int4_hnsw scalar quantization to cut vector memory footprint by 75-87%, and tune m, ef_construction, and hybrid BM25 + RRF pipelines. |
| Licensing & Platform Architecture | Engineering teams need an objective evaluation of Elasticsearch 8.14+ licensing (AGPLv3/SSPL/Elastic License v2), Elastic Cloud vs self-managed ECK, or OpenSearch compatibility trade-offs. | Deliver a vendor-neutral licensing, TCO, and feature-compatibility matrix comparing Elastic Cloud, self-managed Elasticsearch on Kubernetes (ECK), and Apache 2.0 OpenSearch alternatives. |
Engagement Cadence & Response Targets
Structured architecture sprints and 15-minute P1 escalation.
Whether you engage JusDB for a focused 2-week architecture audit or transition into ongoing 24x7 search reliability operations under a 24x7 support agreement, every severity tier is backed by named search engineers.
Production database down, primary unreachable, data corruption risk, or active failover event.
Severe latency spike, replication lag climbing rapidly, connection exhaustion, or degraded HA node.
Non-critical slow query regression, staging issue, backup warning, or replication check alert.
Configuration question, index advisory, planned schema review, or general operational request.
Initial-response targets, not resolution guarantees. The target for your engagement is set in your service agreement. Business-hours consulting coverage runs 03:30–12:30 UTC; 24x7 incident response is available where your service agreement includes it.
Read-Only Cluster Telemetry
We collect _cluster/settings, _cat/shards, _ilm/policy, _nodes/stats/jvm,breaker, and slow-log traces without impacting production traffic.
Shard & Heap Bottleneck Model
We model shard-to-heap ratios, mapping field growth, segment merge debt, and hot-warm-cold storage economics across your retention window.
Templates, ILM & Query Refactors
You receive production-ready index templates, Data Stream rollover policies, JVM options, and rewritten search/aggregation queries.
Guided Rollout & Reindex Cutover
Our engineers pair with your team during rolling node restarts, ILM transitions, and atomic alias cutovers.
Consulting Scope
What is included in an Elasticsearch consulting engagement.
Every engagement produces concrete configurations, including index templates, ILM lifecycle JSON, JVM flags, and reindex scripts tailored to your document volume and query SLOs.
Shard Sizing & Cluster Topology
Right-size primary and replica shards to the 20-50 GB sweet spot, isolate dedicated master, ingest, coordinating, and tiered data roles, and enforce zone-aware shard allocation across availability zones.
Hot-Warm-Cold-Frozen ILM Design
Design Data Streams and Index Lifecycle Management (ILM) policies with byte-size rollover, warm _shrink and forcemerge, and cold/frozen searchable snapshots on object storage.
JVM Heap, G1GC & Query Tuning
Eliminate parent circuit breaker exceptions, profile slow queries and aggregations via the Search Profiler API, optimize BM25 mappings, and tune bulk write thread pools and refresh intervals.
Vector Search & Hybrid RAG Architecture
Architect Lucene dense_vector HNSW indices, configure int8_hnsw scalar quantization, size off-heap memory for vector graphs, and tune hybrid lexical + semantic search relevance.
Upgrades, ECK & Licensing Strategy
Plan rolling upgrades from Elasticsearch 6.x/7.x to 8.x, review Elastic Cloud vs self-managed ECK on Kubernetes, or assess cross-engine migration paths with documented rollback runbooks.
24x7 Production Search Operations
Transition from consulting into ongoing reliability operations with a 15 minutes P1 response target under a 24x7 support agreement, snapshot restore drills, and Prometheus/OpenTelemetry alerting.
Inside an Elasticsearch Architecture Review
From 14,200 small shards to a deterministic ILM lifecycle.
Step through a real remediation sequence on a 15-node Elasticsearch 8.15 cluster: diagnosing oversharding and parent circuit breaker trips, rolling out a 40 GB Data Stream ILM policy, and verifying the right-sized production state.
ELASTICSEARCH SHARD & ILM REMEDIATION SIMULATOR
Application and routing
Daily index patterns with 5 primary shards and 1 replica across dozens of microservices created 14,200 shards averaging 180 MB each. Cluster state updates blocked master nodes and parent circuit breakers tripped during peak aggregations.
Daily index patterns with 5 primary shards and 1 replica across dozens of microservices created 14,200 shards averaging 180 MB each. Cluster state updates blocked master nodes and parent circuit breakers tripped during peak aggregations.
Application and routing
JusDB migrates time-series indices to Data Streams backed by an Index Lifecycle Management (ILM) policy that rolls over at 40 GB primary shard size or 7 days, shrinks warm shard counts, and force-merges read-only indices to 1 Lucene segment.
JusDB migrates time-series indices to Data Streams backed by an Index Lifecycle Management (ILM) policy that rolls over at 40 GB primary shard size or 7 days, shrinks warm shard counts, and force-merges read-only indices to 1 Lucene segment.
Application and routing
After reindexing legacy indices and enforcing ILM rollover, total shard count dropped by 91%, JVM heap stabilized at 64% with sub-25ms G1GC pauses, and search p99 latency improved from 2.94s to 42ms.
After reindexing legacy indices and enforcing ILM rollover, total shard count dropped by 91%, JVM heap stabilized at 64% with sub-25ms G1GC pauses, and search p99 latency improved from 2.94s to 42ms.
Audit Deliverable Specimen: Index Template & Strict Mapping Guard
# JusDB Elasticsearch Architecture Audit: Finding Specimen
# Cluster: es-prod-analytics (Elasticsearch 8.15, 15 data nodes, 64 GB RAM / 31 GB Heap)
## 1. Verify Shard Count per Node & Average Shard Size
GET _cat/allocation?v&s=shards:desc
GET _cat/shards?v&h=index,shard,prirep,state,store,docs,node&s=store:desc
## 2. Inspect JVM Heap Pressure, OldGC Pauses & Circuit Breakers
GET _nodes/stats/jvm,breaker,thread_pool?filter_path=nodes.*.name,nodes.*.jvm.mem.heap_used_percent,nodes.*.jvm.gc.collectors,nodes.*.breakers.parent,nodes.*.thread_pool.write.rejected
## 3. Remediation: Index Template with Strict Mapping & 40GB ILM Rollover
PUT _index_template/logs-app-template
{
"index_patterns": ["logs-app-*"],
"data_stream": {},
"template": {
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1,
"index.lifecycle.name": "jusdb-hot-warm-cold-policy",
"index.refresh_interval": "30s",
"index.mapping.total_fields.limit": 1000,
"index.codec": "best_compression"
},
"mappings": {
"dynamic": "strict",
"properties": {
"@timestamp": { "type": "date" },
"service": { "type": "keyword" },
"level": { "type": "keyword" },
"trace_id": { "type": "keyword", "doc_values": false },
"message": { "type": "match_only_text" }
}
}
}
}Zero-Friction Collaboration
Clear ownership between your team and JusDB.
Managed cloud databases automate much of the infrastructure layer, but your team still owns query performance, capacity, cost, and incident decisions.
Cloud Accounts, Keys & Search Product Logic
- Set business priorities, application requirements, and release plans
- Approve production changes, except routine actions you have pre-approved
- Own your cloud accounts, access policies, and vendor relationships
- Decide when maintenance can happen and how quickly service must recover
- Move the work in-house or to another provider after your agreement's notice period, with your data returned or deleted and the transition help your agreement sets out (Terms, section 8)
Shard Architecture, ILM, JVM Tuning & 24x7 Escalation
- Monitor database health, investigate alerts, and involve the right people
- Review query performance, replication, capacity, and backup health
- Carry out approved changes and update the operating instructions they affect
- Report completed work, open risks, and recommended next steps
| Evaluation Vector | JusDB DBRE | In-House DBA | Legacy Agency | Developer Generalist |
|---|---|---|---|---|
| Primary Shard Sizing & Capacity Modeling (30–50GB) | Enforces the strict 30–50GB primary shard standard, eliminates small-shard Lucene segment metadata bloat, and models memory-to-disk ratios to maximize query cache residency. | Defaults to arbitrary shard counts per index (e.g., 5 shards per index), spawning thousands of 500MB shards that exhaust master node heap and degrade search scatter-gather. | Recommends vertical node scaling and adding costly RAM without analyzing Lucene segment counts or consolidating oversharded indices. | Leaves index templates at system defaults without capacity modeling; triggers cluster state red alerts during high-concurrency aggregation queries. |
| Index Lifecycle Management (ILM Hot/Warm/Cold/Frozen) | Architects automated multi-tier ILM policies transitioning time-series data across hot local NVMe, warm SSD, cold HDD, and frozen searchable snapshots in S3/GCS. | Operates brittle external cron jobs or legacy Curator scripts that fail silently, leaving multi-terabyte indices stranded indefinitely on expensive hot nodes. | Deploys homogeneous single-tier storage clusters and performs emergency manual index deletions only after 85% disk watermark alerts trigger. | No lifecycle tiering; allows log and telemetry indices to accumulate on primary data disks until filesystem 100% full crashes take nodes offline. |
| Lucene Mapping Explosion Prevention & Dynamic Field Governance | Enforces strict mapping schemas, dynamic field governance rules, runtime fields, and caps index.mapping.total_fields.limit to prevent cluster-state serialization stalls. | Permits unconstrained dynamic mappings (dynamic: true); ingested payloads with thousands of random nested JSON keys bloat cluster state to hundreds of megabytes. | Arbitrarily raises index.mapping.total_fields.limit to 10,000+ when ingestion fails, worsening master node heap saturation and slow cluster state publication. | Ingests unstructured JSON payloads without schema design; triggers Lucene field type collisions, unindexed fields, and node restart serialization delays. |
| JVM Heap Ergonomics & 31GB Compressed OOPs Cap | Caps JVM heap strictly at <31GB to guarantee Compressed Ordinary Object Pointers (Compressed OOPs) stay active, maximizing off-heap Lucene filesystem page cache and tuning G1GC pauses. | Allocates 64GB+ heap to Elasticsearch nodes, disabling Compressed OOPs, losing 50% effective pointer density, and triggering catastrophic multi-minute GC pauses. | Leaves default heap configurations or allocates 50% blindly without tuning circuit breaker thresholds, search threadpools, or off-heap filesystem cache ratios. | Allocates 90% of host RAM to JVM heap, starving the underlying Linux OS kernel and Lucene page cache, leading to severe disk paging and node timeouts. |
| Cross-Cluster Replication (CCR) & Disaster Recovery | Designs active-passive and active-active Cross-Cluster Replication (CCR) with auto-following index patterns, sub-second RPO replication streams, and automated health heartbeats. | Relies solely on nightly snapshot restores with 24-hour RPO, requiring multi-hour downtime to restore terabytes of data during an availability zone outage. | Attempts DIY dual-writing from application microservices without transaction rollback coordination or distributed sequence tracking, causing silent data drift. | Assumes single-cluster multi-AZ replication is disaster recovery; suffers total data loss or protracted downtime when regional infrastructure fails. |
| Enterprise Security Hardening (TLS, RBAC, SAML) | Hardens clusters with FIPS-compliant TLS node-to-node transport encryption, fine-grained Document and Field Level Security (DLS/FLS), SAML/OIDC SSO federation, and immutable audit trails. | Uses default elastic superuser credentials shared across application services with basic HTTP authentication and unencrypted internal node transport. | Places unauthenticated Elasticsearch behind an external Nginx proxy without native cluster transport TLS or internal role-based access controls. | Disables security entirely (xpack.security.enabled: false) to bypass TLS certificate validation issues during local and staging development. |
Elasticsearch Support
Emergency unassigned-shard recovery, circuit breaker remediation, and named on-call DBRE coverage.
Upgrades & ReindexElasticsearch Migration
Elasticsearch 7.x to 8.x upgrades, verified reindex pipelines, and Elasticsearch-to-OpenSearch migrations.
Cloud-Native SearchElasticsearch on Kubernetes
ECK Operator architecture, Local NVMe PV provisioning, rolling upgrades, and PDB safety.
Procurement and security
Ready for legal and security review.
- Mutual NDA before any access
- Named individual accounts only
- Read-only access by default
- ISO 27001 & SOC 2 Type II: aligned, certification in progress
Mutual NDA before any access
We sign a mutual NDA before viewing schemas, logs, slow-query captures, or cloud topology.
Least-privilege, named credentials
Engineers connect through your bastion, VPN, or SSM using named accounts with read-only permissions by default.
Change control & approval gates
Production DDL, parameter changes, and failovers follow your change-management workflow with rollback steps prepared first.
Named leadership & direct escalation
Engagements are led by named senior database reliability engineers in India with direct Slack, Teams, and PagerDuty channels.
Elasticsearch Consulting FAQ
Common questions on shard sizing, ILM, and vector search.
Direct engineering answers on primary shard sizing, parent circuit breakers, Lucene HNSW quantization, and licensing reviews.
Still have a question?
Ask it on a scoping call. No production access needed.
Book a 30-min scoping call (opens Google Calendar in a new tab)What is the ideal shard size and shard-per-node count in Elasticsearch?
Why does Elasticsearch throw CircuitBreakingException: [parent] Data too large?
How do you design Hot-Warm-Cold-Frozen Index Lifecycle Management (ILM) tiers?
How should we size memory for Elasticsearch dense_vector (HNSW) search?
Can JusDB help us evaluate Elasticsearch 8.14+ AGPLv3/SSPL licensing vs Elastic Cloud vs OpenSearch?
How do you prevent mapping explosions on multi-tenant log clusters?
What are your response targets if we transition into ongoing Elasticsearch operations?
How does a JusDB Elasticsearch consulting engagement start?
Related Search & OLAP Practices
Explore adjacent search and analytics engineering.
Elasticsearch Database Reliability & Operations Hub
Explore JusDB's complete Elasticsearch reliability engineering, shard lifecycle, and support capabilities.
OpenSearch ComparisonOpenSearch Consulting & Architecture Services
Evaluating Apache 2.0 OpenSearch alongside Elasticsearch? Review our OpenSearch architecture practice.
OLAP AlternativeClickHouse Consulting for Log Analytics
Comparing Elasticsearch against ClickHouse for high-volume observability and log retention? See our ClickHouse practice.
Start With an Elasticsearch Diagnostic
Right-size your Elasticsearch shards and stabilize JVM heap pressure.
Book a 30-minute architecture call with a senior search engineer or request a read-only shard, ILM, and JVM heap audit.
Related Elasticsearch Services
Explore more ways our Elasticsearch experts can help with your database infrastructure.







