Free audit

View Audit Scope
JusDBOur engineering team is in India; business-hours coverage is Mon–Fri 09:00–18:00 IST (UTC+5:30), and 24x7 on-call is available under your service agreement. · P1 15 minutes · P2 30 minutes

Elasticsearch consulting for 20-50 GB shards, ILM tiering, and low-latency search.

JusDB provides Elasticsearch consulting services and production search operations for full-text search, observability log stacks, and dense-vector RAG workloads on Elastic Cloud, Kubernetes (ECK), and self-managed clusters. Search engineers right-size shards to 20-50 GB, design Hot-Warm-Cold-Frozen Index Lifecycle Management (ILM) policies, tune G1GC JVM heaps, and evaluate AGPLv3/SSPL/Elastic licensing under your service agreement.

ELASTICSEARCH CLUSTER DIAGNOSTIC

es-master-01

Active

Cluster state updates: <15ms · Total shards: 1,180 (24 per data node) · Pending tasks: 0 · Snapshot repository s3://es-prod-snapshots verified.

Targets: 20-50 GB primary shards · <=31 GB JVM heap · Hot-Warm-Cold-Frozen ILM

Selected clients

  • Innoviti
  • Blitz
  • Freecharge
  • Nango
  • Recochain
  • Screen Facts
  • Nexora
  • WeCredit

Production Search Bottlenecks

Six failure modes that stall Elasticsearch clusters.

Most Elasticsearch incidents trace back to oversharding from daily index patterns, JVM heap saturation from unbounded aggregations, or missing Index Lifecycle Management (ILM) policies. Here is how our search engineers diagnose and fix them.

Failure ModeWhat You See in ProductionJusDB Engineering Action
Oversharding & Small ShardsThousands of sub-gigabyte shards from daily index patterns exhaust JVM heap, slow down master cluster-state publication, and increase query coordinator fan-out latency.Consolidate indices into Data Streams with ILM rollover at max_primary_shard_size: 40gb, use the _shrink API on warm indices, and keep shard counts below 20 shards per GB of configured JVM heap.
JVM Heap & Circuit Breaker TripsData nodes hit 85-95% JVM heap utilization, trigger multi-second G1GC stop-the-world pauses, and reject requests with CircuitBreakingException: [parent] Data too large.Cap JVM heap at 50% of host RAM (never exceeding 31 GB to preserve compressed OOPs), eliminate high-cardinality text fielddata, convert wide nested aggregations to composite paginations, and tune G1GC region sizing.
Hot-Warm-Cold-Frozen Tiering Cost30 to 365 days of observability logs or audit events sit on hot NVMe instances because ILM phase transitions or snapshot repositories are misconfigured.Architect automated Hot-Warm-Cold-Frozen ILM policies: ingest on NVMe hot nodes, shrink and forcemerge on dense warm storage, and mount S3-backed searchable snapshots for cold and frozen retention.
Mapping Explosion & Refresh OverheadUnbounded JSON payloads create 5,000+ dynamic fields per index, stalling master nodes on mapping updates while default 1-second refresh intervals saturate disk IOPS.Enforce dynamic: strict or runtime fields for sparse attributes, use flattened or match_only_text types for log payloads, and tune refresh_interval to 30s for bulk ingestion.
Dense Vector (kNN / HNSW) Memory SizingAdding dense_vector HNSW indices for semantic search or RAG exhausts OS page cache, causing disk thrashing and 500ms+ vector query latencies.Right-size off-heap OS page cache for Lucene HNSW graphs, enable int8_hnsw or int4_hnsw scalar quantization to cut vector memory footprint by 75-87%, and tune m, ef_construction, and hybrid BM25 + RRF pipelines.
Licensing & Platform ArchitectureEngineering teams need an objective evaluation of Elasticsearch 8.14+ licensing (AGPLv3/SSPL/Elastic License v2), Elastic Cloud vs self-managed ECK, or OpenSearch compatibility trade-offs.Deliver a vendor-neutral licensing, TCO, and feature-compatibility matrix comparing Elastic Cloud, self-managed Elasticsearch on Kubernetes (ECK), and Apache 2.0 OpenSearch alternatives.

Engagement Cadence & Response Targets

Structured architecture sprints and 15-minute P1 escalation.

Whether you engage JusDB for a focused 2-week architecture audit or transition into ongoing 24x7 search reliability operations under a 24x7 support agreement, every severity tier is backed by named search engineers.

P115 minutes

Production database down, primary unreachable, data corruption risk, or active failover event.

P230 minutes

Severe latency spike, replication lag climbing rapidly, connection exhaustion, or degraded HA node.

P360 minutes

Non-critical slow query regression, staging issue, backup warning, or replication check alert.

P490 minutes

Configuration question, index advisory, planned schema review, or general operational request.

Initial-response targets, not resolution guarantees. The target for your engagement is set in your service agreement. Business-hours consulting coverage runs 03:30–12:30 UTC; 24x7 incident response is available where your service agreement includes it.

Step 01

Read-Only Cluster Telemetry

We collect _cluster/settings, _cat/shards, _ilm/policy, _nodes/stats/jvm,breaker, and slow-log traces without impacting production traffic.

Step 02

Shard & Heap Bottleneck Model

We model shard-to-heap ratios, mapping field growth, segment merge debt, and hot-warm-cold storage economics across your retention window.

Step 03

Templates, ILM & Query Refactors

You receive production-ready index templates, Data Stream rollover policies, JVM options, and rewritten search/aggregation queries.

Step 04

Guided Rollout & Reindex Cutover

Our engineers pair with your team during rolling node restarts, ILM transitions, and atomic alias cutovers.

Consulting Scope

What is included in an Elasticsearch consulting engagement.

Every engagement produces concrete configurations, including index templates, ILM lifecycle JSON, JVM flags, and reindex scripts tailored to your document volume and query SLOs.

Shard Sizing & Cluster Topology

Right-size primary and replica shards to the 20-50 GB sweet spot, isolate dedicated master, ingest, coordinating, and tiered data roles, and enforce zone-aware shard allocation across availability zones.

Hot-Warm-Cold-Frozen ILM Design

Design Data Streams and Index Lifecycle Management (ILM) policies with byte-size rollover, warm _shrink and forcemerge, and cold/frozen searchable snapshots on object storage.

JVM Heap, G1GC & Query Tuning

Eliminate parent circuit breaker exceptions, profile slow queries and aggregations via the Search Profiler API, optimize BM25 mappings, and tune bulk write thread pools and refresh intervals.

Vector Search & Hybrid RAG Architecture

Architect Lucene dense_vector HNSW indices, configure int8_hnsw scalar quantization, size off-heap memory for vector graphs, and tune hybrid lexical + semantic search relevance.

Upgrades, ECK & Licensing Strategy

Plan rolling upgrades from Elasticsearch 6.x/7.x to 8.x, review Elastic Cloud vs self-managed ECK on Kubernetes, or assess cross-engine migration paths with documented rollback runbooks.

24x7 Production Search Operations

Transition from consulting into ongoing reliability operations with a 15 minutes P1 response target under a 24x7 support agreement, snapshot restore drills, and Prometheus/OpenTelemetry alerting.

Inside an Elasticsearch Architecture Review

From 14,200 small shards to a deterministic ILM lifecycle.

Step through a real remediation sequence on a 15-node Elasticsearch 8.15 cluster: diagnosing oversharding and parent circuit breaker trips, rolling out a 40 GB Data Stream ILM policy, and verifying the right-sized production state.

ELASTICSEARCH SHARD & ILM REMEDIATION SIMULATOR

Application and routing

Daily index patterns with 5 primary shards and 1 replica across dozens of microservices created 14,200 shards averaging 180 MB each. Cluster state updates blocked master nodes and parent circuit breakers tripped during peak aggregations.

14,200 small shards · 89% JVM Heap

    Daily index patterns with 5 primary shards and 1 replica across dozens of microservices created 14,200 shards averaging 180 MB each. Cluster state updates blocked master nodes and parent circuit breakers tripped during peak aggregations.

    Audit Deliverable Specimen: Index Template & Strict Mapping Guard

    # JusDB Elasticsearch Architecture Audit: Finding Specimen
    # Cluster: es-prod-analytics (Elasticsearch 8.15, 15 data nodes, 64 GB RAM / 31 GB Heap)
    
    ## 1. Verify Shard Count per Node & Average Shard Size
    GET _cat/allocation?v&s=shards:desc
    GET _cat/shards?v&h=index,shard,prirep,state,store,docs,node&s=store:desc
    
    ## 2. Inspect JVM Heap Pressure, OldGC Pauses & Circuit Breakers
    GET _nodes/stats/jvm,breaker,thread_pool?filter_path=nodes.*.name,nodes.*.jvm.mem.heap_used_percent,nodes.*.jvm.gc.collectors,nodes.*.breakers.parent,nodes.*.thread_pool.write.rejected
    
    ## 3. Remediation: Index Template with Strict Mapping & 40GB ILM Rollover
    PUT _index_template/logs-app-template
    {
      "index_patterns": ["logs-app-*"],
      "data_stream": {},
      "template": {
        "settings": {
          "number_of_shards": 3,
          "number_of_replicas": 1,
          "index.lifecycle.name": "jusdb-hot-warm-cold-policy",
          "index.refresh_interval": "30s",
          "index.mapping.total_fields.limit": 1000,
          "index.codec": "best_compression"
        },
        "mappings": {
          "dynamic": "strict",
          "properties": {
            "@timestamp": { "type": "date" },
            "service":    { "type": "keyword" },
            "level":      { "type": "keyword" },
            "trace_id":   { "type": "keyword", "doc_values": false },
            "message":    { "type": "match_only_text" }
          }
        }
      }
    }

    Zero-Friction Collaboration

    Clear ownership between your team and JusDB.

    Managed cloud databases automate much of the infrastructure layer, but your team still owns query performance, capacity, cost, and incident decisions.

    Customer Retains

    Cloud Accounts, Keys & Search Product Logic

    • Set business priorities, application requirements, and release plans
    • Approve production changes, except routine actions you have pre-approved
    • Own your cloud accounts, access policies, and vendor relationships
    • Decide when maintenance can happen and how quickly service must recover
    • Move the work in-house or to another provider after your agreement's notice period, with your data returned or deleted and the transition help your agreement sets out (Terms, section 8)
    JusDB Executes

    Shard Architecture, ILM, JVM Tuning & 24x7 Escalation

    • Monitor database health, investigate alerts, and involve the right people
    • Review query performance, replication, capacity, and backup health
    • Carry out approved changes and update the operating instructions they affect
    • Report completed work, open risks, and recommended next steps
    Evaluation Vector
    JusDB DBRE
    In-House DBALegacy AgencyDeveloper Generalist
    Primary Shard Sizing & Capacity Modeling (30–50GB)Enforces the strict 30–50GB primary shard standard, eliminates small-shard Lucene segment metadata bloat, and models memory-to-disk ratios to maximize query cache residency.Defaults to arbitrary shard counts per index (e.g., 5 shards per index), spawning thousands of 500MB shards that exhaust master node heap and degrade search scatter-gather.Recommends vertical node scaling and adding costly RAM without analyzing Lucene segment counts or consolidating oversharded indices.Leaves index templates at system defaults without capacity modeling; triggers cluster state red alerts during high-concurrency aggregation queries.
    Index Lifecycle Management (ILM Hot/Warm/Cold/Frozen)Architects automated multi-tier ILM policies transitioning time-series data across hot local NVMe, warm SSD, cold HDD, and frozen searchable snapshots in S3/GCS.Operates brittle external cron jobs or legacy Curator scripts that fail silently, leaving multi-terabyte indices stranded indefinitely on expensive hot nodes.Deploys homogeneous single-tier storage clusters and performs emergency manual index deletions only after 85% disk watermark alerts trigger.No lifecycle tiering; allows log and telemetry indices to accumulate on primary data disks until filesystem 100% full crashes take nodes offline.
    Lucene Mapping Explosion Prevention & Dynamic Field GovernanceEnforces strict mapping schemas, dynamic field governance rules, runtime fields, and caps index.mapping.total_fields.limit to prevent cluster-state serialization stalls.Permits unconstrained dynamic mappings (dynamic: true); ingested payloads with thousands of random nested JSON keys bloat cluster state to hundreds of megabytes.Arbitrarily raises index.mapping.total_fields.limit to 10,000+ when ingestion fails, worsening master node heap saturation and slow cluster state publication.Ingests unstructured JSON payloads without schema design; triggers Lucene field type collisions, unindexed fields, and node restart serialization delays.
    JVM Heap Ergonomics & 31GB Compressed OOPs CapCaps JVM heap strictly at <31GB to guarantee Compressed Ordinary Object Pointers (Compressed OOPs) stay active, maximizing off-heap Lucene filesystem page cache and tuning G1GC pauses.Allocates 64GB+ heap to Elasticsearch nodes, disabling Compressed OOPs, losing 50% effective pointer density, and triggering catastrophic multi-minute GC pauses.Leaves default heap configurations or allocates 50% blindly without tuning circuit breaker thresholds, search threadpools, or off-heap filesystem cache ratios.Allocates 90% of host RAM to JVM heap, starving the underlying Linux OS kernel and Lucene page cache, leading to severe disk paging and node timeouts.
    Cross-Cluster Replication (CCR) & Disaster RecoveryDesigns active-passive and active-active Cross-Cluster Replication (CCR) with auto-following index patterns, sub-second RPO replication streams, and automated health heartbeats.Relies solely on nightly snapshot restores with 24-hour RPO, requiring multi-hour downtime to restore terabytes of data during an availability zone outage.Attempts DIY dual-writing from application microservices without transaction rollback coordination or distributed sequence tracking, causing silent data drift.Assumes single-cluster multi-AZ replication is disaster recovery; suffers total data loss or protracted downtime when regional infrastructure fails.
    Enterprise Security Hardening (TLS, RBAC, SAML)Hardens clusters with FIPS-compliant TLS node-to-node transport encryption, fine-grained Document and Field Level Security (DLS/FLS), SAML/OIDC SSO federation, and immutable audit trails.Uses default elastic superuser credentials shared across application services with basic HTTP authentication and unencrypted internal node transport.Places unauthenticated Elasticsearch behind an external Nginx proxy without native cluster transport TLS or internal role-based access controls.Disables security entirely (xpack.security.enabled: false) to bypass TLS certificate validation issues during local and staging development.

    Procurement and security

    Ready for legal and security review.

    Mutual NDA before any access, least-privilege credentials in your vault, and security controls aligned with ISO 27001 and SOC 2 Type II.
    • Mutual NDA before any access
    • Named individual accounts only
    • Read-only access by default
    • ISO 27001 & SOC 2 Type II: aligned, certification in progress
    • Mutual NDA before any access

      We sign a mutual NDA before viewing schemas, logs, slow-query captures, or cloud topology.

    • Least-privilege, named credentials

      Engineers connect through your bastion, VPN, or SSM using named accounts with read-only permissions by default.

    • Change control & approval gates

      Production DDL, parameter changes, and failovers follow your change-management workflow with rollback steps prepared first.

    • Named leadership & direct escalation

      Engagements are led by named senior database reliability engineers in India with direct Slack, Teams, and PagerDuty channels.

    Elasticsearch Consulting FAQ

    Common questions on shard sizing, ILM, and vector search.

    Direct engineering answers on primary shard sizing, parent circuit breakers, Lucene HNSW quantization, and licensing reviews.

    Still have a question?

    Ask it on a scoping call. No production access needed.

    Book a 30-min scoping call (opens Google Calendar in a new tab)

    What is the ideal shard size and shard-per-node count in Elasticsearch?

    For most search and log analytics workloads, primary and replica shards should be sized between 20 GB and 50 GB (with 30-40 GB as the typical rollover target for time-series Data Streams). A data node should hold fewer than 20 shards per GB of configured JVM heap, meaning a node with a 31 GB heap should host no more than 600 shards. Keeping shards in this range prevents master cluster-state bottlenecks and maximizes Lucene segment merge efficiency.

    Why does Elasticsearch throw CircuitBreakingException: [parent] Data too large?

    The parent circuit breaker trips when combined JVM heap usage across fielddata, in-flight bulk/search requests, and Lucene segment memory exceeds the configured limit (typically 95% of real memory or 70% without real-memory bounding). Common culprits include oversharding, unpaginated terms aggregations over high-cardinality fields, or enabling fielddata on text fields. JusDB resolves this by shrinking shard counts, converting deep aggregations to composite aggregations, and tuning heap and G1GC settings.

    How do you design Hot-Warm-Cold-Frozen Index Lifecycle Management (ILM) tiers?

    We configure Data Streams and ILM policies that ingest active indices on hot NVMe nodes, roll over when primary shards reach 40 GB, shrink shard counts and force-merge to 1 segment on warm nodes after 7 days, and mount S3/GCS/Azure Blob searchable snapshots on cold and frozen tiers for 30-365+ day retention. This typically reduces cluster storage and compute spend by 35-55%.

    How should we size memory for Elasticsearch dense_vector (HNSW) search?

    Lucene HNSW graphs reside off-heap in the operating system page cache rather than inside the JVM heap. For float32 vectors, memory required is roughly num_vectors * dimensions * 4 bytes * 1.25. On Elasticsearch 8.x, we strongly recommend int8_hnsw (or int4_hnsw) scalar quantization, which reduces off-heap vector memory requirements by 75% to 87% with minimal recall impact, and we ensure data nodes allocate at least 50% of system RAM to the OS page cache.

    Can JusDB help us evaluate Elasticsearch 8.14+ AGPLv3/SSPL licensing vs Elastic Cloud vs OpenSearch?

    Yes. Following Elastic's addition of AGPLv3 alongside SSPL and Elastic License v2 in Elasticsearch 8.14+, many organizations want an engineering-led review of licensing obligations, X-Pack feature availability (ILM, Security, searchable snapshots, vector quantization), and total cost of ownership across self-managed Elasticsearch, Elastic Cloud, and OpenSearch.

    How do you prevent mapping explosions on multi-tenant log clusters?

    When microservice teams emit unvalidated JSON logs, dynamic mapping can generate thousands of fields per index, bloating the cluster state and stalling the active master node. We enforce index templates with dynamic: strict or runtime fields, use the flattened field type for arbitrary JSON metadata maps, replace full text with match_only_text where positional scoring isn't needed, and cap index.mapping.total_fields.limit.

    What are your response targets if we transition into ongoing Elasticsearch operations?

    Every production support and managed DBRE contract includes severity-tiered response targets: P1 Critical (cluster red, unassigned primaries, circuit breaker outage) within 15 minutes, P2 High within 30 minutes, P3 Medium within 60 minutes, and P4 Low within 90 minutes under a 24x7 support agreement.

    How does a JusDB Elasticsearch consulting engagement start?

    Engagements begin with a read-only diagnostic audit of your cluster settings, _cat/shards geometry, ILM policies, JVM heap and GC telemetry, and slow-log profiles. Within 5 to 10 business days, you receive a prioritized findings report with copy-pasteable index templates, ILM JSON policies, and a phased remediation roadmap.

    Related Elasticsearch Services

    Explore more ways our Elasticsearch experts can help with your database infrastructure.