Free audit

View Audit Scope
ScyllaDB ConsultingSeastar Shard-per-Core · 6.x Tablets · Hot Partition & Compaction Review

ScyllaDB Consulting & Architecture Review

JusDB provides ScyllaDB consulting, architecture reviews, hot partition remediation, and compaction strategy optimization for high-throughput, low-latency wide-column NoSQL estates. Our database reliability engineers calibrate the C++ Seastar shard-per-core engine (cpuset pinning and NVMe iotune), design ScyllaDB 6.x Tablet-based keyspaces, eliminate large-partition shard skew, configure Workload Prioritization service levels, and lead migrations from Apache Cassandra and Amazon DynamoDB.

The first 30-minute call covers your databases, current ownership, and required coverage. No production access is needed. Or request a free audit of one database instance.

  • NDA before access
  • Read-only by default
  • Written scope before work starts

ScyllaDB Seastar & schema control plane

Select a ScyllaDB subsystem

Seastar C++ Shard-per-Core & iotune

cpuset + Direct I/O

ScyllaDB's Seastar framework assigns one pinned C++ reactor thread, memory pool, and NIC/NVMe queue per physical vCPU core, bypassing the Linux OS page cache via XFS Direct I/O.

Requires running scylla_io_setup (iotune) against local NVMe RAID-0 arrays so io_properties.yaml matches exact hardware IOPS and bandwidth ceilings.

Inspect CQL schema & Seastar audit specimen

Verified against ScyllaDB 6.x Seastar engine, Raft Tablets, and NVMe iotune specifications.

ScyllaDB subsystems covered
Seastar cpuset & iotune, system.large_partitions, TWCS/ICS/STCS compaction, ScyllaDB 6.x Tablets, and Workload Prioritization
Initial-response targets
15 minutes for a critical (P1) incident under a 24x7 support agreement; 2 to 4 hours in business hours. See details
Coverage hours
Business hours (03:30–12:30 UTC) by default; 24x7 on-call available under a 24x7 support agreement

Selected clients

  • Innoviti
  • Blitz
  • Freecharge
  • Nango
  • Recochain
  • Screen Facts
  • Nexora
  • WeCredit

Production failure modes

Common ScyllaDB shard-skew and compaction bottlenecks we resolve

ScyllaDB's shard-per-core C++ engine delivers massive throughput when partition keys and NVMe queues are balanced, but a single hot partition, tombstone storm, or mismatched compaction strategy will stall an individual vCPU reactor.

ScyllaDB production symptoms, root causes, and first consulting actions
ScyllaDB / Seastar signalWhat is usually happeningWhere we start
Shard 7 at 99% CPUOne Seastar reactor core saturates while node average CPU is 22%High-velocity queries target a single tenant_id or device_id partition key, routing every request to the exact same Murmur3 token and Seastar vCPU shard.Run nodetool toppartitions and query system.large_partitions, add a time or hash bucket column to the composite partition key ((tenant_id, bucket)), and verify shard-aware client port 19042.
system.large_partitions > 100MBReactor stalls (> 50ms) and cache eviction spikes during partition readsUnbounded clustering rows accumulate inside a single partition over months, forcing Seastar's LSA (Log-Structured Allocator) to deserialize massive index structures in one reactor slice.Audit system.large_partitions, system.large_rows, and system.large_cells; introduce time-bucketed partition keys and bound clustering range scans with paging.
Compaction Backlog > 80Read p99 degrades as SSTable count per read climbs above 15Heavy overwrites under SizeTieredCompactionStrategy (STCS) or uncalibrated io_properties.yaml starve compaction I/O shares, forcing reads to merge dozens of SSTables on the fly.Re-run scylla_io_setup (iotune) to verify NVMe IOPS/bandwidth, evaluate TWCS for time-series or ICS for large datasets, and inspect Prometheus scylla_compaction_manager_pending_tasks.
Tombstone Overread TimeoutRange queries fail with ReadTimeoutException despite small result setsUsing ScyllaDB tables like a message queue (inserting and deleting rows or writing null values into wide partitions) forces scans to skip past 100,000+ tombstones before finding live rows.Eliminate explicit null binds in prepared statements, replace row-level deletes with TimeWindowCompactionStrategy (TWCS) + default_time_to_live, and tune gc_grace_seconds with Scylla Manager repair.
Coordinator Hop PenaltyApplication p99 is 4x higher than ScyllaDB storage-proxy internal latencyApplications use a legacy non-shard-aware Cassandra driver or connect through a generic round-robin load balancer, forcing every query to hop across internal node/CPU shards.Deploy Scylla shard-aware drivers (Java, Go, Rust, Python, Node.js) connecting directly to port 9042 and shard-aware port 19042 with TokenAwarePolicy.
Batch / Analytics StarvationSpark analytics or CQL full-table scans spike real-time OLTP p99 latencyBackground Spark jobs, CDC consumers, or ad-hoc ALLOW FILTERING queries share the same Seastar I/O and CPU scheduling group as user-facing checkout traffic.Configure Workload Prioritization (CREATE SERVICE LEVEL with shares=100 vs shares=1000, timeout, and workload_type='batch') and attach batch roles to the low-share level.

Consulting scope

What does our ScyllaDB consulting service include?

We work directly on your CQL schemas, Seastar hardware profiles, compaction strategies, and shard-aware application drivers with reproducible telemetry evidence.

  • Hot partition & CQL data model review

    In ScyllaDB's shard-per-core architecture, data modeling directly dictates CPU balance. We redesign skewed partition keys and clustering schemas for uniform core utilization.

    • Audit system.large_partitions, system.large_rows, and nodetool toppartitions telemetry
    • Design composite bucketed partition keys, Secondary Indexes vs Materialized Views, and TTLs

    system.large_partitions · nodetool toppartitions · Murmur3 bucketing · CQL

    ScyllaDB performance tuning
  • Compaction strategy review (STCS, TWCS, ICS, LCS)

    Choosing the wrong compaction strategy doubles disk footprint and spikes read p99. We align each table's compaction strategy with its exact read/write/TTL profile.

    • Transition time-series tables to TWCS with aligned compaction_window_size and TTLs
    • Evaluate Incremental Compaction Strategy (ICS) and LCS to reclaim disk space and bound read amplification

    TWCS · ICS · STCS ·LCS · gc_grace_seconds · tombstone compaction

  • Seastar shard-per-core, cpuset & iotune calibration

    ScyllaDB bypasses the OS kernel cache and drives NVMe queues directly. We verify that your hardware, interrupts, and Seastar flags match your instance class.

    • Calibrate scylla_cpuset_setup, scylla_io_setup (iotune), and io_properties.yaml on NVMe RAID-0
    • Isolate IRQ interrupts from Seastar reactor cores and tune LSA cache vs memtable RAM

    scylla_io_setup · io_properties.yaml · cpuset.conf · XFS Direct I/O

  • ScyllaDB 6.x Tablets & Workload Prioritization

    Adopt ScyllaDB 6.x Raft-based Tablets for rapid elastic scaling and isolate real-time OLTP SLAs from background analytics using service levels.

    • Assess keyspace readiness for ScyllaDB 6.x Tablets (initial_tablets sizing & LWT/Counter checks)
    • Configure CREATE SERVICE LEVEL shares and timeouts to fence Spark/batch workloads

    ScyllaDB 6.x Tablets · Raft topology · CREATE SERVICE LEVEL · ATTACH SERVICE LEVEL

  • Multi-DC topology, consistency & shard-aware drivers

    We optimize replication factors, quorum consistency levels, and client driver routing so queries hit the exact vCPU shard owning the data in one network hop.

    • Architect NetworkTopologyStrategy across multi-AZ racks with LOCAL_QUORUM reads/writes
    • Tune shard-aware drivers on port 19042, speculative execution policies, and connection pools

    NetworkTopologyStrategy · LOCAL_QUORUM · Port 19042 · TokenAwarePolicy

    ScyllaDB on Kubernetes
  • Cassandra & DynamoDB (Alternator) migration advisory

    Replacing a 50-node JVM Cassandra ring or cutting DynamoDB costs via ScyllaDB Alternator? We model hardware sizing and dual-write / SSTableloader cutovers.

    • Size 5x-to-10x node consolidation from Apache Cassandra onto NVMe ScyllaDB instances
    • Architect DynamoDB-to-Alternator API migrations with Scylla Migrator (Spark) validation

    Scylla Migrator · Alternator API · sstableloader · dual-write verification

    ScyllaDB migration services

Shared responsibility & response targets

Where generic cloud or vendor guidance stops, and what JusDB delivers

Adding more nodes to a ScyllaDB cluster will never fix a single-partition hotspot or an STCS tombstone storm; we fix the root CQL schema and Seastar I/O configuration.

Comparison between generic ScyllaDB defaults and JusDB consulting deliverables
ScyllaDB domainWhere generic guidance stopsWhat JusDB engineers deliver
Hot partition & shard-skew resolutionAdvises scaling up the entire cluster (adding expensive NVMe nodes) when only a single vCPU shard is pinned at 99%.Identifies the exact offending partition key in system.large_partitions and refactors the CQL key with deterministic bucketing.
Compaction strategy & disk sizingLeaves all tables on default SizeTieredCompactionStrategy (STCS), wasting 50% disk capacity and accumulating tombstones.Audits every table's mutation/TTL pattern, migrating time-series data to TWCS and high-density tables to ICS or LCS.
NVMe storage & Seastar iotune calibrationDeploys ScyllaDB on uncalibrated EBS volumes or generic cloud disks without running scylla_io_setup.Sizes local NVMe RAID-0 (AWS i3en/i4i, GCP Z3/N2D, Azure Lsv3) with XFS Direct I/O and calibrated io_properties.yaml.
Client driver shard-awareness (Port 19042)Stops at the server boundary while applications connect via legacy Cassandra drivers that incur 95%+ coordinator hops.Upgrades and tunes shard-aware drivers connecting to port 19042 so requests route directly to the owning vCPU core.
OLTP vs analytics isolationRecommends spinning up a separate analytics datacenter ring just to run daily Spark jobs or backfills.Configures ScyllaDB Workload Prioritization service levels (shares + timeouts) to run mixed workloads safely on one ring.
Vendor-neutral deployment modelPushes proprietary ScyllaDB Cloud managed lock-in even when self-managed EC2/GKE or scylla-operator costs 50% less.Supports ScyllaDB Open Source, ScyllaDB Enterprise, ScyllaDB Cloud, and Kubernetes scylla-operator with zero bias.

Published initial-response targets by severity

Standard business hours: 03:30–12:30 UTC

Initial-response targets are 2 to 4 hours during covered business hours, Mon–Fri 09:00–18:00 IST (UTC+5:30). Where your service agreement includes 24x7 on-call, the initial-response target for P1 incidents is 15 minutes. Targets are not resolution guarantees.

SeverityInitial responseDefinitionChannel & coverage
P1 · Critical15 minutesProduction database down, primary unreachable, data corruption risk, or active failover event.24x7 on-call (under your service agreement) · Direct PagerDuty, Opsgenie, or phone page plus incident bridge
P2 · High30 minutesSevere latency spike, replication lag climbing rapidly, connection exhaustion, or degraded HA node.24x7 on-call (under your service agreement) · On-call alert plus shared Slack or Teams channel
P3 · Medium60 minutesNon-critical slow query regression, staging issue, backup warning, or replication check alert.24x7 on-call (under your service agreement) · Shared Slack or Teams channel or support ticket
P4 · Low90 minutesConfiguration question, index advisory, planned schema review, or general operational request.24x7 on-call (under your service agreement) · Shared Slack or Teams channel or support ticket

Our engineering team is in India; business-hours coverage is Mon–Fri 09:00–18:00 IST (UTC+5:30), and 24x7 on-call is available under your service agreement.

Initial-response targets, not resolution guarantees. The target for your engagement is set in your service agreement.

Interactive shard & compaction simulator

How partition bucketing and TWCS eliminate Seastar shard stalls

Step through a single-shard hot partition and STCS compaction backlog, TWCS and service-level remediation, and a fully balanced ScyllaDB 6.x Tablet topology.

ScyllaDB Seastar shard-per-core & compaction simulator

Select a state to inspect per-core CPU balance, compaction queues, and client routing hops.

Shard-Aware Driver (Port 19042), Murmur3 Token Ring & NVMe Queues

Unbucketed partition key (sensor_id) routes 40% of writes to Shard 3 on Node 1; STCS compaction falls behind and reads touch 24 SSTables.

Shard 3 at 99% CPU · 420 MB partition · Read p99 68ms
  • scylla-node-1 · Shard 3 (Pinned Core)

    Owns hot partition sensor_id='fleet-global'

    SATURATED · 99% CPU · 88ms reactor stalls

  • scylla-node-1 · Shards 0..2, 4..15

    15 sibling Seastar cores on same i4i.4xlarge

    IDLE · 18% average CPU utilization

  • NVMe RAID-0 Storage (STCS default)

    85 pending compactions · 420 MB large partition

    BLOATED · High read & write amplification

Because ScyllaDB shards data by CPU core, a single large or hot partition cannot use the other 15 cores on the machine. Adding nodes to the cluster does not fix a single-partition bottleneck.

Representative consulting deliverable

How a ScyllaDB hot-partition and compaction audit is delivered

Every engagement includes the system.large_partitions evidence, the refactored CQL schema and compaction DDL, and the Workload Prioritization service-level rules.

Sample finding · 480 MB hot partition on Shard 3 & STCS write amplificationScyllaDB 6.1 · AWS i4i.4xlarge (3-AZ)

Finding

Unbucketed telemetry partition key and STCS compaction push LOCAL_QUORUM p99 to 74ms

Querying system.large_partitions exposed 19 partitions exceeding 350 MiB (2.1M rows each) pinned to Seastar Shard 3. Meanwhile, default STCS merged 90-day TTL SSTables with fresh writes. Bucketing by day + shard salt and switching to TWCS cut p99 to 1.1ms.

LOCAL_QUORUM read p9974.2ms to 1.1ms
NVMe write amplification6.8x to 1.4x

1. CQL diagnostic query & refactored Tablet + TWCS schema with partition bucketing

cqlsh · large partition audit & bucketed TWCS table

-- 1. Identify partitions > 100 MB causing Seastar reactor stalls
SELECT keyspace_name, table_name, partition_key, partition_size, rows
FROM system.large_partitions
WHERE partition_size > 104857600 ALLOW FILTERING;

-- 2. Enable ScyllaDB 6.x Raft Tablets on the keyspace across 3 AZ racks
CREATE KEYSPACE IF NOT EXISTS telemetry_prod
WITH replication = {
  'class': 'NetworkTopologyStrategy',
  'us-east-1': 3
} AND tablets = {'enabled': true, 'initial': 128};

-- 3. Composite (tenant_id, day_bucket, shard_salt) key + TWCS 1-day window
CREATE TABLE telemetry_prod.device_events (
  tenant_id uuid,
  day_bucket int,      -- e.g. 20261109
  shard_salt tinyint,  -- 0..15 deterministic hash mod
  event_ts timestamp,
  device_id text,
  payload blob,
  PRIMARY KEY ((tenant_id, day_bucket, shard_salt), event_ts, device_id)
) WITH CLUSTERING ORDER BY (event_ts DESC, device_id ASC)
  AND compaction = {
    'class': 'TimeWindowCompactionStrategy',
    'compaction_window_unit': 'DAYS',
    'compaction_window_size': 1
  }
  AND default_time_to_live = 7776000
  AND gc_grace_seconds = 86400;

2. Workload Prioritization service levels (OLTP vs Spark analytics)

cqlsh · Seastar service level isolation

CREATE SERVICE LEVEL sl_realtime_oltp WITH shares = 1000 AND timeout = 500ms AND workload_type = 'interactive';
CREATE SERVICE LEVEL sl_spark_batch   WITH shares = 100  AND timeout = 10s   AND workload_type = 'batch';

ATTACH SERVICE LEVEL sl_realtime_oltp TO app_checkout_role;
ATTACH SERVICE LEVEL sl_spark_batch   TO spark_etl_role;

Ways to work together

How to engage JusDB for ScyllaDB consulting

Start with a free read-only audit of your ScyllaDB schema and Seastar metrics, scope a focused hot-partition or migration project, or put your ring under ongoing DBRE coverage.

Scoped consulting projectThis page

Fix shard skew, tune compaction, or migrate to ScyllaDB.

A fixed-scope engagement delivering CQL schema refactors, Seastar hardware tuning, ScyllaDB 6.x Tablet upgrades, or Cassandra/DynamoDB migrations.

Common ScyllaDB scopes agreed in writing first

  • Hot partition & CQL refactoring
  • TWCS / ICS compaction tuning
  • Seastar cpuset & NVMe iotune
  • ScyllaDB 6.x Tablets & SLAs
  • Cassandra / DynamoDB migration
Free ScyllaDB audit

One ScyllaDB keyspace and ring, audited read-only.

We inspect your system.large_partitions table, per-shard CPU balance, compaction backlog, and NVMe io_properties.yaml, then deliver a written report within one week.

What we check on your ScyllaDB ring

  • Hot partitions, large rows/cells & tombstone ratios
  • Compaction strategy (STCS/TWCS/ICS) vs disk headroom
  • Seastar cpuset, NVMe iotune & port 19042 routing
Ongoing ScyllaDB DBRE & support

Named engineers covering your ScyllaDB estate.

Proactive Scylla Monitoring Stack alerting, Scylla Manager repair and S3 backup scheduling, and incident response under a 24x7 support agreement.

  1. Page on-callavailable under a 24x7 support agreement
  2. 15 minP1 initial-response target
  3. Written RCAafter every P1

ScyllaDB platforms, cloud hardware, and migration sources we support

For procurement and security teams

Mutual NDA first, read-only CQL grants, and written scope

We work inside your security boundary via bastion hosts, VPN, or temporary cloud IAM roles. Your team approves every schema migration or scylla.yaml change and keeps every runbook.Raguraman, Engineering Manager · Allen D, Lead Database SRE
  • Mutual NDA
  • Written statement of work
  • Vendor-neutral advisory
  • Read-only CQL & Prometheus
  • NDA before access

    Signed before any diagnostic review. The 30-minute scoping call requires no credentials.

  • Written statement of work

    Covered clusters, deliverables, maintenance windows, and fees agreed in writing first.

  • Read-only by default

    Initial audits require only read-only access to system_* tables, nodetool status, and Prometheus metrics.

  • Deliverables you keep

    Every CQL migration script, scylla.yaml diff, and service-level configuration stays with your team. Meet the JusDB team.

JusDB's security practices are aligned with ISO 27001 and SOC 2 Type II, and JusDB is pursuing both certifications (see Terms, section 12). Read the stated status.

Technical FAQ

ScyllaDB consulting common questions

Answers on hot partition bucketing, STCS vs TWCS vs ICS compaction, ScyllaDB 6.x Tablets, shard-aware port 19042, and Workload Prioritization.

Still have a question?

Ask it on a scoping call. No production access needed.

Book a 30-min scoping call (opens Google Calendar in a new tab)

What does a JusDB ScyllaDB consulting engagement include?

Our ScyllaDB consulting engagements cover comprehensive architecture and CQL schema reviews, hot partition and large partition remediation (system.large_partitions), compaction strategy optimization (STCS, TWCS, ICS, LCS), Seastar shard-per-core CPU pinning and NVMe iotune calibration, ScyllaDB 6.x Tablet migration readiness, Workload Prioritization service levels, Scylla Manager repair scheduling, and migrations from Apache Cassandra or Amazon DynamoDB.

How do you diagnose and fix hot partitions and large partitions in ScyllaDB?

Because ScyllaDB hashes each partition key via Murmur3 to a single vCPU shard on each replica node, a partition exceeding 100 MB or receiving disproportionate traffic pins one CPU core at 99% and triggers Seastar reactor stalls. We query system.large_partitions, system.large_rows, and system.large_cells alongside live nodetool toppartitions sampling, then redesign the CQL PRIMARY KEY with a deterministic time bucket or hash salt (for example PRIMARY KEY ((tenant_id, yyyymmdd, bucket_id), event_ts)) so reads and writes fan out across all CPU cores.

Which ScyllaDB compaction strategy should we use: STCS, TWCS, LCS, or ICS?

Each strategy matches a specific workload: (1) TimeWindowCompactionStrategy (TWCS) is mandatory for append-only time-series data with uniform TTLs, as it drops expired SSTable time windows intact without rewrite amplification; (2) LeveledCompactionStrategy (LCS) suits read-heavy workloads with frequent in-place row updates; (3) Incremental Compaction Strategy (ICS, available in ScyllaDB Enterprise/Cloud) replaces STCS by splitting SSTables into uniform 1 GB fragments, cutting temporary disk space overhead from 50% down to ~15%; and (4) STCS fits write-intensive workloads with ample NVMe headroom.

What are ScyllaDB 6.x Tablets and when should we enable them?

Introduced in ScyllaDB 6.0, Tablets replace static vnode token ranges with fine-grained, Raft-managed storage units that split, merge, and migrate dynamically per CPU shard. Unlike vnode streaming - which requires hours of cluster-wide cleanup when adding a node - Tablets rebalance elastically in minutes. Before enabling Tablets on a keyspace (WITH tablets = {'enabled': true}), we verify that the keyspace does not rely on vnode-only features such as Counter tables or legacy Materialized Views/LWT constraints.

Why is shard-aware port 19042 important for ScyllaDB client drivers?

Standard Cassandra drivers are token-aware (they know which physical node owns a partition) but not shard-aware (they do not know which of the 32 or 64 vCPU cores on that node owns the token). Connecting to standard port 9042 means 97% of requests on a 32-core node land on the wrong CPU core and require an internal inter-core relay. ScyllaDB's shard-aware drivers connect to port 19042 (or use source-port hashing) so every client thread maintains a direct socket to each Seastar vCPU shard.

How does ScyllaDB Workload Prioritization prevent batch jobs from hurting OLTP latency?

Using CQL service levels (CREATE SERVICE LEVEL batch_sl WITH shares = 100 AND workload_type = 'batch'), ScyllaDB's Seastar CPU and I/O schedulers allocate disk bandwidth and reactor polling slices proportionally across competing roles. Even if a Spark connector or full-table scan floods the cluster with concurrent requests, real-time OLTP roles assigned to a 1000-share service level preempt batch I/O queues.

Which cloud instance families do you recommend for self-managed ScyllaDB?

ScyllaDB's Seastar engine is built around local NVMe SSDs formatted with XFS in RAID-0 and calibrated via scylla_io_setup. On AWS, we recommend i4i, i3en, or Graviton3 im4gn/is4gen instances; on Google Cloud, Z3 or N2D/C3 with Local SSDs; and on Azure, Lsv3 / Lasv3 series VMs.

What are your response targets for ScyllaDB consulting and production incidents?

Initial-response targets are 2 to 4 hours during covered business hours, Mon–Fri 09:00–18:00 IST (UTC+5:30). Where your service agreement includes 24x7 on-call, the initial-response target for P1 incidents is 15 minutes. Targets are not resolution guarantees.

Next step

Talk to an engineer about your ScyllaDB schema and Seastar ring

Tell us your ScyllaDB version, instance family, and whether you are resolving hot partitions, tuning compaction, or migrating from Cassandra/DynamoDB. The first 30-minute call covers your databases, current ownership, and required coverage. No production access is needed. Or request a free audit of one database instance, subject to availability: mutual NDA before read-only access, a prioritized report within one week after access is ready, and no obligation.

NDA before access · read-only by default · written scope first

Related ScyllaDB Services

Explore more ways our ScyllaDB experts can help with your database infrastructure.