Upgrade · NVMe storage

Enterprise NVMe storage: a step change in performance, with infrastructure constraints.

Q: Can I add NVMe to my existing server?

It depends on the model. A server with a standard SAS/SATA backplane will not take NVMe unless the backplane is replaced with a tri-mode version. Servers that are already prepared for it (Dell PowerEdge R750 with tri-mode backplane, HPE ProLiant DL380 Gen11 with NVMe-ready bays, Lenovo SR650 V2) take the drives directly. Send us the exact model and we will come back to you on feasibility.

Q: Does NVMe behind a tri-mode controller reach the same performance as NVMe direct on PCIe?

No. NVMe direct on PCIe Gen4 x4 does 7 GB/s and 1M IOPS; NVMe behind a tri-mode controller with cache is limited by the controller (typically 1-2 GB/s, a few hundred k IOPS). But tri-mode gives you hardware RAID, protected cache and a single management console: for mid-market workloads that want easy protection it is an excellent trade-off.

Q: Does NVMe hot-swap really work?

Yes, on enterprise U.2 and U.3 (Dell, HPE, Lenovo): it is a native feature of the protocol. Not on consumer NVMe (M.2) — M.2 was never designed for hot-swap. The more recent enterprise U.3 SSDs support hot-add without taking the system down, with the operating system picking up the drive (automatic on Linux; on Windows a rescan is sometimes needed).

Q: PCIe Gen4 vs Gen5 — is the difference worth it?

For IOPS-bound workloads (databases, virtualisation) the difference is marginal: Gen4 already saturates most workloads. For raw throughput (analytics, ML training, file servers with large datasets) Gen5 does make a difference, doubling sequential throughput. The CPUs that support Gen5 are Xeon Sapphire Rapids and later, EPYC Genoa and later.

Q: Can I run software RAID (Linux mdadm, Windows Storage Spaces) on NVMe?

Yes — and it is in fact the modern pattern for getting the most out of NVMe: software-defined storage exposes NVMe IOPS far more efficiently than a hardware RAID controller. ZFS on NVMe is excellent for database workloads. Windows Storage Spaces Direct and VMware vSAN are designed for exactly this: NVMe + SDS.

Q: Endurance: how many DWPD do enterprise NVMe drives handle?

Enterprise NVMe SSDs are rated by workload: Read-Intensive (1 DWPD), Mixed-Use (3 DWPD), Write-Intensive (10 DWPD). For write-heavy OLTP databases, go Mixed-Use. Read-Intensive is fine for VM repositories and file servers. Write-Intensive only for log shippers / metadata-heavy work. Picking the wrong class can cause wear-out in 1-2 years instead of 5+.

Enterprise NVMe replaces SAS/SATA with a native PCIe protocol: ten to a hundred times more IOPS, latency in single-digit microseconds instead of milliseconds. But buying NVMe drives is not enough — you need a compatible backplane, free PCIe Gen4/5 slots, redundant power for high-wattage drives and hot-swap support on U.2/U.3.

Request a quote How we proceed →

Why NVMe

The performance jump vs SAS/SATA.

IOPS

Enterprise SAS 12G SSD: 100-200k IOPS
Enterprise SATA SSD: 80-100k IOPS
Enterprise NVMe Gen4: 1M-1.6M IOPS · 10× SAS

Latency

Enterprise SAS/SATA SSD: 100-300 µs
Enterprise NVMe: 10-30 µs · 10× lower
NVMe-oF over the network: 30-100 µs (fine for distributed scenarios)

Throughput

SAS 12G: 1.2 GB/s per drive
SATA 6G: 550 MB/s per drive
NVMe Gen4 x4: 7 GB/s per drive
NVMe Gen5 x4: 14 GB/s (coming)

Infrastructure constraints

What it actually takes.

1 · Tri-mode backplane

A "SAS/SATA only" backplane will not take NVMe. You need a tri-mode backplane (SAS/SATA/NVMe). PowerEdge R750 with a configurable NVMe back zone; HPE DL380 Gen11 NVMe-ready; Lenovo SR650 V2 with an NVMe-ready backplane. On older models (early R740 releases) the backplane has to be replaced.

2 · Available PCIe lanes

Every U.2/U.3 NVMe drive takes 4 PCIe lanes. A 2-socket Xeon Scalable Gen3 system typically has 64 CPU lanes plus chipset: each NVMe slot costs 4 lanes. We check the lane budget including any 25G/100G NICs, GPUs and tri-mode storage controllers.

3 · Power

Enterprise NVMe wattage: U.2 draws 7-15 W under load, with higher peaks. 24× U.2 means 240-360W for the drives alone. On systems whose PSUs are already at the limit, adding NVMe pushes past the budget: you have to move up to larger PSUs.

4 · Cooling

Enterprise NVMe drives generate non-trivial heat under load. Systems with standard airflow cope; high-density systems (dense 1U) need performance fans and updated thermal management. Cooking NVMe drives accelerates wear-out and triggers thermal throttling.

5 · Form factor (U.2, U.3, M.2, EDSFF)

U.2 (2.5") is the classic; U.3 is backward compatible and tri-mode; M.2 is boot/cache only (no hot-swap); EDSFF E1.S/E3 are emerging for data center density. The standard on enterprise servers is enterprise U.2/U.3 for storage and M.2 for boot.

6 · Drivers and OS

Modern operating systems support NVMe natively (Linux kernel 3.13+, Windows Server 2012 R2+). Older systems (Windows Server 2008, RHEL 6) need vendor drivers. VMware ESXi 6.5+ supports NVMe natively.

NVMe-ready server models

The systems we work on directly.

# Dell PowerEdge — generation by generation R740xd / R740xd2 NVMe bays in front (4-12 U.2 typical) R750 / R750xa Configurable tri-mode backplane R760 / R760xa Gen16 · PCIe Gen5 · EDSFF coming R6515 / R7515 EPYC entry · optional NVMe bays R6525 / R7525 EPYC dual · NVMe-ready # HPE ProLiant DL380 Gen10/11 NVMe-ready bays, tri-mode option DL360 Gen10/11 Dense 1U, limited NVMe options DL385 Gen10/11 EPYC · NVMe bays available DL580 Gen10 4-socket · many NVMe slots # Lenovo ThinkSystem SR650 V2 / V3 2U · configurable backplane SR650 V3 Gen16 · PCIe Gen5 SR635 V2 / SR655 V2 EPYC · NVMe-ready SR860 V3 4-socket · many NVMe slots # Supermicro AS-2125HS-TNR 2U EPYC NVMe focus SYS-220BT-DNTR BigTwin NVMe A+ Server NVMe EDSFF E1.S form factor # EOL servers · limited upgrade R730/R730xd PCIe AIC NVMe only (no direct bay) DL380 Gen9 AIC NVMe available, no hot-swap SR630/SR650 v1 Limited NVMe, no tri-mode controller

How the job runs

Five phases.

1 · Platform audit

We check the model, the existing backplane, free PCIe lanes, free AIC slots, PSU budget and current cooling. We establish whether the migration is plug-in or needs a backplane upgrade.

2 · Drive selection and DWPD class

Read-intensive, mixed-use or write-intensive depending on the workload. Per-drive capacity consistent with the chosen class and the lane budget. Vendors: Samsung PM9A3/PM9B1, Kioxia CD8 / CM7, Micron 7450/9400, Solidigm D7-PS1010, Western Digital DC SN840.

3 · Backplane / controller upgrade if needed

Replacing a SAS-only backplane with a tri-mode one where required. Installing a tri-mode controller if hardware RAID is needed. Updated SAS cabling (SlimSAS, MCIO).

4 · Drive installation and configuration

Physical installation of the NVMe drives in the bays. Storage configuration: hardware RAID via tri-mode, HBA passthrough for SDS, or mdadm/Storage Spaces if software-defined. Drivers and firmware updated.

5 · Validation and benchmarking

fio benchmarks for the typical workload, drive temperature checks under load, hot-swap validation if required, a written performance baseline. Data migration from the previous storage in an agreed window.

Anonymised real case

PostgreSQL database migration from SAS 10K to U.3 NVMe.

Datacenter customer in Lombardy, multi-tenant SaaS application on PostgreSQL 15 with an 8 TB active dataset. PowerEdge R750xs with 8× SAS 12G 10K 2.4 TB in RAID 10, PERC H755 controller. OLTP query latency in the 8-30 ms range, batch reporting that took 4-6 hours overnight and saturated the storage.

Solution: the R750xs was already NVMe-ready in the front bays. We replaced the storage pool with 6× Samsung PM9A3 U.2 NVMe 3.84 TB Mixed-Use in software RAID 10 via mdadm (no tri-mode controller, to get full NVMe speed). The PERC H755 was kept as an HBA in passthrough.

Migration: the new storage was set up in parallel, pg_basebackup to the new pool, swap during the overnight maintenance window. The database was online on the new storage after 90 minutes of total downtime (including post-switch vacuum/analyze).

Result: 95th-percentile OLTP query latency from 30 ms to 4 ms. Overnight batch from 4-6 hours to 50 minutes. Storage IOPS from 18k peak to 380k peak. ROI calculated at 8 months against the cost of scaling out onto an additional node.

# Pre · 8× SAS 10K 2.4 TB · RAID 10 · PERC H755 4K random read 62,000 IOPS 4K random write 41,000 IOPS 128K seq read 2.4 GB/s PG OLTP p95 28 ms Batch reporting 4h 40min # Post · 6× Samsung PM9A3 3.84 TB · RAID 10 mdadm 4K random read 420,000 IOPS 4K random write 285,000 IOPS 128K seq read 14.2 GB/s PG OLTP p95 3.8 ms Batch reporting 47 min

Cost drivers

Three main line items in an NVMe quote.

Enterprise NVMe drives — the largest item. It varies with capacity, DWPD class (Read-Intensive < Mixed-Use < Write-Intensive) and vendor (Samsung/Kioxia/Micron premium, Solidigm and WD competitive).
Backplane / controller upgrade if needed — on systems that are not NVMe-ready this is a significant additional cost.
Data migration and labour — an agreed maintenance window, any online sync with vSphere Storage vMotion or pg_basebackup, post-migration validation.

FAQ

The questions we get most often.

Can I add NVMe to my existing server?

It depends on the model. A server with a standard SAS/SATA backplane will not take NVMe unless the backplane is replaced with a tri-mode version. Servers that are already prepared for it (Dell PowerEdge R750 with tri-mode backplane, HPE ProLiant DL380 Gen11 with NVMe-ready bays, Lenovo SR650 V2) take the drives directly. Send us the exact model and we will come back to you on feasibility.

Does NVMe behind a tri-mode controller reach the same performance as NVMe direct on PCIe?