Learn OpenStack · reference/decision-guides
Choose a Cinder Block Storage Backend
Cinder supports many storage backends via a driver model. The decision affects performance, HA, operational complexity, and cost. The most common choices are LVM/iSCSI (the reference driver), Ceph RBD, NFS, and vendor-specific drivers.
Feature Comparison
| Feature | LVM/iSCSI | Ceph RBD | NFS | Vendor (Pure/NetApp/Dell EMC/HPE) |
|---|---|---|---|---|
| High availability | No (single cinder-volume node) | Yes (distributed, N+K redundancy) | Depends on NFS server HA | Yes (array HA built in) |
| Performance | Local disk speed | Distributed, scales with OSDs | Network + NFS server speed | Varies — often excellent, NVMe arrays very fast |
| Snapshots | LVM snapshots (slow, CoW) | Copy-on-write (near-instant) | Backend dependent | Native (near-instant) |
| Volume cloning | LVM full copy (slow) | Shallow clone (fast, CoW) | Full copy | Native (fast) |
| Replication | No | Built-in (mirroring) | Backend dependent | Yes (synchronous/asynchronous) |
| Live migration support | Limited (iSCSI shared access issues) | Full (shared RBD pool) | Yes (shared filesystem) | Depends on driver |
| Boot from volume | Yes | Yes (preferred with Nova RBD) | Yes | Yes |
| Thin provisioning | Yes (LVM thin pools) | Yes | Depends | Yes |
| Multi-attach | No (iSCSI single-initiator default) | Yes (with RBD multi-attach) | Yes | Depends |
| Encryption (LUKS) | Yes | Yes | Yes | Depends |
| Max tested volume size | ~16 TB (LVM practical) | Petabytes | NFS server limit | Array dependent |
| Complexity | Low | High (Ceph cluster required) | Low | Medium (vendor driver + array) |
| Cost | Low (commodity hardware) | Medium-High (dedicated Ceph cluster) | Low (if NFS already exists) | High (storage array + licenses) |
| Operational tooling | vgdisplay, lvdisplay, iscsiadm | ceph, rbd, ceph dashboard | Standard NFS tools | Vendor CLI/GUI |
| Best for | Dev/test, small single-node clouds | Production scale-out clouds | Simple shared storage | Enterprise with existing array |
Recommendation by Use Case
Development, Testing, All-in-One
Use LVM/iSCSI. Ships as the Cinder default, zero additional infrastructure. Use thin-provisioned LVM to avoid pre-allocating disk.
# Check if a thin pool exists
lvdisplay | grep "LV Pool"
# If not, create one:
lvcreate --thin -L 100G vg-name/thinpool-name
Production, Scale-Out Cloud (primary recommendation)
Use Ceph RBD. Provides distributed redundancy, fast snapshots, fast clones, and native live-migration support. If you are already running Ceph for images (Glance) or compute ephemeral disks (Nova), adding Cinder RBD is low incremental cost.
Prerequisites:
- A running Ceph cluster with a dedicated pool for Cinder (e.g. volumes)
- A Ceph client keyring for Cinder: ceph auth get-or-create client.cinder
- A Ceph client keyring for Nova (to access Cinder volumes during attach): ceph auth get-or-create client.nova
- librbd and ceph-common installed on all compute nodes
# Verify Ceph pool exists
ceph osd lspools | grep volumes
# Check Cinder can reach the Ceph cluster
sudo -u cinder rbd --id cinder --keyring /etc/ceph/ceph.client.cinder.keyring ls volumes
Simple Shared Storage (existing NFS)
Use NFS. If your organization already runs a robust NFS server (NetApp, EMC, or even Linux NFS with DRBD), the Cinder NFS driver requires minimal setup. Performance and HA depend entirely on the NFS server.
Enterprise with Existing Storage Array
Use the vendor driver. Pure Storage, NetApp ONTAP, Dell EMC PowerStore/PowerMax, HPE Nimble, and others have certified Cinder drivers. These provide native array features (instant snapshots, deduplication, replication) with OpenStack management.
Check the Cinder driver support matrix for your specific array and OpenStack release.
Mixed: Dev/test and Production in Same Cloud
Use multiple backends. Cinder supports multiple backends simultaneously. Use LVM for a standard volume type and Ceph RBD for a performance volume type:
# /etc/cinder/cinder.conf
[DEFAULT]
enabled_backends = lvm-backend,ceph-backend
default_volume_type = standard
[lvm-backend]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-volumes
volume_backend_name = LVM
[ceph-backend]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
rbd_pool = volumes
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot = false
rbd_max_clone_depth = 5
rbd_store_chunk_size = 4
rados_connect_timeout = -1
rbd_user = cinder
rbd_secret_uuid = <LIBVIRT_SECRET_UUID>
volume_backend_name = Ceph
Then create volume types:
openstack volume type create standard
openstack volume type set standard --property volume_backend_name=LVM
openstack volume type create performance
openstack volume type set performance --property volume_backend_name=Ceph
iSCSI Configuration Reference
# /etc/cinder/cinder.conf — LVM backend
[lvm-backend]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-volumes
iscsi_protocol = iscsi
iscsi_helper = lioadm # or tgtadm
volume_backend_name = LVM
# Verify iSCSI target service
systemctl status tgt # tgtd
systemctl status rtslib-fb-targetctl # LIO
# List targets
tgtadm --lld iscsi --mode target --op show # tgtd
targetcli ls # LIO
# Check active iSCSI sessions from compute nodes
iscsiadm -m session
Ceph RBD Configuration Reference
# /etc/cinder/cinder.conf — Ceph backend
[ceph-backend]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
rbd_pool = volumes
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot = false
rbd_max_clone_depth = 5
rbd_store_chunk_size = 4
rados_connect_timeout = -1
rbd_user = cinder
rbd_secret_uuid = <LIBVIRT_SECRET_UUID>
volume_backend_name = Ceph
# Create the Ceph pool
ceph osd pool create volumes 128
ceph osd pool application enable volumes rbd
# Create the Cinder keyring
ceph auth get-or-create client.cinder \
mon 'allow r' \
osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
# Register the secret in libvirt (on each compute node)
cat > /tmp/secret.xml <<EOF
<secret ephemeral="no" private="no">
<uuid><LIBVIRT_SECRET_UUID></uuid>
<usage type="ceph">
<name>client.cinder secret</name>
</usage>
</secret>
EOF
virsh secret-define --file /tmp/secret.xml
virsh secret-set-value --secret <LIBVIRT_SECRET_UUID> \
--base64 $(ceph auth get-key client.cinder)
Operational Checks
# List Cinder backends and their status
openstack volume service list
# Check volume counts per backend
openstack volume list --all-projects -f json | \
jq 'group_by(."Volume Type") | map({type: .[0]."Volume Type", count: length})'
# Ceph pool usage
ceph df
rados df
# LVM usage
vgdisplay cinder-volumes
lvdisplay | grep -E "LV Name|LV Size|Pool"