Replication Guide

ZFS send/receive replication for volumes and LUNs — dataset-level protection with near-zero RPO
OviOS Linux v6 retadm ZFS Send/Receive · SSH
← Home ← Back to Docs

1 How Replication Works

retadm manages ZFS send/receive replication between two OviOS nodes over SSH. It replicates individual volumes (NFS/SMB datasets) and LUNs (iSCSI block devices) to a destination pool on a secondary system — providing an independent, up-to-date copy of your data at a remote site.

ℹ️

Dataset-level, not pool-level. Replication targets individual volumes and LUNs, not entire pools. This lets you selectively replicate only the datasets you need to protect.

Source
192.168.0.149
data/smbvol · esx/lun01
SSH · ZFS
Destination
192.168.0.111
pool: repl

What Gets Replicated

  • ZFS volumes (NFS and SMB datasets)
  • ZFS zvols (iSCSI LUNs)
  • All ZFS snapshots associated with the dataset
  • Optionally: OviOS configuration (LUN mappings, share definitions, users) via autosync

What Does Not Get Replicated

  • Entire pools — replication is per-dataset
  • The OS itself — only data on the pools is replicated
  • iSCSI target definitions and service config — unless autosync.enable is on

2 Prerequisites

Requirement Details
Two OviOS nodes Source and destination, both reachable over the network
Destination pool A ZFS pool on the destination node to receive replicated datasets
Root SSH access options ssh.allow.root on must be set on both nodes
Passwordless SSH Key-based auth configured in both directions (source → dest and dest → source)
⚠️

Passwordless SSH must work in both directions — source to destination, and destination to source. retadm uses the connection in both directions during initialization and repair operations.

3 Step 1 — Passwordless SSH Setup

This step must be completed on both the source and destination nodes before retadm will work.

On Both Nodes — Enable Root SSH

ovios-shell ➤ options ssh.allow.root on
Configuration ssh.allow.root updated.

ovios-shell ➤ service ssh restart
Executing restart on ssh...

On the Source Node — Generate SSH Key

SOURCE — 192.168.0.149
~]$ sudo ssh-keygen
Generating public/private ed25519 key pair.
Enter file in which to save the key (/root/.ssh/id_ed25519): [press Enter]
Enter passphrase (empty for no passphrase): [press Enter]
Enter same passphrase again: [press Enter]
Your identification has been saved in /root/.ssh/id_ed25519
Your public key has been saved in /root/.ssh/id_ed25519.pub

On the Destination Node — Generate SSH Key

DESTINATION — 192.168.0.111
~]$ sudo ssh-keygen
# Follow the same prompts, leave passphrase empty

Exchange Keys — Source Trusts Destination

SOURCE — 192.168.0.149
~]$ sudo ssh-copy-id 192.168.0.111
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed
root@192.168.0.111's password: [enter root password]
Number of key(s) added: 1

# Verify the connection works without a password:
~]$ sudo ssh 192.168.0.111

Exchange Keys — Destination Trusts Source

DESTINATION — 192.168.0.111
~]$ sudo ssh-copy-id 192.168.0.149
The authenticity of host '192.168.0.149' can't be established.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
root@192.168.0.149's password: [enter root password]
Number of key(s) added: 1

# Verify the connection works without a password:
~]$ sudo ssh 192.168.0.149
💡

Always verify the passwordless connection works in both directions by running sudo ssh <remote-ip> before proceeding. If it prompts for a password, the key was not installed correctly.

4 Step 2 — Initialize Replication

Run retadm init from the source node to start the initial full replication of a dataset. The wizard checks connectivity to the destination, then prompts for the source dataset and target pool/name on the destination.

SOURCE — 192.168.0.149

Running init command on the source node. Use the arrow keys to navigate the list of datasets and select the dataset you want to replicate. Press Enter to select the dataset. Or you can run retadm init <pool/dataset> to skip the autoselect of the dataset. 

ovios-shell ➤ retadm init

Destination Host (IP/Name): 192.168.0.111
✔ data/smbvol                          # select the source dataset
Remote Pool Name: repl                  # pool on the destination
Remote Dataset Name: smbvol             # name for the dataset on the destination

[+] Full replication started in background.
Monitor /var/log/ovios/repl.log

The initial full replication runs in the background. Check progress:

ovios-shell ➤ retadm status

DATASET                   DESTINATION       STATUS
-----------------------------------------------------------------
data/smbvol               192.168.0.111     idle
esx/lun01                 192.168.0.111     running_full
ℹ️

Status values: running_full means the initial baseline transfer is in progress. idle means the last operation completed successfully and the dataset is waiting for the next incremental. Monitor /var/log/ovios/repl.log for detailed progress on large datasets.

Repeat retadm init for each dataset you want to replicate. In the example environment, both data/smbvol and esx/lun01 are initialized separately.

During the initialization you will be asked whether to replicate the properties or not. If you choose yes, if your source is an iSCSI LUN the mapping properties will be transferred to the destination node.

If you choose no, you will have to manually map the LUN on the destination node.

If your source dataset is a volume, and you choose yes, SMB and NFS options will be replicated to the destination node.

If you choose no, you will have to manually configure SMB and NFS on the destination node.

For the iSCSI LUN you can replicate with the properties enabled, and if you want to apply the targets and iSCSI settings on the destination node, you will have to run restore luns on the destination after the initial replication.

If your destination node has completely different iSCSI settings, than the source node, it is recommended to choose no, and configure the iSCSI settings manually on the destination node.

5 Step 3 — Incremental Replication

After the initial full replication completes, subsequent runs send only the changes since the last snapshot — making them fast and bandwidth-efficient regardless of dataset size.

Run an incremental replication manually:

ovios-shell ➤ retadm inc data/smbvol
ovios-shell ➤ retadm inc esx/lun01

Use --dry-run to preview what would be sent without actually transferring data:

ovios-shell ➤ retadm inc data/smbvol --dry-run

6 Step 4 — Scheduling with Cron

Incremental replication should be automated via cron to achieve a low RPO (Recovery Point Objective). The command for use in cron is:

/usr/bin/ovios-shell retadm inc pool/dataset

Open the crontab editor:

ovios-shell ➤ edit-cron

Example: replicate both datasets every 5 minutes:

# OviOS replication — every 5 minutes
*/5 * * * *  /usr/bin/ovios-shell retadm inc data/smbvol
*/5 * * * *  /usr/bin/ovios-shell retadm inc esx/lun01
💡

Enable the cron service first if it is not already running: options cron.enable on. For large datasets, set the interval longer than the expected transfer time to avoid overlapping jobs.

7 Command Reference

ovios-shell ➤ retadm
usage: retadm [status|init|inc|reset|doctor|repair|watch] [dataset] [--dry-run]
Command Description
retadm status Show all configured replication tasks with their current status (idle, running_full, running_inc, error)
retadm init Initialize a new replication task — performs the initial full baseline send to the destination
retadm inc [dataset] Send incremental changes since the last snapshot for the specified dataset
retadm watch [dataset] Tail live replication progress for a dataset
retadm doctor [dataset] Diagnose replication issues — checks SSH connectivity, snapshot state, and destination pool
retadm repair [dataset] Attempt to repair a broken replication relationship (e.g. after a destination pool was rolled back)
retadm reset [dataset] Remove the replication task and its tracking snapshots — requires re-initialization with a new full send

All commands that take a dataset accept the --dry-run flag to preview the operation without making changes.

8 Configuration Sync

By default, retadm replicates data only. OviOS configuration — iSCSI target definitions, LUN mappings, SMB users, share definitions — lives separately and must be handled explicitly.

When autosync.enable is turned on, OviOS automatically pushes configuration changes to the destination node whenever they occur: new LUN mappings, new SMB users, new share exports, and so on. This means the destination is always ready to serve without manual reconfiguration in the event of a failover.

ovios-shell ➤ options autosync.enable on
Approach Behaviour Best For
autosync on VERY IMPORTANT:
During retadm init ALWAYS choose YES to replicate properties!
All config changes on the source are automatically mirrored to the destination.
Therefore your destination POOL and LUN / VOLUME names must be identical to the destination.
Ex: if you replicate the dataset at: pool/lun01 on the source, then the destination must have the same dataset at: pool/lun01 (NOT pool/lun01-failover)
Why: because autosync will sync the targets, mappings, SMB users, SMB shares exactly as they appear on the source.
If all these conditions are met, failover requires only starting services on the destination. 		Production environments; fast failover requirement
autosync off The destination maintains its own independent configuration. You manage it separately. Active-active setups; destinations with a different share/target layout
💡

When autosync is enabled, the Cluster Sync Logs panel in the OviOS web interface shows the sync status between partners in real time.

9 Failover Considerations

When a failover is needed, the destination node must take over serving the replicated datasets. The exact steps depend on whether autosync was enabled on the source.

With autosync Enabled

  1. On the destination, import the replicated pools if not already imported. 
    service zfs start
    For SMB and NFS volumes, make sure they are mounted:
    service zfs mount-all
  2. Run restore to apply the synced OviOS configuration: 
    restore all
  3. Start all services: 
    service reset
  4. Verify everything is online: 
    ovios hc

Without autosync

The destination will have the data but not the source's configuration. You will need to manually recreate iSCSI targets, LUN mappings, and share definitions on the destination before clients can connect. This is appropriate when the destination intentionally has a different configuration (e.g. different initiator ACLs, different share names).

Quick reference

  1. On the destination, import the replicated pools if not already imported. 
    service zfs start
    For SMB and NFS volumes, make sure they are mounted:
    service zfs mount-all
  2. Start and / or enable all necessary services: 
    service iscsi start
    service smb start
    service nfs start
    Or user the options commands to enable the services.
  3. Run restore to apply the synced OviOS configuration, including iSCSI targets and LUN mappings (for LUNs which have been replicated with properties = yes) 
    restore all
    This will create the targets automatically. However, if the destination already has its own mappings and iSCSI config it is best to map the LUNs manually to not end up with duplicate lun IDs on the same target.
  4. Verify everything is online: 
    ovios hc
⚠️

Replicated LUNs on the destination are received as read-only until the replication relationship is broken or the dataset is promoted. Attempting to write to a replicated dataset without promoting it first will fail.

💡

Run retadm doctor regularly to catch replication issues before they become a problem. A healthy replication relationship you've never tested is still untested — schedule a quarterly failover drill.

OviOS Linux v6 — Storage made simple.