Major Changes and Features

Documentation about major changes and features is also included in the doc/features/ directory of GlusterFS repository.


The AFR reporting via CLI has been improved. This feature provides a coherent mechanism to present heal status,information and the logs associated. This makes the end user more aware of healing status and provides statistics.

For more information refer here.


This feature provides ability to take snapshots of files in GlusterFS. File snapshot is supported on the files of QCOW2/QED format.

This feature adds better integration with Openstack Cinder, and in general ability to take snapshots of files (typically VM images)

For more information refer here.


This feature add a new translator which is designed to provide direct access to files in glusterfs using its GFID

For more information refer here.

Prevent NFS restart on Volume change

Earlier any volume change (volume option, volume start, volume stop, volume delete,brick add, etc) required restarting NFS server.

With this feature, it is no longer required to restart NFS server, thereby providing better usability with no disrupts in NFS connections


This feature provides support upto 65536 quota configurations per volume.


This feature provides read-ahead support for directories to improve sequential directory read performance.


zerofill feature allows creation of pre-allocated and zeroed-out files on GlusterFS volumes by offloading the zeroing part to server and/or storage (storage offloads use SCSI WRITESAME), thereby achieves quick creation of pre-allocated and zeroed-out VM disk image by using server/storage off-loads.

For more information refer here.


This feature attempts to identify storage/file system failures and disable the failed brick without disrupting the rest of the NODE operation.

This adds a health-checker that periodically checks the status of the filesystem (implies checking of functional storage-hardware).

For more information refer here.

Changelog based distributed geo-replication

New improved geo-replication which makes use of all the nodes in the master volume. Unlike previous version of geo-replication where all changes were detected and synced on a single node in master volume, now each node of master volume participates in the geo-replication.

Change Detection - Now geo-rep makes use of changelog xlator to detect the set of files which needs to be synced. Changelog xlator runs per brick and when enabled, records each fops which modifies the files. geo-rep consumes these journal created by this xlator and syncs the files identified as 'changed' to slave.

Distributed nature - Each of the nodes take the repsonsibility of syncing the data which is present in that node. In case of replicated volume, one of them will be 'Active'ly syncing the data, while the other one is 'Passive'.

Syncing Method - Apart from the using rsync as the syncing method, now there tar+ssh syncing method, which can be leveraged by the workload where there is large amount of smallfiles.

Improved block device translator

This feature provides a translator to use logical volumes to store VM images and expose them as files to QEMU/KVM.

The Volume group is represented as directory and logical volumes as files.

Remove brick CLI Change

remove-brick CLI earlier used to remove the brick forcefully ( without data migration ), when called without any arguments. This mode of 'remove-brick' cli, without any arguments has been deprecated.

Experimental Features

The following features are experimental with this release:

  • RDMA-connection manager (RDMA-CM).
  • support for NUFA translator.
  • disk-encryption
  • On-Wire Compression + Decompression [CDC]

Minor Improvements:

  • Old graphs are cleaned up by FUSE clients

  • New command "volume status tasks" introduced to track asynchronous tasks like rebalance and remove-brick

  • glfs_readdir(), glfs_readdirplus(), glfs_fallocate(), glfs_discard() APIs support added in libgfapi

  • Per client RPC throttling added in rpc server

  • Communication between cli and glusterd happens over unix domain socket

  • Information on connected NFS clients is persistent across NFS restarts.

  • Hardlink creation failures with SMB addressed

  • Non-local clients function with nufa volumes

  • Configurable option added to mount.glusterfs to use kernel-readdirp with fuse client

  • AUTH support for exported nfs sub-directories added

Known Issues:

  • The following configuration changes are necessary for qemu and samba integration with libgfapi to work seamlessly:

         1) gluster volume set <volname> server.allow-insecure on
         2) Edit /etc/glusterfs/glusterd.vol to contain this line:
                option rpc-auth-allow-insecure on
        Post 1), restarting the volume would be necessary.
        Post 2), restarting glusterd would be necessary.
  • RDMA connection manager needs IPoIB for connection establishment. More details can be found here.

  • For Block Device translator based volumes open-behind translator at the client side needs to be disabled.

  • libgfapi clients calling glfs_fini before a successfull glfs_init will cause the client to hang as reported here. The workaround is NOT to call glfs_fini for error cases encountered before a successfull glfs_init.