Accessing Gluster volume via SMB Protocol
Layered product Samba is used to export the Gluster volume and ctdb for providing the high availability Samba. Here are the steps to configure Highly Available Samba cluster to export Gluster volume.
Note: These configuration steps are applicable to Samba version = 4.1. and Gluster Version >= 3.7. ctdb >= 2.5**
Step 1: Choose the servers that will export the Gluster volume.
The servers may/may not be part of the trusted storage pool. Preferable number of servers is <=4. Install Samba and ctdb packages on these servers.
Step 2: Enable/Disable the auto export of Gluster volume via SMB
# gluster volume set VOLNAME user.smb disable/enable
Step 3: Setup the CTDB Cluster:
Create a ctdb meta volume with replica N, N being the number of the servers that are used as Samba servers. This volume will host only a zero byte lock file, hence choose the minimal sized bricks. To create the n replica volume run the following command:
# gluster volume create <volname> replica n <ipaddr/host name>:/<brick_patch>.... N times
In the following files, replace "all" in the statement "META=all" to the newly created volume name.
Start the ctdb volume
# gluster vol start <volname>
Verify the following:
If the following lines are added in smb.conf file in all the nodes running samba/ctdb:
clustering = yes idmap backend = tdb2
If the ctdb volume is mounted at /gluster/lock on all the nodes that runs ctdb/samba
If the mount entry for ctdb volume is added in /etc/fstab
If file /etc/sysconfig/ctdb exists on all the nodes that runs ctdb/samba
Create /etc/ctdb/nodes files on all the nodes that runs ctdb/samba, and add the IPs of all these nodes in the file.
#cat /etc/ctdb/nodes 10.16.157.0 10.16.157.3 10.16.157.6 10.16.157.8The IPs listed here are the private IPs of Samba/ctdb servers, which should be a private non-routable subnet and are only used for internal cluster traffic. For more details refer to the ctdb man page.
Create /etc/ctdb/public_addresses files on all the nodes that runs ctdb/samba, and add the virtual IPs in the following format:
<virtual IP><routing prefix> <node interface>
Eg: #cat /etc/ctdb/public_addresses 192.168.1.20/24 eth0 192.168.1.21/24 eth0
Step 4: Suggested settings before exporting the volume
Permit insecure ports for clients to brick and clients to glusterd connections
# gluster volume set VOLNAME server.allow-insecure
Also edit /etc/glusterfs/glusterd.vol on all nodes in trusted storage pool and add the following setting:
option rpc-auth-allow-insecure on
Restart glusterd service on all nodes:
systemctl restart glusterd
Disable metadata caching in gluster client:
gluster volume set VOLNAME stat-prefetch off
To ensure lock and IO coherency:
#gluster volume set VOLNAME storage.batch-fsync-delay-usec 0
If using Samba 4.X version add the following line in smb.conf for all gluster volume or in the global section
kernel share modes = no kernel oplocks = no map archive = no map hidden = no map read only = no map system = no store dos attributes = yes
Note: Setting 'store dos attributes = no' is recommended if archive/hidden/read-only dos attributes are not used. This can give better performance.
- If SELinux is enabled and enforcing, try the following command if ctdb fails.
# setsebool -P use_fusefs_home_dirs 1 # setsebool -P samba_load_libgfapi 1
Step 5: Mount the volume using SMB
If no Active directory setup add the user on all the samba server and set the password
# adduser USERNAME # smbpasswd -a USERNAME
Start the ctdb, smb and other related services:
# systemctl re/start ctdb # ctdb status # ctdb ip # ctdb ping -n all
To verify if the volume exported by samba can be accessed by a user:
# smbclient //<hostname>/gluster-<volname> -U <username>%<password>
To mount on a linux system:
# mount -t cifs -o user=<username>,pass=<password> //<Virtual IP>/gluster-<volname> /<mountpoint>
To mount on Windows system:
>net use <device:> \\<Virtual IP>\gluster-<volname>
\\<Virtual IP>\gluster-<volname> from windows explorer.