Release notes for Gluster 7.0

This is a major release that includes a range of code improvements and stability fixes along with a few features as noted below.

A selection of the key features and changes are documented in this page. A full list of bugs that have been addressed is included further below.

Announcements

  1. Releases that receive maintenance updates post release 7 are, 5, 6 and 7 (reference)

  2. Release 7 will receive maintenance updates around the 10th of every month for the first 3 months post release (i.e Sep'19, Oct'19, Nov'19). Post the initial 3 months, it will receive maintenance updates every 2 months till EOL.

Major changes and features

Highlights

  • Several stability fixes addressing,
  • coverity, clang-scan, address sanitizer and valgrind reported issues
  • removal of unused and hence, deprecated code and features
  • Performance Improvements

Features

1. Rpcbind not required in glusterd.service when gnfs isn't built.

2. Latency based read child to improve read workload latency in a cluster, especially in a cloud setup. Also provides a load balancing with the outstanding pending request.

3. Glusterfind: integrate with gfid2path, to improve performance.

4. Issue #532: Work towards implementing global thread pooling has started

5. This release includes extra coverage for glfs public APIs in our regression tests, so we don't break anything.

Major issues

None

Bugs addressed

Bugs addressed since release-6 are listed below.

  • #789278: Issues reported by Coverity static analysis tool
  • #1098991: Dist-geo-rep: Invalid slave url (::: three or more colons) error out with unclear error message.
  • #1193929: GlusterFS can be improved
  • #1241494: [Backup]: Glusterfind CLI commands need to verify the accepted names for session/volume, before failing with error(s)
  • #1512093: Value of pending entry operations in detail status output is going up after each synchronization.
  • #1535511: Gluster CLI shouldn't stop if log file couldn't be opened
  • #1542072: Syntactical errors in hook scripts for managing SELinux context on bricks #2 (S10selinux-label-brick.sh + S10selinux-del-fcontext.sh)
  • #1573226: eventsapi: ABRT report for package glusterfs has reached 10 occurrences
  • #1580315: gluster volume status inode getting timed out after 30 minutes with no output/error
  • #1590385: Refactor dht lookup code
  • #1593224: [Disperse] : Client side heal is not removing dirty flag for some of the files.
  • #1596787: glusterfs rpc-clnt.c: error returned while attempting to connect to host: (null), port 0
  • #1622665: clang-scan report: glusterfs issues
  • #1624701: error-out {inode,entry}lk fops with all-zero lk-owner
  • #1628194: tests/dht: Additional tests for dht operations
  • #1633930: ASan (address sanitizer) fixes - Blanket bug
  • #1634664: Inconsistent quorum checks during open and fd based operations
  • #1635688: Keep only the valid (maintained/supported) components in the build
  • #1642168: changes to cloudsync xlator
  • #1642810: remove glupy from code and build
  • #1648169: Fuse mount would crash if features.encryption is on in the version from 3.13.0 to 4.1.5
  • #1648768: Tracker bug for all leases related issues
  • #1650095: Regression tests for geo-replication on EC volume is not available. It should be added.
  • #1651246: Failed to dispatch handler
  • #1651439: gluster-NFS crash while expanding volume
  • #1651445: [RFE] storage.reserve option should take size of disk as input instead of percentage
  • #1652887: Geo-rep help looks to have a typo.
  • #1654021: Gluster volume heal causes continuous info logging of "invalid argument"
  • #1654270: glusterd crashed with seg fault possibly during node reboot while volume creates and deletes were happening
  • #1659334: FUSE mount seems to be hung and not accessible
  • #1659708: Optimize by not stopping (restart) selfheal deamon (shd) when a volume is stopped unless it is the last volume
  • #1664934: glusterfs-fuse client not benefiting from page cache on read after write
  • #1670031: performance regression seen with smallfile workload tests
  • #1672480: Bugs Test Module tests failing on s390x
  • #1672711: Upgrade from glusterfs 3.12 to gluster 4/5 broken
  • #1672727: Fix timeouts so the tests pass on AWS
  • #1672851: With parallel-readdir enabled, deleting a directory containing stale linkto files fails with "Directory not empty"
  • #1674389: [thin arbiter] : rpm - add thin-arbiter package
  • #1674406: glusterfs FUSE client crashing every few days with 'Failed to dispatch handler'
  • #1674412: listing a file while writing to it causes deadlock
  • #1675076: [posix]: log the actual path wherever possible
  • #1676400: rm -rf fails with "Directory not empty"
  • #1676430: distribute: Perf regression in mkdir path
  • #1676736: tests: ./tests/bugs/distribute/bug-1161311.t times out
  • #1676797: server xlator doesn't handle dict unserialization failures correctly
  • #1677559: gNFS crashed when processing "gluster v profile [vol] info nfs"
  • #1678726: Integer Overflow possible in md-cache.c due to data type inconsistency
  • #1679401: Geo-rep setup creates an incorrectly formatted authorized_keys file
  • #1679406: glustereventsd does not start on Ubuntu 16.04 LTS
  • #1680587: Building RPM packages with _for_fedora_koji_builds enabled fails on el6
  • #1683352: remove experimental xlators informations from glusterd-volume-set.c
  • #1683594: nfs ltp ftest* fstat gets mismatch size as except after turn on md-cache
  • #1683816: Memory leak when peer detach fails
  • #1684385: [ovirt-gluster] Rolling gluster upgrade from 3.12.5 to 5.3 led to shard on-disk xattrs disappearing
  • #1684404: Multiple shd processes are running on brick_mux environmet
  • #1685027: Error handling in /usr/sbin/gluster-eventsapi produces IndexError: tuple index out of range
  • #1685120: upgrade from 3.12, 4.1 and 5 to 6 broken
  • #1685414: glusterd memory usage grows at 98 MB/h while running "gluster v profile" in a loop
  • #1685944: WORM-XLator: Maybe integer overflow when computing new atime
  • #1686371: Cleanup nigel access and document it
  • #1686398: Thin-arbiter minor fixes
  • #1686568: [geo-rep]: Checksum mismatch when 2x2 vols are converted to arbiter
  • #1686711: [Thin-arbiter] : send correct error code in case of failure
  • #1687326: [RFE] Revoke access from nodes using Certificate Revoke List in SSL
  • #1687705: Brick process has coredumped, when starting glusterd
  • #1687811: core dump generated while running the test ./tests/00-geo-rep/georep-basic-dr-rsync-arbiter.t
  • #1688068: Proper error message needed for FUSE mount failure when /var is filled.
  • #1688106: Remove implementation of number of files opened in posix xlator
  • #1688116: Spurious failure in test ./tests/bugs/glusterfs/bug-844688.t
  • #1688287: ganesha crash on glusterfs with shard volume
  • #1689097: gfapi: provide an option for changing statedump path in glfs-api.
  • #1689799: [cluster/ec] : Fix handling of heal info cases without locks
  • #1689920: lots of "Matching lock not found for unlock xxx" when using disperse (ec) xlator
  • #1690753: Volume stop when quorum not met is successful
  • #1691164: glusterd leaking memory when issued gluster vol status all tasks continuosly
  • #1691616: client log flooding with intentional socket shutdown message when a brick is down
  • #1692093: Network throughput usage increased x5
  • #1692612: Locking issue when restarting bricks
  • #1692666: ssh-port config set is failing
  • #1693575: gfapi: do not block epoll thread for upcall notifications
  • #1693648: Geo-re: Geo replication failing in "cannot allocate memory"
  • #1693692: Increase code coverage from regression tests
  • #1694820: Geo-rep: Data inconsistency while syncing heavy renames with constant destination name
  • #1694925: GF_LOG_OCCASSIONALLY API doesn't log at first instance
  • #1695327: regression test fails with brick mux enabled.
  • #1696046: Log level changes do not take effect until the process is restarted
  • #1696077: Add pause and resume test case for geo-rep
  • #1696136: gluster fuse mount crashed, when deleting 2T image file from oVirt Manager UI
  • #1696512: glusterfs build is failing on rhel-6
  • #1696599: Fops hang when inodelk fails on the first fop
  • #1697316: Getting SEEK-2 and SEEK7 errors with [Invalid argument] in the bricks' logs
  • #1697486: bug-1650403.t && bug-858215.t are throwing error "No such file" at the time of access glustershd pidfile
  • #1697866: Provide a way to detach a failed node
  • #1697907: ctime feature breaks old client to connect to new server
  • #1697930: Thin-Arbiter SHD minor fixes
  • #1698078: ctime: Creation of tar file on gluster mount throws warning "file changed as we read it"
  • #1698449: thin-arbiter lock release fixes
  • #1699025: Brick is not able to detach successfully in brick_mux environment
  • #1699176: rebalance start command doesn't throw up error message if the command fails
  • #1699189: fix truncate lock to cover the write in tuncate clean
  • #1699339: With 1800+ vol and simultaneous 2 gluster pod restarts, running gluster commands gives issues once all pods are up
  • #1699394: [geo-rep]: Geo-rep goes FAULTY with OSError
  • #1699866: I/O error on writes to a disperse volume when replace-brick is executed
  • #1700078: disablle + reenable of bitrot leads to files marked as bad
  • #1700865: FUSE mount seems to be hung and not accessible
  • #1701337: issues with 'building' glusterfs packages if we do 'git clone --depth 1'
  • #1701457: ctime: Logs are flooded with "posix set mdata failed, No ctime" error during open
  • #1702131: The source file is left in EC volume after rename when glusterfsd out of service
  • #1702185: coredump reported by test ./tests/bugs/glusterd/bug-1699339.t
  • #1702299: Custom xattrs are not healed on newly added brick
  • #1702303: Enable enable fips-mode-rchecksum for new volumes by default
  • #1702952: remove tier related information from manual pages
  • #1703020: The cluster.heal-timeout option is unavailable for ec volume
  • #1703629: statedump is not capturing info related to glusterd
  • #1703948: Self-heal daemon resources are not cleaned properly after a ec fini
  • #1704252: Creation of bulkvoldict thread logic is not correct while brick_mux is enabled for single volume
  • #1704888: delete the snapshots and volume at the end of uss.t
  • #1705865: VM stuck in a shutdown because of a pending fuse request
  • #1705884: Image size as reported from the fuse mount is incorrect
  • #1706603: Glusterfsd crashing in ec-inode-write.c, in GF_ASSERT
  • #1707081: Self heal daemon not coming up after upgrade to glusterfs-6.0-2 (intermittently) on a brick mux setup
  • #1707700: maintain consistent values across for options when fetched at cluster level or volume level
  • #1707728: geo-rep: Sync hangs with tarssh as sync-engine
  • #1707742: tests/geo-rep: arequal checksum comparison always succeeds
  • #1707746: AFR-v2 does not log before attempting data self-heal
  • #1708051: Capture memory consumption for gluster process at the time of throwing no memory available message
  • #1708156: ec ignores lock contention notifications for partially acquired locks
  • #1708163: tests: fix bug-1319374.c compile warnings.
  • #1708926: Invalid memory access while executing cleanup_and_exit
  • #1708929: Add more test coverage for shd mux
  • #1709248: [geo-rep]: Non-root - Unable to set up mountbroker root directory and group
  • #1709653: geo-rep: With heavy rename workload geo-rep log if flooded
  • #1710054: Optimize the glustershd manager to send reconfigure
  • #1710159: glusterd: While upgrading (3-node cluster) 'gluster v status' times out on node to be upgraded
  • #1711240: [GNFS] gf_nfs_mt_inode_ctx serious memory leak
  • #1711250: bulkvoldict thread is not handling all volumes while brick multiplex is enabled
  • #1711297: Optimize glusterd code to copy dictionary in handshake code path
  • #1711764: Files inaccessible if one rebalance process is killed in a multinode volume
  • #1711820: Typo in cli return string.
  • #1711827: test case bug-1399598-uss-with-ssl.t is generating crash
  • #1712322: Brick logs inundated with [2019-04-27 22:14:53.378047] I [dict.c:541:dict_get] (-->/usr/lib64/glusterfs/6.0/xlator/features/worm.so(+0x7241) [0x7fe857bb3241] -->/usr/lib64/glusterfs/6.0/xlator/features/locks.so(+0x1c219) [0x7fe857dda219] [Invalid argumen
  • #1712668: Remove-brick shows warning cluster.force-migration enabled where as cluster.force-migration is disabled on the volume
  • #1712741: glusterd_svcs_stop should call individual wrapper function to stop rather than calling the glusterd_svc_stop
  • #1713730: Failure when glusterd is configured to bind specific IPv6 address. If bind-address is IPv6, *addr_len will be non-zero and it goes to ret = -1 branch, which will cause listen failure eventually
  • #1714098: Make debugging hung frames easier
  • #1714415: Script to make it easier to find hung frames
  • #1714973: upgrade after tier code removal results in peer rejection.
  • #1715921: uss.t tests times out with brick-mux regression
  • #1716695: Fix memory leaks that are present even after an xlator fini [client side xlator]
  • #1716766: [Thin-arbiter] TA process is not picking 24007 as port while starting up
  • #1716812: Failed to create volume which transport_type is "tcp,rdma"
  • #1716830: DHT: directory permissions are wiped out
  • #1717757: WORM: Segmentation Fault if bitrot stub do signature
  • #1717782: gluster v get all still showing storage.fips-mode-rchecksum off
  • #1717819: Changes to self-heal logic w.r.t. detecting metadata split-brains
  • #1717953: SELinux context labels are missing for newly added bricks using add-brick command
  • #1718191: Regression: Intermittent test failure for quick-read-with-upcall.t
  • #1718273: markdown formatting errors in files present under /doc directory of the project
  • #1718316: Ganesha-gfapi logs are flooded with error messages related to "gf_uuid_is_null(gfid)) [Invalid argument]" when lookups are running from multiple clients
  • #1718338: Upcall: Avoid sending upcalls for invalid Inode
  • #1718848: False positive logging of mount failure
  • #1718998: Fix test case "tests/basic/afr/split-brain-favorite-child-policy.t" failure
  • #1720201: Healing not proceeding during in-service upgrade on a disperse volume
  • #1720290: ctime changes: tar still complains file changed as we read it if uss is enabled
  • #1720615: [RHEL-8.1] yum update fails for rhel-8 glusterfs client packages 6.0-5.el8
  • #1720993: tests/features/subdir-mount.t is failing for brick_mux regrssion
  • #1721385: glusterfs-libs: usage of inet_addr() may impact IPv6
  • #1721435: DHT: Internal xattrs visible on the mount
  • #1721441: geo-rep: Fix permissions for GEOREP_DIR in non-root setup
  • #1721601: [SHD] : logs of one volume are going to log file of other volume
  • #1722541: stale shd process files leading to heal timing out and heal deamon not coming up for all volumes