Page 1 of 1

Directory index full

Posted: 2020/06/30 20:21:33
by jscarville
I recently tried to copy a bunch of files from an S3 container to local storage on one of our file servers. I used the aws utilty.

All the files (~11 million) are in one directory so I expected a bit of trouble but not like this. After 10,213,694 files the process stopped with an out of space error. The drive has plenty of space so I dug a bit deeper and found the following error message:

Code: Select all

EXT4-fs warning (device dm-2): ext4_dx_add_entry: Directory index full!
About 7,000 times...

This is one of our older CentOS 6 servers but is up-to-date on patches. The filesystem was formatted as ext4 with journalling.

I tried unmounting the volume and running fsck which reports the volume as clean. So I figure there is no file system corruption.

Is there some way I can expand the directory index and complete the copy?

Code: Select all

# df -hP /dev/mapper/vg_apg-apg
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/vg_apg-apg  4.5T  3.3T  943G  79% /NFS/apg

Code: Select all

# df -hiP /dev/mapper/vg_apg-apg
Filesystem             Inodes IUsed IFree IUse% Mounted on
/dev/mapper/vg_apg-apg   288M  9.8M  279M    4% /NFS/apg

Re: Directory index full

Posted: 2020/06/30 20:58:20
by TrevorH

Re: Directory index full

Posted: 2020/06/30 21:52:28
by jscarville
Thank you.

Would it make more sense to just migrate the existing data to an XFS formatted volume and download the rest of the files to there?