mkfs.ext4 unbelievably slow formatting 1TB partition

[LMHmedchem]

Hello,

I just installed a new 2TB Samsung 860 EVO for some added space. I am replacing an older 128GB SSD, so I started by imaging over the old drive. I wanted to add 3 partitions in addition to the one I imaged from the old drive. I added the first two using GParted, but GParted took a long time to finish formatting so I decided to use the terminal to add the last partition which is ~1TB in size.

I did the following. The drive is /sdg so I created an unformatted partition.

Code: Select all

sudo fdisk /dev/sdg
-option "p"
   Device Boot      Start         End      Blocks   Id  System
/dev/sdg1   *        4096   511991549   255993727    7  HPFS/NTFS/exFAT
/dev/sdg2       511995904   716795903   102400000   83  Linux
/dev/sdg3       716800000  1740799999   512000000   83  Linux

The print option shows the original NTFS partition and the two new partitions I created with GParted. I then used the "n" option to add a new partition,

Code: Select all

sudo fdisk /dev/sdg
-option "n"
Partition type: p   primary
First sector: 1740804096
Last sector: 3907029167

-option "p"
   Device Boot      Start         End      Blocks   Id  System
/dev/sdg1   *        4096   511991549   255993727    7  HPFS/NTFS/exFAT
/dev/sdg2       511995904   716795903   102400000   83  Linux
/dev/sdg3       716800000  1740799999   512000000   83  Linux
/dev/sdg4      1740804096  3907029167  1083112536   83  Linux

The print option now shows a 4th partition starting 2MB after the third partition and running to the end. I used the "w" option to write to the partition table and restarted.

The lsblk --fs command now shows a 4th partition that does not have a file system.

Code: Select all

NAME   FSTYPE LABEL              UUID                                 MOUNTPOINT
 sdg                                                                   
├─sdg1 ntfs   win_VM             01CE9B8721B085B0                     
├─sdg2 ext4   linux_VM           5bd4d199-568c-4dbe-a159-024d1c761008 
├─sdg3 ext4   linux_data         a89334f7-59b7-4d04-b89b-a5a30c379644 
└─sdg4

I have tried to format the 4th partition with,

Code: Select all

sudo mkfs.ext4 /dev/sdg4

So far, this has run for more than 10 hours and still has not finished. The current terminal output is

Code: Select all

Discarding device blocks: 243273728/27077813470778134

The listed block values have changed over the 10 hours, so I guess it is doing something. Oddly, the system monitor shows the mkfs.ext4 process as "disk sleep" for CPU usage. The system load is more or less 2 cores at 100%, but there is nothing in the process table that is listed as using any CPU to speak of. There is also only about 1GB of RAM usage. I would expect something like this to use allot of RAM and CPU, especially if it is taking forever.

I will let it go overnight and hopefully it will finish but this is taking a very long time. At what point should I start to worry that there is a problem?

LMHmedchem

[TrevorH]

Discarding device blocks: 243273728/27077813470778134

I make 27077813470778134 somewhere around 24627TB. I think you may have a have a hardware problem.

[LMHmedchem]

I make 27077813470778134 somewhere around 24627TB. I think you may have a have a hardware problem.

This did finish some time overnight. This is the final output.

Code: Select all

Discarding device blocks: done                            
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
67698688 inodes, 270778134 blocks
13538906 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2420113408
8264 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
	102400000, 214990848

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

The partition looks correct in GParted, meaning that the size is what I was expecting. How would I go about checking the filesystem to make sure that everything is good?

At the moment, I can open the new partitions in dolphin but I cannot write (I cannot copy any folders into the new partition or create new folders). I just get a permission denied error.

One of the partitions is mounted at,

Code: Select all

/run/media/$USER/linux_VM

Checking the permission, that directory is owned by root,

Code: Select all

drwxr-xr-x. 3 root   root   4096 Feb 10 13:17 linux_VM

Do I just chown on the directory in the location where it is mounted?

Code: Select all

sudo chown -R $USER:$USER /run/media/$USER/linux_VM

LMHmedchem

[hunter86_bg]

Why your partitions are having a gap inbetween ?
This can cause disk misalignment which can cause slow performance.

[LMHmedchem]

Why your partitions are having a gap in between ?
This can cause disk misalignment which can cause slow performance.

When I first started using SSDs, I found that there could be a problem in restoring a partition from an image. At times (after the SSD was getting older) I would attempt to restore and get an error saying that the target partition was not large enough to write the image. Apparently, SSD cells can fail over time and the storage controller on the SSD will just skip these non-function cells in IO. Having a few non-functioning cells on the target device makes the partition appear fractionally smaller than it was when the image was taken. This causes the restore to fail. Leaving a couple of MB padding between partitions on an SSD allows me to enlarge the partition later if I need to to fix this issue.

Maybe this doesn't make any sense any more (or maybe it never did). I haven't noticed any performance issues. Why would it matter which cylinder the partition starts at as long as it lines up with the partition table? Is there some way I can test if this is an issue?

LMHmedchem

[hunter86_bg]

Why your partitions are having a gap in between ?
This can cause disk misalignment which can cause slow performance.

When I first started using SSDs, I found that there could be a problem in restoring a partition from an image. At times (after the SSD was getting older) I would attempt to restore and get an error saying that the target partition was not large enough to write the image. Apparently, SSD cells can fail over time and the storage controller on the SSD will just skip these non-function cells in IO. Having a few non-functioning cells on the target device makes the partition appear fractionally smaller than it was when the image was taken. This causes the restore to fail. Leaving a couple of MB padding between partitions on an SSD allows me to enlarge the partition later if I need to to fix this issue.

Maybe this doesn't make any sense any more (or maybe it never did). I haven't noticed any performance issues. Why would it matter which cylinder the partition starts at as long as it lines up with the partition table? Is there some way I can test if this is an issue?

LMHmedchem

Here is a nice article about the topic.

[LMHmedchem]

I am going to mark this as solved because I think that it just takes a long time to format a large partition in ext4 using the method I chose. I would expect at least 60 minutes for every 100GB and possibly much more. I haven't been able to identify any problems with the partition so far, so I guess everything is aright.

If someone disagrees with this length of time, I would appreciate knowing about that.

LMHmedchem

[hunter86_bg]

I'm partitioning ext4 on larger (yet faster) storage - and it never took me an hour.

You can try writing a file (larger than the RAM) or using bonie++ to benchmark your drive and see any deviations.
If the disk performs as expect it - you are OK.

[LMHmedchem]

I ran bonnie++ with the following arguments.

Code: Select all

sudo bonnie++ -u user_name -d /run/media/user_name/qm_scratch/

This directory is on the ~1TB SSD partition that took so long to format. It was set up to use as a scratch disk but is empty at the moment. The hardware is a 2TB Samsung 860 EVO. I presume that setting -d to point to a directory on the SSD will result in the SSD being tested. I wasn't sure what to use as a value for the -s argument. I believe that it defaults to RAM*2, so that would be 32GB.

These are the results,

Code: Select all

Version      1.97   ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost.local 31G   618  99 263812  18 135393  20  1486  99 342229  21 +++++ +++
Latency             24059us   89517us     499ms   10518us    1167us    3387us
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
localhost.locald 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency               362us     257us     571us     237us      32us     325us

For comparison, I ran the same test on the SSD where the OS is mounted. This is a 256GB Samsung 850 Pro. These are the results for the /root SSD.

Code: Select all

Version      1.97   ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
localhost.local 31G   616  99 508745  34 256370  25  1480  99 625045  33 +++++ +++
Latency             21603us     125ms    2160ms   11886us   11816us    2454us
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
localhost.locald 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency              1955us     435us     432us     251us       7us     653us

I don't really know how to read these results. It looks like the values are very similar for some categories, but not for others. What does this look like to you?

If there is a problem here, how would I go about fixing it?

LMHmedchem

[hunter86_bg]

For me the performance is 50% lower than the other ssd.
You should expect better performance.

Move all data away , then create 1 single partition (use all defaults).
Next - create ext4 filesystem (use defaults) and mount it.
Repeat the bonie++ test with the same options (adjust only the mount point)

Next remount with 'noatime' option:

Code: Select all

mount -o remount,noatime /mountpoint/of/disk

And test with bonie++.

Next rebuild the FS - this time use 'mkfs.xfs' .
Mount with 'noatime' option and test with bonie++.

Last, you can change thr I/O scheduler:

Code: Select all

  echo noop > /sys/block/sdXYZ/queue/scheduler

Note: replace sdXYZ with the device name.

Last test with Bonie++

Usually XFS is performing better than ext4.
Mount option noatime increases performance - but no access times are updated.
noop &none I/O schedulers perform better than deadline (default in RHEL7/CentOS7)

Your worst enemy is the disk allignment. If you need to leave a gap - it should be divisible by 4096.