Problem Multipath Centos 6.5 - Help Me

Issues related to applications and software problems
dcantos
Posts: 13
Joined: 2018/06/01 17:43:46

Problem Multipath Centos 6.5 - Help Me

Post by dcantos » 2018/06/01 17:50:10

Hello,

Once again writing to you as you always clear me of great doubt, a question I do not know if you have worked with SAN networks with IBM with the Storwize V3700, I commented that my Zimbra already has little space left and what I did was to buy a module of fiber and buy 3 hard drives to make a Raid 5 and have 2Tb available storage by means of the ZExtras I manage the HSM but just when I was going to configure the array by the fiber module in my server centos I find the novelty that is repeated the arrangement I leave a few lines of what it shows me:

With the command: fdisk -l the following is shown having storage duplicity.

Disk /dev/sdd: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sde: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdf: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdg: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdh: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdi: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdj: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdk: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


/dev/sda4 à/
Tmpfs à /dev/shm
/dev/sda2 à/boot
/dev/sda1 à/boot/efi
/dev/sdb1 à/opt
/dev/sdc1 à/mnt/StoreZimbra2

Image

https://www.centos.org/forums/download/ ... ew&id=2190

when I installed the multipath driver I changed the units

Image
https://www.centos.org/forums/download/ ... ew&id=2190

/dev/sdi4
Tmpfs
/dev/sdi2
/dev/sdi1
/dev/mapper/mpathcp1
/dev/mapper/mpathcp1

root@mail ~]# fdisk -l

WARNING: GPT (GUID Partition Table) detected on '/dev/sdi'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdi: 299.0 GB, 298999349248 bytes
255 heads, 63 sectors/track, 36351 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdi1 1 36352 291991551+ ee GPT
Partition 1 does not start on physical sector boundary.

Disk /dev/sdk: 499.0 GB, 498999492608 bytes
255 heads, 63 sectors/track, 60666 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xd8f31c0a

Device Boot Start End Blocks Id System
/dev/sdk1 1 60666 487299613+ 83 Linux
Partition 1 does not start on physical sector boundary.

WARNING: GPT (GUID Partition Table) detected on '/dev/sdj'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdj: 897.0 GB, 896998047744 bytes
255 heads, 63 sectors/track, 109053 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdj1 1 109054 875974655+ ee GPT
Partition 1 does not start on physical sector boundary.

Disk /dev/sdc: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdd: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdf: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sde: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdh: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sda: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdb: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sdg: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/mpathc: 499.0 GB, 498999492608 bytes
255 heads, 63 sectors/track, 60666 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xd8f31c0a

Device Boot Start End Blocks Id System
/dev/mapper/mpathcp1 1 60666 487299613+ 83 Linux
Partition 1 does not start on physical sector boundary.

WARNING: GPT (GUID Partition Table) detected on '/dev/mapper/mpatha'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/mapper/mpatha: 897.0 GB, 896998047744 bytes
255 heads, 63 sectors/track, 109053 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/mapper/mpathap1 1 109054 875974655+ ee GPT
Partition 1 does not start on physical sector boundary.

Disk /dev/mapper/mpathd: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/mpathcp1: 499.0 GB, 498994804224 bytes
255 heads, 63 sectors/track, 60665 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Alignment offset: 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/mpathap1: 897.0 GB, 896995950592 bytes
255 heads, 63 sectors/track, 109053 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

I do not know if something else is missing in the configuration, the truth is the first time I make this configuration, I do not know if you can help me please. is a server Zimbra centos 6.5


In if I thought that it is not normal that only should leave a single arrangement not several, talking to another friend told me that you had to install the multipath driver for your version and guided me with the following links:

https://www.nextstep4it.com/setup-dm-mu ... el-centos/
Attachments
2.png
2.png (72.4 KiB) Viewed 3041 times
1.png
1.png (13.78 KiB) Viewed 3041 times

tunk
Posts: 1205
Joined: 2017/02/22 15:08:17

Re: Problem Multipath Centos 6.5 - Help Me

Post by tunk » 2018/06/03 14:20:01

First, you really should update to CentOS 6.9 - 6.5 got it's last update 3.5 years ago.
I don't have much experience with multipath, but I think each volume should show up as /dev/mapper/mpathX and two corresponding devices (/dev/sdY+Z).
What's the output of: multipath -ll
How is /etc/multipath.conf and /etc/fstab?

dcantos
Posts: 13
Joined: 2018/06/01 17:43:46

Re: Problem Multipath Centos 6.5 - Help Me

Post by dcantos » 2018/06/04 14:49:23

tunk wrote:First, you really should update to CentOS 6.9 - 6.5 got it's last update 3.5 years ago.
I don't have much experience with multipath, but I think each volume should show up as /dev/mapper/mpathX and two corresponding devices (/dev/sdY+Z).
What's the output of: multipath -ll
How is /etc/multipath.conf and /etc/fstab?
Attached as requested, I commented that the command multipath -ll was not done because I have stopped the multipath service to not affect anything with the services I have running in this case (Zimbra).

cat /etc/multipath.conf
# This is a basic configuration file with some examples, for device mapper
# multipath.
# For a complete list of the default configuration values, see
# /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf.defaults
# For a list of configuration options with descriptions, see
# /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf.annotated
#
# REMEMBER: After updating multipath.conf, you must run
#
# service multipathd reload
#
# for the changes to take effect in multipathd

## By default, devices with vendor = "IBM" and product = "S/390.*" are
## blacklisted. To enable mulitpathing on these devies, uncomment the
## following lines.
#blacklist_exceptions {
# device {
# vendor "IBM"
# product "S/390.*"
# }
#}

## Use user friendly names, instead of using WWIDs as names.
defaults {
user_friendly_names yes
}
##
## Here is an example of how to configure some standard options.
##
#
#defaults {
# udev_dir /dev
# polling_interval 10
# path_selector "round-robin 0"
# path_grouping_policy multibus
# getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
# prio alua
# path_checker readsector0
# rr_min_io 100
# max_fds 8192
# rr_weight priorities
# failback immediate
# no_path_retry fail
# user_friendly_names yes
#}
##
## The wwid line in the following blacklist section is shown as an example
## of how to blacklist devices by wwid. The 2 devnode lines are the
## compiled in default blacklist. If you want to blacklist entire types
## of devices, such as all scsi devices, you should use a devnode line.
## However, if you want to blacklist specific devices, you should use
## a wwid line. Since there is no guarantee that a specific device will
## not change names on reboot (from /dev/sda to /dev/sdb for example)
## devnode lines are not recommended for blacklisting specific devices.
##
#blacklist {
# wwid 26353900f02796769
# devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
# devnode "^hd[a-z]"
#}
#multipaths {
# multipath {
# wwid 3600508b4000156d700012000000b0000
# alias yellow
# path_grouping_policy multibus
# path_checker readsector0
# path_selector "round-robin 0"
# failback manual
# rr_weight priorities
# no_path_retry 5
# }
# multipath {
# wwid 1DEC_____321816758474
# alias red
# }
#}
#devices {
# device {
# vendor "COMPAQ "
# product "HSV110 (C)COMPAQ"
# path_grouping_policy multibus
# getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
# path_checker readsector0
# path_selector "round-robin 0"
# hardware_handler "0"
# failback 15
# rr_weight priorities
# no_path_retry queue
# }
# device {
# vendor "COMPAQ "
# product "MSA1000 "
# path_grouping_policy multibus
# }
#}
________________________
#
# /etc/fstab
# Created by anaconda on Mon Oct 13 05:14:44 2014
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=98ca7c8f-d347-40b4-b466-f27251bb2b12 / ext4 defaults 1 1
UUID=223514cb-b822-49f3-94b3-59e01414342c /boot ext4 defaults 1 2
UUID=5606-DCE4 /boot/efi vfat umask=0077,shortname=winnt 0 0
UUID=b10d46c5-2024-4020-a69f-9f62be6874aa /opt ext4 defaults 1 2
UUID=39ee24bf-0442-4663-84b1-6b5892b400db swap swap defaults 0 0
UUID=1f980079-6b9f-4b20-93e5-907292424e20 /mnt/StoreZimbra2 ext4 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
192.168.0.20:/volume1/SRVCORREO /mnt/backup nfs defaults 0 0



tell me what process or configuration I need in the file. multipath configuration I have commented on the lines but that I had not if something else I need.

tunk
Posts: 1205
Joined: 2017/02/22 15:08:17

Re: Problem Multipath Centos 6.5 - Help Me

Post by tunk » 2018/06/05 14:09:50

I have a V3700 on a CentOS 7.5 system.
I cannot remember doing anything special, maybe except editing /etc/multipath.conf:
defaults {
user_friendly_names yes
}
blacklist {
devnode "^sd[a-f]"
}
Where sd[a-f] are normal disks.

You have eight 2.2TB /dev/sdX disks, and I would start looking at that; maybe there's something in the setup on the V3700.
You also have /dev/mapper/mpatha 897GB and /dev/mapper/mpathc 499GB, each with only one /dev/sdX. Is this something you've setup on the V3700?

dcantos
Posts: 13
Joined: 2018/06/01 17:43:46

Re: Problem Multipath Centos 6.5 - Help Me

Post by dcantos » 2018/06/05 16:05:53

tunk wrote:
2018/06/05 14:09:50
I have a V3700 on a CentOS 7.5 system.
I cannot remember doing anything special, maybe except editing /etc/multipath.conf:
defaults {
user_friendly_names yes
}
blacklist {
devnode "^sd[a-f]"
}
Where sd[a-f] are normal disks.

You have eight 2.2TB /dev/sdX disks, and I would start looking at that; maybe there's something in the setup on the V3700.
You also have /dev/mapper/mpatha 897GB and /dev/mapper/mpathc 499GB, each with only one /dev/sdX. Is this something you've setup on the V3700?

Hello Trunks,

I think he does not understand me or I do not understand.

First of all I do not have 8 2.2tb disc, I only have one of 2.2tb, but in the centos the same storage is repeated 8 times, I understand that configuring the multipath driver of those 8 discs makes me one only to be able to configure and mount the disc .

second "You also have /dev/mapper/mpatha 897GB and /dev/mapper/mpathc 499GB, each with only one /dev/sdX. Is this something you've setup on the V3700?" Regarding your question those discs are local only when I installed the multipath driver I did not know that the disc was blacklisted and in the documentation of centos already reading it tells me that if I do not blacklist the disc /dev/sdx it it changes temporarily to /dev/mapper/ mpathXX and that's what happened because that's local disk.

I understand that if I configure the multipath driver that repeat of 8 disc becomes only one.

I leave that part of the documentation that says:

2.1. Multipath Device Identifiers
Each multipath device has a World Wide Identifier (WWID), which is guaranteed to be globally unique and unchanging. By default, the name of a multipath device is set to its WWID. Alternately, you can set the user_friendly_names option in the multipath configuration file, which sets the alias to a node-unique name of the form mpathn.

For example, a node with two HBAs attached to a storage controller with two ports via a single unzoned FC switch sees four devices: /dev/sda, / dev/sdb, dev/sdc, and /dev/ sdd. DM-Multipath creates a single device with a unique WWID that reroutes I / O to those four devices depending on the multipath configuration. When the user_friendly_names configuration option is set to yes, the name of the multipath device is set to /dev/ mpath/mpathn.

For information on the multipath configuration defaults, including the user_friendly_names configuration option, see section 4.3, "Configuration File Defaults".

You can also set the name of a multipath device to a name of your choosing by using the alias option in the multipath section of the multipath configuration file. For information on the multipaths of the multipath configuration file, see Section 4.4, "Multipaths Device Configuration Attributes".

tunk
Posts: 1205
Joined: 2017/02/22 15:08:17

Re: Problem Multipath Centos 6.5 - Help Me

Post by tunk » 2018/06/05 16:33:49

Have you made a partition on /dev/mapper/mpathd and created a file system on that partition?

dcantos
Posts: 13
Joined: 2018/06/01 17:43:46

Re: Problem Multipath Centos 6.5 - Help Me

Post by dcantos » 2018/06/05 17:05:49

no, the partition / dev / mapper / multipathzx because before it was dev / sdX but the multipath I change that for not blocking it are logical disk the only disk per fiber is the 2.2tb in the first part is the images of the before and after if you understand me.

tunk
Posts: 1205
Joined: 2017/02/22 15:08:17

Re: Problem Multipath Centos 6.5 - Help Me

Post by tunk » 2018/06/05 17:23:35

Sorry, don't follow you.
Below is my output from fdisk -l.
I seem to remember that if you create a partition on one of them it should show up on the other two as well
(since it's the same disk).

Disk /dev/sdh: 16492.7 GB, 16492674416640 bytes, 32212254720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 32768 bytes / 32768 bytes
Disk label type: gpt
Disk identifier: xxx
# Start End Size Type Name
1 34 32212254686 15T Microsoft basic primary
Partition 1 does not start on physical sector boundary.
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdg: 16492.7 GB, 16492674416640 bytes, 32212254720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 32768 bytes / 32768 bytes
Disk label type: gpt
Disk identifier: xxx
# Start End Size Type Name
1 34 32212254686 15T Microsoft basic primary
Partition 1 does not start on physical sector boundary.
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/mapper/mpatha: 16492.7 GB, 16492674416640 bytes, 32212254720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 32768 bytes / 32768 bytes
Disk label type: gpt
Disk identifier: xxx
# Start End Size Type Name
1 34 32212254686 15T Microsoft basic primary
Partition 1 does not start on physical sector boundary.

dcantos
Posts: 13
Joined: 2018/06/01 17:43:46

Re: Problem Multipath Centos 6.5 - Help Me

Post by dcantos » 2018/06/05 17:39:14

So, is it normal for me to repeat the disc, even if it is the same disc? but I have no problem with the integrity of the data?

dcantos
Posts: 13
Joined: 2018/06/01 17:43:46

Re: Problem Multipath Centos 6.5 - Help Me

Post by dcantos » 2018/06/05 17:58:21

You can show the result of the command df -Th.

And the multipath .conf file please.

Post Reply