My earlier post Online resizing of Oracle volumes on Pure FlashArray was focused on resizing Pure FlashArray volumes that were mounted directly or through Oracle ASM. What if the volume is part of LVM and you wanted to extend the size of a logical volume?
In this post I will go over how simple it is to extend the Pure volume that is part of a LVM.
There are lot of sites in the web that talks about LVM in Linux and I wouldn’t get into that in this post but will focus on extending a logical volume that is part of a LVM.
The common use case is to extend the logical volume in a root volume to extend the space for /home or swap as the default installation of Linux (Red Hat or Oracle Linux) creates the root volume with 50GB, swap with 4GB and the rest for /home when the underlying volume is at least 50GB.
I recently installed Oracle Linux on a bare metal server with SAN boot. I had used a Pure FlashArray volume of 200GB to host the boot LUN. As part of the installation Oracle Linux had created 50GB of root volume, 4 GB of swap space and the remaining 145GB was allocated towards /home.
FlashArray LVM Details
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
[root@oranode1 ~]# pvdisplay —– Physical volume —– PV Name /dev/mapper/boot_lun_node1p2 VG Name ol PV Size 199.51 GiB / not usable 0 Allocatable yes PE Size 4.00 MiB Total PE 51073 Free PE 15 Allocated PE 51058 PV UUID GOlYxE–SBRV–fWKs–r7ri–0Y0t–0Zbk–CgDcAz [root@oranode1 ~]# vgdisplay —– Volume group —– VG Name ol System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 4 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 1 Act PV 1 VG Size 199.50 GiB PE Size 4.00 MiB Total PE 51073 Alloc PE / Size 51058 / 199.45 GiB Free PE / Size 15 / 60.00 MiB VG UUID OC99UO–1AJZ–kgLt–cEUL–Ssdo–BW9p–gYo3Au [root@oranode1 ~]# lvdisplay —– Logical volume —– LV Path <strong>/dev/ol/swap</strong> LV Name swap VG Name ol LV UUID hRFcUs–EOlO–cLxC–CGiq–A0HZ–C4UJ–uVPW9O LV Write Access read/write LV Creation host, time oranode1, 2017–01–04 15:56:08 –0800 LV Status available # open 2 LV Size 4.00 GiB Current LE 1024 Segments 1 Allocation inherit Read ahead sectors auto – currently set to 8192 Block device 252:4 —– Logical volume —– LV Path /dev/ol/home LV Name home VG Name ol LV UUID O6xNfE–miDJ–HB3T–G15u–sdco–YdYB–2JljMs LV Write Access read/write LV Creation host, time oranode1, 2017–01–04 15:56:08 –0800 LV Status available # open 0 LV Size 145.45 GiB Current LE 37234 Segments 1 Allocation inherit Read ahead sectors auto – currently set to 8192 Block device 252:5 —– Logical volume —– LV Path /dev/ol/root LV Name root VG Name ol LV UUID eLihog–m02B–Jg74–pA6x–8Qlu–gqdY–jqCQlt LV Write Access read/write LV Creation host, time oranode1, 2017–01–04 15:56:09 –0800 LV Status available # open 1 LV Size 50.00 GiB Current LE 12800 Segments 1 Allocation inherit Read ahead sectors auto – currently set to 8192 Block device 252:3 |
I realized I wanted to increase the size of the swap volume to 20GB instead of 4GB. There are various ways the swap size can be increased like allocating space from different filesystem and mark it as swap or add a new volume altogether and setup swap. I chose to extend the logical volume (/dev/ol/swap) from 4GB to 20GB but as you can see, there is no space left in the physical volume to be allocated.
Here is the high level sequence to add space and extend the logical volume.
- Extend the pure volume (through GUI or CLI)
- Rescan the device on the Linux host to reflect the new size
- Add a new partition to the boot lun (in this instance as partition 3)
- Create physical volume out of the new partition
- Extend the Volume group with the new physical volume
- Extend the logical volume
Here is how the updated logical volume will look like pictorially.
What is the benefit with this approach?
One could argue that they can create a new Pure volume altogether, attach to the host, create a physical volume out of it and add it to the volume group, which would work perfectly fine but it means a new change to the operating environment with a new volume. If you had lot of scripts to manage your operations (be it backup or cloning ) and your existing volume was not part of a protection group, now you have to modify multiple places to reflect the new volume which makes up a logical grouping. Rather this would make it easier to perform the change without impacting any of these scripts.
FlashArray Detailed steps
- Edit the Pure volume in Pure GUI and increase it from 200GB to 250GB. You can also do the same through CLI.
1purevol setattr —size 250G fs_bootvol - Rescan the scsi devices with -s option on the Linux host to reflect the new size.
1234567891011121314151617181920212223242526272829303132[root@oranode1 ~]# rescan-scsi-bus.sh -sScanning SCSI subsystem for new devicesSearching for resized LUNsRESIZED: Host: scsi0 Channel: 00 Id: 04 Lun: 01Vendor: PURE Model: FlashArray Rev: 483Type: Direct–Access ANSI SCSI revision: 06RESIZED: Host: scsi0 Channel: 00 Id: 05 Lun: 01Vendor: PURE Model: FlashArray Rev: 483Type: Direct–Access ANSI SCSI revision: 06RESIZED: Host: scsi1 Channel: 00 Id: 04 Lun: 01Vendor: PURE Model: FlashArray Rev: 483Type: Direct–Access ANSI SCSI revision: 06RESIZED: Host: scsi1 Channel: 00 Id: 05 Lun: 01Vendor: PURE Model: FlashArray Rev: 483Type: Direct–Access ANSI SCSI revision: 060 new or changed device(s) found.4 remapped or resized device(s) found.[0:0:4:1][0:0:5:1][1:0:4:1][1:0:5:1]0 device(s) removed.[root@oranode1 ~]# multipath -llboot_lun_node1 (3624a93704b53f8cc7922442b000110a6) dm–0 PURE ,FlashArraysize=200G features=‘0’ hwhandler=‘0’ wp=rw`–+– policy=‘queue-length 0’ prio=1 status=active|– 0:0:4:1 sda 8:0 active ready running|– 0:0:5:1 sdb 8:16 active ready running|– 1:0:4:1 sdc 8:32 active ready running`– 1:0:5:1 sdd 8:48 active ready running - Multipath command still shows 200G as the size. To reflect the updated size, run the following command. As you can see the new size is now reflected at the OS level.Note: boot_lun_nod1 is the alias for the boot lun that is on Pure.
1234567891011[root@oranode1 ~]# multipathd -k’resize map boot_lun_node1′ok[root@oranode1 ~]# multipath -llJan 04 17:52:16 | zram0: No fc_remote_port device for ‘rport–1:-1-0’boot_lun_node1 (3624a93704b53f8cc7922442b000110a6) dm–0 PURE ,FlashArraysize=250G features=‘0’ hwhandler=‘0’ wp=rw`–+– policy=‘queue-length 0’ prio=1 status=active|– 0:0:4:1 sda 8:0 active ready running|– 0:0:5:1 sdb 8:16 active ready running|– 1:0:4:1 sdc 8:32 active ready running`– 1:0:5:1 sdd 8:48 active ready running - Create a new partition, in this instance the 3rd partition on the boot lun as type 8e (Linux LVM) using fdisk.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172[root@oranode1 ~]# fdisk /dev/mapper/boot_lun_node1Welcome to fdisk (util–linux 2.23.2).Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Command (m for help): nPartition type:p primary (2 primary, 0 extended, 2 free)e extendedSelect (default p): pPartition number (3,4, default 3):First sector (419430400–524287999, default 419430400):Using default value 419430400Last sector, +sectors or +size{K,M,G} (419430400–524287999, default 524287999):Using default value 524287999Partition 3 of type Linux and of size 50 GiB is setCommand (m for help): tPartition number (1–3, default 3): 3Hex code (type L to list all codes): L0 Empty 24 NEC DOS 81 Minix / old Lin bf Solaris1 FAT12 27 Hidden NTFS Win 82 Linux swap / So c1 DRDOS/sec (FAT–2 XENIX root 39 Plan 9 83 Linux c4 DRDOS/sec (FAT–3 XENIX usr 3c PartitionMagic 84 OS/2 hidden C: c6 DRDOS/sec (FAT–4 FAT16 <32M 40 Venix 80286 85 Linux extended c7 Syrinx5 Extended 41 PPC PReP Boot 86 NTFS volume set da Non–FS data6 FAT16 42 SFS 87 NTFS volume set db CP/M / CTOS / .7 HPFS/NTFS/exFAT 4d QNX4.x 88 Linux plaintext de Dell Utility8 AIX 4e QNX4.x 2nd part 8e Linux LVM df BootIt9 AIX bootable 4f QNX4.x 3rd part 93 Amoeba e1 DOS accessa OS/2 Boot Manag 50 OnTrack DM 94 Amoeba BBT e3 DOS R/Ob W95 FAT32 51 OnTrack DM6 Aux 9f BSD/OS e4 SpeedStorc W95 FAT32 (LBA) 52 CP/M a0 IBM Thinkpad hi eb BeOS fse W95 FAT16 (LBA) 53 OnTrack DM6 Aux a5 FreeBSD ee GPTf W95 Ext‘d (LBA) 54 OnTrackDM6 a6 OpenBSD ef EFI (FAT-12/16/10 OPUS 55 EZ-Drive a7 NeXTSTEP f0 Linux/PA-RISC b11 Hidden FAT12 56 Golden Bow a8 Darwin UFS f1 SpeedStor12 Compaq diagnost 5c Priam Edisk a9 NetBSD f4 SpeedStor14 Hidden FAT16 <3 61 SpeedStor ab Darwin boot f2 DOS secondary16 Hidden FAT16 63 GNU HURD or Sys af HFS / HFS+ fb VMware VMFS17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fc VMware VMKCORE18 AST SmartSleep 65 Novell Netware b8 BSDI swap fd Linux raid auto1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid fe LANstep1c Hidden W95 FAT3 75 PC/IX be Solaris boot ff BBT1e Hidden W95 FAT1 80 Old MinixHex code (type L to list all codes): 8eChanged type of partition ‘Linux‘ to ‘Linux LVM‘Command (m for help): wThe partition table has been altered!Calling ioctl() to re–read partition table.WARNING: Re–reading the partition table failed with error 22: Invalid argument.The kernel still uses the old table. The new table will be used atthe next reboot or after you run partprobe(8) or kpartx(8)Syncing disks.[root@oranode1 ~]# ls -ltr /dev/mapper/boot*lrwxrwxrwx 1 root root 7 Jan 4 17:52 /dev/mapper/boot_lun_node1p2 –> ../dm–2lrwxrwxrwx 1 root root 7 Jan 4 17:52 /dev/mapper/boot_lun_node1p1 –> ../dm–1lrwxrwxrwx 1 root root 7 Jan 4 17:54 /dev/mapper/boot_lun_node1 –> ../dm–0 - As reported by fdisk, the partition table doesn’t reflect the 3rd partition. Run kpartx command to re-read the partition table.
123456[root@oranode1 ~]# kpartx -a /dev/mapper/boot_lun_node1[root@oranode1 ~]# ls -ltr /dev/mapper/boot*lrwxrwxrwx 1 root root 7 Jan 4 17:54 /dev/mapper/boot_lun_node1 –> ../dm–0lrwxrwxrwx 1 root root 7 Jan 4 17:56 /dev/mapper/boot_lun_node1p2 –> ../dm–2lrwxrwxrwx 1 root root 7 Jan 4 17:56 /dev/mapper/boot_lun_node1p1 –> ../dm–1lrwxrwxrwx 1 root root 7 Jan 4 17:56 /dev/mapper/boot_lun_node1p3 –> ../dm–6 - Now create the physical volume using the 3rd partition that we created.
123456789101112131415161718192021222324252627[root@oranode1 ~]# pvcreate /dev/mapper/boot_lun_node1p3Physical volume “/dev/mapper/boot_lun_node1p3” successfully created[root@oranode1 ~]# pvdisplay—– Physical volume —–PV Name /dev/mapper/boot_lun_node1p2VG Name olPV Size 199.51 GiB / not usable 0Allocatable yesPE Size 4.00 MiBTotal PE 51073Free PE 15Allocated PE 51058PV UUID GOlYxE–SBRV–fWKs–r7ri–0Y0t–0Zbk–CgDcAz—– Physical volume —–PV Name /dev/mapper/boot_lun_node1p3VG Name olPV Size 50.00 GiB / not usable 0Allocatable yesPE Size 4.00 MiBTotal PE 12799Free PE 12799Allocated PE 0PV UUID 5u4WNT–bljS–cUq3–18xP–yMzT–fxVD–9kV58d - Extend the volume group with the physical volume that we just created. As you can see vgdisplay shows the free space available as 50GB.
12345678910111213141516171819202122232425[root@oranode1 ~]# vgextend ol /dev/mapper/boot_lun_node1p3Volume group “ol” successfully extended[root@oranode1 ~]# vgdisplay—– Volume group —–VG Name olSystem IDFormat lvm2Metadata Areas 2Metadata Sequence No 5VG Access read/writeVG Status resizableMAX LV 0Cur LV 3Open LV 3Max PV 0Cur PV 2Act PV 2VG Size 249.50 GiBPE Size 4.00 MiBTotal PE 63872Alloc PE / Size 51058 / 199.45 GiB<strong>Free PE / Size 12814 / 50.05 GiB</strong>VG UUID OC99UO–1AJZ–kgLt–cEUL–Ssdo–BW9p–gYo3Au - Now we can extend the logical volume by 16G (to get to the total of 20GB) as the underlying volume group has free space available.
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758[root@oranode1 ~]# lvextend -L +16G /dev/ol/swapSize of logical volume ol/swap changed from 4.00 GiB (1024 extents) to 20.00 GiB (5120 extents).Logical volume swap successfully resized.[root@oranode1 ~]# lvdisplay—– Logical volume —–LV Path /dev/ol/swapLV Name swapVG Name olLV UUID hRFcUs–EOlO–cLxC–CGiq–A0HZ–C4UJ–uVPW9OLV Write Access read/writeLV Creation host, time oranode1, 2017–01–04 15:56:08 –0800LV Status available# open 2LV Size 20.00 GiBCurrent LE 5120Segments 3Allocation inheritRead ahead sectors auto– currently set to 8192Block device 252:4—– Logical volume —–LV Path /dev/ol/homeLV Name homeVG Name olLV UUID O6xNfE–miDJ–HB3T–G15u–sdco–YdYB–2JljMsLV Write Access read/writeLV Creation host, time oranode1, 2017–01–04 15:56:08 –0800LV Status available# open 1LV Size 145.45 GiBCurrent LE 37234Segments 1Allocation inheritRead ahead sectors auto– currently set to 8192Block device 252:5—– Logical volume —–LV Path /dev/ol/rootLV Name rootVG Name olLV UUID eLihog–m02B–Jg74–pA6x–8Qlu–gqdY–jqCQltLV Write Access read/writeLV Creation host, time oranode1, 2017–01–04 15:56:09 –0800LV Status available# open 1LV Size 50.00 GiBCurrent LE 12800Segments 1Allocation inheritRead ahead sectors auto– currently set to 8192Block device 252:3 - In this example, we extended the swap space and to reflect the swap space on the system, perform the following. If this is on any other logical volume that is mounted, use the appropriate command (like xfs_growfs or resize2fs) to extend the filesystem.
123456789101112131415161718192021[root@oranode1 ~]# swapoff -v /dev/ol/swapswapoff /dev/ol/swap[root@oranode1 ~]# mkswap /dev/ol/swapmkswap: /dev/ol/swap: warning: wiping old swap signature.Setting up swapspace version 1, size = 20971516 KiBno label, UUID=92439b82–9660–4d05–b7e7–77c74b53a1e8[root@oranode1 ~]# swapon -vaswapon /dev/mapper/ol–swapswapon: /dev/mapper/ol–swap: found swap signature: version 1, page–size 4, same byte orderswapon: /dev/mapper/ol–swap: pagesize=4096, swapsize=21474836480, devsize=21474836480[root@oranode1 ~]# cat /proc/swapsFilename Type Size Used Priority/dev/dm–4 partition 20971516 0 –1[root@oranode1 ~]# grep SwapTotal /proc/meminfoSwapTotal: 20971516 kB
As you can see, Pure Storage has simplified the step 1 (resizing) significantly. It is not about just resizing but in general Pure Storage has simplified various aspects of storage that Storage Administrators have lot more time in their hand to do other activities including keeping up with the trend.
Disclaimer: Even though the following sequence of steps were validated in our environment, please perform your own due diligence by testing this in a test environment. Improper use or wrong sequence of steps can cause irreversible effects and possible data loss which neither I nor the storage vendor can support.
