技术标签: 运维
In this article we will learn
Earlier we had a situation wherein the LVM metadata from one of our CentOS 8 node was missing. Due to this all the logical volumes, volume groups and physical volumes mapped to that LVM metadata was not visible on the Linux server. So we had to restore LVM metadata from the backup using vgcfgrestore. I will share the steps to reproduce the scenario i.e. manually delete the LVM metadata and then steps to recover LVM2 partition, restore PV, restore VG and restore LVM metadata in Linux using vgcfgrestore.
vgcfgbackup can be used to manually create LVM backups, as these backups are very helpful and can also be used in LVM Disaster Recovery.
ALSO READ:
Step by Step Guide to perform LVM backup and restore using LVM snapshot (RHEL/CentOS 7/8)
How to boot a Linux host using LVM snapshot with BOOM Utility to verify the Snapshot content (CentOS/RHEL 8)
Before we go ahead with the steps to recover LVM2 partition in Linux, we must first prepare Lab Environment with logical volumes. Next we will manually delete lvm metadata to reproduce the issue scenario.
I have created a Virtual Machine with CentOS 8 OS using Oracle VirtualBox which is installed on a Linux server. Next I added an additional virtual disk to this VM which is mapped to /dev/sdb
.
Still installing Linux manually?
I would recommend to configure one click installation using Network PXE Boot Server. Using PXE server you can install Oracle Virtual Machines or KVM based Virtual Machines or any type of physical server without any manual intervention saving time and effort.
The first step is to create physical volume using pvcreate
What is Software Testing Why Testing is Important
[root@centos-8 ~]# pvcreate /dev/sdb Physical volume "/dev/sdb" successfully created.
Next create a new Volume Group, we will name this VG as test_vg
.
[root@centos-8 ~]# vgcreate test_vg /dev/sdb Volume group "test_vg" successfully created
List the available volume groups using vgs
. I currently have two volume groups wherein rhel
volume group contains my system LVM2 partitions
[root@centos-8 ~]# vgs VG #PV #LV #SN Attr VSize VFree rhel 1 2 0 wz--n- <14.50g 0 test_vg 1 0 0 wz--n- <8.00g <8.00g <-- new VG
Create a new logical volume test_lv1
under our new volume group test_vg
[root@centos-8 ~]# lvcreate -L 1G -n test_lv1 test_vg Logical volume "test_lv1" created.
Create ext4 file system on this new logical volume
[root@centos-8 ~]# mkfs.ext4 /dev/mapper/test_vg-test_lv1 mke2fs 1.44.6 (5-Mar-2019) Creating filesystem with 262144 4k blocks and 65536 inodes Filesystem UUID: c2d6eff5-f32f-40d4-88a5-a4ffd82ff45a Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Allocating group tables: done Writing inode tables: done Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done
List the available volume groups along with the mapped storage device. Here as you see test_vg
is mapped to /dev/sdb
[root@centos-8 ~]# vgs -o+devices VG #PV #LV #SN Attr VSize VFree Devices rhel 1 2 0 wz--n- <14.50g 0 /dev/sda2(0) rhel 1 2 0 wz--n- <14.50g 0 /dev/sda2(239) test_vg 1 1 0 wz--n- <8.00g <7.00g /dev/sdb(0)
Similarly you can see the new logical volume test_lv1
is mapped to /dev/sdb
device
[root@centos-8 ~]# lvs -o+devices LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices root rhel -wi-ao---- 13.56g /dev/sda2(239) swap rhel -wi-ao---- 956.00m /dev/sda2(0) test_lv1 test_vg -wi-a----- 1.00g /dev/sdb(0) <-- new Logical Volume
We will put some data into our logical volume to make sure there are no data loss after we recover LVM2 partition, restore PV and restore VG using LVM metadata in the next steps.
Advertisement
[root@centos-8 ~]# mkdir /test [root@centos-8 ~]# mount /dev/mapper/test_vg-test_lv1 /test/
Create a dummy file and note down the md5sum value of this file
[root@centos-8 ~]# touch /test/file [root@centos-8 ~]# md5sum /test/file d41d8cd98f00b204e9800998ecf8427e /test/file
Next un-mount the logical volume
[root@centos-8 ~]# umount /test/
To manually delete LVM metadata in Linux you can use various tools such as wipefs
, dd
etc. wipefs
can erase filesystem, raid or partition-table signatures (magic strings) from the specified device to make the signatures invisible for libblkid. wipefs does not erase the filesystem itself nor any other data from the device.
WARNING:
Execute this command wisely and is not recommended to be executed in production environments as it will delete all the file system signature of the device.
In this example we will use wipefs to delete LVM metadata from /dev/sdb
device. Since the device in question /dev/sdb
is in use by Volume Group, we have to use -f
to forcefully wipe the LVM metadata
[root@centos-8 ~]# wipefs --all --backup -f /dev/sdb /dev/sdb: 8 bytes were erased at offset 0x00000218 (LVM2_member): 4c 56 4d 32 20 30 30 31
We have used --backup
so that before deleting the LVM metadata, wipefs will create a backup of the ext4 signature containing LVM metadata under the home folder of the user who is executing the command. Since we used root user, our LVM metadata backup is stored under root user's home folder.
[root@centos-8 ~]# ls -l /root/wipefs-sdb-0x00000218.bak -rw------- 1 root root 8 Apr 5 13:45 /root/wipefs-sdb-0x00000218.bak
HINT:
To restore lvm metadata stored in the file system signature from the backup we can use dd if=~/wipefs-sdb-0x00000218.bak of=/dev/sdb seek=$((0x00000218)) bs=1 conv=notrunc
Next you can verify that all the logical volumes, volume groups and physical volume part of /dev/sdb
is missing from the Linux server
[root@centos-8 ~]# lvs -o+devices LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices root rhel -wi-ao---- 13.56g /dev/sda2(239) swap rhel -wi-ao---- 956.00m /dev/sda2(0) <--Our Logical volume no more visible
[root@centos-8 ~]# vgs VG #PV #LV #SN Attr VSize VFree rhel 1 2 0 wz--n- <14.50g 0 <-- test_vg no more visible
[root@centos-8 ~]# pvs PV VG Fmt Attr PSize PFree /dev/sda2 rhel lvm2 a-- <14.50g 0 <-- /dev/sdb no more visible
Similarly with lsblk
also we can verify that there are no LVM2 partitions under /dev/sdb
[root@centos-8 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 15G 0 disk ├─sda1 8:1 0 512M 0 part /boot └─sda2 8:2 0 14.5G 0 part ├─rhel-root 253:0 0 13.6G 0 lvm / └─rhel-swap 253:1 0 956M 0 lvm [SWAP] sdb 8:16 0 8G 0 disk sr0 11:0 1 1024M 0 rom sr1 11:1 1 1024M 0 rom
lvm.conf
file./etc/lvm/backup
file and the metadata archives are stored in the /etc/lvm/archive
file./etc/lvm/archive
file are kept and how many archive files are kept is determined by parameters you can set in the lvm.conf
file./etc/lvm
directory in the backup./etc/lvm/backup
file with the vgcfgbackup
command.vgcfgrestore
command.To list the available backups of LVM metadata use vgcfgrestore --list
. Currently we have three backup stages where the last backup was taken after we created test_lv1
logical volume.
[root@centos-8 ~]# vgcfgrestore --list test_vg File: /etc/lvm/archive/test_vg_00000-1327770182.vg VG name: test_vg Description: Created *before* executing 'vgcreate test_vg /dev/sdb' Backup Time: Sun Apr 5 13:43:26 2020 File: /etc/lvm/archive/test_vg_00001-1359568949.vg VG name: test_vg Description: Created *before* executing 'lvcreate -L 1G -n test_lv1 test_vg' Backup Time: Sun Apr 5 13:44:02 2020 File: /etc/lvm/backup/test_vg VG name: test_vg Description: Created *after* executing 'lvcreate -L 1G -n test_lv1 test_vg' Backup Time: Sun Apr 5 13:44:02 2020
So we will use the last backup i.e. /etc/lvm/backup/test_vg
to restore LVM metadata till the stage where test_lv1
was created.
IMPORTANT NOTE:
In my case the physical volume was also missing hence I am creating a new Physical Volume, but if in your case your Physical Volume is present and only Volume Groups and Logical Volumes are missing then you can ignore this step.
You must perform proper pre-checks and take backup of your file system before executing these steps in production environment to prevent any data loss.
/etc/lvm/backup/test_vg
"SBJi2o-jG2O-TfWb-3pyQ-Fh6k-fK6A-AslOg1
is the UUID of the missing PV so I will use this to restore PV in Linuxphysical_volumes { pv0 { id = "SBJi2o-jG2O-TfWb-3pyQ-Fh6k-fK6A-AslOg1" device = "/dev/sdb" # Hint only status = ["ALLOCATABLE"] flags = [] dev_size = 16777216 # 8 Gigabytes pe_start = 2048 pe_count = 2047 # 7.99609 Gigabytes } }
Next again it is important that you test the physical volume restore. We use --test
mode to verify the operation. With --test
commands will not update LVM metadata. This is implemented by disabling all metadata writing but nevertheless returning success to the calling function.
So here I have provided the same UUID of /dev/sdb
as we collected earlier, followed by the backup file we want to use to restore PV and then the device name using which we will perform pvcreate. The pvcreate
command overwrites only the LVM metadata areas and does not affect the existing data areas.
[root@centos-8 ~]# pvcreate --test --uuid "SBJi2o-jG2O-TfWb-3pyQ-Fh6k-fK6A-AslOg1" --restorefile /etc/lvm/backup/test_vg /dev/sdb TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated. WARNING: Couldn't find device with uuid SBJi2o-jG2O-TfWb-3pyQ-Fh6k-fK6A-AslOg1. Physical volume "/dev/sdb" successfully created.
With --test
mode we know that the command execution is successful. So we will run the same command without --test
to restore PV in real.
[root@centos-8 ~]# pvcreate --uuid "SBJi2o-jG2O-TfWb-3pyQ-Fh6k-fK6A-AslOg1" --restorefile /etc/lvm/backup/test_vg /dev/sdb WARNING: Couldn't find device with uuid SBJi2o-jG2O-TfWb-3pyQ-Fh6k-fK6A-AslOg1. Physical volume "/dev/sdb" successfully created.
Next verify the list of available Physical Volumes
[root@centos-8 ~]# pvs PV VG Fmt Attr PSize PFree /dev/sda2 rhel lvm2 a-- <14.50g 0 /dev/sdb lvm2 --- 8.00g 8.00g <-- Now /dev/sdb is visible
pvcreate
, we will execute vgcfgrestore with --test
mode to check the if restore VC would be success or fail.[root@centos-8 ~]# vgcfgrestore --test -f /etc/lvm/backup/test_vg test_vg TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated. Restored volume group test_vg.
As we see that the command execution in --test
mode was successful so now we can safely execute our command to restore VG and recover LVM2 partition in Linux using vgcfgrestore
.
[root@centos-8 ~]# vgcfgrestore -f /etc/lvm/backup/test_vg test_vg Restored volume group test_vg.
Using vgs
your can check if restore VG was successful.
[root@centos-8 ~]# vgs VG #PV #LV #SN Attr VSize VFree rhel 1 2 0 wz--n- <14.50g 0 test_vg 1 1 0 wz--n- <8.00g <7.00g <-- test_vg is not visible
Next verify the if you were able to restore deleted lvm and recover LVM2 partition using lvs
.
[root@centos-8 ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert root rhel -wi-ao---- 13.56g swap rhel -wi-ao---- 956.00m test_lv1 test_vg -wi------- 1.00g <-- our logical volume is also visible
Next activate the volume group test_vg
[root@centos-8 ~]# vgchange -ay test_vg 1 logical volume(s) in volume group "test_vg" now active
The most crucial part, make sure there was no data loss in the entire process to restore PV, restore VG, restore LVM metadata and recover LVM2 partition.
[root@centos-8 ~]# mount /dev/mapper/test_vg-test_lv1 /test/
If we are able to mount the logical volume so it means our ext4 file system signature is intact and not lost or else the mount would fail.
[root@centos-8 ~]# ls -l /test/ total 16 -rw-r--r-- 1 root root 0 Apr 5 13:45 file drwx------ 2 root root 16384 Apr 5 13:44 lost+found
Our test file exists and the md5sum
matches the value of what we had taken before deleting the LVM metadata
[root@centos-8 ~]# md5sum /test/file d41d8cd98f00b204e9800998ecf8427e /test/file <-- same as earlier
So overall restore PV, restore VG, restore LVM metadata and recover LVM2 partition was successful.
Lastly I hope the steps from the article to recover LVM2 partition using vgcfgrestore on Linux was helpful. So, let me know your suggestions and feedback using the comment section.
References:
Restore deleted LVM metadata
Performing Logical Volume Backup and restore using vgcfgbackup and vgcfgrestore
Related Searches: lvm backup and restore in linux, lvm disaster recovery, how to restore vg in linux, how to restore pv in linux, how to restore lvm metadata in linux
文章浏览阅读343次。五种原始的变量类型1.Undefined--未定义类型 例:var v;2.String -- ' '或" "3.Boolean4.Number5.Null--空类型 例: var v=null;Number中:NaN -- not a number非数本身是一个数字,但是它和任何数字都不相等,代表非数,它和自己都不相等判断是不是NaN不能用=_curry函数未定义
文章浏览阅读1.2w次,点赞2次,收藏17次。兑换码编码设计当前各个业务系统,只要涉及到产品销售,就离不开大大小小的运营活动需求,其中最普遍的就是兑换码需求,无论是线下活动或者是线上活动,都能起到良好的宣传效果。兑换码:由一系列字符组成,每一个兑换码对应系统中的一组信息,可以是优惠信息(优惠券),也可以是相关奖品信息。在实际的运营活动中,要求兑换码是唯一的,每一个兑换码对应一个优惠信息,而且需求量往往比较大(实际上的需求只有预期_优惠券编码规则
文章浏览阅读45次。C语言程序设计实训教程教学课件作者周林ch04结构化程序设计课件.ppt* * 4.1 选择结构程序设计 4.2 循环结构程序设计 4.3 辅助控制语句 第四章 结构化程序设计 4.1 选择结构程序设计 在现实生活中,需要进行判断和选择的情况是很多的: 如果你在家,我去拜访你 如果考试不及格,要补考 如果遇到红灯,要停车等待 第四章 结构化程序设计 在现实生活中,需要进行判断和选择的情况..._在现实生活中遇到过条件判断的问
文章浏览阅读999次。幻数使用说明 在驱动程序中实现的ioctl函数体内,实际上是有一个switch{case}结构,每一个case对应一个命令码,做出一些相应的操作。怎么实现这些操作,这是每一个程序员自己的事情。 因为设备都是特定的,这里也没法说。关键在于怎样组织命令码,因为在ioctl中命令码是唯一联系用户程序命令和驱动程序支持的途径 。 命令码的组织是有一些讲究的,因为我们一定要做到命令和设备是一一对应的,利_ioctl-number.txt幻数说明
文章浏览阅读399次。键盘按下“Shift+Ctrl+p” 输入: C++Configurations,选择JSON界面做如下改动:1.首先把 “/usr/include”,放在最前2.查看C++路径,终端输入gcc -v -E -x c++ - /usr/include/c++/5 /usr/include/x86_64-linux-gnu/c++/5 /usr/include/c++/5/backward /usr/lib/gcc/x86_64-linux-gnu/5/include /usr/local/_orb-slam3 include 报错
文章浏览阅读129次。本系列的最后一篇,因未有精力写更多的入门教程,上篇已经抛出书单,有兴趣的朋友可阅读好书来成长,此系列主讲有理由爱Sqlserver的论证性文章,希望读者们看完后,可自行做出判断,Sqlserver是否真的合适自己,目的已达成。渴望自动化及使用场景笔者所最能接触到的群体为Excel、PowerBI用户群体,在Excel中,我们知道可以使用VBA、VSTO来给Excel带来自动化操作..._sqlsever 数据分析
文章浏览阅读294次,点赞6次,收藏4次。教育智脑)建立学校的全连接中台,对学校运营过程中的数据进行处理和标准化管理,挖掘数据的价值。能:一、原先孤立的系统聚合到一个统一的平台,实现单点登录,统一身份认证,方便管理;三、数据共享,盘活了教育大数据资源,通过对外提供数。的方式构建教育的通用服务能力平台,支撑教育核心服务能力的沉淀和共享。物联网将学校的各要素(人、机、料、法、环、测)全面互联,数据实时。智慧校园解决方案,赋能教学、管理和服务升级,智慧教育体系,该数据平台具有以下几大功。教育大数据平台底座:教育智脑。教育大数据平台,以中国联通。_高校智慧大脑
文章浏览阅读9.5k次,点赞2次,收藏27次。分治法,动态规划法,贪心算法这三者之间有类似之处,比如都需要将问题划分为一个个子问题,然后通过解决这些子问题来解决最终问题。但其实这三者之间的区别还是蛮大的。贪心是则可看成是链式结构回溯和分支界限为穷举式的搜索,其思想的差异是深度优先和广度优先一:分治算法一、基本概念在计算机科学中,分治法是一种很重要的算法。字面上的解释是“分而治之”,就是把一个复杂的问题分成两_算法概念实例
文章浏览阅读5.6k次。考研篇emmmmm,这是我随笔篇章的第二更,原本计划是在中秋放假期间写好的,但是放假的时候被安排写一下单例模式,做了俩机试题目,还刷了下PAT的东西,emmmmm,最主要的还是因为我浪的很开心,没空出时间来写写东西。 距离我考研结束已经快两年了,距离今年的考研还有90天左右。 趁着这个机会回忆一下青春,这一篇会写的比较有趣,好玩,纯粹是为了记录一下当年考研中发生的有趣的事。 首先介绍..._考研调剂抑郁
文章浏览阅读438次。SpringMVC文章目录SpringMVC1、SpringMVC简介1.1 什么是MVC1.2 什么是SpringMVC1.3 SpringMVC的特点2、HelloWorld2.1 开发环境2.2 创建maven工程a>添加web模块b>打包方式:warc>引入依赖2.3 配置web.xml2.4 创建请求控制器2.5 创建SpringMVC的配置文件2.6 测试Helloworld2.7 总结3、@RequestMapping注解3.1 @RequestMapping注解的功能3._class org.springframework.web.filter.characterencodingfilter is not a jakart
文章浏览阅读4.9k次。gdb 远程调试的一个问题:Don't know how to run. Try "help target".它在抱怨不知道怎么跑,目标是什么. 你需要为它指定target remote 或target extended-remote例如:target extended-remote 192.168.1.136:1234指明target 是某IP的某端口完整示例如下:targ..._don't know how to run. try "help target".
文章浏览阅读85次。习题 11、算法描述主要是用两种基本方法:第一是自然语言描述,第二是使用专用工具进行算法描述2、c 语言程序的结构如下:1、c 语言程序由函数组成,每个程序必须具有一个 main 函数作为程序的主控函数。2、“/*“与“*/“之间的内容构成 c 语言程序的注释部分。3、用预处理命令#include 可以包含有关文件的信息。4、大小写字母在 c 语言中是有区别的。5、除 main 函数和标准库函数以..._c语言语法0x1e