Description- Increase or decrease the size of static partition in linux & integrating LVM with Hadoop
To do so, we have to follow some steps —
- We will attach a hard disk from the virtual box to the os.
2. Before beginning, we first check the current state of our storage and we see that 8gb HDD is attached — command-fdisk -l.
3. we need to create physical volume for the recently attached storage by the command — pvcreate /dev/sdb
we will confirm by the command pvdisplay.
4. Now, we need to create volume group. for that, the command is — vgcreate mdk /dev/sdb #here mdk is the group name.
we can confirm by the command — vgdisplay.
5. Now, we need to create the logical volume from the volume group created earlier by the command — lvcreate — size 4G — name lvm2 mdk #lvm2 is the name of the logical volume it can be anything.
#we can confirm by the command — lvdisplay
6. We need to format the created logical partition so that its format can be synced with the base os directory. for this, the command is — mkfs.ext4 /dev/mdk/lvm2.
7. We need to start the namenode of the cluster after configuring core-site.xml and hdfs-site.xml. command — hadoop start namenode
to see that namenode is started, we can use hadoop-daemon.sh dfsadmin -report command
8. Now, we need to mount the newly created logical volume with the static directory that we have created. after mounting, we need to start the hadoop service for datanode by command — hadoop-daemon.sh start datanode.
we can confirm by jps command.
the location of the mounted folder needs to be given in the hdfs-site.xml file.
Description- Automating LVM Partition using python script.
