So far in this series of articles I’ve covered the following items.
Part 1 – initial setup and creating a VM
Part 2 – add a license key, enable VM automatic startup and shutdown and unregister a VM
Part 3 – create a virtual switch and configure a firewall VM
Part 4 – install an update for ESXi
The server has been running for a bit over 90 days and I haven’t had to use the VI client once. This server is my current main production ESXi host and has been running a number of VMs that are critical to my home lab. In the coming weeks, I plan to migrate about 1.6 TB of file server data into an RDM, but before that could occur I needed to create a new datastore and migrate some VMs from one datastore to another.
Creating a datastore
1) The first step of the process to migrate VMs was to create a new datastore. The install had been done to a 500 RAID 1 array, but the default datastore had been created an a 2000 GB RAID 5 array. fdisk -l shows these 2 arrays. fdisk was run as shown below to create a partition and then set the type to fb (or vmfs)
# fdisk -l
Disk /dev/disks/mpx.vmhba3:C0:T0:L0: 2000.0 GB, 2000043376640 bytes
255 heads, 63 sectors/track, 243158 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/disks/mpx.vmhba3:C0:T0:L0p1 1 243159 1953167328+ fb VMFS
Disk /dev/disks/naa.600508e000000000f881e4e840e52a08: 499.9 GB, 499999834112 bytes
64 heads, 32 sectors/track, 476837 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/disks/naa.600508e000000000f881e4e840e52a08p1 5 900 917504 5 Extended
/dev/disks/naa.600508e000000000f881e4e840e52a08p4 * 1 4 4080 4 FAT16 <32M
/dev/disks/naa.600508e000000000f881e4e840e52a08p5 5 254 255984 6 FAT16
/dev/disks/naa.600508e000000000f881e4e840e52a08p6 255 504 255984 6 FAT16
/dev/disks/naa.600508e000000000f881e4e840e52a08p7 505 614 112624 fc VMKcore
/dev/disks/naa.600508e000000000f881e4e840e52a08p8 615 900 292848 6 FAT16
Partition table entries are not in disk order
# fdisk /dev/disks/naa.600508e000000000f881e4e840e52a08
The number of cylinders for this disk is set to 476837.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)Command (m for help): pDisk /dev/disks/naa.600508e000000000f881e4e840e52a08: 499.9 GB, 499999834112 bytes
64 heads, 32 sectors/track, 476837 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System
/dev/disks/naa.600508e000000000f881e4e840e52a08p1 5 900 917504 5 Extended
/dev/disks/naa.600508e000000000f881e4e840e52a08p4 * 1 4 4080 4 FAT16 <32M
/dev/disks/naa.600508e000000000f881e4e840e52a08p5 5 254 255984 6 FAT16
/dev/disks/naa.600508e000000000f881e4e840e52a08p6 255 504 255984 6 FAT16
/dev/disks/naa.600508e000000000f881e4e840e52a08p7 505 614 112624 fc VMKcore
/dev/disks/naa.600508e000000000f881e4e840e52a08p8 615 900 292848 6 FAT16
Partition table entries are not in disk order
Command (m for help): n
Command action
l logical (5 or over)
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (901-476837, default 901): 901
Last cylinder or +size or +sizeM or +sizeK (901-476837, default 476837): 476837
Command (m for help): t
Partition number (1-8): 2
Hex code (type L to list codes): fb
Changed system type of partition 2 to fb (VMFS)
Command (m for help): w
The partition table has been altered!Calling ioctl() to re-read partition table
2) The datastore was then created with the vmkfstools command. Note that the command was run with the volume ID (vml). That can be obtained by running ls -l /vmfs/devices/disks/. I also specified a block size of 8 MB. If you don’t know how large the VMDK files will be that you need to create, just go with 8 MB. With the default of 1 MB, the maximum VMDK that you’ll be able to create is 256 GB. You can’t change it later unless you reformat. With 2 MB as the block size your VMDK can be up to 512 GB, with 4 MB it can be 1024 GB and with 8 MB it can be 2048 GB.
# vmkfstools -C vmfs3 -b 8M -S "datastore2" /vmfs/devices/disks/vml.0200000000600508e000000000f881e4e840e52a084c6f67696361:2
Creating vmfs3 file system on "vml.0200000000600508e000000000f881e4e840e52a084c6f67696361:2" with blockSize 8388608 and volume label "datastore2".
Successfully created new volume: 4aea17ed-5267d790-647d-001a9253e63a
TIP: Before you start to use the datastore, you might need to run service.sh restart. .
Migrating a VM from one datastore to another
1) On the new datastore I created a folder with the command mkdir DC05 and then made that the current directory.
2) I used vimcmd vmsvc/getallvms to get the VM id for DC05 and then ran vim-cmd vmsvc/power.shutdown 96 (where 96 is the VM id for DC05). Since DC05 had VMware Tools installed ESXi was able to shutdown the VM cleanly. Otherwise I would have used vim-cmd vmsvc/power.off 96.
3) I ran ps | grep DC05 to check if processes were still running for this VM. The first time I ran the command I received the below output and thus knew the VM was still running. After a few minutes the command showed no process for DC05 and I knew the VM had powered down. I could have also checked the vmware.log file for the VM.
8236 7077 Worker#0:DC05.vmx /bin/vmx
7078 vmm0:DC05.vmx
7108 7077 mks:DC05.vmx /bin/vmx
7109 7077 vcpu-0:DC05.vmx /bin/vmx
4) I then ran vim-cmd vmsvc/unregister 96 to remove DC05 from the hosts inventory.
5) I then moved the files from the source datastore with the move command: mv ../../datastore1/DC05/<filename> ../DC05/ with the exception of the VMDK file.
6) I made a copy of the VMDK file for DC05 with the command vmkfstools -i ../../datastore1/DC05/DC05.vmdk ../DC05/DC05.vmdk. Note: vmkfstools is not snapshot aware so you’ll want to commit or delete any snapshots before you move your VM. Also vmkfstools is better performing that mv or cp as it only requires a SCSI reservation on the destination LUN when the VMDK file is created. Both mv and cp will require many SCSI reservations as the destination file will slowly grow to match the size of the source VMDK. Also with using vmkfstools -i you retain the source VMDK should something go wrong with the migration.
7) To register the VM I ran vim-cmd solo/registervm /vmfs/volumes/datastore2/DC05/DC05.vmx and then vim-cmd vmsvc/getallvms to get the VM id for DC05 as it did change.
TIP: If you have configured auto startup or shutdown for a VM (see Part 2), unregistering a VM will remove the entry from the auto start list. You’ll want to backup the file before migrating a VM and then edit it after to have the correct VM id.
8) I was then ready to power on the VM. I ran vim-cmd vmsvc/power.on 272 and got the below.
Powering on VM:
(vim.fault.UnsupportedVmxLocation) {
dynamicType = <unset>,
faultCause = (vmodl.MethodFault) null,
msg = "Attempted to power On a virtual machine with the .vmx file not stored on a NAS or VMFS version 3 datastore. The virtual machine files must be relocated or VMFS upgraded.",
}
I then ran vim-cmd hostsvc/datastore/summary datastore2 and received the output below.
Datastore not found.
I first tried to vim-cmd hostsvc/refresh_services, but output for datastore2 was still not shown. I then ran services.sh restart and then the datastore/summary command would output this.
# vim-cmd hostsvc/datastore/summary datastore2
(vim.Datastore.Summary) {
dynamicType = <unset>,
datastore = 'vim.Datastore:4aea17ed-5267d790-647d-001a9253e63a',
name = "datastore2",
url = "/vmfs/volumes/4aea17ed-5267d790-647d-001a9253e63a",
capacity = 499021512704,
freeSpace = 465097981952,
uncommitted = 1073741824,
accessible = true,
multipleHostAccess = <unset>,
type = "VMFS",
}
9) I was then ready to start the VM again with the command: vim-cmd vmsvc/power.on 272 and I got the expected message of powered on.
10) I was expecting that a message would be displayed for the VM that one would usually see in the vSphere client asking if the VM had been moved or copied. I used the command vim-cmd vmsvc/message to respond to that. In the below example I choose 1 (I moved it) as the question response the second time I ran the command. When you specify vmsvc/message you have to use the VM id, followed by the message ID (0 is this case) followed by the response (with the options of 0, 1 or 2 in this case). This VM was up and running in under a minute and I was ready to proceed to other VMs.
/sbin # vim-cmd vmsvc/message 272
Virtual machine message 0:
msg.uuid.altered:This virtual machine may have been moved or copied
In order to configure certain management and networking features VMware ESX needs to know which.
Did you move this virtual machine, or did you copy it?
If you don't know, answer "I copied it".
0. Cancel (Cancel)
1. I _moved it (I _moved it)
2. I _copied it (I _copied it) [default]
/sbin # vim-cmd vmsvc/message 272 0 1