10-03-2020, 06:20 AM
There are times when we want to replicate our own online VPS locally whatever the reason behind that decision; this HowTo tutorial is just about this process done for a QEMU/KVM guest system.
Some Context:
During last June I decided to terminate my VPS-9 hosting of a GIS project that I've deployed there over a period of 2 months. The general experience was fair but I've never let it run on autopilot mainly because of CPU issues (stealness issues and CPU spikes (under certain circumstances) with time spans edging the sponsor's AUP.
Because everything was already perfectly setup and working just fine, I decided to replicate my VPS-9 locally, on my Fedora Server 32, which is running KVM.
The replication process consists in two steps:
Cloning your VPS storage device over the network
As with pretty much anything in Linux, there are many ways to clone a block device over the network. But the most simple and straightforward way is to use the old and time-tested DD tool over SSH.
Before showing the command I've used, I must first stress on the fact that while you have to keep your VPS online, it must be kept in a quasi idle state. That means all its services must be turned off, everything!.. Of course you'll still have some residual activity, but what we aim for is that the filesystem should not be altered too much while we're copying the entire block device. You must remember that transferring 100 GiB (the size of VirMach VPS-9) over the network will take hours!
Now for the command:
What this command does is first initiate the SSH login via our public key authentication then, when done, run the dd command as a sudoer. But for this to work you'll have to let your sudoer run commands without entering the password everytime [1].
Now let's see what the bellow subcommand does:
This command, if successful, will end up generating a gzipped stream of our block device that will feed the next portion of the code highlighted in the code bellow.
This portion of the code will simply get the streamed feed and store it locally as the 'phoenix.img.gz' image file.
Back to the dd command. it was set to copy our block device, which was /dev/vda, in 16 MB block size while ignoring all read errors and constantly updating us on its progress.
That's a hell of a command!.. Isn't it?
Here is the whole output in my case, from start to finish, some 30 hours latter:
Once I have my block device on my PC, I can now start setting up my VPS-9 locally; but that for the next post.
Stay tuned!...
------------------------
[1]-Generally, it's just a matter of appending this line at the end of the '/etc/sudoers' file:
Edited!..
Some Context:
During last June I decided to terminate my VPS-9 hosting of a GIS project that I've deployed there over a period of 2 months. The general experience was fair but I've never let it run on autopilot mainly because of CPU issues (stealness issues and CPU spikes (under certain circumstances) with time spans edging the sponsor's AUP.
Because everything was already perfectly setup and working just fine, I decided to replicate my VPS-9 locally, on my Fedora Server 32, which is running KVM.
The replication process consists in two steps:
- The first thing we have to do is to clone VPS-9 virtual disk (ie /dev/vda) over the network using dd.
- Once we get our block device then we can setup our KVM VPS by mimicking the same configuration used by VirMach.
Cloning your VPS storage device over the network
As with pretty much anything in Linux, there are many ways to clone a block device over the network. But the most simple and straightforward way is to use the old and time-tested DD tool over SSH.
Before showing the command I've used, I must first stress on the fact that while you have to keep your VPS online, it must be kept in a quasi idle state. That means all its services must be turned off, everything!.. Of course you'll still have some residual activity, but what we aim for is that the filesystem should not be altered too much while we're copying the entire block device. You must remember that transferring 100 GiB (the size of VirMach VPS-9) over the network will take hours!
Now for the command:
Code: (Select All)
ssh -i ~/.ssh/phoenix-ecdsa-key xxx.xxx.xxx.xxx -p SSH-PORT "sudo dd if=/dev/vda bs=16M conv=sparse,notrunc,noerror status=progress | gzip -1 -" | dd bs=16M of=phoenix.img.gzWhat this command does is first initiate the SSH login via our public key authentication then, when done, run the dd command as a sudoer. But for this to work you'll have to let your sudoer run commands without entering the password everytime [1].
Now let's see what the bellow subcommand does:
Code: (Select All)
sudo dd if=/dev/vda bs=16M conv=sparse,notrunc,noerror status=progress | gzip -1 -Code: (Select All)
<....> | dd bs=16M of=phoenix.img.gzBack to the dd command. it was set to copy our block device, which was /dev/vda, in 16 MB block size while ignoring all read errors and constantly updating us on its progress.
That's a hell of a command!.. Isn't it?
Here is the whole output in my case, from start to finish, some 30 hours latter:
Code: (Select All)
[me@local media]$ ssh -i ~/.ssh/phoenix-ecdsa-key xxx.xxx.xxx.xxx -p SSH-PORT "sudo dd if=/dev/vda bs=16M conv=sparse,notrunc,noerror status=progress | gzip -1 -" | dd bs=16M of=phoenix.img.gz
107374182400 bytes (107 GB, 100 GiB) copied, 118118 s, 909 kB/s
6400+0 records in
6400+0 records out
107374182400 bytes (107 GB, 100 GiB) copied, 118118 s, 909 kB/s
0+2428890 records in
0+2428890 records out
47201888523 bytes (47 GB, 44 GiB) copied, 118125 s, 400 kB/sOnce I have my block device on my PC, I can now start setting up my VPS-9 locally; but that for the next post.
Stay tuned!...
------------------------
[1]-Generally, it's just a matter of appending this line at the end of the '/etc/sudoers' file:
Code: (Select All)
sudoerUserName ALL=(ALL) NOPASSWD: ALLEdited!..
