07-16-2020, 09:34 PM
hi,
I wanted to do an offline version of a very good mathematics help site (if you are interested.cut the knot org). so i used httrack and got it on my vps. It was almost 12gb.
Next, being the weird person that i am, i wanted to run lrztar -z on it to see how far i could compress it.
For the lrzip-lrztar novice (i am one too, apparently), lrztar archives the directory and then runs normal compression algorithm on that file with slight modification. it even finds repeated data chunks that are really far far away from one another. so kind of extended range on finding algorithm. the bigger the file, the better the output. with -z switch it is the max compression using zpaq at the cost of huge ram and time usage.
Now the problem is it gets very high CPU usage which i was unable to keep in control using cpulimit. So i stopped after trying for 2 minutes. (didn't try tinkering with cgroups though).
Here is my question,
do you guys have any idea how i could run that on my vps safely ?
I wanted to do an offline version of a very good mathematics help site (if you are interested.cut the knot org). so i used httrack and got it on my vps. It was almost 12gb.
Next, being the weird person that i am, i wanted to run lrztar -z on it to see how far i could compress it.
For the lrzip-lrztar novice (i am one too, apparently), lrztar archives the directory and then runs normal compression algorithm on that file with slight modification. it even finds repeated data chunks that are really far far away from one another. so kind of extended range on finding algorithm. the bigger the file, the better the output. with -z switch it is the max compression using zpaq at the cost of huge ram and time usage.
Now the problem is it gets very high CPU usage which i was unable to keep in control using cpulimit. So i stopped after trying for 2 minutes. (didn't try tinkering with cgroups though).
Here is my question,
do you guys have any idea how i could run that on my vps safely ?