Rollback an apt-get upgrade if something goes wrong on Debian / Ubuntu Linux

Rolling back to the previous version may solve the problem or free the disk space. Both CentOS/RHEL support rollback feature, but I learned hard way both Debian and Ubuntu does not support rollback feature yet.

Know the problem before choosing the solution

I remotely administer a Ubuntu 14.04.3 LTS server in one of those dirt cheap clouds, and I will periodically use apt-get command to install packages or do upgrades. I wanted to set up “Planet Venus” ( a flexible feed aggregator ) software on my system. For some weird reason, I ran:
$ sudo apt-get -y --install-suggests install planet-venus
I should have stopped the installation. But, I was too tired and overworked that day. The result was a disaster. I ran out of disk space, and it installed 1764 packages on the system. My planet-venus installation broke down too. It was one of those days where I made mistakes and forgot to take snapshot/backups. Long story short:

Related: This is not the first time I f*cked up. See my 10 Unix command line mistakes.

Solution

I tried Google and found this wiki page not so helpful. A couple of mailing list and forum posts did not help at all. Finally, a hint come from Twitter:

Does anyone know a simple way to rollback packages on @Ubuntu Linux 14.04 LTS server?messed up.. I need to rollback over 1k+ packages

— nixCraft (@nixcraft) January 17, 2016

How to rollback an apt-get upgrade from command line?

I quickly ran:
# grep -A 2 'Start-Date: 2016-01-17 07:56:42' /var/log/apt/history.log
Sample output (full dump here):

Fig.01: history.log to rescue

Rollback / undo an apt-get install command

Rest was easy.

Create the list:

grep -A 2 'Start-Date: 2016-01-17 07:56:42' /var/log/apt/history.log | tail -1 >/tmp/packages.txt

grep -A 2 ‘Start-Date: 2016-01-17 07:56:42’ /var/log/apt/history.log | tail -1 >/tmp/packages.txt

Edit the /tmp/packages.txt file and delete Install: word:

vi /tmp/packages.txt

vi /tmp/packages.txt

OR

sed -i 's/Install://' /tmp/packages.txt

sed -i ‘s/Install://’ /tmp/packages.txt

Finally, I need to clean up a few things:

tr ',' 'n' < /tmp/packages.txt | sed '/automatic)/d' | awk '{ print $1}' > /tmp/final.packages.txt wc -l /tmp/final.packages.txt

tr ‘,’ ‘n’ /tmp/final.packages.txt
wc -l /tmp/final.packages.txt

Sample outputs:

1764 /tmp/final.packages.txt

Delete the packages

Now, I have an entire list of all packages installed on that unfaithful day

# less /tmp/final.packages.txt 
libmaa3:amd64
ant:amd64
libmimic0:amd64
dc:amd64
libparse-yapp-perl:amd64
gir1.2-clutter-1.0:amd64
libjna-java:amd64
python-egenix-mxbeebase:amd64
libxkbcommon-x11-0:amd64
libmpeg2-4:amd64
libopencv-core2.4:amd64
libdvdread4:amd64
libhunspell-1.3-0:amd64
fonts-lobster:amd64
libtotem-plparser18:amd64
libodbcinstq4-1:amd64
jed-common:amd64
....
..
...
xfonts-cyrillic:amd64
postgresql:amd64
db5.3-util:amd64
libopencore-amrnb0:amd64
firebird2.5-examples:amd64
libboost-random1.54-dev:amd64
libtbb2:amd64
libwxgtk2.8-0:amd64
libc6-x32:amd64
magicfilter:amd64

Just uninstall it:

# Run as root # Store packages name in $p p="$(</tmp/final.packages.txt)" # Nuke it apt-get --purge remove $p #clears out the local repository of retrieved package files apt-get clean # Just in case ... apt-get autoremove # Verify disk space df -H

# Run as root
# Store packages name in $p
p=”$(</tmp/final.packages.txt)"# Nuke it
apt-get –purge remove $p#clears out the local repository of retrieved package files
apt-get clean# Just in case …
apt-get autoremove# Verify disk space
df -H

Conclusion

To help yourself, you must be yourself. Be the best that you can be. When you make a mistake, learn from it, pick yourself up and move on. –Dave Pelzer

I learned that:

1. The best time to backup is before you do major stuff on the server.
2. Think twice. Hit enter once.
3. Never trust blindly the apt-get or any command that has -y option.
4. Always make the snapshot. Unfortunately, this box still uses ext4. There is no option to set my filesystem to BTRFS/ZFS (Linux on ZFS) with this cloud server provider. So I’m stuck with ext4 for now.