Best way to backup and restore?

General eFa discussion
CatHerder
Posts: 2
Joined: 11 Jun 2016 13:48

Best way to backup and restore?

Post by CatHerder »

So I’ve been a very happy EFA user/admin for years. I installed the VMware image a million years ago, and have been updating ever since. Most, but not all, of the upgrades were clean. However today, in updating to the latest, /some/ of thekernel updates failed due to an out of memory error, and after I rebooted to clean things up, the actual application updates ran leaving the kernel updates in a questionable state.

Now, everything /seems/ to be fine, but I’m a bit nervous about the systems state moving forward. Hence, what I’d really like to do is backup my configuration, install a new VM with the latest image, and restore the configuration from backup.

What’s the best process to do that?

Thanks!
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

This deserves a how-to. I think I'll take a stab at it :D

Here we go...note this is involved, since EFA-Backup is not yet portable without some help (https://github.com/E-F-A/v3/issues/324)

1) Run EFA-Backup

Code: Select all

sudo /usr/local/sbin/EFA-Backup -backup
2) Loosen permissions most recent backup file (to prepare to copy)

Code: Select all

sudo chmod +r /var/EFA/backup/<backup-MMDDYYYY-HHMMSS.tar.gz>
Where <backup-MMDDYYYY-HHMMSS.tar.gz> is the name of the backup file

3) Copy off the backup file using your favorite tool, such as WinSCP.

4) Restrict the backup file after copy (optional, but recommended)

Code: Select all

sudo chmod go-r /var/EFA/backup/<backup-MMDDYYYY-HHMMSS.tar.gz>
Where <backup-MMDDYYYY-HHMMSS.tar.gz> is the name of the backup file

5) If you plan to keep the existing quarantine, copy all of /var/spool/MailScanner/quarantine (optional and may take a while, depending on size)

6) Document settings (hint: look at /etc/EFA-Config)

If any customizations were made, now is a good time to document them to reapply them.

7) Shut down old appliance (Pro Tip: use hypervisor to set NIC to an isolated net so that you can power on again and reference later)

8) Deploy a new appliance (same version)

9) Configure new appliance with the same ip address and settings as the original using EFA Intial Configuration.

10) Use WinSCP or similar to copy the backup file to the new appliance.

Note that you will likely get a Security Breach warning. This is because the new appliance does not have the same public key at the moment. Ignore/Accept/Update and continue.

11) Move the backup file to the backup area, if necessary

Code: Select all

sudo mv /home/<efauser>/<backup-MMDDYYYY-HHMMSS.tar.gz> /var/EFA/backup
12) Patch func_systemrestore to allow for importing

Code: Select all

   sudo wget -O /var/EFA/lib/EFA-Configure/func_systemrestore https://raw.githubusercontent.com/E-F-A/v3/master/build/EFA/lib-EFA-Configure/func_systemrestore
13) Execute EFA-Configure and perform a System Restore using the backup file

Code: Select all

sudo /usr/local/sbin/EFA-Configure
14) Keep the old appliance around for a while, if you can, just in case we missed something :dance:
CatHerder
Posts: 2
Joined: 11 Jun 2016 13:48

Re: Best way to backup and restore?

Post by CatHerder »

Shawn,
Thanks a million for this! I hope to be able to try to run through this and get you feedback tonight or tomorrow night.

Really appreciate it!
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

Hi,

I am just forced to do a complete restore from a still running install (3.0.2.3). The setup is a manual build running as VM on XenServer 6.5SP1.

While trying to perform the announced update to version 3.0.2.4 it came out that the file system of the VM is heavily damaged and could not be repaired . (The source for this issue is based on the live migrations of the running VM to a different XenServer host with slightly different CPU.)

Now I am building again a replacment VM with the goal to preserve all the settings and quarantine messages. I will follow the directions given here, but I have one problem. The new build is the current version and the backup files are from the previous one.

Could this work? :?
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

ajmind wrote: 28 Sep 2017 14:39 Hi,

Now I am building again a replacment VM with the goal to preserve all the settings and quarantine messages. I will follow the directions given here, but I have one problem. The new build is the current version and the backup files are from the previous one.
I think we need an archive of previous versions. You are running your own build, right?

Use this kickstart: http://dl.efa-project.org/build/3.0.2.3/ks.cfg
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

My damaged install is coming from a version before 3.0.1.9 with mysql DB

the restore script breaks on several points due to the lack of a mysql db.

could you help again?
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

I have started now a kickstart install of version 3.0.1.8 which has mysql still in use...
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

Precisely which version is the broken install?
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

The broken install is version 3.0.2.3
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

ajmind wrote: 29 Sep 2017 13:44 My damaged install is coming from a version before 3.0.1.9 with mysql DB
The broken install is version 3.0.2.3
I'm confused.
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

Why confused?

The original install was made with a version <3.0.1.8 and was subsequently updated until version 3.0.2.3.

Pre 3.0.1.9 versions were using mysql, beginning with 3.0.1.9 mariadb was introduced.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

So why doesn't the backup/restore above work on 3.0.2.3? It is using mariadb.
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

After the initial setup with EFA-Configure of the new appliance and copying the backup with the restore script no mysql daemon was running nor available.

I could simply not run the DB from the backup!
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

It sounds like you do do not have a good build, and something went wrong during attempting to build 3.0.2.3.

When you run the build, everything is logged to /var/log/EFA and can be retrieved prior to running the initial setup. You may want to try rebuilding and reviewing this log before proceeding.
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

Ups, I grabbed the Centos minmal 6.8 instead of 6.9!

I will create a new VM with the correct CentOs version ...

So new attempt but no success, the build.log shows

Code: Select all

Loaded plugins: fastestmirror, security                                         
Setting up Install Process                                                      
Loading mirror speeds from cached hostfile                                      
 * EFA: dl.efa-project.org                                                      
 * base: ftp.fau.de                                                             
 * epel: ftp.fau.de                                                             
 * extras: mirror.wiuwiu.de                                                     
 * updates: centos.intergenia.de                                                
[b]No package MariaDB-server available.                                            
No package MariaDB-client available.  [/b]    
Therefore I could not reach my goal :-(

I build now my own VM with Centos minimal 6.9 and amend the build script to use the 3.0.2.3 version.

We will see what happens.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

How are you calling the build script, and which build script are you using?
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

Finally my VM is back and running smoothly :-)

It was a long and time consuming journey and I have learned a lot about kickstart and eFa!

As the setup with Kickstart has failed due to the non availabilty of a mariaDB mirror I have started to build it according the setup Guide:
https://wiki.efa-project.org/doku.php? ... stallation

After this step I have modified the content of this script in order to use the 3.0.2.3 version:

https://raw.githubusercontent.com/E-F-A ... ut-ks.bash

Some of my configurations were not part of the eFa backup so I have copied them manually into the correct locations.

A big issue was that no mails were going through eFa until I have figured out that I do have to postmap some files from the /etc/postfix directory.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

Can you share which configurations you had to copy?
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

I will report the steps tomorrow, I need first a break from eFa 😉
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

My restore process with some problems from the EFA Backup:

1. Import of quarantine:
5) If you plan to keep the existing quarantine, copy all of /var/spool/MailScanner/quarantine
From the old VM:

Code: Select all

drwxr-xr-x 34 postfix sqlgrey 4096 29. Sep 04:22 quarantine
The new VM:

Code: Select all

drwxr-xr-x. 32 postfix mtagroup 4096  4. Okt 04:00 quarantine
So manually copying files needs the correction with

Code: Select all

chown -R postfix:mtagroup /var/spool/MailScanner/quarantine/ 
2. Missing files in Backup:

Code: Select all

lrwxrwxrwx 1 root root       30 28. Sep 11:35 reports -> /usr/share/MailScanner/reports/[language folder]
this folder and content is not part of the EFA-backup, so language specific customization is lost.

I have copied them from my own backup.

3. Postfix problems

no mails are going in and out, quite a few errors are reported in

Code: Select all

\var\log\maillog
postfix ist claiming about missing or the age of

*.db files located in

Code: Select all


/etc/postfix

client_access
client_access.db
helo_access.db
recipient_access
recipient_access.db
sasl_passwd
sasl_passwd.db
sender_access
sender_access.db
transport
transport.db
virtual
virtual.db
You should know what to do, simply

Code: Select all

[root@IT1MAILGW2 postfix]# postmap /etc/postfix/client_access
for all flat files with the same name as *.db files.

That's it so far.

PS: One thing missed which is only nice to have:

The mailgraph statistics from the old VM are missing and I do not know where they located and how to restore...
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

One open issue after restore has shown up today:

Code: Select all

/etc/cron.weekly/trim-sql-awl-weekly:

ERROR 1045 (28000): Access denied for user 'sa_user'@'localhost' (using
password: YES)
Any hint how to solve this?
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Best way to backup and restore?

Post by shawniverson »

Open that cron job and see if there is a password embedded in it.

If there is, change it to the one in /etc/EFA-Config.

Let me know if this works.
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Best way to backup and restore?

Post by ajmind »

Yep, it works.

The file

Code: Select all

/usr/sbin/trim-awk
contains a password, which was not coming from the recent restore backup. (Simply not part of the backup process.)
User avatar
BOOZy
Posts: 39
Joined: 04 Oct 2017 13:17

Re: Best way to backup and restore?

Post by BOOZy »

If you're using VMWare, why aren't you using Veeam?
Or the free edition of you can't afford it.
Post Reply