Best way to backup and restore?

[CatHerder]

So I’ve been a very happy EFA user/admin for years. I installed the VMware image a million years ago, and have been updating ever since. Most, but not all, of the upgrades were clean. However today, in updating to the latest, /some/ of thekernel updates failed due to an out of memory error, and after I rebooted to clean things up, the actual application updates ran leaving the kernel updates in a questionable state.

Now, everything /seems/ to be fine, but I’m a bit nervous about the systems state moving forward. Hence, what I’d really like to do is backup my configuration, install a new VM with the latest image, and restore the configuration from backup.

What’s the best process to do that?

Thanks!

[shawniverson]

This deserves a how-to. I think I'll take a stab at it

Here we go...note this is involved, since EFA-Backup is not yet portable without some help (https://github.com/E-F-A/v3/issues/324)

1) Run EFA-Backup

Code: Select all

sudo /usr/local/sbin/EFA-Backup -backup

2) Loosen permissions most recent backup file (to prepare to copy)

Code: Select all

sudo chmod +r /var/EFA/backup/<backup-MMDDYYYY-HHMMSS.tar.gz>

Where <backup-MMDDYYYY-HHMMSS.tar.gz> is the name of the backup file

3) Copy off the backup file using your favorite tool, such as WinSCP.

4) Restrict the backup file after copy (optional, but recommended)

Code: Select all

sudo chmod go-r /var/EFA/backup/<backup-MMDDYYYY-HHMMSS.tar.gz>

Where <backup-MMDDYYYY-HHMMSS.tar.gz> is the name of the backup file

5) If you plan to keep the existing quarantine, copy all of /var/spool/MailScanner/quarantine (optional and may take a while, depending on size)

6) Document settings (hint: look at /etc/EFA-Config)

If any customizations were made, now is a good time to document them to reapply them.

7) Shut down old appliance (Pro Tip: use hypervisor to set NIC to an isolated net so that you can power on again and reference later)

8) Deploy a new appliance (same version)

9) Configure new appliance with the same ip address and settings as the original using EFA Intial Configuration.

10) Use WinSCP or similar to copy the backup file to the new appliance.

Note that you will likely get a Security Breach warning. This is because the new appliance does not have the same public key at the moment. Ignore/Accept/Update and continue.

11) Move the backup file to the backup area, if necessary

Code: Select all

sudo mv /home/<efauser>/<backup-MMDDYYYY-HHMMSS.tar.gz> /var/EFA/backup

12) Patch func_systemrestore to allow for importing

Code: Select all

   sudo wget -O /var/EFA/lib/EFA-Configure/func_systemrestore https://raw.githubusercontent.com/E-F-A/v3/master/build/EFA/lib-EFA-Configure/func_systemrestore

13) Execute EFA-Configure and perform a System Restore using the backup file

Code: Select all

sudo /usr/local/sbin/EFA-Configure

14) Keep the old appliance around for a while, if you can, just in case we missed something

[CatHerder]

Shawn,
Thanks a million for this! I hope to be able to try to run through this and get you feedback tonight or tomorrow night.

Really appreciate it!

[ajmind]

Hi,

I am just forced to do a complete restore from a still running install (3.0.2.3). The setup is a manual build running as VM on XenServer 6.5SP1.

While trying to perform the announced update to version 3.0.2.4 it came out that the file system of the VM is heavily damaged and could not be repaired . (The source for this issue is based on the live migrations of the running VM to a different XenServer host with slightly different CPU.)

Now I am building again a replacment VM with the goal to preserve all the settings and quarantine messages. I will follow the directions given here, but I have one problem. The new build is the current version and the backup files are from the previous one.

Could this work?

[shawniverson]

Hi,

Now I am building again a replacment VM with the goal to preserve all the settings and quarantine messages. I will follow the directions given here, but I have one problem. The new build is the current version and the backup files are from the previous one.

I think we need an archive of previous versions. You are running your own build, right?

Use this kickstart: http://dl.efa-project.org/build/3.0.2.3/ks.cfg

[ajmind]

My damaged install is coming from a version before 3.0.1.9 with mysql DB

the restore script breaks on several points due to the lack of a mysql db.

could you help again?

[ajmind]

I have started now a kickstart install of version 3.0.1.8 which has mysql still in use...

[shawniverson]

Precisely which version is the broken install?

[ajmind]

The broken install is version 3.0.2.3

[shawniverson]

My damaged install is coming from a version before 3.0.1.9 with mysql DB

The broken install is version 3.0.2.3

I'm confused.

[ajmind]

Why confused?

The original install was made with a version <3.0.1.8 and was subsequently updated until version 3.0.2.3.

Pre 3.0.1.9 versions were using mysql, beginning with 3.0.1.9 mariadb was introduced.

[shawniverson]

So why doesn't the backup/restore above work on 3.0.2.3? It is using mariadb.

[ajmind]

After the initial setup with EFA-Configure of the new appliance and copying the backup with the restore script no mysql daemon was running nor available.

I could simply not run the DB from the backup!

[shawniverson]

It sounds like you do do not have a good build, and something went wrong during attempting to build 3.0.2.3.

When you run the build, everything is logged to /var/log/EFA and can be retrieved prior to running the initial setup. You may want to try rebuilding and reviewing this log before proceeding.

[ajmind]

Ups, I grabbed the Centos minmal 6.8 instead of 6.9!

I will create a new VM with the correct CentOs version ...

So new attempt but no success, the build.log shows

Code: Select all

Loaded plugins: fastestmirror, security                                         
Setting up Install Process                                                      
Loading mirror speeds from cached hostfile                                      
 * EFA: dl.efa-project.org                                                      
 * base: ftp.fau.de                                                             
 * epel: ftp.fau.de                                                             
 * extras: mirror.wiuwiu.de                                                     
 * updates: centos.intergenia.de                                                
[b]No package MariaDB-server available.                                            
No package MariaDB-client available.  [/b]    

Therefore I could not reach my goal

I build now my own VM with Centos minimal 6.9 and amend the build script to use the 3.0.2.3 version.

We will see what happens.

[shawniverson]

How are you calling the build script, and which build script are you using?

[ajmind]

Finally my VM is back and running smoothly

It was a long and time consuming journey and I have learned a lot about kickstart and eFa!

As the setup with Kickstart has failed due to the non availabilty of a mariaDB mirror I have started to build it according the setup Guide:
https://wiki.efa-project.org/doku.php? ... stallation

After this step I have modified the content of this script in order to use the 3.0.2.3 version:

https://raw.githubusercontent.com/E-F-A ... ut-ks.bash

Some of my configurations were not part of the eFa backup so I have copied them manually into the correct locations.

A big issue was that no mails were going through eFa until I have figured out that I do have to postmap some files from the /etc/postfix directory.

[shawniverson]

Can you share which configurations you had to copy?

[shawniverson]

https://github.com/E-F-A/v3/issues/395

[ajmind]

I will report the steps tomorrow, I need first a break from eFa

[ajmind]

My restore process with some problems from the EFA Backup:

1. Import of quarantine:

5) If you plan to keep the existing quarantine, copy all of /var/spool/MailScanner/quarantine

From the old VM:

Code: Select all

drwxr-xr-x 34 postfix sqlgrey 4096 29. Sep 04:22 quarantine

The new VM:

Code: Select all

drwxr-xr-x. 32 postfix mtagroup 4096  4. Okt 04:00 quarantine

So manually copying files needs the correction with

Code: Select all

chown -R postfix:mtagroup /var/spool/MailScanner/quarantine/ 

2. Missing files in Backup:

Code: Select all

lrwxrwxrwx 1 root root       30 28. Sep 11:35 reports -> /usr/share/MailScanner/reports/[language folder]

this folder and content is not part of the EFA-backup, so language specific customization is lost.

I have copied them from my own backup.

3. Postfix problems

no mails are going in and out, quite a few errors are reported in

Code: Select all

\var\log\maillog

postfix ist claiming about missing or the age of

*.db files located in

Code: Select all

/etc/postfix

client_access
client_access.db
helo_access.db
recipient_access
recipient_access.db
sasl_passwd
sasl_passwd.db
sender_access
sender_access.db
transport
transport.db
virtual
virtual.db

You should know what to do, simply

Code: Select all

[root@IT1MAILGW2 postfix]# postmap /etc/postfix/client_access

for all flat files with the same name as *.db files.

That's it so far.

PS: One thing missed which is only nice to have:

The mailgraph statistics from the old VM are missing and I do not know where they located and how to restore...

[ajmind]

One open issue after restore has shown up today:

Code: Select all

/etc/cron.weekly/trim-sql-awl-weekly:

ERROR 1045 (28000): Access denied for user 'sa_user'@'localhost' (using
password: YES)

Any hint how to solve this?

[shawniverson]

Open that cron job and see if there is a password embedded in it.

If there is, change it to the one in /etc/EFA-Config.

Let me know if this works.

[ajmind]

Yep, it works.

The file

Code: Select all

/usr/sbin/trim-awk

contains a password, which was not coming from the recent restore backup. (Simply not part of the backup process.)

[BOOZy]

If you're using VMWare, why aren't you using Veeam?
Or the free edition of you can't afford it.