efa-project.org


https://forum.efa-project.org/

Best way to backup and restore?

https://forum.efa-project.org/viewtopic.php?t=2703

Page 1 of 2

Best way to backup and restore?

Posted: 17 Sep 2017 16:56
by CatHerder
So I’ve been a very happy EFA user/admin for years. I installed the VMware image a million years ago, and have been updating ever since. Most, but not all, of the upgrades were clean. However today, in updating to the latest, /some/ of thekernel updates failed due to an out of memory error, and after I rebooted to clean things up, the actual application updates ran leaving the kernel updates in a questionable state.

Now, everything /seems/ to be fine, but I’m a bit nervous about the systems state moving forward. Hence, what I’d really like to do is backup my configuration, install a new VM with the latest image, and restore the configuration from backup.

What’s the best process to do that?

Thanks!

Re: Best way to backup and restore?

Posted: 18 Sep 2017 00:18
by shawniverson
This deserves a how-to. I think I'll take a stab at it :D

Here we go...note this is involved, since EFA-Backup is not yet portable without some help (https://github.com/E-F-A/v3/issues/324)

1) Run EFA-Backup

Code: Select all

sudo /usr/local/sbin/EFA-Backup -backup
2) Loosen permissions most recent backup file (to prepare to copy)

Code: Select all

sudo chmod +r /var/EFA/backup/<backup-MMDDYYYY-HHMMSS.tar.gz>
Where <backup-MMDDYYYY-HHMMSS.tar.gz> is the name of the backup file

3) Copy off the backup file using your favorite tool, such as WinSCP.

4) Restrict the backup file after copy (optional, but recommended)

Code: Select all

sudo chmod go-r /var/EFA/backup/<backup-MMDDYYYY-HHMMSS.tar.gz>
Where <backup-MMDDYYYY-HHMMSS.tar.gz> is the name of the backup file

5) If you plan to keep the existing quarantine, copy all of /var/spool/MailScanner/quarantine (optional and may take a while, depending on size)

6) Document settings (hint: look at /etc/EFA-Config)

If any customizations were made, now is a good time to document them to reapply them.

7) Shut down old appliance (Pro Tip: use hypervisor to set NIC to an isolated net so that you can power on again and reference later)

8) Deploy a new appliance (same version)

9) Configure new appliance with the same ip address and settings as the original using EFA Intial Configuration.

10) Use WinSCP or similar to copy the backup file to the new appliance.

Note that you will likely get a Security Breach warning. This is because the new appliance does not have the same public key at the moment. Ignore/Accept/Update and continue.

11) Move the backup file to the backup area, if necessary

Code: Select all

sudo mv /home/<efauser>/<backup-MMDDYYYY-HHMMSS.tar.gz> /var/EFA/backup
12) Patch func_systemrestore to allow for importing

Code: Select all

   sudo wget -O /var/EFA/lib/EFA-Configure/func_systemrestore https://raw.githubusercontent.com/E-F-A/v3/master/build/EFA/lib-EFA-Configure/func_systemrestore
13) Execute EFA-Configure and perform a System Restore using the backup file

Code: Select all

sudo /usr/local/sbin/EFA-Configure
14) Keep the old appliance around for a while, if you can, just in case we missed something :dance:

Re: Best way to backup and restore?

Posted: 18 Sep 2017 13:37
by CatHerder
Shawn,
Thanks a million for this! I hope to be able to try to run through this and get you feedback tonight or tomorrow night.

Really appreciate it!

Re: Best way to backup and restore?

Posted: 28 Sep 2017 14:39
by ajmind
Hi,

I am just forced to do a complete restore from a still running install (3.0.2.3). The setup is a manual build running as VM on XenServer 6.5SP1.

While trying to perform the announced update to version 3.0.2.4 it came out that the file system of the VM is heavily damaged and could not be repaired . (The source for this issue is based on the live migrations of the running VM to a different XenServer host with slightly different CPU.)

Now I am building again a replacment VM with the goal to preserve all the settings and quarantine messages. I will follow the directions given here, but I have one problem. The new build is the current version and the backup files are from the previous one.

Could this work? :?

Re: Best way to backup and restore?

Posted: 28 Sep 2017 20:36
by shawniverson
ajmind wrote: 28 Sep 2017 14:39 Hi,

Now I am building again a replacment VM with the goal to preserve all the settings and quarantine messages. I will follow the directions given here, but I have one problem. The new build is the current version and the backup files are from the previous one.
I think we need an archive of previous versions. You are running your own build, right?

Use this kickstart: http://dl.efa-project.org/build/3.0.2.3/ks.cfg

Re: Best way to backup and restore?

Posted: 29 Sep 2017 13:44
by ajmind
My damaged install is coming from a version before 3.0.1.9 with mysql DB

the restore script breaks on several points due to the lack of a mysql db.

could you help again?

Re: Best way to backup and restore?

Posted: 29 Sep 2017 20:41
by ajmind
I have started now a kickstart install of version 3.0.1.8 which has mysql still in use...

Re: Best way to backup and restore?

Posted: 29 Sep 2017 21:08
by shawniverson
Precisely which version is the broken install?

Re: Best way to backup and restore?

Posted: 30 Sep 2017 04:19
by ajmind
The broken install is version 3.0.2.3

Re: Best way to backup and restore?

Posted: 30 Sep 2017 12:35
by shawniverson
ajmind wrote: 29 Sep 2017 13:44 My damaged install is coming from a version before 3.0.1.9 with mysql DB
The broken install is version 3.0.2.3
I'm confused.

Re: Best way to backup and restore?

Posted: 30 Sep 2017 15:17
by ajmind
Why confused?

The original install was made with a version <3.0.1.8 and was subsequently updated until version 3.0.2.3.

Pre 3.0.1.9 versions were using mysql, beginning with 3.0.1.9 mariadb was introduced.

Re: Best way to backup and restore?

Posted: 30 Sep 2017 15:52
by shawniverson
So why doesn't the backup/restore above work on 3.0.2.3? It is using mariadb.

Re: Best way to backup and restore?

Posted: 30 Sep 2017 17:53
by ajmind
After the initial setup with EFA-Configure of the new appliance and copying the backup with the restore script no mysql daemon was running nor available.

I could simply not run the DB from the backup!

Re: Best way to backup and restore?

Posted: 01 Oct 2017 12:57
by shawniverson
It sounds like you do do not have a good build, and something went wrong during attempting to build 3.0.2.3.

When you run the build, everything is logged to /var/log/EFA and can be retrieved prior to running the initial setup. You may want to try rebuilding and reviewing this log before proceeding.

Re: Best way to backup and restore?

Posted: 01 Oct 2017 19:58
by ajmind
Ups, I grabbed the Centos minmal 6.8 instead of 6.9!

I will create a new VM with the correct CentOs version ...

So new attempt but no success, the build.log shows

Code: Select all

Loaded plugins: fastestmirror, security                                         
Setting up Install Process                                                      
Loading mirror speeds from cached hostfile                                      
 * EFA: dl.efa-project.org                                                      
 * base: ftp.fau.de                                                             
 * epel: ftp.fau.de                                                             
 * extras: mirror.wiuwiu.de                                                     
 * updates: centos.intergenia.de                                                
[b]No package MariaDB-server available.                                            
No package MariaDB-client available.  [/b]
Therefore I could not reach my goal :-(

I build now my own VM with Centos minimal 6.9 and amend the build script to use the 3.0.2.3 version.

We will see what happens.

Re: Best way to backup and restore?

Posted: 02 Oct 2017 11:15
by shawniverson
How are you calling the build script, and which build script are you using?

Re: Best way to backup and restore?

Posted: 02 Oct 2017 11:27
by ajmind
Finally my VM is back and running smoothly :-)

It was a long and time consuming journey and I have learned a lot about kickstart and eFa!

As the setup with Kickstart has failed due to the non availabilty of a mariaDB mirror I have started to build it according the setup Guide:
https://wiki.efa-project.org/doku.php? ... stallation

After this step I have modified the content of this script in order to use the 3.0.2.3 version:

https://raw.githubusercontent.com/E-F-A ... ut-ks.bash

Some of my configurations were not part of the eFa backup so I have copied them manually into the correct locations.

A big issue was that no mails were going through eFa until I have figured out that I do have to postmap some files from the /etc/postfix directory.

Re: Best way to backup and restore?

Posted: 02 Oct 2017 13:44
by shawniverson
Can you share which configurations you had to copy?

Re: Best way to backup and restore?

Posted: 02 Oct 2017 13:45
by shawniverson
https://github.com/E-F-A/v3/issues/395

Re: Best way to backup and restore?

Posted: 03 Oct 2017 08:41
by ajmind
I will report the steps tomorrow, I need first a break from eFa 😉

Re: Best way to backup and restore?

Posted: 04 Oct 2017 14:46
by ajmind
My restore process with some problems from the EFA Backup:

1. Import of quarantine:
5) If you plan to keep the existing quarantine, copy all of /var/spool/MailScanner/quarantine
From the old VM:

Code: Select all

drwxr-xr-x 34 postfix sqlgrey 4096 29. Sep 04:22 quarantine
The new VM:

Code: Select all

drwxr-xr-x. 32 postfix mtagroup 4096  4. Okt 04:00 quarantine
So manually copying files needs the correction with

Code: Select all

chown -R postfix:mtagroup /var/spool/MailScanner/quarantine/
2. Missing files in Backup:

Code: Select all

lrwxrwxrwx 1 root root       30 28. Sep 11:35 reports -> /usr/share/MailScanner/reports/[language folder]
this folder and content is not part of the EFA-backup, so language specific customization is lost.

I have copied them from my own backup.

3. Postfix problems

no mails are going in and out, quite a few errors are reported in

Code: Select all

\var\log\maillog
postfix ist claiming about missing or the age of

*.db files located in

Code: Select all


/etc/postfix

client_access
client_access.db
helo_access.db
recipient_access
recipient_access.db
sasl_passwd
sasl_passwd.db
sender_access
sender_access.db
transport
transport.db
virtual
virtual.db
You should know what to do, simply

Code: Select all

[root@IT1MAILGW2 postfix]# postmap /etc/postfix/client_access
for all flat files with the same name as *.db files.

That's it so far.

PS: One thing missed which is only nice to have:

The mailgraph statistics from the old VM are missing and I do not know where they located and how to restore...

Re: Best way to backup and restore?

Posted: 09 Oct 2017 08:17
by ajmind
One open issue after restore has shown up today:

Code: Select all

/etc/cron.weekly/trim-sql-awl-weekly:

ERROR 1045 (28000): Access denied for user 'sa_user'@'localhost' (using
password: YES)
Any hint how to solve this?

Re: Best way to backup and restore?

Posted: 09 Oct 2017 17:15
by shawniverson
Open that cron job and see if there is a password embedded in it.

If there is, change it to the one in /etc/EFA-Config.

Let me know if this works.

Re: Best way to backup and restore?

Posted: 10 Oct 2017 09:40
by ajmind
Yep, it works.

The file

Code: Select all

/usr/sbin/trim-awk
contains a password, which was not coming from the recent restore backup. (Simply not part of the backup process.)

Re: Best way to backup and restore?

Posted: 09 Nov 2017 09:56
by BOOZy
If you're using VMWare, why aren't you using Veeam?
Or the free edition of you can't afford it.
All times are UTC
Page 1 of 2

Powered by phpBB® Forum Software © phpBB Limited