Best method to deploy new EFA v4

[MattS]

The time has finally come where we're going to deploy a replacement for our v3 based EFA server and I'm wondering what's the most straightforward reliable way to deploy a new install. I've followed the evolution of v4 but have also seen numerous fixes and tweaks needed to configuration files post-install to fix bugs and whilst that seems to have mostly settled down now, I wonder if that applies to all types of install? I'm going to be doing this entirely remotely so would much rather not have to wrestle with too many post-install problems. On the bright side, I'm not bothered about migrating anything from our existing v3 install so no messing around trying to migrate blacklist/whitelists, etc

Yes, that probably does equate to me being a bit lazy and seeking the easiest path but needs must.

[MattS]

So I ended up going with the curl method which seemed to work fine on a CentOS 7.8-2003 Minimal install on a fresh VM.

Only issue I've had is downloading the GeoIP2 lite database. I've obtained a licence key and installed that through EFA-Configure okay but when I try to download it through the web interface I constantly get a curl timeout error:

Downloading file, please wait...
Error occurred while downloading GeoIP data file: cURL error 28: Operation timed out after 10001 milliseconds with 1518595 out of 1931948 bytes received
Download complete, unpacking files...

There is a partial file download but it's obviously corrupt and can't be unzipped/installed. Is there any way I can change the curl timeout to allow it to fully download? Or is there a way to install via the CLI?

[MattS]

Hmmmm. Maybe it hasn't gone so well after all.

Still got whatever this geoip download problem is which might be why I can't see anything listed in the recent messages screen (assuming it behaves in a similar way as it did in 3.x).

But, I thought I'd test emails from an external source before changing any MX records and used Wormly to test an email to the IP address

Code: Select all

Connecting...
Connection: opening to 45.75.xxx.xxx:25, timeout=300, options=array (
      	         )
Connection: opened
SERVER -> CLIENT: 220 mx.obfuscated.com ESMTP Postfix
CLIENT -> SERVER: EHLO tools.wormly.com
SERVER -> CLIENT: 250-mx.obfuscated.com
      	         250-PIPELINING
      	         250-SIZE 133169152
      	         250-ETRN
      	         250-STARTTLS
      	         250-ENHANCEDSTATUSCODES
      	         250-8BITMIME
      	         250-DSN
      	         250-SMTPUTF8
      	         250 CHUNKING
CLIENT -> SERVER: STARTTLS
SERVER -> CLIENT: 220 2.0.0 Ready to start TLS
2020-06-14 23:02:15	SMTP Error: Could not connect to SMTP host.
CLIENT -> SERVER: QUIT
SERVER -> CLIENT: MIA
      	         �Ni`f~D[oޟ/u��
      	         ����6�(\>GO?Vz�t�v�#�p='m�t��[s���iq�S�����9oMZܳȂa(rn��J����9Ic݉�F���s��.��Ƭ`�7�{AhčxE�Zi���3.��ܡ���-G:�%W�c$�Lq�d��c]��"�H�C�4!�dȺ=�P~�#�=A��񻆁�G�,���YY����妌0��ρqbx�)E��"}4i5�$�ڈ?l8&�co'�٬߰�D�Lb�S,C��G��\eik��̸߫��(t�E�R�4�6xݑ�-��=xw7�J�\�A1e��q~�
SMTP ERROR: QUIT command failed: MIA
      	         �Ni`f~D[oޟ/u��
      	         ����6�(\>GO?Vz�t�v�#�p='m�t��[s���iq�S�����9oMZܳȂa(rn��J����9Ic݉�F���s��.��Ƭ`�7�{AhčxE�Zi���3.��ܡ���-G:�%W�c$�Lq�d��c]��"�H�C�4!�dȺ=�P~�#�=A��񻆁�G�,���YY����妌0��ρqbx�)E��"}4i5�$�ڈ?l8&�co'�٬߰�D�Lb�S,C��G��\eik��̸߫��(t�E�R�4�6xݑ�-��=xw7�J�\�A1e��q~�
Connection: closed
2020-06-14 23:02:15	SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting
Message sending failed.

So it looks like it's doing it's initial handshake with Postfix okay then it's going wrong somewhere further down the line.

maillog on the efa machine looks like this:

Code: Select all

Jun 15 00:02:15 mx postfix/smtpd[27471]: connect from tools.wormly.com[172.104.20.135]
Jun 15 00:02:15 mx postfix/smtpd[27471]: SSL_accept error from tools.wormly.com[172.104.20.135]: 0
Jun 15 00:02:15 mx postfix/smtpd[27471]: warning: TLS library problem: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca:s3_pkt.c:1493:SSL alert number 48:
Jun 15 00:02:15 mx postfix/smtpd[27471]: lost connection after STARTTLS from tools.wormly.com[172.104.20.135]
Jun 15 00:02:15 mx postfix/smtpd[27471]: disconnect from tools.wormly.com[172.104.20.135] ehlo=1 starttls=0/1 commands=1/2

[MattS]

Looks like I have the same problem with my existing 3.2.6 install so maybe it's actually a Wormly issue or something with SSL configuration common to 3.2.6 and 4.0.2.

[henk]

Hi MattS,

As you did provide minimal details...

Since you are not able to download the GeoIP2 lite database, did you check DNS (Unbound) after the initial install?


(I've done several Efa4 kickstart installations, with the CentOS-7-x86_64-Minimal-1908.iso, without issues.)

[pdwalker]

I just recently installed efa4 in parallel with my previous efa3 installation.

I did a complete clean install, and then made sure everything was working first.

Once I did that, then some unimportant domains were configured to go through the new box where I could then verify the operation and make whatever fixes were necessary

And then when I was happy, I switched everything over while leaving my v3 box running - just in case.

If there are still no issues, I'll shutdown the efa3 box at the end of the month.

viewtopic.php?f=5&t=4325

The log files are your friends - check them to make sure there are no weird errors. And enable fail2ban to cut out the unnecessary requests in maillog.

[MattS]

Hi MattS,

As you did provide minimal details...

Since you are not able to download the GeoIP2 lite database, did you check DNS (Unbound) after the initial install?


(I've done several Efa4 kickstart installations, with the CentOS-7-x86_64-Minimal-1908.iso, without issues.)

Thanks Henk. Not sure how to check unbound is working correctly but doing an nslookup via localhost resolves external names okay so I assume that means it's doing something properly.

I _think_ this curl problem with GeoIP2 is purely a bandwidth issue and curl crapping out if it takes 10 seconds or longer to download the file. I get a varying file sized between 1.5mb and 1.7mb but it always dies at 10000ms. A manual work around would be handy though as I could force curl to allow more time. For some reason, the datacentre has only provisioned a 1mbps connection (worse still it's capped and not burstable) to this rack and there's already a fair amount of background chatter over this connection that will be eating into that 1Mbps. They're currently claiming it might take as long as two weeks for them to flick the proverbial switch to upgrade it to something sensible. Painful when I've got a 500Mbps/150mbps connection here at home.

Any idea on the TLS issue? I'm assuming/hoping it's probably because I'm not using LetsEncrypt.

[MattS]

viewtopic.php?f=5&t=4325

The log files are your friends - check them to make sure there are no weird errors. And enable fail2ban to cut out the unnecessary requests in maillog.

I'd been watching with interest, knowing the potential for some pain was fast approaching.

[pdwalker]

If unbound is working, then the following command should return the following results:

Code: Select all

[root@efa4 ~]# dig -t txt 2.0.0.127.multi.uribl.com

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-16.P2.el7_8.6 <<>> -t txt 2.0.0.127.multi.uribl.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 57955
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1480
;; QUESTION SECTION:
;2.0.0.127.multi.uribl.com.	IN	TXT

;; AUTHORITY SECTION:
multi.uribl.com.	195	IN	SOA	uribl.com. admins.dnswl.org. 1461111961 7200 7200 604800 60

;; Query time: 5 msec
;; SERVER: 10.10.1.1#53(10.10.1.1)
;; WHEN: Tue Jun 16 15:11:05 HKT 2020
;; MSG SIZE  rcvd: 106

If unbound is not configured properly, then you'll see this... (I'm querying a public dns server so the query will fail - just like unbound will fail if it is not configured with recursion properly)

Code: Select all

[root@efa4 ~]# dig -t txt 2.0.0.127.multi.uribl.com @1.1.1.1

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-16.P2.el7_8.6 <<>> -t txt 2.0.0.127.multi.uribl.com @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2522
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1452
;; QUESTION SECTION:
;2.0.0.127.multi.uribl.com.	IN	TXT

;; ANSWER SECTION:
2.0.0.127.multi.uribl.com. 2100	IN	TXT	"127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 108.162.223.80]"

;; Query time: 231 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Tue Jun 16 15:11:20 HKT 2020
;; MSG SIZE  rcvd: 205

[pdwalker]

I'd been watching with interest, knowing the potential for some pain was fast approaching.

It was more painful than I wanted it to be, but much less painful than I thought it would be because efa4 takes care of a lot of things automatically that I had to setup manually in efa3.

It was also a good exercise in cleaning up and documenting my efa installation a little better.

[henk]

There's a simple basic test script to check DNS . viewtopic.php?t=3032