Page 1 of 1

Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

Posted: 29 Jan 2020 19:05
by ajmind
This morning trouble started on our eFa 4.x box:

Code: Select all

 LOCAL: Could not create socket directory: /var/run/clamd.socket: Permission denied
Jan 29 19:16:38  clamd[7374]: ERROR: LOCAL: Could not create socket directory: /var/run/clamd.socket: Permission denied
Jan 29 19:16:38  clamd[7374]: ERROR: LOCAL: Socket file /var/run/clamd.socket/clamd.sock could not be bound: No such file or directory
Jan 29 19:16:38  clamd[7374]: LOCAL: Socket file /var/run/clamd.socket/clamd.sock could not be bound: No such file or directory
Regardless what I am reading and doing this problem persists. No messages are processed as

Code: Select all

postfix/master[23237]: warning: process /usr/libexec/postfix/pickup pid 20458 exit status 127
Jan 29 10:21:19 IT1MAILGW1 postfix/master[23237]: warning: /usr/libexec/postfix/pickup: bad command startup -- throttling
How to solve this issue as no e-mails are getting through this box anymore.
Could I disable virus scanning that messages are delivered. (As a temporary solution!).

Uptated subject due to further examination of problem

Problems have been identified after a reboot of the EFa VM:

Code: Select all

[ERROR] mysqld: Can't create/write to file '/run/mariadb/mariadb.pid' (Errcode: 2 "No such file or directory")
2020-01-29 11:49:54 140330304825088 [ERROR] Can't start server: can't create PID file: No such file or directory

Code: Select all

[suexec:notice] [pid 1571:tid 140156932184256] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Wed Jan 29 11:49:52.571676 2020] [cgid:error] [pid 1647:tid 140156932184256] (2)No such file or directory: AH01243: Couldn't bind unix domain socket /run/httpd/cgisock.1571

[Wed Jan 29 11:49:52.572863 2020] [core:error] [pid 1571:tid 140156932184256] (2)No such file or directory: AH00099: could not create /run/httpd/httpd.pid
[Wed Jan 29 11:49:52.572893 2020] [core:error] [pid 1571:tid 140156932184256] AH00100: httpd: could not log pid to file /run/httpd/httpd.pid
Manually creating the required dirs with correct permissions clears the error (mariadb, httpd), but
clamd@scan claims still not be able to create

Code: Select all

/var/run/clamd.socket: Permission denied
even it is there and the permission is set to

Code: Select all

clamscan:mtagroup
A further reboot will start over again with these problems mentioned above.

As an emergency solution I have switched to my old eFA 3.0.26 VM which is now serving our organisation.

However, there are still e-mails stuck in the quarantine of the eFa 4.01. VM, which I believe they have not been delivered yet.

Any advise how to find the root cause of this problem and how to solve it would be appreciated.

Re: Problem with clamd.socket permission since today

Posted: 29 Jan 2020 23:18
by shawniverson
Examine /etc/clamd.d/scan.conf:

Code: Select all

# Path to a local socket file the daemon will listen on.
# Default: disabled (must be specified by a user)
LocalSocket /var/run/clamd.socket/clamd.sock
Run this as root:

Code: Select all

mkdir -p /var/run/clamd.socket
chown -R clamscan:mtagroup /var/run/clamd.socket
echo "d /run/clamd.socket 0750 clamscan mtagroup -" > /etc/tmpfiles.d/clamd.socket.conf

Re: Problem with clamd.socket permission since today

Posted: 30 Jan 2020 02:58
by ajmind
It seems to be that all services needed could not create the socket file in /var/run:

Code: Select all

● mariadb.service - MariaDB 10.1 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─limit.conf, override.conf
   Active: failed (Result: exit-code) since Do 2020-01-30 03:37:57 CET; 16min ago
  Process: 1642 ExecStart=/usr/libexec/mysqld --basedir=/usr $MYSQLD_OPTS $_WSREP_NEW_CLUSTER (code=exited, status=1/FAILURE)
  Process: 1594 ExecStartPre=/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
  Process: 1494 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
 Main PID: 1642 (code=exited, status=1/FAILURE)
   Status: "Starting Innodb crash recovery"

Jan 30 03:37:53 systemd[1]: Starting MariaDB 10.1 database server...
Jan 30 03:37:53 mysql-check-socket[1494]: Socket file /var/lib/mysql/mysql.sock exists.
Jan 30 03:37:54 mysql-check-socket[1494]: No process is using /var/lib/mysql/mysql.sock, which means it is a garbage, so it will be removed automatically.
Jan 30 03:37:54 mysql-prepare-db-dir[1594]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done.
Jan 30 03:37:54 mysql-prepare-db-dir[1594]: If this is not the case, make sure the /var/lib/mysql is empty before running mysql-prepare-db-dir.
Jan 30 03:37:55 mysqld[1642]: 2020-01-30  3:37:55 139729601505472 [Note] /usr/libexec/mysqld (mysqld 10.1.41-MariaDB) starting as process 1642 ...
Jan 30 03:37:57  systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Jan 30 03:37:57 systemd[1]: Failed to start MariaDB 10.1 database server.
Jan 30 03:37:57 sysemd[1]: Unit mariadb.service entered failed state.
Jan 30 03:37:57 systemd[1]: mariadb.service failed.

I have also created the needed dir

Code: Select all

mkdir -p /var/run/mariadb
chown -R mysql:mysql /var/run/mariadb
this has worked but after a reboot everything is gone.

Update
last morning ther was a yum update performed for
sqlite.x86_64 3.7.17-8.el7_7.1

Could this be a reason that all the needed services are failing?

Re: Problem with clamd.socket permission since today

Posted: 30 Jan 2020 04:45
by ajmind
shawniverson wrote: 29 Jan 2020 23:18 Examine /etc/clamd.d/scan.conf:

Code: Select all

mkdir -p /var/run/clamd.socket
chown -R clamscan:mtagroup /var/run/clamd.socket
echo "d /run/clamd.socket 0750 clamscan mtagroup -" > /etc/tmpfiles.d/clamd.socket.conf
Done but still no luck.

Code: Select all

ERROR: LOCAL: Socket file /var/run/clamd.socket/clamd.sock could not be bound: Permission denied

Re: Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

Posted: 30 Jan 2020 11:52
by shawniverson
Try toggling selinux off to check and see if maybe you have a problem with selinux.

Code: Select all

setenforce 0

Re: Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

Posted: 30 Jan 2020 12:30
by ajmind
shawniverson wrote: 30 Jan 2020 11:52 Try toggling selinux off to check and see if maybe you have a problem with selinux.

Code: Select all

setenforce 0
Yep, this has solved these issues regarding creating pid and socket files.

The question is now, why it is happened, and what will happen after reboot?

Update

Further testing have shown:

Code: Select all

/etc/selinux/config:
SELINUX=enforcing
This breaks after reboot my eFA VM.

Changing to

Code: Select all

/etc/selinux/config:
SELINUX=permissive
returns everything back to normal. However, I do not have changed something in this context. Why it has started to create problems.

Mmmh, I have installed recently the free Sophos AV engine...

Further reading a post in
https://serverfault.com/a/705725

explains a little bit for me the problem. So I have followed that suggestion

Code: Select all

restorecon -r -v /

changed back:
/etc/selinux/config:
SELINUX=enforcing

reboot
Voila, my eFa VM returns back with everything as nearly 36 hours before. so now I am confident to decommission my old backup eFa 3.0.2.6 and to restart my eFa VM 4.0.1.

Thank you for helping me out of this hassle!
:dance:

Re: Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

Posted: 31 Jan 2020 10:42
by shawniverson
:dance: :violin: