Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

General eFa discussion
Post Reply
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

Post by ajmind »

This morning trouble started on our eFa 4.x box:

Code: Select all

 LOCAL: Could not create socket directory: /var/run/clamd.socket: Permission denied
Jan 29 19:16:38  clamd[7374]: ERROR: LOCAL: Could not create socket directory: /var/run/clamd.socket: Permission denied
Jan 29 19:16:38  clamd[7374]: ERROR: LOCAL: Socket file /var/run/clamd.socket/clamd.sock could not be bound: No such file or directory
Jan 29 19:16:38  clamd[7374]: LOCAL: Socket file /var/run/clamd.socket/clamd.sock could not be bound: No such file or directory
Regardless what I am reading and doing this problem persists. No messages are processed as

Code: Select all

postfix/master[23237]: warning: process /usr/libexec/postfix/pickup pid 20458 exit status 127
Jan 29 10:21:19 IT1MAILGW1 postfix/master[23237]: warning: /usr/libexec/postfix/pickup: bad command startup -- throttling
How to solve this issue as no e-mails are getting through this box anymore.
Could I disable virus scanning that messages are delivered. (As a temporary solution!).

Uptated subject due to further examination of problem

Problems have been identified after a reboot of the EFa VM:

Code: Select all

[ERROR] mysqld: Can't create/write to file '/run/mariadb/mariadb.pid' (Errcode: 2 "No such file or directory")
2020-01-29 11:49:54 140330304825088 [ERROR] Can't start server: can't create PID file: No such file or directory

Code: Select all

[suexec:notice] [pid 1571:tid 140156932184256] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Wed Jan 29 11:49:52.571676 2020] [cgid:error] [pid 1647:tid 140156932184256] (2)No such file or directory: AH01243: Couldn't bind unix domain socket /run/httpd/cgisock.1571

[Wed Jan 29 11:49:52.572863 2020] [core:error] [pid 1571:tid 140156932184256] (2)No such file or directory: AH00099: could not create /run/httpd/httpd.pid
[Wed Jan 29 11:49:52.572893 2020] [core:error] [pid 1571:tid 140156932184256] AH00100: httpd: could not log pid to file /run/httpd/httpd.pid
Manually creating the required dirs with correct permissions clears the error (mariadb, httpd), but
clamd@scan claims still not be able to create

Code: Select all

/var/run/clamd.socket: Permission denied
even it is there and the permission is set to

Code: Select all

clamscan:mtagroup
A further reboot will start over again with these problems mentioned above.

As an emergency solution I have switched to my old eFA 3.0.26 VM which is now serving our organisation.

However, there are still e-mails stuck in the quarantine of the eFa 4.01. VM, which I believe they have not been delivered yet.

Any advise how to find the root cause of this problem and how to solve it would be appreciated.
Last edited by ajmind on 30 Jan 2020 10:32, edited 2 times in total.
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Problem with clamd.socket permission since today

Post by shawniverson »

Examine /etc/clamd.d/scan.conf:

Code: Select all

# Path to a local socket file the daemon will listen on.
# Default: disabled (must be specified by a user)
LocalSocket /var/run/clamd.socket/clamd.sock
Run this as root:

Code: Select all

mkdir -p /var/run/clamd.socket
chown -R clamscan:mtagroup /var/run/clamd.socket
echo "d /run/clamd.socket 0750 clamscan mtagroup -" > /etc/tmpfiles.d/clamd.socket.conf
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Problem with clamd.socket permission since today

Post by ajmind »

It seems to be that all services needed could not create the socket file in /var/run:

Code: Select all

● mariadb.service - MariaDB 10.1 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─limit.conf, override.conf
   Active: failed (Result: exit-code) since Do 2020-01-30 03:37:57 CET; 16min ago
  Process: 1642 ExecStart=/usr/libexec/mysqld --basedir=/usr $MYSQLD_OPTS $_WSREP_NEW_CLUSTER (code=exited, status=1/FAILURE)
  Process: 1594 ExecStartPre=/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
  Process: 1494 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
 Main PID: 1642 (code=exited, status=1/FAILURE)
   Status: "Starting Innodb crash recovery"

Jan 30 03:37:53 systemd[1]: Starting MariaDB 10.1 database server...
Jan 30 03:37:53 mysql-check-socket[1494]: Socket file /var/lib/mysql/mysql.sock exists.
Jan 30 03:37:54 mysql-check-socket[1494]: No process is using /var/lib/mysql/mysql.sock, which means it is a garbage, so it will be removed automatically.
Jan 30 03:37:54 mysql-prepare-db-dir[1594]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done.
Jan 30 03:37:54 mysql-prepare-db-dir[1594]: If this is not the case, make sure the /var/lib/mysql is empty before running mysql-prepare-db-dir.
Jan 30 03:37:55 mysqld[1642]: 2020-01-30  3:37:55 139729601505472 [Note] /usr/libexec/mysqld (mysqld 10.1.41-MariaDB) starting as process 1642 ...
Jan 30 03:37:57  systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Jan 30 03:37:57 systemd[1]: Failed to start MariaDB 10.1 database server.
Jan 30 03:37:57 sysemd[1]: Unit mariadb.service entered failed state.
Jan 30 03:37:57 systemd[1]: mariadb.service failed.

I have also created the needed dir

Code: Select all

mkdir -p /var/run/mariadb
chown -R mysql:mysql /var/run/mariadb
this has worked but after a reboot everything is gone.

Update
last morning ther was a yum update performed for
sqlite.x86_64 3.7.17-8.el7_7.1

Could this be a reason that all the needed services are failing?
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Problem with clamd.socket permission since today

Post by ajmind »

shawniverson wrote: 29 Jan 2020 23:18 Examine /etc/clamd.d/scan.conf:

Code: Select all

mkdir -p /var/run/clamd.socket
chown -R clamscan:mtagroup /var/run/clamd.socket
echo "d /run/clamd.socket 0750 clamscan mtagroup -" > /etc/tmpfiles.d/clamd.socket.conf
Done but still no luck.

Code: Select all

ERROR: LOCAL: Socket file /var/run/clamd.socket/clamd.sock could not be bound: Permission denied
User avatar
shawniverson
Posts: 3644
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:

Re: Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

Post by shawniverson »

Try toggling selinux off to check and see if maybe you have a problem with selinux.

Code: Select all

setenforce 0
ajmind
Posts: 53
Joined: 28 Mar 2017 15:26

Re: Problem with eFa 4.0.1 after reboot- no pid or socket files are created in /var/run...

Post by ajmind »

shawniverson wrote: 30 Jan 2020 11:52 Try toggling selinux off to check and see if maybe you have a problem with selinux.

Code: Select all

setenforce 0
Yep, this has solved these issues regarding creating pid and socket files.

The question is now, why it is happened, and what will happen after reboot?

Update

Further testing have shown:

Code: Select all

/etc/selinux/config:
SELINUX=enforcing
This breaks after reboot my eFA VM.

Changing to

Code: Select all

/etc/selinux/config:
SELINUX=permissive
returns everything back to normal. However, I do not have changed something in this context. Why it has started to create problems.

Mmmh, I have installed recently the free Sophos AV engine...

Further reading a post in
https://serverfault.com/a/705725

explains a little bit for me the problem. So I have followed that suggestion

Code: Select all

restorecon -r -v /

changed back:
/etc/selinux/config:
SELINUX=enforcing

reboot
Voila, my eFa VM returns back with everything as nearly 36 hours before. so now I am confident to decommission my old backup eFa 3.0.2.6 and to restart my eFa VM 4.0.1.

Thank you for helping me out of this hassle!
:dance:
Post Reply