milter cpu at 100%

[nicola.piazzi]

During work i found that mysqll cpu was at 100% for hours
There was same select running forever each time i watch with a different messageid
The only way to stop it was to rename program and then kill its pid
mv /usr/bin/mailwatch/tools/Postfix_relay/mailwatch_milter_relay.php /usr/bin/mailwatch/tools/Postfix_relay/mailwatch_milter_relay.php2

Probably an msmilter issue
After killing and restarting problem goes on


MariaDB [(none)]> SHOW FULL PROCESSLIST;
+-----+-------------+-----------+-------------+---------+------+--------------------------+-----------------------------------------------------------------------------------------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+-----+-------------+-----------+-------------+---------+------+--------------------------+-----------------------------------------------------------------------------------------------------+----------+
| 1 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 2 | system user | | NULL | Daemon | NULL | InnoDB purge coordinator | NULL | 0.000 |
| 3 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 4 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 5 | system user | | NULL | Daemon | NULL | InnoDB shutdown handler | NULL | 0.000 |
| 126 | root | localhost | NULL | Query | 0 | Init | SHOW FULL PROCESSLIST | 0.000 |
| 139 | mailwatch | localhost | mailscanner | Sleep | 39 | | NULL | 0.000 |
| 142 | mailwatch | localhost | mailscanner | Query | 0 | Sending data | SELECT id from `maillog` where messageid='<B95873D8-8B54-410D-B66F-1F97E7D6A2AA@gmail.com>' LIMIT 1 | 0.000 |
| 214 | sa_user | localhost | sa_bayes | Sleep | 13 | | NULL | 0.000 |
| 228 | sa_user | localhost | sa_bayes | Sleep | 8 | | NULL | 0.000 |
| 242 | sa_user | localhost | sa_bayes | Sleep | 3 | | NULL | 0.000 |
+-----+-------------+-----------+-------------+---------+------+--------------------------+-----------------------------------------------------------------------------------------------------+----------+

[nicola.piazzi]

with a rapid look to the program seems that it uses some entries from maillog
so is possible that priogram made same thing forever depending on what is in the log

i solved with
logrotate -vf /etc/logrotate.conf

but there is in act a potential bug

[nicola.piazzi]

Today same thing
program search a string that at now contains about 21000 recs and do a select
[root@EFA42 log]# grep postfix\/cleanup maillog | wc
21052 224074 3943351

but it do it forever, i lauched same program manually and stays forever

[nicola.piazzi]

Probably i found something

I dont know how this program work but i think that it make a select for all lines containing postfix\/cleanup, and they are 21000

problem is in function doit($input)
it solves putting sleep(10); before the SELECT but obviously i cant know how i do

[nicola.piazzi]

i think that better thing is to watermark maillog file grepping with date/time

[shawniverson]

Following...this looks like a bug

[nicola.piazzi]

do you think that at now i can have a problem with that delay in source ?

[shawniverson]

No, but you can disable that script for now or modify it, it works relay stats from the gui. I'd like to work with you to resolve it.

[nicola.piazzi]

How can i diable that script ?
What data will not be available in gui disabling it ?
If you have some modifications i can try its 4 you in production (efa receiving 12k messages7day)

[shawniverson]

Code: Select all

sudo systemctl disable postfix_relay && systemctl stop postfix_relay
sudo systemctl disable milter_relay && systemctl stop milter_relay

The relay status will stop appearing in the GUI for message details when you do this.

I think the problem as you have indicated is the service trying to search the entire maillog for the message id each iteration. It would make more sense to search only a small portion of it, such as the last few hours or so. I can work on this and try to get the service modified to do that.

[nicola.piazzi]

yes, read a portion can be better thing

[nicola.piazzi]

hallo shawn, you re enabled postfix_relay on 4.0.4.11 release ?
and source is corrected now ?

thanks

[nicola.piazzi]

Not solved, seems same source, now i disable services

[shawniverson]

Not solved yet, still working on it, yeah, disable it.