Date Rollback Condition is BAD

General eFa discussion
Post Reply
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Date Rollback Condition is BAD

Post by shawniverson »

My NTP server was rebuilt today, and for a brief time was handing out a time in the past.

EFA saw it and synced to the past.

It appears to be walking through all emails in the quarantine????

Processed: 2,082,528 90.8Gb
Clean: 2,081,286 99.9%
Viruses: 0 0.0%
Top Virus: None
Blocked files: 1,242 0.1%
Others: 0 0.0%
Spam: 0 0.0%
High Scoring Spam: 0 0.0%
MCP: 0 0.0%
High Scoring MCP: 0 0.0%

Oh no!!!!!

:o :o :o :o :o
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

Okay, it looks like the system is stabilizing now that the time is correct...

CPU utilization is still high due to incoming queue being backlogged...

Monitoring to see what happens...
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

Something wrong...this appears to have started prior to the NTP server rebuild.

MailWatch shows the system looping through previously quarantined messages and marking them as clean repeatedly...

System is rapidly declining on disk space.

Watching incoming queue...it is trending downward slowly.
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

Ok, found clamd dead. Was retrying scanning on a batch of messages over and over.

Started clamd...

MailScanner processes are turning zombie...
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

Moved /var/spool/postfix/hold --> /var/spool/postfix/hold.chk

Created new hold

System now normal...something corrupt in hold queue? Checking...
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

Nothing corrupt in hold queue...all messages eventually processed.

However, clamd went down again part way....as soon as it did, the looping started again.

I had to stop MailScanner, start clamd, and start MailScanner.

Will be monitoring this very closely.
Top
ramtech
Posts: 56
Joined: 20 Sep 2013 01:31

Re: Date Rollback Condition is BAD

Post by ramtech »

Sounds less than fun.

Looking forward reading to the progress.
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

It appears that a combination of CPU overload and too many MailScanner threads may be at fault...I think that clamd is getting pushed to its limit with the large number of incoming messages.

I have reduced my MailScanner threads to 5. Will test with a large queue again and see what happens.
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

Yep, too many MailScanner children was the culprit.

:naughty:
Top
User avatar
shawniverson
Posts: 3783
Joined: 13 Jan 2014 23:30
Location: Indianapolis, Indiana USA
Contact:
Contact shawniverson

Re: Date Rollback Condition is BAD

Post by shawniverson »

************Update**************

http://forum.efa-project.org/viewtopic.php?f=13&t=552
Top
Post Reply

Return to “Discussion”