efa-project.org

ok, I've gone from 1/7 the speed to 1/2 the speed (vm vs physical machine mysql performance). I have two arrays in the vm host, a 2 drive raid1 array and a 6 drive raid6 array. The raid6 array has a faster readwrite performance than the raid1 array. The raw vm image for efa was on the slower raid1 a...

I've used sysbench to do some performance testing on mysql. Short version: - MySQL performance inside my efa kvm is 1/7th the speed of the mysql performance on another older physical machine. - io wait % in the physical machine (and vm host machine) is very low. - io wait % inside the vm is very hig...

times without and with --no-sync # su - apache -bash-4.1$ time /usr/local/bin/sa-learn -p /etc/MailScanner/spam.assassin.prefs.conf --ham --file /var/spool/MailScanner/quarantine/20150402/nonspam/07A1B181E17.A009C Learned tokens from 1 message(s) (1 message(s) examined) real 1m7.466s user 0m2.987s s...

I made the following change in /var/www/html/mailscanner/functions.php just to get an idea of how long the UI was taking to train diff -c functions.php functions.php.org *** functions.php    2015-04-02 15:59:20.040362566 +0800 --- functions.php.org    2015-04-02 16:00:15.688371543 +0800 ************...

[edit: ignore this]
Is there an address I can send the results to? A lot of this information cannot be posted in public form, and there is too much to sanitize.

shawniverson wrote:You can add a spamassassin rule to local.cf to override this score...

being pendantic; the file is in:

/etc/mail/spamassassin/local.cf

Question: do we need to restart spamassassin after changing the rules?

shawniverson wrote:Try running sa-learn with --no-sync and report back please.

like so?

Agreed it is too slow. Don't know why. Still working on it.

You're seeing the same thing I am. While the system is running sa-learn, the rest of the system grinds to a halt, or becomes very slow and unresponsive. It is like the whole system waits until sa-learn completes. It seems to take about 30 seconds per message for sa-learn to run. It's not an SQL tabl...

The UI is called "mailwatch" ( https://sourceforge.net/projects/mailwatch/ ) and does not appear to be terribly active at the moment. It might be worth giving the author a prod over there.

added to the wiki

First, edit your post and remove the password.

What was the load average during that time? And was top showing iowait during that time?

If so, we're both experiencing the same thing.

I'm still looking.

Can you run the following command a couple of times and tell me what kind of queries to which database you are seeing? mysqladmin -uroot -p<EFA mysql root password from /etc/EFA-Config> processlist run it 5 or 10 times. what ones in the "db" column show active queries? ("Command" column = "Query") T...

todd,

Do you get a lot of non English language mail into your system?

Shawn, I expect if he does as you say, the load will drop to near 0.

I'm now almost certain that the problem is because of an unoptimal mysql configuration. I'm playing with the configurations now to see if I can improve the performance without making too many changes.

Isn't the internet a wonderful invention? (except for spam, of course)

It should be plenty of hardware, that's not the problem. The problem is elsewhere.

I'm doing some testing now to see if I can isolate the issue.

The page marked 125 is 141 pages in. There is another document added to the front of the PDF which throws off the page numbering.

As for the 16 processes, with a max of 10 set in EFA, I'm not sure. I'd have to dig in and look.

It looks like the mailscanner process is getting hung up on i/o somewhere.

I have an idea. I'll test it in a couple of hours and let you know if it makes a difference for me.

SA Learn causes a lot of IO wait for me

Hmmm...

Interesting. I am getting slower performance now. I am seeing a lot of iowait.

Let me see if I can figure this out...

3.0.0.7, build from the centos netinstall selecting the SA Learn option of "As Spam+Report" results in the same error warn: reporter: razor2 report failed: No such file or directory report requires authentication at /usr/local/share/perl5/Mail/SpamAssassin/Plugin/Razor2.pm (sorry, I didn't save the ...

As a wild ass guess, I'd suspect mysql being the bottleneck to mailscanner. Perhaps there is a way to debug the time it takes for Mailscanner to handle requests. That information could give you a better indication of where Mailscanner is being held up. Once you know that, then solving the problem be...

makes sense and works for me.

If you look at page 141 of the MailScanner User Guide and Training Manual , you can see their recommendations. So how many should you use? It'll depend on the workload and where the processes are contending for resources. I'd be inclined to go with 1 or 2 per CPU core (3 for hyperthreaded cores) and...