TxRep.pm error when trying to learn HAM from GUI

[ovizii]

I go to my EFA GUI, open a message and at the bottom I chose: SA Learn HAM, click OK, see this error:

Code: Select all

SA Learn: Use of uninitialized value $msgscore in addition (+) at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/TxRep.pm line 1415., Use of uninitialized value $msgscore in subtraction (-) at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Plugin/TxRep.pm line 1415., Learned tokens from 0 message(s) (1 message(s) examined)

Found a few bugs online related to this but am not a good enough coder to patch with the suggested fixes.

[shawniverson]

TxRep has not been thoroughly tested in 3.0.1.1, and is disabled by default. There may be bugs present.

[ovizii]

I saw a post here announcing it as a drop in replacement for AWL so I thought I'd give it a try.

If you're interested, here are a few links to the same/similar bug:
http://www.eee.hku.hk/~kmpoon/lists/spa ... 07481.html

someone in this thread mentions:

More testing shows that the issue only occurs with txrep_track_messages turned on for txrep.
And it has to be a message that generates a new msg_id from the bayes.pm get_msgid call.

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7164
there seems to be a commit and a solution, I'm just not sure how to get it

[ovizii]

OK; I solved that problem and got past that error by disabling:
txrep_track_messages 0

On the author's site: http://truxoft.com/resources/txrep.htm he also statest that some soem administrator settigns should not be set from a user's preferences file so being unsure I moved them from my own .cf file to mailscanner.cf:

As you can see, I have commented the AWL lines and added the lines for TXREP

Code: Select all

#Begin E.F.A. mods for MySQL
bayes_store_module              Mail::SpamAssassin::BayesStore::SQL
bayes_sql_dsn                   DBI:mysql:sa_bayes:localhost
bayes_sql_username              sa_user
bayes_sql_password              secret
bayes_sql_override_username     mailwatch
#auto_whitelist_factory          Mail::SpamAssassin::SQLBasedAddrList
#user_awl_dsn                    DBI:mysql:sa_bayes:localhost
#user_awl_sql_username           sa_user
#user_awl_sql_password           secret
#End E.F.A. mods for MySQL

dcc_home /var/dcc
bayes_ignore_header X-ICTProConsult-MailScanner-EFA
bayes_ignore_header X-ICTProConsult-MailScanner-EFA-SpamCheck
bayes_ignore_header X-ICTProConsult-MailScanner-EFA-SpamScore
bayes_ignore_header X-ICTProConsult-MailScanner-EFA-Information

# TXREP DATABASE details
txrep_factory                   Mail::SpamAssassin::SQLBasedAddrList
user_awl_sql_table              txrep
user_awl_dsn                    DBI:mysql:sa_bayes:localhost
user_awl_sql_username           sa_user
user_awl_sql_password           secret

and yet when analysing a message from the command line with spamassassin I see errors like this one:

Code: Select all

May 14 23:48:33.231 [17444] info: auto-whitelist: sql-based add_score/update 1|-20|root|87.156.160.40|none: SQL error: Table 'sa_bayes.txrep' doesn't exist

[ovizii]

Problem apparently solved with a newer version of TXRep.pm from here: http://svn.apache.org/viewvc/spamassass ... n/TxRep.pm (Revision 1720440)

There is still something wrong though as adding the header:

Code: Select all

add_header all TxRep _TXREP_IP_

only adds a header of: _TXREP_IP_ so it doesn't actually translate that variable into the value it should represent.


Also, I don't see any evidence of TXREP being used. How could I check its actually in use?

[ovizii]

And talking about auto whitelisting I found this setting inside /etc/Mailscanner/Mailscanner.conf

Do I need to disable this too if I have disabled the AWL SA plugin in favor of TxRep?

Code: Select all

# Set this option to "yes" to enable the automatic whitelisting functions
# available within SpamAssassin. This will cause addresses from which you
# get real mail, to be marked so that it will never incorrectly spam-tag
# messages from those addresses.
# To disable whitelisting, you must set "use_auto_whitelist 0" in your
# spam.assassin.prefs.conf file as well as set this to no.
SpamAssassin Auto Whitelist = yes

[ovizii]

I don't think TxRep is working properly at all. According to its documentation it needs a table called txrep.

I checked and there is none:
mysql> show tables;

Code: Select all

+-----------------------+
| Tables_in_mailscanner |
+-----------------------+
| audit_log             |
| blacklist             |
| inq                   |
| maillog               |
| mcp_rules             |
| mtalog                |
| mtalog_ids            |
| outq                  |
| sa_rules              |
| saved_filters         |
| spamscores            |
| user_filters          |
| users                 |
| whitelist             |
+-----------------------+
14 rows in set (0.00 sec)

Here is an instructions on how to create that table: http://truxoft.com/resources/txrep.htm

To install a new SQL table for TxRep, save the 'CREATE' SQL command shown below into a file named txrep_mysql.sql, and use the following command. You can also simply run the SQL command from within the respective database area in PhpMyAdmin:

mysql -h <hostname> -u <adminusername> -p <databasename> < txrep_mysql.sql
Enter password: <adminpassword>

CREATE TABLE txrep (
username varchar(100) NOT NULL default '',
email varchar(255) NOT NULL default '',
ip varchar(40) NOT NULL default '',
count int(11) NOT NULL default '0',
totscore float NOT NULL default '0',
signedby varchar(255) NOT NULL default '',
PRIMARY KEY (username,email,signedby,ip)
) ENGINE=MyISAM;

I get an error however:

Code: Select all

mysql -h localhost -u root -p mailscanner < txrep_mysql.txt
Enter password:
ERROR 1071 (42000) at line 1: Specified key was too long; max key length is 1000 bytes

Any more ideas?

[ovizii]

partially solved by realizing I'm an idiot.
I was looking at the wrong DB.

Code: Select all

mysql> use sa_bayes;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+--------------------+
| Tables_in_sa_bayes |
+--------------------+
| awl                |
| bayes_expire       |
| bayes_global_vars  |
| bayes_seen         |
| bayes_token        |
| bayes_vars         |
+--------------------+
6 rows in set (0.00 sec)

and then this worked to create the table:

Code: Select all

mysql -h localhost -u root -p sa_bayes < txrep_mysql.txt

[ovizii]

So now, I can see that TXREP is working perfectly. It is adding scores to my incoming emails. Also, inside the DB named sa_bayes there is a table called txrep containing all the logged emails. Seems perfect except for one thing:

The user names logged in there are either postfix or apache.

I assume everything coming IN is logged with postfix and everything that gets released or learned/unlearned via the mailwatch GUi gets logged with apache?

Screenshot:

txrep.png (106.06 KiB) Viewed 7578 times

postfix has 320 entries and apache 176

So keeping in mind that txrep logs as postfix or apache what should I do with these txrep settings?

Code: Select all

# uncomment below to make TxRep use one big global pool for everyone.
# if it is commented each user has his own data pool
# user_awl_sql_override_username        SomeName

# looks like it is possible to be grouped per domain
# @example.com user_awl_sql_override_username SomeName

# User vs global storage range [0..10]          (default: 0)
# 5 means the user rep is weighted 5x the global rep
# txrep_user2global_ratio 5

So the questions is which reputation does txrep use in this case? There is no user rep so it looks for the global one, does it take the postfix one or the apache one?
And what would happen if I set this parameter:

Code: Select all

user_awl_sql_override_username        TxRep