Negative Caching - Automated recipient verification

[LBJ]

G'day All,

I just set up an EFA installation, and after working around a socket incompatibility between EFA and opendkim/opendmarc viewtopic.php?p=19491#p19491, it's running quite nicely for the most part.

The Automated Recipient Verification was working correctly in allowing email for valid addresses, and rejecting email for non-valid addresses.

Once it was running, I was curious as to what error EFA returns if it's ever temporarily unable to reach the target server to perform its Automated Recipient Verification. After firewalling its IP from the target server, I was pleased to see it returned a transient "450 4.1.1 Recipient address rejected: unverified address: No user at this address" error, and not a permanent 5xx error.

That was good.

However, when the sending server next retried to send the item to the valid email address on the target domain, EFA continued to immediately return the 450 transient rejection without any detectable attempt to verify the address with the target server which was now reachable. This has continued for over an hour so far. Other addresses are verified as expected, but the address which previously failed continues to be reported as invalid.

Is there somewhere EFA is caching negative verifications?

[pdwalker]

Have a read here:

http://www.postfix.org/ADDRESS_VERIFICA ... ml#caching

Can you check your postfix settings and see if you've configured a caching database?

[LBJ]

G'day pdwalker,

I just came back to update this post with a solution, and what you suggested is certainly related. Thank you very much.

Restarting postfix, and even rebooting, made no difference. However, after 3 hours, with no configuration changes, the negatively cached address was tested again and incoming email was subsequently accepted.

My test setup is a plain vanilla installation, and I made no mods to main.cf.

The critical default postfix configuration items causing this are...

Code: Select all

# postconf address_verify_negative_cache
address_verify_negative_cache = yes
# postconf address_verify_negative_refresh_time
address_verify_negative_refresh_time = 3h
# postconf address_verify_map
address_verify_map = btree:$data_directory/verify_cache

The important sections from the postfix documentation are...

Code: Select all

address_verify_negative_cache (default: yes)

    Enable caching of failed address verification probe results. When this feature is enabled, the cache may pollute quickly with garbage. When this feature is disabled, Postfix will generate an address probe for every lookup.

    This feature is available in Postfix 2.1 and later.
address_verify_negative_expire_time (default: 3d)

    The time after which a failed probe expires from the address verification cache.

    Specify a non-zero time value (an integral value plus an optional one-letter suffix that specifies the time unit). Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks). The default time unit is d (days).

    This feature is available in Postfix 2.1 and later.
address_verify_negative_refresh_time (default: 3h)

    The time after which a failed address verification probe needs to be refreshed.

    Specify a non-zero time value (an integral value plus an optional one-letter suffix that specifies the time unit). Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks). The default time unit is h (hours).

    This feature is available in Postfix 2.1 and later.
 
 Address verification database

To improve performance, the Postfix verify(8) daemon can save address verification results to a persistent database. This is enabled by default with Postfix 2.7 and later. The address_verify_map (NOTE: singular) configuration parameter specifies persistent storage for sender or recipient address verification results. If you specify an empty value, all address verification results are lost after "postfix reload" or "postfix stop".

    # Example 1: Default setting for Postfix 2.7 and later.
    # Note: avoid hash files here. Use btree or lmdb instead.
    /etc/postfix/main.cf:
        address_verify_map = btree:$data_directory/verify_cache

    # Example 2: Shared persistent lmdb: cache (Postfix 2.11 or later).  
    # Disable automatic cache cleanup in all Postfix instances except
    # for one instance that will be responsible for cache cleanup.
    /etc/postfix/main.cf:
        address_verify_map = lmdb:$data_directory/verify_cache
        # address_verify_cache_cleanup_interval = 0

    # Example 3: Shared persistent btree: cache (Postfix 2.9 or later).  
    # Disable automatic cache cleanup in all Postfix instances except
    # for one instance that will be responsible for cache cleanup.
    /etc/postfix/main.cf:
        address_verify_map = proxy:btree:$data_directory/verify_cache
        # address_verify_cache_cleanup_interval = 0

    # Example 4: Shared memory cache (requires Postfix 2.9 or later).
    # Disable automatic cache cleanup in all Postfix instances.
    # See memcache_table(5) for details.
    /etc/postfix/main.cf:
        address_verify_map = memcache:/etc/postfix/verify-memcache.cf
        address_verify_cache_cleanup_interval = 0

    # Example 5: Default setting for Postfix 2.6 and earlier.
    # This uses non-persistent storage only.
    /etc/postfix/main.cf:
        address_verify_map =

For our installation, things work nicely with...

Code: Select all

# postconf "address_verify_negative_cache = no"
# postconf "address_verify_map ="

That gives us no negative caching, and non persistent positive caching.

[pdwalker]

Interesting.

I say that because my system doesn't use address verification, and none of the cache entries are defined in my main.cf.

I wonder how those settings got into your configuration?

Anyway, it's working now and the problem is resolved.

[LBJ]

G'day pdwalker,

You won't see them in the main.cf file since they're the default values. Postfix contains a range of default values for configuration items without any entries existing in the main.cf file.

To check for them, use the postconf utility from the CLI...

Code: Select all

# postconf address_verify_negative_cache
address_verify_negative_cache = yes

# postconf address_verify_negative_refresh_time
address_verify_negative_refresh_time = 3h

# postconf address_verify_map
address_verify_map = btree:$data_directory/verify_cache

Those CLI commands simply display the current value (either default or modified) for the configuration items of address_verify_negative_cache, address_verify_negative_refresh_time and address_verify_map respectively.

They're the default values for the current release of postfix and are in force with or without any visible configuration item in the main.cf file. To change them from the default, you can either manually edit the main.cf file or use the postconf utility to take care of it for you as per...

Code: Select all

postconf "address_verify_negative_cache = no"

In older versions of postfix, the postconf command required the -e switch to edit the value, but the current and recent versions don't.

That above CLI command will modify any existing related configuration entry in the main.cf file, or otherwise just append "address_verify_negative_cache = no" to the file to override the default value.