Tech Tip (LISTSERV) – Issue 1 – 2006
Q: How can I best configure LISTSERV's Auto-Delete mechanism?
Answer by Liam Kelly
Senior Consulting Analyst, L-Soft
Properly configuring and tuning LISTSERV's auto-delete mechanism has multiple benefits for list owners and site administrators alike. It reduces the amount of time the list owner needs to spend handling bounces manually; it reduces the resources wasted by the SMTP server attempting delivery to invalid addresses; and it helps to prevent list mailings from being misidentified as spam due to high bounce rates. Setting up auto-delete for your LISTSERV lists is as simple as adding a keyword to the list header, but choosing the most appropriate and effective setting for your list can be confusing. In this Tech Tip, we'll discuss the different Auto-Delete options available and how to choose the best set of options for your LISTSERV list.
The Auto-Delete Keyword
To enable automatic deletion of invalid addresses for your list, we'll need to add a line to the list header. The format of the auto-delete keyword is as follows:
Auto-Delete=Yes|No, [Manual|Semi-Auto|Full-Auto], [Delay(x)], [Max(x)], [Probe(x)]
Setting the first parameter to "No" turns off automatic deletion for the list, and LISTSERV will instead forward all delivery errors to the list owner (or Errors-To) address for manual processing. Unless the list is set to "Validate=No", this is the default behavior. Setting the first parameter to "Yes" tells LISTSERV that we'd like to use automatic delivery error processing for the list.
Once we've told LISTSERV that we'd like to use auto-delete, we need to specify the type of auto-deletion to use. "Manual" mode doesn't actually delete subscriber addresses. With auto-delete set to "Manual", LISTSERV will automatically process bounced messages and compile a Daily Error Monitoring Report (DEMR) that gets sent to the list owner for action. LISTSERV itself takes no further action on the bounced addresses. In "Semi-Auto" mode, LISTSERV will automatically delete invalid addresses from the list per the parameters set forth by the "Delay" and "Max" settings. If LISTSERV can't automatically process the bounce or if the bounce error is a temporary error (like "Mailbox Full"), the bounce gets passed along to the list owner for manual processing. In "Full-Auto" mode, permanent errors that LISTSERV can't automatically process only show up on the DEMR, and the bounce messages themselves are not delivered to the list owner. Temporary errors are discarded.
Once we've determined the auto-delete mode, we can assign settings for the "Delay", "Max", and "Probe" settings. "Delay" and "Max" are settings that tune the bounce threshold at which LISTSERV will automatically remove invalid addresses. The "Probe" setting determines what portion of list messages get sent in a special "Passive Probe" format. We'll look at each of these settings in some detail below.
Delay and Max: Your Partners in List Hygiene
The "Delay" and "Max" settings work together to determine at what point LISTSERV will delete a bouncing email address from the list. "Delay" tells LISTSERV how many days to monitor a bouncing address before taking action on it. "Max" allows us to tell LISTSERV some maximum number of bounces to allow within that monitoring period. For instance, if we set Delay(8) and Max(10), we're telling LISTSERV that if it sees a bounce for a subscriber, the subscriber should be monitored for eight days. If on the eighth day LISTSERV receives another bounce for that subscriber, the address will be removed from the list. If LISTSERV receives ten bounces for the subscriber before the end of the eight day monitoring period, the subscriber instead will be removed on the tenth bounced message. If, on the other hand, no additional bounces are received during the eight-day monitoring period, LISTSERV will stop monitoring that subscriber after the eighth day (until the address starts bouncing again).
Tuning the "Delay" and "Max" parameters is easiest on lists that receive a large amount of daily traffic. For instance, on a list that receives an average of ten posts each day, we might set something like:
- Auto-Delete = Full-Auto, Delay(4), Max(20)
With the setting above, once we've received a bounce for a subscriber, we'll monitor the address for four days. If we receive twenty bounces for that address within the four-day period or if we receive bounced mail for it on the fourth day, the subscriber will be automatically deleted. Otherwise, the address will be removed from the Daily Error Monitoring report after the fourth day without any new bounces. We might adjust the "Delay" and "Max" parameters depending on how much traffic comes through the list and how zealous we want to be in removing addresses that bounce.
Tuning the Auto-Delete keyword is more difficult for lists that have less traffic. For example, if we have an announcement-style list that only receives postings once each week, the example setting above would mean that invalid addresses never get removed from the list, because there would never be a list posting four days after the initial bounce, and there would never be twenty postings within that four-day period. For a weekly newsletter list, something like the following would be more appropriate:
- Auto-Delete = Full-Auto, Delay(8), Max(2)
With this setting, we'll place bounced addresses on the monitoring report for eight days. If the address bounces twice within eight days (two consecutive newsletters), we'll delete it from the list. If it only bounces once and the second newsletter gets delivered with no bounce, the address will be removed from the monitoring report before the next mailing.
If we want to implement a zero-tolerance policy for bounced email addresses on our list, we can set:
- Auto-Delete = Full-Auto, Delay(0), Max(1)
In this case, subscribers are never monitored, and instead get removed on the first bounced message.
LISTSERV's auto-delete mechanism is not infallible. We may find out that we have email addresses on our Daily Error Monitoring Report that aren't subscribed to our list and hence never get auto-deleted. The problem is often that someone is subscribed under an email address that forwards to some second address, and the second address is generating bounces back to LISTSERV. The address gets placed on the DEMR, but since the bounce is for an address that LISTSERV doesn't recognize, the originally subscribed email address doesn't ever get deleted.
To get around this problem, we can tell LISTSERV to send some portion of each list posting as "passive probes". A passive probe is a message with a specially formed return address that allows LISTSERV to tell from the return address what the originally subscribed address was. The "Probe" parameter of the auto-delete keyword tells LISTSERV what portion of the list to probe on any given day. For a probe value of 'x', LISTSERV will send 1/x of the first post of the day as passive probes, giving preference to new subscribers or addresses that it has not recently probed. A simplistic example: on a list with 60 subscribers that averages one post per day and a Probe setting of (30), the first list posting of each day would contain an average of two passive probes (for two individual subscribers - 1/30th of the list membership). The rest of the subscribers would receive regular bulk SMTP (BSMTP) messages. We can lower the Probe interval to probe each user more often, but note that passive probes use more server resources to generate and mail than regular list postings. To turn off passive probes entirely, set the Probe parameter to (0). To probe every address on each day's first posting, set the Probe parameter to (1).
Generally, passive probes are transparent to subscribers. They appear as regular list postings, but with a special return address specific to that recipient. If a passive probe bounces back to LISTSERV, the server follows up with a message to the subscriber explaining that the probe failed. If that follow-up message also bounces, LISTSERV will add the address to the DEMR for the period of time specified by the Delay parameter. LISTSERV will continue to probe the address with the first posting of each day until it either stops bouncing or is auto-deleted.
Now that we know how each of LISTSERV's auto-delete parameters works, let's look at some real-life examples. For an active discussion list of 60 or so members that gets 10-20 posts per day, we might set something like:
- Auto-Delete = Semi-Auto, Delay(4), Max(20), Probe(30)
This tells LISTSERV to do semi-automatic bounce handling (temporary errors get sent to the list owner), monitor users for four days, delete them if twenty bounces are received within that four-day period, and probe each user about once each month (1/30 of the list each day).
For a weekly newsletter with 500 subscribers, we might set:
- Auto-Delete = Full-Auto, Delay(8), Max(2), Probe(12)
Here we process bounces fully automatically, monitor users for just over a week, delete them on the second newsletter bounce, and probe each user about once every three months (1/12 of the list each week).
Finally, for a monthly newsletter with 2,000 subscribers, we might set:
- Auto-Delete = Full-Auto, Delay(0), Max(1), Probe(12)
In this example, we delete invalid addresses on the first hard bounce, and probe each list member address once each year (1/12 of the list each month).
Proper configuration and tuning of LISTSERV's auto-delete feature can take much of the hassle out of list ownership, with the added benefits of better delivery rates and reduced server load. For more information about LISTSERV's auto-delete mechanism, see the LISTSERV Site Manager's Manual.