SmartList Mailing List Management Software

SmartList FAQ (Frequently Asked Questions)

Last Updated: May 26, 2021
Revision: 1.55

Table of Contents

1: General

    1.1: Introduction:
    1.2: History of this document:
    1.3: What is SmartList?
    1.4: Is there more documentation available?
    1.5: How does SmartList work?
    1.6: Where can I get SmartList?
    1.7: Is there a SmartList Support mailing list?
    1.8: Is there an archive of the mailing list?
    1.9: What's the purpose of the rc.local.* files?
    1.10: How can I see what's going on (diagnostics)?

2: Basic List Administration

    2.1: What does "uncomment" mean?
    2.2: How do I make the list Open/Closed?
    2.3: How can I disable automatic unsubscription?
    2.4: How can I prevent certain people from automatic unsubscription?
    2.5: How do people subscribe/unsubscribe to the list?
    2.6: How do I change the admin password for my list?
    2.7: How do I enable 'auto help'?
    2.8: How can I tell SmartList to notify me when someone subscribes/unsubscribes?
    2.9: How can I automatically subscribe people who post to the list?
    2.10: How can I add a short message file to the top or the bottom of all the mailing list messages?
    2.11: How can I keep the automatically added messages from multiplying when people quote them all the time?
    2.12: How can I automatically send files to new subscribers?
    2.13: Can I add comments to the addresses in the dist file?
    2.14: How does someone access the archive server?
    2.15: What does "delink" mean?
    2.16: How do I add another list of accept addresses?
    2.17: Hello, Is there a list of x-commands somewhere?
    2.18: How to track bounces from addresses not on list?

3: Remote Administration

    3.1: How do I use X-Command for remote administration?
    3.2: Can I issue X-Commands from the message body?
    3.3: How do I subscribe/unsubscribe people to my list?
    3.4: How do I get a list of the subscribers?
    3.5: How can I prevent the accidental multiple subscription of an address with x_command?
    3.6: Is there a web interface for administering SmartList?
    3.7: Why isn't SmartList adding/removing names even though it says it is?

4: Restrictions and Security

    4.1: How can I restrict who can subscribe to my list?
    4.2: How can I restrict who can post to my list?
    4.3: How can I prevent that someone (un)subscribes someone else without their knowledge?
    4.4: Is there a possibility to authenticate postings with SmartList?
    4.5: How can I restrict who can access the archive server?
    4.6: Can SmartList confirm (un)subscriptions?
    4.7: How can I allow posts only from a certain domain, without using the accept file?
    4.8: How to bounce messages with a certain word in the subject?
    4.9: How to automatically notify posters of messages bounced for being too large?
    4.10: How do I allow digest recipients to post to the regular list?
    4.11: How to restrict postings from a particular sender?

5: Digests

    5.1: How do I turn a non-digest list into a digest?
    5.2: How often do digests get sent out?
    5.3: Can I force a digest to be sent every day, even if it is small?
    5.4: How can I add a message to the beginning of every digest?
    5.5: Is there a way to make the digest go out without being an attachment?

6: Interaction of SmartList with Message Transfer Agents (MTAs)

    6.1: Does SmartList run with Postfix?
    6.2: Does SmartList run with qmail?
    6.3: Sendmail can't run flist. Why? (smrsh)
    6.4: How do I get SmartList to work with different virtual accounts?

7: Addons and enhancements for SmartList

    7.1: Is it possible to have more than one list maintainer?
    7.2: Can I remotely update files used by SmartList like help.txt?
    7.3: How can a subscriber temporarily suspend subscription?
    7.4: Where can I find more SmartList information and utilities?
    7.5: Can SmartList send individualized messages to each list recipient?

8: Miscellaneous

    8.1: How can I make my list moderated?
    8.2: How can I change the messages that SmartList sends to people?
    8.3: How can I add "[listname]" to all of the mailing list messages for easy filtering?
    8.4: How can I add a custom header to all of the mailing list messages?
    8.5: How can I remove some headers from all of the mailing list messages?
    8.6: Is there an easy way to remove duplicates from the dist file?
    8.7: How can I keep people from sending HTML/attachments/other junk to the list?
    8.8: How do I make replies go to the list instead of to the sender?
    8.9: How can I make replies go to the original sender?
    8.10: Why doesn't putting "Reply-To: $listaddr" in my rc.custom file work for all subscribers?
    8.11: How can I include the number of subscribers in the footer I'm attaching to each list mail?
    8.12: How to send an auto-reply back when someone posts?
    8.13: Why are MIME and HTML emails a problem for SmartList?

SmartList FAQ (Frequently Asked Questions)


1: General


1.1: Introduction:

Welcome to the SmartList FAQ.

You are strongly advised to read the official SmartList Manual in advance. Many questions in this FAQ are only pointing to the appropriate chapter in the Manual.

This document is a collection of helpful hints and tips for getting the most out of SmartList. The sources for this information are numerous.

If you have a comment or suggestion regarding this FAQ, please forward it to the Smartlist User's mailing list (for the list info, see http://mailman.rwth-aachen.de/mailman/listinfo/smartlist).

Disclaimer: All information in this FAQ is provided AS IS and without any warranty of any kind. No person involved in this document assumes any liability for damages that occur as a result of using this information. USE THIS INFORMATION ONLY AT YOUR OWN RISK.

It is suggested that you create a test list, and try changes on that test list before you place them into production.


1.2: History of this document:

This document was originally written by jason@mindwell.com, then picked up and modified/enhanced by Werner Reisberger (werner@pure.ch), later was updated/ maintained by Peter Hartzler (pete@hartzler.net). Currently this FAQ is located at https://www.animalgenome.org/SmartList/, maintained by Zhiliang Hu since 6 Oct 2005.

The photo of the Catalina Macaw in the document logo is by Judy Leach ( http://www.petparrot.com/) and is used with her permission.


1.3: What is SmartList?

SmartList is an email list management system based on procmail.

The current version of SmartList is 3.15, available at https://github.com/procmail-org/SmartList


1.4: Is there more documentation available?

The best documentation is the SmartList manual contained within the SmartList sources. Every list maintainer should first read this manual to understand how SmartList works and to save a lot of time if problems occur later.

The SmartList manual is available at https://www.animalgenome.org/Smartlist/Manual.html as well as in the .etc directory of a normal SmartList install.

Next, the distribution comes with considerable amounts of example code, which is to be found in the .examples directory of your installation.

There are actually several FAQs that people have put together, and you might very well find some useful bits in the others. There appears to be a considerable overlap in these documents, but what the heck. :) Check out:

You can also search in the list archive(s) (see question "Is there an archive of the mailing list?")

Finally, SmartList is heavily based on procmail, and in fact many questions about how to do specialized things in SmartList end up being mainly procmail questions. You might want to check out:


1.5: How does SmartList work?

SmartList is based on a set of procmail scripts to process emails suitable for the purpose of a mailing list. If you want to know how the emails will be processed take a look into the SmartList manual section "3m. Schematic overview of what goes on behind the scenes"


1.6: Where can I get SmartList?

SmartList needs also Procmail.

The latest version of Procmail can be found at: http://www.procmail.org or ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail/procmail.tar.gz

The latest version of Smartlist can be found at:
http://www.procmail.org/SmartList-3.15.tar.gz
or
ftp://ftp.procmail.org/pub/procmail/SmartList-3.15.tar.gz

The procmail homepage, http://www.procmail.org , includes a list of mirror sites at which this and related package should be obtainable, as well as links to PGP signatures for the above.


1.7: Is there a SmartList Support mailing list?

Yes. for more information, see http://MailMan.RWTH-Aachen.DE/mailman/listinfo/smartlist

Another mailing list, for developers is

smartlist-dev-request@cuci.nl

Before you post to these or any other support lists you are strongly encouraged to read and understand Eric S. Raymond's excellent advice on "How To Ask Questions The Smart Way" at http://www.catb.org/~esr/faqs/smart-questions.html

NOTE: This is NOT a link to a help desk for this or any other software project!


1.8: Is there an archive of the mailing list?

Yes, at

http://mailman.rwth-aachen.de/pipermail/smartlist/


1.9: What's the purpose of the rc.local.* files?

They let you enhance or alter the function of SmartList by calling procmail scripts at different states of email processing.


1.10: How can I see what's going on (diagnostics)?

If SmartList sends out messages to the maintainer or the list, it adds message headers beginning with "X-Diagnostics:". Every maintainer should read these header lines to recognize problems with the list.

SmartList usually creates by default in every list a log file called "log". If you want to trace a problem with a list you can set the variable VERBOSE in rc.custom to "on". Subsequently, every processed message will create a lot of debugging information in the log file.

Don't forget to turn it off later.


2: Basic List Administration


2.1: What does "uncomment" mean?

This question seems very trivial but experience showed that many people have problems understanding how to (de)activate a variable in the rc.custom file.

To uncomment a line in the file, remove all hashmarks (#) from the front of the line (if there is more than one, be sure and remove them all), and change the value if necessary. For example, the moderate section in the standard distribution looks like this:

#moderated_flag
##moderated_flag=       yes             # uncomment this to make the list
                                        # moderated (you must create a

For example, to turn your list into a moderated list, remove the two hash marks from the line that says "moderated_flag = yes", so the section now looks like this:

#moderated_flag
moderated_flag=       yes               # uncomment this to make the list
                                        # moderated (you must create a

Note that, anything after a hashmark on a line is considered a comment, and anything before any hashmarks (if any) is uncommented. So, you can uncomment a feature setting, and still have a comment on that line, out towards the end.

To turn something off, uncomment the corresponding flag/variable name and don't assign anything to it. This turns the flag off by removing it from memory.

NOTE: The config files ship with the current running default commented out with a single hash (#), so you can see what the default is. You should probably leave that line alone. The line after that, commented with two hash marks, is typically the one you want to uncomment to change SmartList's behavior. For example, this default configuration allows non-members to send messages to the list:

#foreign_submit =       yes
##foreign_submit                        # uncomment this line if you
                                        # want to restrict submitting to
                                        # people on the accept list

Remember that in these cases if you uncomment the line that is single-hashed, it makes no difference, since the single-hashed commented line shows you what the default in rc.init is. So if that is what you want, you can leave things alone (both lines commented), since it's already set that way. If you want to change the flag, you would uncomment the double-hashed lines, like:

#foreign_submit =       yes
foreign_submit                          # uncomment this line if you
                                        # want to restrict submitting to
                                        # people on the accept list

NOTE Be sure the text editor you use saves text in "UNIX" text format, with only a newline character at the end of each line, and not a carriage-return or carriage-return newline pair. Many windows and Mac based text editors insert that carriage-return, and this can break your configuration files.

If you end up with carriage returns in your text files, you can remove them with perl, like this: (this particular example assumes the file you want to clean is rc.local.s20, change this file to be the one that needs cleaning):

perl -p -i -e 's/\r//' rc.local.s20

2.2: How do I make the list Open/Closed?

To make the list closed subscription (i.e. the maintainer has to approve subscriptions), uncomment the auto_subscribe line in rc.custom, and set it to read:

auto_subscribe = no
To make the list closed for unsubscriptions, do the same thing with the auto_unsubscribe line.

If you're reading this, then perhaps you'd also be interested in section 8.1: How can I make my list moderated? .


2.3: How can I disable automatic unsubscription?

  1. Set the variable auto_unsubscribe in rc.custom to an empty value.
  2. Remove the line
    (Only addresses below this line can be automatically removed)
    from the dist file.
NOTE: Occasionally someone encounters trouble where their list won't allow people to unsubscribe. The first thing to check if you have this trouble is that your dist file contains the line
(Only addresses below this line can be automatically removed)

NOTE: See Section 2.1: What does "uncomment" mean? for a discussion of text file formatting issues that can BREAK your text files.


2.4: How can I prevent certain people from automatic unsubscription?

Everything in the dist file preceding the line containing: '(Only addresses below this line can be automatically removed)' is write protected from changes by multigram (i.e. these addresses can never be automatically/accidentally unsubscribed).


2.5: How do people subscribe/unsubscribe to the list?

By sending a message to the listname-request address with the message 'subscribe' or 'unsubscribe' in the body or the subject of the mail, if the list is open, they will be added to it, if it is a closed list the request will be forwarded to the list maintainer.


2.6: How do I change the admin password for my list?

The password is stored in the rc.custom file.


2.7: How do I enable 'auto help'?

By uncommenting the appropriate "auto_help" line in the rc.custom file the list will respond to every undecipherable request message as if it requested help. Messages that will still get through to the maintainer are those:


2.8: How can I tell SmartList to notify me when someone subscribes/unsubscribes?

By setting the cc_requests or cc_unsubrequests variables in the rc.custom file to yes (making sure to uncomment it).


2.9: How can I automatically subscribe people who post to the list?

Instead of rejecting submissions by people not on the accept (dist) list, you can enable "force_subscribe". This will cause people submitting mails to the list to be autosubscribed to the list if they were not in the dist file.


2.10: How can I add a short message file to the top or the bottom of all the mailing list messages?

Uncomment the line "RC_LOCAL_SUBMIT_20 = rc.local.s20" from your rc.custom file, and add the following to your rc.local.s20 file:

# # Adding a disclaimer in front of every mail: #

:0 fhw | cat - header.txt

# # Appending a footer to every outgoing mail: #

:0 fbw | cat - footer.txt

You will also need to create a file called header.txt or one called footer.txt, which contains the text to add to the message (this file can be empty though, so you can create the files, but only fill them in when you need to add something)

There's a catch, however. If you are using HTML/MIME formatting in your messages then you need to also read Section 8.13: Why are MIME and HTML emails a problem for SmartList?


2.11: How can I keep the automatically added messages from multiplying when people quote them all the time?

A small refinement here: if you add something like this:

#
# Appending a disclaimer to every outgoing mail:
#

:0 fbBw * ! to unsubscribe | cat - footer.txt

then (assuming that the phrase "to unsubscribe" appears in your footer.txt file), the footer won't be repeated if some bozo list member quotes an entire message without trimming the footer. (Answer submitted by Doug Landauer)

An even cleaner solution, developed by Paul Thomas and Era Eriksson is to pull in content from your actual footer.txt to check and see if that text already exists in the message. This has the advantage of tending to automatically track changes you might make to your footer.txt file...

TEXT=`sed -n '2 p' <footer.txt`
:0 fbBw
* $ ! $\TEXT
| cat - footer.txt
What this does is load the variable TEXT with a line of text from your footer.txt file (the second line, in this example), and use that data for its test. Note that you need two $'s -- one to say "here's the variable I want to interpolate" ($TEXT) and one in front of the condition to say "interpolate variables in this condition". Also note that $\TEXT is a special form of $TEXT where regex specials will be escaped in the interpolated value. So if the second line of the footer says "*** cut here ***" then that is not a valid regular expression, but "\*\*\* cut here \*\*\*" is.

2.12: How can I automatically send files to new subscribers?

You can create a file named "subscribe.files" in the list directory. It can contain any number of archive-server commands. The results (i.e. the files requested) will be sent to the new subscriber.


2.13: Can I add comments to the addresses in the dist file?

Yes. See Chapter 5 of the SmartList manual for the exact format and limitations.

However, according to Paul Thomas:

"Perhaps it should be noted that comments in the dist file confuse multigram and those addresses have to be removed by hand should they desired to be unsubsribed, start bouncing, etc."
Another clue comes from Donald MacDougall who suggests:
Put the real name in between parentheses thus:

emailname@emailaddress (Real Name)

2.14: How does someone access the archive server?

See Chapter 4 of the SmartList manual.

Additionally, there are web interfaces to allow searching archives. A combination of Mhonarc, Wilma and Glimpse was reported to work well, and more information is available on the Mhonarc site.


2.15: What does "delink" mean?

delink is a shell script in the .bin directory of your SmartList installation which lets you remove the hard link from a file. This is important for list files like rc.init which are by default hard linked across all lists.

Usage: delink filename

A set of files which are hard linked together have the quality that a change made to any of those files will be reflected in all of them. In other words, all files in a hard linked set are always identical. This is because linked files are the same file, simply pointed to from different places on the machine. So, to make changes to only one file of a link set, you must first get that file to not simply be a reference pointer to the other link members, but be a distinct file.

If you want to make one file in a hard linked set different from the others, perhaps to customize one list differently from the others, then you must first remove that file from the hard linked set, using the delink tool or the equivalent.

You can use the showlink tool to get some idea of what a file is hard linked to. Note that the showlink tool is only designed to show hard links within the SmartList file tree.

If you cannot run shell commands against your list control files, you may be able to delink a file by copying the file to be delinked to a new name, then moving that copied file back to the name of the original file you were trying to delink.

Alternatively, you can delink a file by downloading it (scp, sftp or ftp), deleting the file from the server, and re-uploading it.


2.16: How do I add another list of accept addresses?

By default, the accept list (those who can submit to the list even if foreign_submit is off) is linked to the dist list. You may want to add another list of valid submitters, such as to allow both list and digest recipients to post to a list.

Additionally, you may want to add yet another accept list, which could include the list owner, or other valid senders:

  1. Unlink rc.submit.
  2. Find the line that ends with:
     -x$listaddr -x$listreq accept accept2
    
  3. Change it to:
     -x$listaddr -x$listreq accept accept2 accept3
    

(Tip o' the hat to Paul Thomas)

Here is another hint, submitted by Scott Pallack:

I use 3accept instead of accept3. My list has both normal (dist linked to accept) and digest (dist linked to accept2) subscribers and I can put additional addresses, such as the listowner, alternative addresses for list members, etc., into the 3accept and still be able to do things like:

grep -i user@domain.net acc*
cat acc*|grep @|sort|uniq|wc
cat acc*|grep @|sort|uniq -d
etc.

2.17: Hello, Is there a list of x-commands somewhere?

Send the X-Command 'help' to your server. ;)


2.18: How to track bounces from addresses not on list?

Through the magic of email forwarding, it's possible for a list to sustain bounces from addresses that are not directly subscribed. The scenario is that address A, subscribed, is configured to forward to address B. For whatever reason, email to address B starts bouncing, and your list gets 'error delivering email to B' even though address B is not on the list. This situation often leaves no useful clues that an admin might use to remove the actual offending email address (A).

The most reliable way to track this one down is by the use of a "probe." This is a special mailing that goes out to all list recipients such that each email is unique. Smartlist is not capable of generating such emails, so an admin must create such a system using their favorite scripting language. Each email body should contain the subscribed address to which it's being sent (address 'A' in our scenario above).

Note: Probe generators are very simple to write, and tend to be highly customized. If you have trouble with the idea of creating such a thing, contact your local guru, or take the time to learn a smattering of Perl, Python, or a similar scripting language. Remember that the only people who will actually receive your probe are recipients in good standing, so the probe should be worded to strengthen your list community...

The pseudocode would look something like this:

	Open dist list file.
	Loop: for each email on list...
		Create unique email message. (e.g. include current address)
		Inject message into email system.
	... Next in loop.
	Close dist list.

A sample email might look like this -- in this example you would replace all instances of 'A@example.com' with each individual email address on your distribution list, one at a time:

--- cut here ---
To: Hapless List Subscriber <A@example.com>
From: Beleagured Email List <List@my-list-server-domain-example.net>
X-Original-Recipient: A@example.com
Subject: Administratrivia

Greetings.

This email is being sent to you at A@example.com

You are receiving this email because you are subscribed to
Beleagured Email List <List@my-list-server-domain-example.net>

Here is some hopefully useful information about this list, to take
the sting out of receiving what is essentially a piece of junk email...
--- cut here ---

In creating a probe email, samples of the offending bounce email may be helpful to craft the probe such that the tracing information survives the bounce process. In order to keep the signal-to-noise ratio higher, some list admins use a probe to send 'Administratrivia' -- list rules, charter, instructions, web sites, etc.

Sometimes, bounce return messages contain so little information that it is not possible to determine the offending address from a simple probe. In such cases, it may be necessary to send out probes to successively narrower subsets of your distribution list, and use a process of elimination to home in on an offending address. This process generates a lot of noise on a list, and can be quite problematic on large or high volume lists.


3: Remote Administration


3.1: How do I use X-Command for remote administration?

See chapter 2 of the SmartList manual.

And no, you can't just put X-Commands in the Subject line of the header.

You either need to use a mail program which lets you create custom headers ("X-Command: "), use the hint below for including commands in the message body, or use another program to emit X-Command email, such as a web interface or a simple perl program.


3.2: Can I issue X-Commands from the message body?

Yes. There is a recipe you can find in .examples/rc.local.r00 that has the effect of causing SmartList to look in the beginning of the message body as well as within the message header for commands. This trick is helpful if you have a MUA that makes it difficult or impossible to send custom headers.

To use this recipe,

  1. Make a copy of .examples/rc.local.r00 called rc.local.r00 and place it in the list directory.
  2. Uncomment the line referencing rc.local.r00 in the list's rc.custom file.

Now X-Commands can go at the top of the message body. Add a blank line between the X-Commands and any other message content (as for instance with moderator approval).

There's a catch, however. If you are using HTML/MIME formatting in your messages then you need to also read Section 8.13: Why are MIME and HTML emails a problem for SmartList?


3.3: How do I subscribe/unsubscribe people to my list?

See chapter 2 of the SmartList manual.


3.4: How do I get a list of the subscribers?

See chapter 2 of the SmartList manual.


3.5: How can I prevent the accidental multiple subscription of an address with x_command?

Modify the x_command script with the following patch:

51,66c51,60
<          $echo $2 | $multigram -c -l32000 dist >/dev/null
<          if test $? -ne 0; then
< 	   $echo "From $2 " >$tmprequest
< 	   $echo "From: request ($listreq)" >>$tmprequest
< 	   $echo "Reply-To: $2" >>$tmprequest
< 	   $echo "To: $listreq" >>$tmprequest
< 	   $echo "Subject: subscribe $2" >>$tmprequest
< 	   $echo "$2" >$tmpfrom
< 	   $test -z "$subscribe_log" ||
< 	    $echo "x_command: subscribe" >>$subscribe_log
< 	    $subscribe <$tmprequest ||
<    	   $echo "X-Diagnostic: Unscreened, not subscribed"
<      else
<        $echo ""
<        $echo "$2 is already subscribed!"
<      fi
---
> 	 $echo "From $2 " >$tmprequest
> 	 $echo "From: request ($listreq)" >>$tmprequest
> 	 $echo "Reply-To: $2" >>$tmprequest
> 	 $echo "To: $listreq" >>$tmprequest
> 	 $echo "Subject: subscribe $2" >>$tmprequest
> 	 $echo "$2" >$tmpfrom
> 	 $test -z "$subscribe_log" ||
> 	  $echo "x_command: subscribe" >>$subscribe_log
> 	 $subscribe <$tmprequest ||
> 	  $echo "X-Diagnostic: Unscreened, not subscribed"


3.6: Is there a web interface for administering SmartList?

Werner Reisberger <werner@pure.ch> has released a web interface for the remote administration/configuration of SmartList lists, called rconfig. The current version is a complete rewrite of older versions, and appears to be quite featureful. (Currently its download site is not available. You may want to write to him for details.)

"ulf1" developed a Smartlist manager in PHP (last updated in 2014). It can be found at
https://ftp.drupal.org/files/projects/smartlist_manager-6.x-1.x-dev.tar.gz

Or, there is a perl cgi script which lets you send all X-Commands via the web. It's called x_command and is available at

http://www.pure.ch/pub/smartlist/

You can also use Matt Wright's "Formmail" script to provide a web based sign-up form. You can get the script at https://www.scriptarchive.com/formmail.html


3.7: Why isn't SmartList adding/removing names even though it says it is?

There are several possible reasons why SmartList might fail to update the dist list. The two most common reasons seem to be:

  1. An incorrect installation (possibly from a faulty package), or
  2. Incorrect file permissions.

Assuming your installation is OK, check the file permissions of your dist file. The dist file must have permissions that allow SmartList to update it. Also check the flist binary. The 's' (setuid/setgid) permissions of the flist binary determine what rights it has when it runs. The dist file must be writable by a process with those permissions.

NOTE: See Section 2.1: What does "uncomment" mean? for a discussion of text file formatting issues that can BREAK your text files.


4: Restrictions and Security


4.1: How can I restrict who can subscribe to my list?

See chapter 3b of the SmartList manual.

In addition you can use the confirm addon to forward all subscription requests at first to the maintainer who has to confirm the request before the new address will be added to the list.


4.2: How can I restrict who can post to my list?

See chapter 3c of the SmartList manual.


4.3: How can I prevent that someone (un)subscribes someone else without their knowledge?

Disable the variable auto_unsubscribe in rc.custom or use the confirm addon.


4.4: Is there a possibility to authenticate postings with SmartList?

No. email is an insecure tool and messages are easy to fake. SmartList cannot make it more secure. Each message could be faked on a list. To make email more secure use PGP or other encryption packages for email usage.


4.5: How can I restrict who can access the archive server?

See chapter 4b of the SmartList manual.


4.6: Can SmartList confirm (un)subscriptions?

Yes, You need an addon to SmartList to confirm subscriptions and/or unsubscriptions. The addon is called confirm and is available at:

http://www.pure.ch/pub/smartlist/

Confirm is used to verify email addresses and intent during subscription and unsubscriptiong operations to a SmartList list. This prevents malicious un/subscription of email addresses and ensures that only working addresses are added to the list.


4.7: How can I allow posts only from a certain domain, without using the accept file?

You need to apply the patch below to rc.submit (delink first) and copy the subcreen script within the .examples directory to yourlist_dir/postscreen.

E. g. if you want to allow all postings from anyuser@friendly.dom and reject all other addresses which are not in the accept file, the lines

case "$1" in
*@friendly.dom            exit 0 ;;       # This one is ok
esac

exit 1 # reject everything else

will be sufficient.

Patch to rc.submit:

102a103,107
>   * 2^0 ? if test -f postscreen; then \
>           screenadress=`formail -xFrom: | sed -e 's/.*<\(.*\)>/\1/' \
>           -e 's/\(.*\) (.*)/\1/'`; \
>           postscreen $screenadress;\
>           fi


4.8: How to bounce messages with a certain word in the subject?

If the string you're filtering against is "foobar", then put something like this in rc.local.s00

    :0
    * ^Subject:[ ]*.*foobar.*
    | formail -rtk -I "Subject: Posting of follow-up rejected (was: $MATCH)"

4.9: How to automatically notify posters of messages bounced for being too large?

"By default, posts that are too long are forwarded to the list owner, who has to write the person manually to explain what's happened. It would be more convenient if SmartList would send them an automated notice..."

Try this in rc.local.s00:

:0 w
* !< $size_limit
|formail -rk -I "X-Loop: $listreq" -I "From: $listreq" \
 -I "Subject: Sorry, message size for $list is max.: $size_limit bytes!" |\
 $SENDMAIL -t

It should bounce the whole message, mentioning the max. message size in the subject.

Tip 'o the hat to Werner Reisberger for this solution.


4.10: How do I allow digest recipients to post to the regular list?

Create a symlink or hardlink from the digest dist list to become the regular list's accept2 file.


4.11: How to restrict postings from a particular sender?

After consulting Section 3b of the manual you may notice that the 'reject' file functionality does not extend to checking posters, only subscribers. Meanwhile, Section 3c assumes that you wish to allow all recipients to be able to post...

To single out a sender for post rejection, you have a couple of Smartlist options. This one comes from Tim Pierce:

diff -u -r1.5 -r1.6
--- rc.submit   1999/06/07 15:31:40     1.5
+++ rc.submit   1999/06/23 17:57:17     1.6
@@ -109,6 +109,20 @@
           multigram -b1 -m -l$submit_threshold -L$domain \
            -x$listaddr -x$listreq $accept_files
    {
+     # Check to see if the poster is on the reject list.  The reject
+     # list overrides the accept list.
+
+     :0
+     * ? formail -X"From " -xFrom: -xReply-To: -xSender: -xResent-From: \
+         -xResent-Reply-To: -xResent-Sender: -xReturn-Path: | \
+         multigram -b1 -l$reject_threshold reject rejectx
+     {
+       :0 wfh
+       | formail -A "X-Diagnostic: Found on the reject list" \
+         -A "X-Diagnostic: `multigram -b1 -l$reject_threshold reject rejectx`"
+
+        HOST=continue_with_rc_request
+     }
 
      INCLUDERC=$RC_LOCAL_SUBMIT_10
Next, and perhaps simplest in some ways, you can simply add a procmail rule (e.g. in rc.local.s00) to drop that sender's postings into the bit bucket...

Finally, "JimO" sends in the following:

"Or, if you're up for a bit of work, and this poster sometimes has something useful to contribute, you can make a mini-moderated subset of your subscribers in an rc.custom.s file. I've got something like this in rc.custom.s20:

:0
* ^((From|Reply-To).*(\
   nit@wit\
  |dumb@dumber\
  |raving@lunatic\
  ))
{    :0
    * ^X-MYMODERATION: mysecretword
    {
        :0 fwh
        | formail -I X-MYMODERATION
    }
    :0 E
    {
        :0 fwh
        | formail -A "X-Diagnostic: Probationary Screen"

        :0
        ! $maintainer
    }
}
Then, if you want to approve a post, just send it on with the X-MYMODERATION header, and hack the Message-ID so it won't get rejected as a duplicate. Needless to say, you want a little shell script for the approval-resending process.

Probably more work than most people want, as usually anyone you don't want to freely post, you don't want on your list anyway, gathering your posters' addresses."


5: Digests


5.1: How do I turn a non-digest list into a digest?

See chapter 3a of the SmartList manual.


5.2: How often do digests get sent out?

See chapter 3a of the SmartList manual.


5.3: Can I force a digest to be sent every day, even if it is small?

See chapter 3a of the SmartList manual.


5.4: How can I add a message to the beginning of every digest?

See chapter 3a of the SmartList manual.


5.5: Is there a way to make the digest go out without being an attachment?

(Thanks to Charlie Summers for this one.)

Yes. Change your subscribers' email client, since it's the client that is causing the problem, not SmartList. (Sensible clients should "explode" the multipart/digest issue into separate messages, not treat the entire digest as an attachment.)

If for some reason this isn't an option, then here is a workaround. In .bin/digest change:

Content-Type: multipart/digest; boundary="----------------------------"
... to:
Content-Type: text/plain; charset=US-ASCII

Note that this will REQUIRE you to reject HTML email, since if you accept it in your list the messages in your digest will look like left-over doggie doo with all the HTML tags all over the place...

This will also affect EVERY digest on your server, unless you understand enough shell scripting to add an if/fi conditional to change it only for a specific list. (It's simple enough to do, but shouldn't be attempted unless you have done some shell scripting.)

Later came this follow-up from Charlie, explaining how to make this (or other) change to one list only, without editing the global scripts:

SmartList (possibly by design, but it seems to me more an unintentional although beneficial side effect of the PATH order) will use scripts first in the list directory, then in the ~slist/.bin/ directory. So if one wanted to change the list-d digest to text/plain while leaving the others multipart/digest, one could copy (cp) the .bin/digest shell script into the list-d/ directory, and alter that copy to taste while leaving the .bin/digest shell script alone for use by the other digests on the server. You can use this method to completely change the digest header look on one digest without affecting any other list, too.

I still like the if/fi method myself since it keeps everything together in one shell script for easy reference and debugging, but may find cp'ing the .bin/digest script a _whole_ lot easier.


6: Interaction of SmartList with Message Transfer Agents (MTAs)


6.1: Does SmartList run with Postfix?

Yes. If you use the sendmail replacement of postfix you don't need to change anything. Just make sure that SmartList can find the sendmail wrapper.

For large lists it may be advantageous to replace choplist with the direct sendmail wrapper call because postfix doesn't need a helper program for the load balancing. Uncomment the appropriate line in rc.init, and remember to add an alias to your postfix aliases db which points to your dist file, as in:

example-dist:  :include:/path/to/your/example/dist
However if you choose to let postfix replace choplist you remove SmartList's ability to filter out undesirable messages -- anybody will be able to email directly to the alias and have their message go onto your list.

You might also find the Postfix FAQ useful, in particular the section on "Protecting internal email distribution lists" http://www.postfix.org/faq.html#internal-list ...


6.2: Does SmartList run with qmail?

Yes. If you use the sendmail replacement of qmail you don't need to change anything within SmartList.

For large lists Tracy R Reed <treed@freeside.ultraviolet.org> suggested the following:

To: djb-qmail@koobera.math.uic.edu, SmartList@Informatik.RWTH-Aachen.DE
Subject: smartlist and qmail
Date: Mon, 29 Sep 1997 14:10:42 -0700 (PDT)

I have noticed that using choplist to feed qmail's sendmail clone (because choplist expects sendmail) to distribute mail to my large mailing list was incurring massive amounts of overhead and really slowing things down. If I could just get smartlist to call qmail-inject directly, it would greatly speed things up. To this end, I modified my rc.init file:

alt_sendmail="smart-qmail dist $listreq"

And put this perl script in ~list/.bin/smart-qmail

#!/usr/bin/perl
$from = $ARGV[1];
$addyfile = $ARGV[0];
while() {
  $message .= $_;
  }

open(BLAH,"<$addyfile") || die "Doh! No address file!\n"; $x = 0; while () { if(/Only addresses below this line can be automatically removed/) { next; } chop; $addy .= " ". $_; $x++;

# 30 is kinda arbitrary, I just don't want to run into any ugly command # line length limits.

if ($x == 30) { open(INJECT,"|/var/qmail/bin/qmail-inject -f$from $addy"); print INJECT $message; close INJECT; $x = 0; $addy = ""; } } open(INJECT,"|/var/qmail/bin/qmail-inject -f$from $addy"); print INJECT $message; close INJECT;

Another note -- Paul Thomas points out that

"Qmail bounces email differently than sendmail. It does a 'local' bounce and a 'remote' bounce. SmartList will process to the best of it's ability a 'remote' bounce, but 'local' bounces are written to a qmail "Mailbox' file and overlooked by SmartList. In short, qmail will work with SmartList but to make everything work right one is looking at a major hack of SmartList."

6.3: Sendmail can't run flist. Why? (smrsh)

If you log an error message something like

554 "|exec /path/to/flist listname"... Service unavailable
the reason is most probably that sendmail is using the smrsh "Sendmail Restricted SHell" environment for shell calls.

The way a message initially gets to the SmartList system is through a sendmail alias that pipes the email to the program. The alias probably looks something like:

listname: "|exec /path/to/flist listname"
However, some configurations of sendmail attempt to increase security by restricting allowable programs to those in a certain directory. In this case, you need to create a symlink to the flist binary in the smrsh directory.

Try these steps:

  1. Find the correct location of your smrsh directory. You may have to examine your smrsh source, or use the strings command to examine your smrsh binary.

  2. Create a symlink to your flist binary in the smrsh directory:
    ln -s /path/to/flist /location/of/smrsh/flist
  3. Change the alias for the list to simply be
    listname: "|flist listname"
    You don't need the path to flist now, since sendmail is only going to look in one place (your smrsh directory). You don't need the exec either.

Don't forget to run newaliases after you modify your aliases file to effect your changes.


6.4: How do I get SmartList to work with different virtual accounts?

I installed SmartList on the different virtual accounts and have a link in "/etc/smrsh" to the "flist" program in the ".bin" directory of each account... It works but only the one linked to "flist"... In order to be able to have links to all the "flist" programs of all the accounts in "/etc/smrsh", I used different links with different names but they just DON'T work... it seems to me like Procmail wants the link in "/etc/smrsh" to be named "flist" ONLY, in order to work...

All the names need to begin with "flist". You could use names like

flist.domain1
flist.domain2
...
(Thanks to Philip Guenther for this one.)

7: Addons and enhancements for SmartList


7.1: Is it possible to have more than one list maintainer?

Yes apply the following patch (from Alan Stebbens) to your rc.request file (don't forget to remove the hard link before). After applying the patch you can use several maintainer addresses in rc.custom separated by a comma (don't include blanks between the addresses).

--- installed/rc.request        Thu Oct 31 00:02:23 1996
+++ rc.request  Fri Oct 18 09:11:58 1996
@@ -291,20 +264,20 @@
#

# is there more than one maintainer? -maintainer_re=$maintainer -maintainers=$maintainer +maintainer_re="$maintainer" +maintainers="$maintainer" Maintainer=Maintainer

:0 * maintainer ?? , -{ maintainer_re=`echo "$maintainer" | sed -e 's/,/|/g' -e 's/\./\\\./g'` - maintainers=`echo "$maintainer" | sed -e 's/,/ /g'` +{ maintainer_re="`echo $maintainer | sed -e 's/,/|/g' -e 's/\./\\\./g'`" + maintainers="`echo $maintainer | sed -e 's/,/ /g'`" Maintainer=Maintainers }

:0 wf:dist.lock -* $^$X_COMMAND:.*$maintainer_re[ ]+$X_COMMAND_PASSWORD

If you have access to your aliases file, another approach is to set the maintainer in rc.request to a "role address" defined in the aliases file to expand to all maintainers, for example "mylist-owner". Maintainers would then use the "mylist-owner" address to issue list commands, and any email generated for the maintainer will be delivered to all maintainers defined in the aliases file.


7.2: Can I remotely update files used by SmartList like help.txt?

There is an addon to SmartList that enables you to do this via email or a web interface. You should only use this package called rconfig if you know how SmartList works. Otherwise you could completely destroy a list. rconfig is available at

http://www.pure.ch/pub/smartlist/


7.3: How can a subscriber temporarily suspend subscription?

There is an addon to SmartList that enables subscribers to suspend their subscription for a given number of days. It's called suspend and is available at http://www.pure.ch/pub/smartlist/


7.4: Where can I find more SmartList information and utilities?

SmartList Utilities and Patches - send email to alan.stebbens@software.com with the subject

"send smartlist library".


7.5: Can SmartList send individualized messages to each list recipient?

No. SmartList is designed to take advantage of the bulk processing capabilities of the MTA. It is not designed for personalized mailings.

It is certainly possible to use the dist list to generate personalized mailings using another program. This can be very useful to track certain types of bounces.


8: Miscellaneous


8.1: How can I make my list moderated?

See chapter 3f in the SmartList manual.

Note that if you have enabled a moderator password (see the moderator_PASSWORD option in the list's rc.custom file), then the format for the approval header element is:

Approved: the_address_of_one_of_the_moderators moderator_password
(Thanks to Merrick Munday for that clarification.)

Please see 3.1: How do I use X-Command for remote administration? and 3.2: Can I issue X-Commands from the message body? for more discussion on working with these header-based commands, and complications caused by MIME formatted messages.

If you're fortunate enough to be on a machine with a command line mail tool such as sendmail/qmail etc, and your email client uses standard text format for message saving and/or storage, then you might be interested in this suggestion courtesy of Martin Mokrejs:

First, save the returned/rejected mail that you wish to approve into a new file, making sure that the headers and any attachments are intact. (E.g. in pine Export does not save the attachment, so perhaps use it's Save function to save it to a new folder, then quit pine or close access to that folder).

Next, edit the file, cleaning the headers as necessary, and adding your approved header:

Approved: maintainer@my.address my-password
then, pipe the mail file again into your mail, e.g.:
sendmail -t < rejected-mail
In this way, the mail will appear as originating from the original author, not from the maintainer.

8.2: How can I change the messages that SmartList sends to people?

The messages that SmartList sends out are linked in the ~slist/listname directory, and can be edited using any available file editor. Note that if you wish to change these files for only one list, then you must first delink the file.

The files that it sends and their uses are:


8.3: How can I add "[listname]" to all of the mailing list messages for easy filtering?

In your rc.custom file, uncomment the line that reads "RC_LOCAL_SUBMIT_20 = rc.local.s20", and make a file called "rc.local.s20" in your list directory, in that file put this:

# Send to the digest BEFORE we do anything stupid...it will send it back to
#  us, but that's ok since we will ignore the duplication
# NOTE: Don't put this part in if your list isn't digested...
# (See note from Charlie Summers below)
:0 c
|/path/to/flist digested-listname

#
# NOW munge the subject, append a header/footer, whatever...

# Be sure to replace "[listname]" with what you would like to be at the
# beginning of the subject line. (Posted to
# SmartList@Informatik.RWTH-Aachen.DE by Leigh Wolenczak ).
SUBJ=`formail -zx Subject:`   # extract the subject
:0fw
* !  Subject:.*\[listname\]
* !  Subject:.Re: *
| formail -I "Subject: [listname] $SUBJ"
On this subject, Charlie Summers points out:

Using rc.local.s20, lets you modify the email after you add it to the digest, thereby helping to avoid needless clutter in the digests. This has the added benefit that we don't have to subscribe the digest list to the undigested list's dist file and waste a call to sendmail to deliver the mail that we can easily pipe directly into flist; it also means the additional Received: header fields caused by mailing the file across the directory aren't an issue. (I can't figure out why something this simple isn't included in the Manual file as The Right Way To Do It.)

Another variation of the above is submitted by Martin Mokrejs:

# Extract the subject ONLY IF the listname is NOT YET present in the Subject
:0fh
* ! Subject:.*\[listname\]
* ! Subject:[   ]*((R|r)(e|E)):[     ]*
{ SUBJ=`formail -zx Subject:` }
  
  # if the preceding recipe succeeded
  :0fawh
  | formail -I "Subject: [listname] $SUBJ"

Another approach, courtesy of Tim Pierce: "Here's what we're using here. (This assumes that a `subject_prepend' variable is defined in rc.custom.)

     :0f
     * subject_prepend ?? .
     * !digest_flag ?? y
     * $!^Subject:.*\\[${subject_prepend}\\].*
     * ^Subject:\/.*
     | formail -I "Subject: [$subject_prepend]$MATCH"
This looks for [$subject_prepend] anywhere in the Subject line, not just at the beginning. So the prepended tag doesn't get repeated. It is possible to get
        Subject: Re: [listname] Re: stuff
but not
        Subject: [listname] Re: [listname] stuff
..."

Some listers have suggested that the above don't quite cut it, and some better solutions are forthcoming. The problem is to NOT change the subject if it's a reply, or otherwise already has the subject tag added...

First, Cary Coutant sent this one in, stating that "This recipe won't add the listname if it's already anywhere in the subject line.":

#
# rc.local.s10
#

#
# Insert the list name into the subject field (if not already there)
#

:0 fhw
* $ ! ^Subject:.*\[$list]
* ^Subject:\/.*
| formail -I "Subject: [$list]:$MATCH"

Jari Aalto has developed, among many other procmail goodies, a recipe to control subjects: http://www.procmail.org/jari/pm-code.html and in particular: http://www.procmail.org/jari/pm-code-body.html#272


8.4: How can I add a custom header to all of the mailing list messages?

Uncomment the line "RC_LOCAL_SUBMIT_20 = rc.local.s20" from your rc.custom file, and add the following to your rc.local.s20 file:

#
# Adding some custom headers (although it will work, this is not
# needed for Reply-To related headers, see the reply_to variable in
# rc.custom):
#

:0 fhw | formail -i "X-Subliminal-Message: SmartList is great" \ -I "X-Mailer: procmail, which is great too :-)"


8.5: How can I remove some headers from all of the mailing list messages?

Uncomment the line "RC_LOCAL_SUBMIT_20 = rc.local.s20" from your rc.custom file, and add the following to your rc.local.s20 file if you want :

# # To get rid of some headers: #

:0 fhw | formail -I X-Mailing-List: -I X-Mailer:

# # To get rid of all X- fields: #

:0 fhw | formail -I X-


8.6: Is there an easy way to remove duplicates from the dist file?

This script was contributed to the SmartList mailing list by Mark David McCreary, you can run this from your list directory (where the dist file is located), and it will remove the duplicates and sort the entries that appear below the "(Only addresses below this line can be automatically removed)" line.

#! /bin/sh
:
#
#   rc.dedup.list
#
#   mdm 11/5/97   borrow code from  David W. Tamkin, with additional
#                 suggestions from Martin Konold and Hal Wine
#
#
#  This script will sort and remove duplicates from a Smartlist
#  distribution file.  Smartlist dist files contain a comment line
#  that looks like this
#
#      (Only addresses below this line can be automatically removed)
#
#  Addresses above and including this line will not be sorted.
#

lockfile=lockfile # /usr/bin/lockfile cat=cat # /bin/cat rm=rm # /bin/rm sed=sed # /bin/sed

# create lock file to discourage SmartList running a job

$lockfile -10 -l3600 -r11 -s30 rc.lock

$sed -e "1,/(Only addresses below this line can be automatically removed)/{ ;w slabove.$$ ;d; } " dist | sort -fu -o slbelow.$$

$cat sl?????.$$ > dist $rm -f sl?????.$$

$rm -rf rc.lock


8.7: How can I keep people from sending HTML/attachments/other junk to the list?

Several good recipes for this have been posted to the SmartList mailing list.

This just in: AOL 6.0 can be made to send a message in non-html format. Select the entire message, right click on it, and select TEXT - NORMAL. (Thanks to Ted Fulmer for that tip.)


8.8: How do I make replies go to the list instead of to the sender?

The "Reply-To:" header element is designed to allow you to do this, and the reply_to variable within your rc.custom file gives you the ability to control this header element in list processed email.

There are several approaches. Arguably the simplest is, in the rc.custom file, uncomment the line:

reply_to = $listaddr

and set it to something like this:

reply_to = "Reply-To: $listaddr"

Paul Thomas notes that:

"The 'Reply-To: ' rule in rc.custom is overwritten if a post to the list already contains a 'Reply-To: ' with a different address. Putting something like:
        :0 fhw
        | formail -I "Reply-To: test@somedomain.com"
in rc.local.s20 might be better."

8.9: How can I make replies go to the original sender?

If you run "announcement" style lists, where you want to discourage list discussion by having replies go to the original author, then in the rc.custom file, uncomment the line:

reply_to = $listaddr

and set it to something like this:

reply_to = "Reply-To: `formail -xFrom:`"

Note that the formail -xFrom: command is enclosed in backticks (`), which cause procmail to run the formail command in a subshell and return that command's results.


8.10: Why doesn't putting "Reply-To: $listaddr" in my rc.custom file work for all subscribers?

Smartlist will not overwrite an existing Reply-to: address, so if the member's mail client already adds a Reply-to:, it will remain in the message that appears on the list.

You can change that behavior with a simple one-byte change of the rc.submit file.

Change the line that says:

        -a"Subject: Unidentified subject!" ${reply_to:+"-a$reply_to"} \
to:
        -a"Subject: Unidentified subject!" ${reply_to:+"-i$reply_to"} \

Note that you will need to delink the rc.submit file first, because otherwise your change will affect all lists run from that directory.

(Thanks to Matt Kemner for this one.)


8.11: How can I include the number of subscribers in the footer I'm attaching to each list mail?

The idea is to keep the footer text in a file called footer.template. In that file you embed $COUNT wherever you want the list population inserted. Then you cause your subscription and unsubscription processing to count the list members, and interpolate footer.template to produce a footer.txt file.

Perhaps your footer.template file might look like this:

---
To unsubscribe, blah blah blah.
This list has $COUNT subscribers.

For this to work, you need to uncomment

RC_LOCAL_REQUEST_20  = rc.local.r20
from either your rc.init or your rc.custom. You also need to have cc_unsubrequests and cc_requests set to yes. Then, add this recipe to the end of normal sub/unsub processing; rc.local.r20:
# Count list members, and update footer.txt using footer.template.
:0
* ^X-Diagnostic: (Processed|Added to the subscriber list)
{
    COUNT=`grep -cv '^(' dist`
    :0cWi
    | sed 's/\$COUNT/'"$COUNT"/g footer.template >footer.txt
}

There's a catch, however. If you are using HTML/MIME formatting in your messages then you need to also read Section 8.13: Why are MIME and HTML emails a problem for SmartList?


8.12: How to send an auto-reply back when someone posts?

Courtesy of Ravi Pina:

Placing this in rc.local.s20 and uncommenting the appropriate line in the rc.custom file will provide this functionality.

# recipe to send an autoreply
 
REPLY_TEXT=/path/to/autoreply.txt
REPLY_SUBJECT="Message Received"

:0 whc
| formail -rtzc -xTo: >tmp.from

:0 ch :autoreply.lock
| (formail -rtzc -I "Subject: $REPLY_SUBJECT" \
-A"Precedence: junk" ; cat $REPLY_TEXT) | \
$SENDMAIL -t `cat tmp.from`

Ravi couldn't stop thinking about this one, so he then sent this:

"I've made the auto-reply a bit more robust and bloated :)

Place the following in rc.local.s20 and uncomment the corresponding line in your unlinked rc.custom. REPLY_TEMP is a text file that gets mailed back to the original sender. It will be sent as MAIL_FROM. If you want to put a user name in the header From: line, do so in HDR_FROM. The subject will be set to REPLY_SUBJECT.

If you want to use the original subject in your reply, place `$SUBJECT' where you want it to appear in the REPLY_TEMP file and set MUNGE_BODY appropriately."

#MUNGE_BODY=no
REPLY_TEMP=/path/to/autoreply.txt
MAIL_FROM="moo@cow.org"
HDR_FROM="moo@cow.org (moo!)"
REPLY_SUBJECT="Message Received"
ORIG_SUB=`formail -zx Subject:`

:0 whc
| formail -rtzc -xTo: >tmp.from ; \
cp $REPLY_TEMP tmp.autoreply

:0 whc
* MUNGE_BODY ?? y
| sed 's/\$SUBJECT/'"$ORIG_SUB"/g $REPLY_TEMP > tmp.autoreply

:0 ch :autoreply.lock
| (formail -rtzc -I "Subject: $REPLY_SUBJECT" -I "From: $HDR_FROM" \ 
-A"Precedence: junk" ; cat tmp.autoreply) | \
$SENDMAIL -t -f $MAIL_FROM `cat tmp.from`

8.13: Why are MIME and HTML emails a problem for SmartList?

This FAQ entry originated in a discussion about problems with embedding X-Commands in the body of a MIME formatted message. (See Section 2.10) The exact same type of trouble arises when you want to add a custom message heading or footer to the body of a message. (Note that "body heading" is not the same as a message header entry!) (See Section 3.2)

Note that HTML formatted email is sent as a MIME formatted message.

Hans-Albert Schneider, by way of Martin Mokrejs contributed this useful explanation of why X-Commands from the message body won't work if the email is divided into MIME segments! This is because MIME segments create extra blank lines between the header and the body; something like this:

Mail header (with To:, Subject:, Date:, etc.)
    (empty line)
MIME separator
MIME header of first component
    (empty line)
body of first MIME component
MIME separator
MIME header of second component
    (empty line)
body of second MIME component
MIME separator

So the body actually starts no sooner than after the second blank line... Now, the details:

X-Commands:
The easy answer is to put the X-Command in the email header. Barring that, you could possibly hack SmartList to look for X-Commands anywhere in a message, but that would be somewhat dangerous.

Body heading:
Text inserted by recipes immediately after the first blank line (i.e. at the start of the body if the message were a regular text email) won't show up because it's before the first MIME separator, and thus doesn't belong to any of the MIME message parts.

You'd need to properly determine which MIME sections to modify and which to leave alone (e.g. leave all attachments and binaries alone). Then you'd need to figure out how to parse each segment to be modified, and insert the text appropriately, e.g. as html before the closing html tag in the html section, and just as text in the text section. Very messy. Probably possible if you're good with perl and understand the MIME format, and if all of your submitters use proper MIME formatting email clients. God save you if someone decides to post the newsletter as a word doc.

Body footer:
Similar to the previous case, but here the text is injected after the final MIME separator, and so again doesn't belong to any of the MIME message parts.

A proper MIME-aware text insertion recipe would have to know how to modify ALL of the text message segments without touching any message segments which contain non-message text data, e.g. attachments.


Procmail Home | SmartList Manual | Mailing List