[Pkg-exim4-users] [Solved] Use a System Filter to AutoBCC/AutoSave Sent Msgs

[groups, freeman]

********************************************
** AutosaveToMailBox_AllRemotelySentItems **
********************************************

The Exim docs said to use a system filter so I did, though it was a bit 
of a challenge.

This email is to outline the config I used so that anyone else can have 
a quicker path to bliss.

I'm using Debian v6.0, with Exim v4.72-6+squeeze3 in un-split configuration.

My goal is to have all (or actually, just some) emails that get SENT 
from an account get auto-bcc'd (actually: automatically saved) into the 
inbox of the email address that sent out the message in the first place. 
(e.g. This is helpful to keep a copy of all outbound messages sent from 
your cellphone, for your records).

First, about my setup.

All my email is retrieved via POP3. All my sent email goes out via Exim. 
Same server. The mailbox is one file per user, located as /var/mail/username

I use Thunderbird on my desktop as the permanent, complete warehouse for 
all emails.

However I have migrated two of my email addresses to a laptop PC also 
running TB, and those addresses are primarily used from there nowadays. 
(They get copied over to the desktop warehouse from time to time to keep 
the warehouse up-to-date). However when I send an email from the 
desktop, using one of the now-laptop-resident addresses, I'd still like 
a copy to show up in the sent folder on the laptop.

Of note: In addition to those two ('upper') email addresses on my 
Blackberry I also have two 'lower' addresses on it as well.

The Blackberry was the instigating factor for me getting into this 
'auto-BCC' stuff: I migrated from the Blackberry_BIS-based email, to the 
new Q10. The old BIS service had a built-in AutoBCC function so that any 
messages sent from the Blackberry could be delivered to another email 
address for eventual filing into the sent folder of the sender's TB 
account. The new Q10 had no such automated function built in, so because 
all my Blackberry-originated emails are routed through my personal Exim 
server I could intercept them on their way out and place a copy directly 
into the mail file of the sender, on my server, for later retrieval and 
placement into the proper 'sent' folder.

Finally, I wanted to NOT have duplicate copies on my laptop or desktop. 
When I send from that device I wanted Exim to be able to realize that a 
copy would be already going into my sent folder (for certain addresses) 
so in that case Exim DIDN'T need to place a copy into the sender's 
mailbox file.

- Regarding the sample config below:
- Four email addresses are of interest to me, because that encompasses 
all that are used outside of the Desktop PC
- I have two email addresses (upper1@ and upper2@) that are located on 
the laptop AND on my Q10
- I have two other email addresses (lower1@ and lower2@) that are 
located on my desktop AND my Q10
- my email addresses are in format of upper1 at applepie.mysite.com 
(possibly upper1 at apple.mysite.com)
- my laptop's hostname is 'laptop'
- my desktop PC(s) are hostnamed MainPC5, MainPC6, PC5 and PC6
- I use auth on my SMTP - if you don't then comment out the 
'sender_host_authenticated' check

So below is the config I used to get the results I wanted.

Though I solved my initial need some questions still remain, if there 
are any takers...
- where in the docs can I read about 'mode_fail_narrower = false' and 
'no_mode_fail_narrower' ... or what do they do?!
- I'd prefer to have the BCC'd addresses remain in the copy of the email 
that gets placed into the sender's mailbox but Exim strips the BCC's. 
However it appears that $recipients contains such addresses - if I had 
more time I'd master the coding to parse them out and then use 
'add_header bcc: xxx at yyy.zzz' or similar since that would then perfectly 
duplicate what happens when you send an email from your actual client, 
and any BCC's addresses get included in the saved copy. Maybe one day.

===============================

Overview:
FWICT I needed to do two distinct steps:
A) activate a system filter
1) create a new file that is the filter
2) adjust /etc/exim4/exim4.conf.template to point to the new file as the 
filter (and the proper transport, too)

B) create a new transport for the filter's use since no baked-in 
transport does what we need

===============================

A1) Create New Filter File:
- filename is /etc/exim4/systemfilter.txt
- permissions for me are 0640
- contents are:
= = = = = = = = = = = = = = = = = = = = = = = =
> # Exim filter <<== do not edit or remove this line!
> ##############################################################
>
> if error_message then
> finish
> endif
>
> logfile /var/log/exim4/system_filter.log 0664
> logwrite "\n"
> logwrite "[$tod_log] SystemFilter - ENTRY - 
> sender_address_local_part=$sender_address_local_part"
>
> if not first_delivery then
> logwrite " 1-first_delivery=false, so just finish"
> finish
> endif
>
> if $sender_address_domain is not applepie.mysite.com then
> logwrite " 2-domain is NOT applepie.mysite.com, so just finish"
> finish
> endif
>
> if $sender_host_authenticated is not plain_saslauthd_server then
> logwrite " 3-auth ($sender_host_authenticated) is not good 
> ('plain_saslauthd_server'), so just finish"
> finish
> endif
>
> if not foranyaddress $header_from: ($thisaddress matches 
> "^(upper1|upper2|lower1|lower2)@") then
> logwrite " 4-not a sought email sendername, so just finish"
> finish
> endif
>
> #logwrite " host=$host"
> #logwrite " host_address=$host_address"
> #logwrite " received_ip_address=$received_ip_address"
> #logwrite " received_port=$received_port"
> #logwrite " self_hostname=$self_hostname"
> #logwrite " sender_fullhost=$sender_fullhost"
> #logwrite " sender_host_address=$sender_host_address"
> #logwrite " sender_host_name=$sender_host_name"
>
> # Certain email addresses have their actual sent
> # folder stored on particular PC's, so...
> # for emails sent from those addresses, on those computers:
> # we don't need to Autosave a copy of what we sent since
> # it will be saved there automatically by the email client
> # (Thunderbird in my case). In fact it is preferable since
> # it avoids duplication (or auto-deletion of duplicates).
> #
> # Candidates are:
> # Sender_field host_name
> # ===============================================================
> # (upper1|upper2) laptop.apple(pie)?.mysite.com
> # (lower1|lower2) (Main)?PC(5|6).apple(pie)?.mysite.com
> #
> #############################################################################
> # Stored on laptop?
> ########################################
> if ${lc:$sender_host_name} matches 
> "^laptop\\.apple(pie)?\\.mysite\\.com\\$" then
> if foranyaddress $header_from: ($thisaddress matches 
> "^(upper1|upper2)@") then
> logwrite " 5-Sent from laptop client (TB) for upper1/upper2, so just 
> finish"
> finish
> endif
> endif
>
> # Stored on Main desktop?
> ########################################
> if ${lc:$sender_host_name} matches 
> "^(Main)?PC(5|6)\\.apple(pie)?\\.mysite\\.com\\$" then
> if foranyaddress $header_from: ($thisaddress matches 
> "^(lower1|lower2)@") then
> logwrite " 6-Sent from MainPC client (TB) for lower1/lower2, so just 
> finish"
> finish
> endif
> endif
>
> unseen save /var/mail/${lc:$sender_address_local_part} 660
>
> logwrite " DONE - we copied the email to the sender's inbox"
> finish
= = = = = = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = = = = = =

A2) Activate New Filter:
- edit file /etc/exim4/exim4.conf.template
- at effectively the very beginning of the file add these 3 lines:
= = = = = = = = = = = = = = = = = = = = = = = =
> system_filter=/etc/exim4/systemfilter.txt
> system_filter_file_transport=AutosaveToMailBox_AllRemotelySentItems
> #NotNeeded system_filter_user=Debian-exim
= = = = = = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = = = = = =

B) Create New Transport:
- edit file /etc/exim4/exim4.conf.template
- right after the comment lines that are beneath the line saying:
begin transports
(so about line 1400 in the file, for me)
add in the following
= = = = = = = = = = = = = = = = = = = = = = = =
> AutosaveToMailBox_AllRemotelySentItems:
> # For system filters,
> # $local_part="system-filter"
> # so it's of no use; instead, derive it from sender_address
> ########################################
> debug_print = "T: {CUSTOM - 'AutosaveToMailBox_AllRemotelySentItems') 
> appendfile for ${sender_address}"
>
> driver = appendfile
>
> #NFG!# file = /var/mail/$local_part
> file = /var/mail/${lc:$sender_address_local_part}
> mode = 0660
>
> user = ${lc:$sender_address_local_part}
> group = mail
>
> delivery_date_add
> envelope_to_add
> return_path_add
>
> # We'll need to add a (custom!) filter in TB to move to the
> # sent folder emails which have this header set to 'true'
> ##################################################################################
> headers_add = X-AutosaveToMailBox_AllRemotelySentItems: true
>
> # Alternatively, one could conceivably filter on:
> # Envelope-to: system-filter
> # since that's a hallmark that the email is
> # an Autosaved copy of a remotely-sent msg
> ##########################################################################
>
> #? mode_fail_narrower = false
> #? no_mode_fail_narrower
>
> # Note!...
> ##########################
> # $recipients
> # This variable contains a list of envelope recipients for a message. 
> A comma and a space separate
> # the addresses in the replacement text. However, the variable is not 
> generally available, to prevent
> # exposure of Bcc recipients in unprivileged users’ filter files. You 
> can use $recipients only in these
> # cases:
> # (1) In a system filter file.
> # ...
= = = = = = = = = = = = = = = = = = = = = = = =

And that has it. Have fun!