Re: Spam

[Damion Alexander <dalexand@bard.edu>]

On 11/12/2012 10:09 AM, Csillag Tamas wrote:
> On Thu, Nov 08, 2012 at 01:59:37PM -0500, Damion Alexander wrote:
>> On 11/08/2012 01:17 PM, Rich Graves wrote:
>>> I do it downstream, with sendmail and mailfromd, just because I'm one of
>>> the few crazies who were working with milters from the beginning.
>>
>> We implemented a few outgoing zimbra mta's with mailfromd a few weeks
>> ago. Our mailfromd script is not perfect but we've prevented nearly all
>> outgoing spam blasts since.
> 
> Can you share your config/script to be used as a starting point?
> 
> Thanks!
> 
> Regards,
>   cstamas
> 

mailfromd.mf = script
mailfromd.conf = config

any __TEXT__ strings are placeholders for actual values.

Basic logic of the script

  if whitelisted accept blindly
  if blacklisted put all email in hold queue for review
  if recipient > GL_TRACK_NUM greylist
  if greylisted and recipient rate > RCPT_GL_MAX/RCPT_MAX_TIME blacklist
  if rate > RCPT_MAX/RCPT_MAX_TIME blacklist

We have a set of Zimbra MTA's to handle incoming mail and a set to
handle mail destined outside of our domains. Users log into the 'inside'
set via clients. The outside set are the ones that use mailfromd via
milters.  Because of this recipient counts are only for non local
domains. i.e. send to 80 people 50 on-campus 30 off and the milter only
sees 30.

whitelist and blacklist db's are Berkeley DB's and can be managed via
perl scripts using 'tie' (steve hillman showed us his scripts for this
purpose).

The good about this config: large blasts of spam never leave the
outgoing servers. Not being put on blacklists outweighs the negatives.

The bad:

- users who are used to sending emails to large BCC fields will always
be caught by this. We untrap about 3-4 mailings a day. They haven't
noticed any delays (yet). We've been trying to move them over to mailing
lists.

- in the middle of a large torrent of spam being sent, the incoming
servers (postfix) will reuse a connection by issuing RESET.  As written
the recipient_count doesn't get reset so any additional email senders
will inherit the previous recipient count and trapped.

- any spammer with the patience of a saint can eventually find the magic
numbers. most are never that patient. Upside is that they will be
extremely (self) rate limited.

- token bucket rate limiting algorithms make my brain hurt

Damion

-- 
Damion Alexander
Manager of Networks and Systems
Bard College
Annandale-on-Hudson, NY 12504
(845) 758-7225

# mailfromd conf

state-directory /var/lib/mailfromd;
relayed-domain-file /usr/local/etc/mailfromd-domains;

listen inet://localhost:10057;

# config for greylist db

database greylist {
    enable true;
    # we don't use greylist in the traditional sense so lower the expiration to 30 mins.
    expire-interval 1800;
}

/*

  mailfromd script for (outgoing) mail servers
 
  if whitelisted accept blindly
  if blacklisted put all email in hold queue for review
  if recipient > GL_TRACK_NUM greylist
  if greylisted and recipient rate > RCPT_GL_MAX/RCPT_MAX_TIME blacklist
  if rate > RCPT_MAX/RCPT_MAX_TIME blacklist

  
  
*/


#pragma regex +extended +icase
#pragma stacksize 128k twice
/* con-tassios is needed to activate is_greylisted */
#pragma greylist con-tassios

require status
require dns
require localdomain
require rateok

  
# times

## greylist
const GL_TRACK_TIME interval("5 min")
## max rcpt over time
const RCPT_MAX_TIME interval("30 min")

# recipient limits

const GL_TRACK_NUM __NUMBER__
const RCPT_MAX __NUMBER__
const RCPT_GL_MAX __NUMBER__

# rates

## max gl time
const GL_TRACK_RATE_INTERVAL (( GL_TRACK_TIME / RCPT_GL_MAX ) * 1000000) 
##  
const RCPT_MAX_RATE_INTERVAL (( RCPT_MAX_TIME / RCPT_MAX ) * 1000000) 

# tags

const GL_TRACK_TAG "gltrack"
const RCPT_MAX_TAG "rcptmax"

# databases
const WHITELIST_DB "/var/lib/mailfromd/whitelist.db"
const BLACKLIST_DB "/var/lib/mailfromd/blacklist.db"

# runtime variables

number rcpt_counter
string this_sender
number iam_quarantined 0


/* Begin block is executed at the beginning of each SMTP transaction.
 */
/* 
begin
do
  set ehlo_domain localdomain()
done
*/


prog envfrom
do
   if ((dbmap(WHITELIST_DB,$1)) or ($1 matches "__MAILLIST_SERVER_DOMAIN__") or ($1 == "<>"))
	echo $1 . " is whitelisted"
	accept
   fi

   set this_sender $1
done


prog envrcpt
do 
   set rcpt_counter rcpt_counter + 1
done

/* Do work in eom.  Although we have gathered most all information
   by eoh the message ID is not available till eom.  Since this runs
   on a remote box, we need something to tie into the actual machine
*/


prog eom
do

 if (dbmap(BLACKLIST_DB,this_sender)) 
	set iam_quarantined 1
	echo this_sender . " was already blacklisted (mid $i)"
	quarantine(this_sender . " was already blacklisted (mid: $i)")
	continue
 fi

 if (not tbf_rate(this_sender . RCPT_MAX_TAG, rcpt_counter, RCPT_MAX_RATE_INTERVAL, RCPT_MAX))
	# blacklist, went over the line
        dbput(BLACKLIST_DB,this_sender,1)
	set iam_quarantined 1
	echo this_sender . " is now blacklisted, over max recpipient/time quota (mid $i)"
	quarantine(this_sender . " is now blacklisted, over max recpt/time quota (mid: $i)")
	continue
 fi

 if (is_greylisted(this_sender . GL_TRACK_TAG))
	if (not tbf_rate(this_sender . GL_TRACK_TAG, rcpt_counter, GL_TRACK_RATE_INTERVAL, RCPT_GL_MAX))
		# no more allowed widgets.  + you went over allowed settings. you go way now
		dbput(BLACKLIST_DB,this_sender,1)
		set iam_quarantined 1
		echo this_sender . " is now blacklisted. over greylist quota (mid $i)"
		quarantine(this_sender . " is now blacklisted. over greylist quota (mid: $i)")
		continue
	else 
		# still greylisted. tempfail
		tempfail 451 4.7.1 "Greylisted for %greylist_seconds_left seconds"
	fi
 fi

 if (rcpt_counter >= GL_TRACK_NUM)
	if (greylist(this_sender . GL_TRACK_TAG,GL_TRACK_TIME))
		if ( tbf_rate(this_sender . GL_TRACK_TAG, rcpt_counter, GL_TRACK_RATE_INTERVAL, RCPT_GL_MAX))
			# if statement handles adding to the rate tracking.

			echo this_sender . " has been greylisted. recipient_count = " . rcpt_counter

		fi
		tempfail 451 4.7.1 "Greylisted for %greylist_seconds_left seconds"
	fi
 fi

done