Due to my frustration with my personal e-mail, I recently implemented some additional spam filtering which was amazingly effective.
In the past 24 hours, the system has stopped 775 spams destined to valid users, and has accepted 37 legitimate messages. Previously it would have accepted those messages. Most would end up in my spam folder to be reviewed later, but others would end up in my inbox.
This decrease in spam hitting my inbox is through Greylisting, implemented using Postgrey.
A properly implemented mail server needs to be able to deal with a temporary delivery failures, and try again later. A system implementing Greylisting will keep track of the source IP address as well as the source and destination e-mail addresses. The first time a unique combination of these three things is seen, the message is given a temporary failure for 10 minutes.
Legitimate mail is eventually delivered. Spammers give up and don't come back.
Saturday, May 03, 2008
Friday, April 11, 2008
I reject your e-mail
I've always heard that the correct behavior of a mail server is to reject e-mail for local undeliverable addresses rather than accepting them and then bouncing. I never put too much thought into it though until recently when I took over management of our e-mail infrastructure.
When the systems were handed to me, their queues generally had about 800-1000 e-mails
to be delivered at any given point. As I dug into why, I found that the majority of those e-mails were outgoing bounce messages which were undeliverable for one reason or another.
After a few changes, those systems are now rejecting around 1,000,000 spam messages a day. These are messages that would previously have been accepted and sent back out on the internet as backscatter.
Rejecting instead of bouncing allowed me to significantly cut down on the amount of processing power, bandwith and disk space used on these systems. Not to mention cutting down on the amount of e-mail that the backscatter victims were receiving.
Now if only everyone else who runs mail servers would figure this out.
When the systems were handed to me, their queues generally had about 800-1000 e-mails
to be delivered at any given point. As I dug into why, I found that the majority of those e-mails were outgoing bounce messages which were undeliverable for one reason or another.
After a few changes, those systems are now rejecting around 1,000,000 spam messages a day. These are messages that would previously have been accepted and sent back out on the internet as backscatter.
Rejecting instead of bouncing allowed me to significantly cut down on the amount of processing power, bandwith and disk space used on these systems. Not to mention cutting down on the amount of e-mail that the backscatter victims were receiving.
Now if only everyone else who runs mail servers would figure this out.
Monday, April 07, 2008
I read your e-mail
Well, a few more months go by and I find myself responsible for the e-mail system of a 1000+ employee multi-national company. It only took me about a month or so before I had a better understanding of how mailflow works within the company than anyone had in several years.
Its amazing how much crap can build up in a system that passes hands a few dozen employees who never quite put in the effort to fully understand or attempt any sort of cleanup.
Its amazing how much crap can build up in a system that passes hands a few dozen employees who never quite put in the effort to fully understand or attempt any sort of cleanup.
Friday, December 14, 2007
Wow, I suck
Two posts and I vanish for over a year. Wow I suck.
Well, since my last post I've decided to leave my role as a Security Engineer. Instead I'm moving back to a Systems Administration role supporting Unix servers. Security still interests me, but at the end of the day what I enjoy is using technology to solve problems for people. In my security role I found myself finding problems without being able to assist in solving them.
So as of January 2nd I get to go back to my roots, but with a greater understanding of security. Should be interesting.
Well, since my last post I've decided to leave my role as a Security Engineer. Instead I'm moving back to a Systems Administration role supporting Unix servers. Security still interests me, but at the end of the day what I enjoy is using technology to solve problems for people. In my security role I found myself finding problems without being able to assist in solving them.
So as of January 2nd I get to go back to my roots, but with a greater understanding of security. Should be interesting.
Friday, June 30, 2006
SSH Security
I run a linux server that uses SSH as an authentication method. And my server is under attack. There has been over 1000 login attempts from a handful of systems in the past 2 days alone. Since June 1st, 19 different systems have attacked my system over 10,000 times. Chances are, if you're running a system that allows SSH authentication, your system is under attack as well.
For the past few years there has been a remarkable increase in the number of SSH brute force attacks. There are automated scripts out there that scan networks for SSH daemons on port 22, and when found, attempt to log into them using dictionaries of common usernames and passwords.
If you look in your logs, you'll likely see attempts that look something like this:
Jun 30 09:51:58 localhost sshd[32357]: Failed password for invalid user mark from 211.171.202.87 port 41435 ssh2
Jun 30 09:52:02 localhost sshd[32359]: Failed password for invalid user tomas from 211.171.202.87 port 42307 ssh2
Jun 30 09:52:06 localhost sshd[32388]: Failed password for invalid user rpm from 211.171.202.87 port 43192 ssh2
Jun 30 09:52:10 localhost sshd[32390]: Failed password for invalid user jean from 211.171.202.87 port 44095 ssh2
If one of these common username/password pairs actually work against your system, the attacker will gain access to your machine, and likely will use your computer to continue their search for more systems to attack. Your system may also be used as a jumping point for other types of attacks. Your data may be stolen. And who knows what will be traced back to you.
So is there anything you can do to protect yourself? Sure there is. You have a number of options, with different levels of effectiveness.
1) Switch to key based authentication. If you generate an SSH key and use it to provide all access to your system then you're effectively immune against these sorts of attacks. Brute forcing a password using SSH may take days or weeks. Attempting to brute force an RSA based SSH key would take years. Note that you need to REQUIRE keys for authentication, not just use keys. As long as your server still accepts passwords, you're still vulnerable. Regardless of what your standard method for logging in is.
2) If key based authentication isn't an option for you, your next best option is to use a strong password. The longer your password is, and the more random it is, the less likely a password based attack will work. In this case, it's important to ensure that ALL users of the system have strong passwords. Consider using an application like John the Ripper to test the strength of passwords if you have a number of users on the system.
3) An additional option is to move your SSH daemon to a different tcp port. By default SSH listens on port 22. A daemon listening on a different port would be harder to find. Note that unless you mix this will strong passwords you're depending on Security through Obscurity. Obscuring a system like this isn't a bad thing, but it is not a fix all. Assume that someone will eventually find the daemon, and the attack will continue.
4) Restrict who can talk to your SSH daemon. If you're lucky enough to only have a few sources of logins, you can use either application level IP restrictions or firewall rules to ensure you limit who can attempt to login. This works wonders if you trust the people on the allowed hosts list.
The final item I wanted to mention isn't so much a prevention method as a community good deed. Most IPs you see in your logs represent a system that has been compromised by a similiar attack. In most cases, the owner of the system is not aware of what has happened. Or if they are aware, their ISP might be interested to find out about their activities.
If you have some spare time, try tracking down an attacking system. Use tools like dig and whois to attempt to track down the owner of the system and the hosting ISP, and notify them about the attack. Include the log files showing their attack, and explain the issue in plain language. Treat them as a victim, and talk to them in a non-accusitory way.
In a best case situation you'll get a response from the person and they will work to solve the problem. In the worst case scenario you won't get any response. Maybe they solved the problem without saying anything. Maybe they're ignoring you. In either case, at least you tried.
For the past few years there has been a remarkable increase in the number of SSH brute force attacks. There are automated scripts out there that scan networks for SSH daemons on port 22, and when found, attempt to log into them using dictionaries of common usernames and passwords.
If you look in your logs, you'll likely see attempts that look something like this:
Jun 30 09:51:58 localhost sshd[32357]: Failed password for invalid user mark from 211.171.202.87 port 41435 ssh2
Jun 30 09:52:02 localhost sshd[32359]: Failed password for invalid user tomas from 211.171.202.87 port 42307 ssh2
Jun 30 09:52:06 localhost sshd[32388]: Failed password for invalid user rpm from 211.171.202.87 port 43192 ssh2
Jun 30 09:52:10 localhost sshd[32390]: Failed password for invalid user jean from 211.171.202.87 port 44095 ssh2
If one of these common username/password pairs actually work against your system, the attacker will gain access to your machine, and likely will use your computer to continue their search for more systems to attack. Your system may also be used as a jumping point for other types of attacks. Your data may be stolen. And who knows what will be traced back to you.
So is there anything you can do to protect yourself? Sure there is. You have a number of options, with different levels of effectiveness.
1) Switch to key based authentication. If you generate an SSH key and use it to provide all access to your system then you're effectively immune against these sorts of attacks. Brute forcing a password using SSH may take days or weeks. Attempting to brute force an RSA based SSH key would take years. Note that you need to REQUIRE keys for authentication, not just use keys. As long as your server still accepts passwords, you're still vulnerable. Regardless of what your standard method for logging in is.
2) If key based authentication isn't an option for you, your next best option is to use a strong password. The longer your password is, and the more random it is, the less likely a password based attack will work. In this case, it's important to ensure that ALL users of the system have strong passwords. Consider using an application like John the Ripper to test the strength of passwords if you have a number of users on the system.
3) An additional option is to move your SSH daemon to a different tcp port. By default SSH listens on port 22. A daemon listening on a different port would be harder to find. Note that unless you mix this will strong passwords you're depending on Security through Obscurity. Obscuring a system like this isn't a bad thing, but it is not a fix all. Assume that someone will eventually find the daemon, and the attack will continue.
4) Restrict who can talk to your SSH daemon. If you're lucky enough to only have a few sources of logins, you can use either application level IP restrictions or firewall rules to ensure you limit who can attempt to login. This works wonders if you trust the people on the allowed hosts list.
The final item I wanted to mention isn't so much a prevention method as a community good deed. Most IPs you see in your logs represent a system that has been compromised by a similiar attack. In most cases, the owner of the system is not aware of what has happened. Or if they are aware, their ISP might be interested to find out about their activities.
If you have some spare time, try tracking down an attacking system. Use tools like dig and whois to attempt to track down the owner of the system and the hosting ISP, and notify them about the attack. Include the log files showing their attack, and explain the issue in plain language. Treat them as a victim, and talk to them in a non-accusitory way.
In a best case situation you'll get a response from the person and they will work to solve the problem. In the worst case scenario you won't get any response. Maybe they solved the problem without saying anything. Maybe they're ignoring you. In either case, at least you tried.
Wednesday, June 28, 2006
An Awkward Start
Well folks, I'm here again on my umpteenth attempt to start a regular blog of some sort. These attempts go back about 6 years ago, and generally ends with my anti-social behavior taking over once again.
A little background on myself. I've spent the last 6-7 years working as a Linux sysadmin for a large company based in the Boston area. Last year I made the transitition from Sysadmin work with Infosec (Information Security).
For those of you who may not be familiar with Information Security, it is a rather large field centered around defending computer systems. And of course in order to protect a system you also need to be familiar with how to attack a system. As they say, know your enemy.
This blog is going to cover some of my life, and some of my work. Unless you're interested in computer security, you'll probably want to move on and read something else more up your alley.
A little background on myself. I've spent the last 6-7 years working as a Linux sysadmin for a large company based in the Boston area. Last year I made the transitition from Sysadmin work with Infosec (Information Security).
For those of you who may not be familiar with Information Security, it is a rather large field centered around defending computer systems. And of course in order to protect a system you also need to be familiar with how to attack a system. As they say, know your enemy.
This blog is going to cover some of my life, and some of my work. Unless you're interested in computer security, you'll probably want to move on and read something else more up your alley.
Subscribe to:
Posts (Atom)