GDFuego

Getting political for a moment

2011-10-25T21:21:00.003-04:00

With all that's going on in the world these days, I've been doing a lot of thinking about my own personal political philosophy and where exactly I lie. I'm definitely liberal on the social side of things, but I had often agreed with the stated beliefs of the Republican party or libertarians (small federal government staying out of the way). I tend to vote Democrat though due to disliking the actual ACTIONS of Republicans and their insistence on bringing religion into government.

In light of the Occupy Wall Street movement, I've been trying to consider what I think it would take to actual improve society. Not necessarily specifics of individual items, but more of a higher level political philosophy. In the past few days I think I've figured it out. Note that I do not necessarily believe that any of this would ever happen, and I'm certainly open to the likelihood that I've missed something which renders my opinions a load of horse shit. I'd be curious to hear what people think.

Corporations

Before I start, I should take a moment to step back and talk about businesses. Corporations tend to fall into two basic classes; public and private. Public companies are companies who have stock which is available for sale to the general public in various stock markets around the world. Private companies may or may not have stock, but any shares which exist are privately held by individuals. This could be a single company owner, partners, or it could be in the hands of a number of parties (venture capitalists for example).

General Electric (GE), Google and McDonalds are all examples of public companies. Facebook and the plumber down the street are both examples of private companies, although Facebook is eventually planning to go public. Who knows, maybe Joe the Plumber is as well.

One of the guiding principals of public companies is the concept of maximizing shareholder value. Companies are LEGALLY required to maximize profits. Private companies are also typically motivated to increase profits as much as possible, although they do have more flexibility if they were motivated to exchange profit margins for improving the community, sharing the wealth, or other purposes.

You can sort of think of a company as a machine designed to funnel the maximum amount of money (profits) from customers to shareholders. More customers, more profit. Lower expenses, more profit. Fewer shareholders, more profit per shareholder. The perfect company would have as few employees as possible and would siphon all money to a single pocket.

Luckily there are inefficiencies in these business machines:

* People aren't going to hand over money without something in return, so there is a need to develop and provide a product or service of some sort. This requires employees to design, build, maintain these products/services and support the customer base. These employees get jobs, benefits and their salaries, taking away from the profit margin. For the employees it means the ability to buy food, clothing, shelter and plenty of non-essentials (like maybe the company's products).

* If you have a high margin product, you can expect to see competitors. These other companies are like sharks in the water who smell blood and come swimming. Probably a bad analogy though since the more competitors there are, the lower prices will be due to increase competition. I certainly wouldn't consider myself "saved" from a shark attack when I see more sharks coming.

* Any profit a company receives gets taxed at federal, state and probably city levels. Companies pay taxes on their profit in addition to things like payroll tax. The company executives pay income tax on any salary they are provided. Shareholders pay taxes on any dividends that the stock pays and capital gains tax if they were to sell the stock. These taxes then are used to pay for government expenses, including money to support the community (schools, roads, welfare, etc).

Companies try to decrease these inefficiencies as much as possible. They shift hiring to geographical locations which are cheaper to operate out of. They use patents, copyright law and mergers & acquisitions to decrease the impact of competitors. They use tax loopholes to shift tax liability to cheaper locations or avoid paying it outright. In some case they even leverage our government to do these things for them. Trade agreements open up new customer bases for companies and also open up new cheap labor for them to exploit. New tax breaks or loopholes are created which allow them to keep more of their profits. New regulations are written under the guise of protecting the public which can often raise the cost of doing business for smaller companies, decrease competition. Computers and other forms of automation have also had a MAJOR impact on decreasing inefficiencies.

Here's the thing though. The rest of us, the "99%", live our lives as a result of those inefficiencies in business. The closer they can get to their ideal, the less of us that will have jobs. No jobs, no money, no food. Ironically, also no ability to buy their damn products, so I guess we'd eventually have the last posthumous laugh.

Back to Politics

Just to be clear for readers out there, I'm not against capitalism as an economic system. Corporations SHOULD be expected to act as I've described. Suggesting that business owners shouldn't try to make as much money as possible is absurd and would just be ignoring human nature.

What I am against is capitalism as a political system.

I believe the role of government should be to protect and ensure the prosperity of the population. Measuring this by the income of business while ignoring the income of the majority of the population doesn't achieve this goal.

Since the majority of the country survives by the inefficiencies of business, it should be the role of government to make businesses LESS efficient.

* Increase the cost of outsourcing jobs or importing products to make building at home the best financial option.

* Increase the minimum wage to something people could actually live off of.

* High progressive taxes on income for businesses in order to remove the incentive for industries to merge into a handful of mega corporations (two companies making $1b in profits in a 30% tax bracket would lose money by merging into a single company making $2b in profits in a 40% tax bracket for example). Progressive taxes on corporate income would also give an advantage to newcomers, increasing the amount of competition.

* Neuter patent and copyright law. Having a patent to allow a research company to recoup their expenses makes sense, but 20 years for inventions in technology is an absurdly long time period. Life of the artist + 70 years, or 95 years for work for hire is absurd for copyright. How does paying an artist's GRANDCHILDREN promote the arts? I'm thinking 5 years for patents and 10-15 for copyright.

* Kill any other de-facto monopolies. If a monopoly is necessary, perhaps it should be public works. For example, the last mile of internet access should be a public works project, funded by tax money. The internet access over those lines could then be run by corporations with a more level playing field for competition.

Basically, screw "incentive" programs. Penalize the undesired behavior, or just make it impossible to start with.

Health Care

When it comes to certain "industries", I'm not sure I see how they could run in a way that benefits mankind in a for profit manner. Health care is a big one, as well as education. Perhaps banking as well (yay credit unions). With hospitals, insurance companies and pharmaceutical companies all trying to maximize profit, the current ballooning cost of health care should be expected. Same with the costs of colleges. Education and health are just too important to treat people as "consumers".

TL;DR

I'm apparently a damn Socialist, or something.

So what am I missing? Other than the fact that the people currently in control would all be grievously injured by making these sorts of changes and would fight it tooth and nail, and the public would all say "NO SOCIALISM!!!"

Python Tivo Library

2010-04-21T09:48:00.002-04:00

I've decided its about time I start really learning to code, so I've picked a project and I'm working to see how much I can actually implement.

The long term goal of the project is to create a way to archive TV shows off of your Tivo and provide a mechanism to transfer the shows back on demand for showing. I'm working on creating it on Python, and as a starting point I've begun work on a general python library for interacting with them. Right now provided an IP address and your media access code it can connect to your tivo and give you a listing of all of your shows.

My eventual goal is to provide it both a web interface and an interface that the Tivo can use as well as add autodiscovery of tivo devices. I'd like to also tie it into pytivo for the show playback. One nice thing is that would allow you to compress the shows for better storage.

If anyone is interested in taking a look I'm hosting it at Sourceforge as a way to do revision control as well as learn Subversion a bit better (I'm more familiar with Perforce right now).

http://tivoarchive.sf.net

New blog available

2009-10-14T23:26:00.001-04:00

Electr0n has setup a new blog for the ##security channel on Freenode, and has asked me to help with some content. I just posted there on Pastebin hacking in light of the recent Hotmail password fiasco.

Check it out.

Career "Advancement"

2009-08-17T20:11:00.002-04:00

About 2 years back I left my job in an InfoSec group. That particular position wasn't the right fit for me anymore, and somehow I didn't think I would be a good fit for a security role in most other organizations. I don't have the pen testing experience needed for most security companies, and the thought of maintaining firewall rules at some retail house would bore the crap out of me.

Since then I've been struggling to find myself career wise. I spent some time in an IT role, and I'm now trying a more customer facing role. Still I find myself happiest in ##security on Freenode, answering people's security questions.

So, what now? Maybe someday I'll figure that out.

The Wrong Tool for the Job

2009-03-05T18:58:00.003-05:00

These days anti-virus and anti-spam are two very crucial components of a well run e-mail system. Due to how often spammers change their techniques, my company outsources this function to a vendor which provides both services. Both functions are designed and work fairly well.

Anti-Virus

For Anti-Virus they seem to run messages through multiple commercial anti-virus scanners on their servers. Messages that trigger positive are quarantined, and a notification will be sent to the site admin and/or the intended recipient of the message notifying them of what happened.

The site admin can report false positives to the vendor who will investigate and release a message if they can confirm that it was in fact a false positive. They also take an action to reduce future false positives based on what they find. These investigations tend to take 24 hours or so.

Anti-Spam

Spam tends to be a bit more subjective, so false positives tend to be higher than with Viruses. Due to this, their anti-spam offering makes it a lot easier to both prevent and deal with these situations.

Spam messages can either be tagged for users to filter on their own, or they can be actively filtered and put into a quarantine on their servers. Unlike quarantined virus mail, quarantined spam can be accessed and released by users directly.

In order to prevent false positives site admins are able to whitelist domains, e-mail addresses or IP addresses for specific mail relays. Whitelisting a domain is typically not a great idea in these days of e-mail address spoofing, but e-mail address and whitelisting relays works fairly well.

Where it falls apart

Sounds good so far, right?

Well, here's where it all goes wrong. It appears that anti-virus vendors have discovered that they can use their scanning engines to pick up certain types of phishing and scam e-mails, essentially adding anti-spam into their anti-virus product.

A phishing or a scam mail is SPAM, not a VIRUS. The difference here cause a big problem when you get spam levels of false positives while removing the user's ability to release their own messages and the site admin's ability to implement an sort of whitelisting.

That's when you start getting end user reports of mail threads with customers going missing. Add in a 24 hour turn around time for releasing the messages when the problem is discovered and you start to consider deep-sixing your vendor.

Wooo.. Kindle

2009-03-03T21:33:00.004-05:00

Being a big reader and a tech gear junkie, I was rather tempted when Amazon announced the Kindle back in 2007. Somehow I managed to hold out on buying it until they announced the 2.0 version in early February. I pre-ordering it right away, and got my hands on it just last week.

So far, I like it. Its thinner than I expected. Definitely very easy to use. I can hold it in one hand and access most of the controls that I need to read a book. The left side has the "Previous Page" and "Next Page" buttons while on the right side the "Previous Page" button is replaced by a "Home" button. Since I tend to read books in one direction, this seems to work fine.

The free built in wireless is great for getting books, and occasionally pulling up text-only web sites. Due to the rather slow refresh on screen changes using it as a regular web browser is a bit tough.

My biggest complaint is the DRM for files through the Kindle store. After being bitten by DRM from the iTunes Music store, I definitely have a bad taste in my mouth over DRM. Luckily there are other options out there.

The first for me is Many Books. They offer a lot of free content in quite a few eBook formats, including both the native Kindle format and Mobibook which the Kindle also supports. They even have a Mobile Interface which works well from the Kindle itself. Most of the content has elapsed copyrights (older books), but there are occasionally newer books either available with sample chapters or content that was published under a Creative Commons License.

Next was O'Reilly. Being a big tech book reader, I have a lot of O'Reilly books.

O'Reilly offers a number of their books in DRM-free E-Book formats, including the Kindle supported Mobibook format. They're not free, but I don't have any objection to paying for content, just having its usage limited by DRM. They even provide free updates to the books as new revisions are published. I just wish they made it a bit easier to get a list of just their books available in E-Book format.

While I definitely like the Kindle, the only thing I'm not sure about at this point is if it was worth the cost or not. The Kindle costs $360. Sony's offering is quite a bit cheaper, although I have no idea how it compares feature wise.

Data Recovery From a Bad Disk

2009-02-27T17:12:00.006-05:00

My wife’s laptop drive failed yesterday, leaving her Windows XP laptop unbootable. IT provided her with a new laptop, but had deemed her data lost. While she does do backups of her data to a USB drive, it had been a while since the last backup so she was a bit concerned. And I of course enjoy a new challenge.

From the various articles I’ve read on data recovery in the past, I knew that the best bet was to make an image of the disk and attempt to recover data off of the image. There’s nothing worse than running a chkdisk/fsck on a partition, and having the attempts to fix the filesystem cause additional filesystem problems.

So how should I make an image? Being a Unix guy, my first thought was dd. DD allows you to copy the complete filesystem off of a partition, and write it to a file. Unfortunately dd can have issues when it attempts to read a block from a disk that is in the process of failing. It will attempt to read again, rather than just moving on to the next block.

A quick Google search brought me to ddrescue, which was designed to deal with this very issue.

Next step is to figure out the best way to actually access the data off of the disk. My first thought was to just pull out the drive and hook it up to my desktop machine. I have an adapter that allows me to plug a laptop drive into a standard IDE cable for a desktop system. I soon discovered that the system was using a SATA drive, and I didn’t have the correct cabling to hook a laptop SATA disk to my desktop, so that plan was shot.

Next thought, Linux live cd. Unfortunately this was a Thinkpad x60s laptop (12" ultra-portable) which doesn’t actually have a CD-ROM drive. There are USB drives for it, but I don’t have one available. That leaves a USB flash drive.

Now to choose what linux image to use. I typically use Ubuntu as a live cd, but I’m not actually sure if they include ddrescue on that. I’m also concerned that Ubuntu might try to auto-mount the bad disk, potentially making the problem worse. So, after a bit of searching I come across System Rescue CD. Its simple, console only and includes ddrescue. Even better, it includes instructions for putting it on a USB disk.

I download the ISO and follow their instructions, and no luck. The USB drive won’t work. I think their instructions could use some work. A quick download of uNetbootin, and I’m on my way. uNetbootin is a generic tool for turning a Linux live CD into a bootable usb drive. I found it a few months ago while trying to install Ubuntu on my eeePC. One more reboot, and I’m good to go.

So now I have the necessary tools to make an image of the bad disk. I just need a place to store the disk image. Its a 60GB drive, so there’s a bit of a storage need here. I don’t have a large enough USB drive on the system, so I need something network enabled. As it turns out, System Rescue CD includes sshfs support, allowing me to mount part of my desktop machine
filesystem remotely. Awesome.

Running ddrescue was easy. Just dd_rescue /dev/sda1 /mnt/desktop. A few hours later, and the data was ready to be accessed. It even reported any bad blocks found on the disk. There turned out to be 120 errored reads, all clumped together on the disk. Based on the initial Windows
boot errors, that part of the disk seemed to hold OS components. Good sign for her data.

Now I have an image of a corrupt NTFS partiton. I used the ntfsfix tool from the ntfsprogs package on Ubuntu to fix the image. Any data from the bad sectors of the disk is going to be gone, but the partition can now be mounted in order to read the rest of the data.

A quick mount with mount -t ntfs-3g image /mnt, and there the data is. Looks like all of her important files were fine. I got to show up the IT folks, and earn me some nice brownie points. Perhaps I'll redeem them for actual brownies.

Information Gathering Using SSH Public keys

2009-02-19T20:35:00.005-05:00

I've been a pretty heavy user of SSH for the past 10 years or so. In that time I've learned a number of tricks including port forwarding in various directions, forwarding SSH agents (and the associated risks) and various key management techniques if you're providing key based authentication to large numbers of systems.

The most interesting trick I've learned with SSH, I haven't really seem talked about much. A former co-worker pointed me to the feasibility of this working with protocol 1 and a hacked up SSH client, but these days it trivially works with both protocol 1 and 2 using the normal OpenSSH client.

The Trick

Generate an RSA SSH key, and delete the private half. The passphrase does not matter since we won't be using the private key at all. ssh-keygen -t rsa -f test -N "" && rm test

Take the public key file (test.pub), and copy it to the authorized_keys file of a remote system.

Set mode 600 on the public key. chmod 600 test.pub

Try to log into the remote system using the public half of the SSH key. ssh -2 -i test.pub user@server

Assuming all went according to plan, you should get prompted with Enter passphrase for key 'test.pub':. Since this is the public half of a key, no passphrase will ever succeed. You do however know that the private half of this key would have allowed you to log in.

In case you're curious, the reason for the chmod 600 is that the SSH client attempts to enforce good permissions for private keys by refusing to use a "private" key with open permissions. Since you're essentially tricking the client into treating a public key as a private key, the same rules apply.

So What?

This trick allows you to do two things:

It allows you to identify what servers a user has access to. If you have access to a person's public key (which are typically not protected since they're PUBLIC), you can determine what servers the person has access to by attempting to log into root, their username or any other account using their public key.

The second piece is a bit more interesting. If your company has a central key repository which is available to all employees, it becomes very easy to test all keys against a specific server in order to determine who has a private key which has access to the system.

In the past I've used this functionality at work in order to determine who can still log into a system which had been down for a considerable amount of time (and had missed some key rotations). A hacker could instead use this functionality to know who's private SSH key they're going to need to steal in order to gain access to the targeted system.

Why it works

The reason this works can be understood by looking at the Public Key Authentication Method of the SSH protocol.

Among other bits of data, the SSH client sends a copy of the public SSH key to the server as part of the authentication process. The server then responds with SSH_MSG_USERAUTH_FAILURE or a SSH_MSG_USERAUTH_PK_OK message. At this point you now know if access would be granted with the private key, but you have not needed to use that private key in any way yet.

This explains why only the public key is needed during the authentication step, but not necessarily why the SSH client makes this so easy for us. I suppose its probably just a quirk of how their key parsing code works.

They could change the code to not allow you to attempt to do private key operations without a private key, but really that just adds a small hurdle to exploiting this small weakness in the protocol. At the end of the day, you're still only as safe as the protections you put in place on your private keys.

Mac OS DNS bug

2009-01-28T20:10:00.005-05:00

I had a bit of an interesting experience the other day while attempting to fail over our Jabber server from our production site to the DR site.

Our two servers each have their own A records in DNS with a TTL of 3600 seconds (1 hour). This long timeout is fine since the IP address of the actual server never really changes.

Access to the service is instead provided by a CNAME record which points to one of those two hostnames. The TTL of the CNAME record is 60 seconds, allowing us to quickly fail over between the two sites as needed.

So the time came, and I had to perform a fail over. I updated the CNAME, and in order to prevent users from being unable to connect, I waited 60 seconds before shutting down the old server and starting up the new one.

From there things went bad. I tried to access the admin console, and failed. I tried to log into the Jabber server, and failed. Finally I hit the admin console through the A record instead of the CNAME, and found that other users had seamlessly failed over.

After a bit of testing I determined that my Linux box and my Windows box both worked fine. The only problem was the Mac that I was making the change from. For some reason, the Mac was holding on to the old IP address.

After some testing, and confirmation from other individuals on their Macs, I think I know what was going on. Using dscacheutil -cachedump -entries, I inspected the local resolver cache.

Here's what I found:


Category         Best Before         Last Access      Hits    Refs       TTL    Neg  DS Node
   ----------  ------------------  ------------------  --------  ------  --------  -----  ---------
         Host   01/28/09 21:07:02   01/28/09 20:18:35        10       4      3600       
                    Key: h_aliases:openfire.domain.fake. ipv4:1
                    Key: h_aliases:openfire.domain.fake ipv4:1
                    Key: h_name:server1.domain.fake ipv4:1

This appears to be reporting that the local resolver cached the server1.domain.fake DNS record, and set an expiration date of the record for "01/28/09 21:07:02". openfire.domain.fake was then set as an alias for that record without retaining its own TTL. This would certainly explain the behavior that I saw.

So it seems to Mac OS X may be incompatible with a fairly common DNS failover technique. I filed a bug, so it'll be interesting to see how long it takes before Apple gets around to fixing it.

Home Media Server

2009-01-12T21:44:00.005-05:00

I'm a bit of a media buff. I own several hundred DVDs, and I'm guessing well over 1000 cds.

Like the rest of the known world, I solved the CD problem years ago. I ripped all of my CDs, and I store them in a few places including my iPod and my home media server. Early on I ripped in ogg format, but quickly regretted it when I bought a Phat Box media player for my car. By the time they supported Ogg format, I had moved onto an iPod.

Eventually I started buying music from the iTunes Music Store since it was so much easier than CDs, and I continued until I discovered the Amazon MP3 Store. Buying in mp3 format instead of AAC is so much easier to deal with, not to mention the lack of DRM.

Recently I read about Sockso. Sockso provides me with a simple web interface for streaming music off of my server so I can listen from work without having everything on local disk. Considering that my music collection is 58GB these days, it certainly saves me some space. Unfortunately Sockso does not support AAC format at this point, so I'm kind of out of luck on my iTunes media (even the non-DRM files).

Recently I started trying to tackle the DVD issue as well. I have a DVD changer, but its just kind of clumsy. It attempts to detect the name of movies from the disc, but rarely succeeds. You can attach a PS2 keyboard and type them in manually, but I eventually had to move the player which required removing the discs (and losing the inputted data).

So I thought I'd apply the same techniques to my movies. I used Handbrake to rip a number of movies and copy them onto my server as well. From there I can copy them into iTunes to watch on my computer, my iPod or transfer to my AppleTV (I apparently buy too much Apple gear).

Most of my media is on my file server which runs linux, so I wanted to see if I can get away without running iTunes. My first option was pyTivo. Its an interesting project. Its a python script that you point to your movie collection. It performs the necessary UDP broadcasts in order to announce your movie share on the network for your TiVo to see, and then converts the movie on demand to a format that TiVo can display properly. pyTivo works pretty well, but the code is in flux, and I'm not sure how much I want to trust it.

My latest try was XBMC. Its a rather nice media player that was created for the original XBox. These days it has also been ported to run on Windows, Linux, the Mac, and AppleTV. It can easily be installed on the AppleTV using ATV USB Creator.

XBMC can recieve a stream from a Universal Plug'n'Play Media Server. In my case I used MediaTomb since it was available straight from Ubuntu. I'm not sure I'd suggest it due to the lack of access control. For now, I'm fine just running it bound to my local network only.

I'm not sure I'm really happy with how all of this is working, but its still a work in progress.

I chose poorly

2008-10-16T22:24:00.002-04:00

I finally took the advice of a co-worker and checked out OpenFire to replace Jabber XCP. After about 30 minutes I had Openfire configured with all of the functionality that had taken me weeks to setup properly in XCP.

Additionally I was able to pre-define buddy groups using AD groups, set a message of the day, send broadcast messages to all logged in users, and perform other handy functions.

If you're looking at implementing a Jabber/XMPP solution, its definitely worth a look.

x509 certs and XMPP servers

2008-07-30T22:00:00.002-04:00

Ok, one last XMPP post for tonight.

I came across an interesting issue with our test XMPP server today. When the server was initially created, it was setup with an x509/SSL certificate that was self signed. That cert expired the other day, and I had to replace it.

Some of the people who were using the server started getting cert warnings while others did not. After a bit of investigating, I found the problem.

There are two ways that an XMPP client can connect to a server.

The first method is simple. You configure your client with your Jabber ID (username@company.com), and define an XMPP server (servername.company.com) to connect to. This is how our early documentation recommeded configuring your clients, but it is not the generally recommended method.

The second method is the preferred one. You define a service record (SRV record) in DNS for _xmpp-client._tcp.company.com which points to your server name. Once your client has your Jabber ID (JID), the client will automatically look up the SRV record, and connect to that service.

Now comes the cert warning. Apparently if you use an SRV record, your x509 cert needs to have a common name (CN) of company.com. If you define a server manually however, your CN needs to be the name of the server.

Our self-signed cert was for company.com. I replaced it by a properly signed cert for servername.company.com, and broke everyone using the preferred configuration method. Seeing the issue, I replaced it with a properly signed cert for company.com, and broke the people following the published documentation.

*sigh*

So now I get to update the documentation and answer the various questions for people. Longer term I'm going to see if its possible to provide certs for both methods. While you can't have two certs on the same port, it may be possible to either use a cert with an altCN, or possible have the SRV port point to a non-standard port so that people who define the server name can connect to the default port number (5222/5223).

Managing your own Chat server

2008-07-30T21:27:00.001-04:00

Recently for work I've been dealing with trying to build a robust IM server environment for the company I work for. The plan was to allow employees to chat with each other without the conversations ever leaving the company. Our hope was to also tie the system into the various public instant messaging systems (AIM/MSN/YIM) so that we could use the same system to communicate with customers.

After looking around at various offerings, the choices boiled down to a Jabber/XMPP based solution or a SIP/SIMPLE service like Microsoft OCS or Lotus Sametime. Since XMPP has more open source libraries for coding against it, we decided to go the XMPP route. Personally I was happy since the peer to peer nature of the SIP protocol introduces a few problems on our network.

Of course, deciding on XMPP doesn't really narrow things down too much. There are a number of open source XMPP/Jabber servers out there, and quite a few commercial ones as well.

We looked at Jabberd2, eJabberd and Jabber XCP.

As far as I can tell, Jabberd2 and Jabber XCP are very similar in codebase, at least judging by the configuration file format. The thing that the commercial Jabber XCP product provides is a nice web interface for configuring the service, a pair of supported clients (one Windows client, one web based client) as well as commercial support.

eJabberd was impressive. It is an open source application written in Erlang, which I wasn't familiar with. Erlang was a programming language created for distributed computing by Ericsson. On smaller installations, it can handle clustering without any form of external database. It automatically manages synchronization between nodes of the cluster.

Version 2.0 of ejabberd also included a very nice web interface for managing the service, and included an inpressive number of plugins. Configuring the service was very very easy.

At the end of the day, we ended up going with Jabber XCP. The main reason is the one feature that no one else was able to provide. The ability to really tie into the public IM systems, or at least one of them. Any XMPP server can tie into other XMPP services (like Google Talk), but Jabber XCP offered the ability to tie into AIM as well.

The majority of Jabber/XMPP servers offer transports for the various IM services. What these transports do is essentially allow you to log into your own AOL/MSN/etc account from within your IM client. So I may be myname@company.com within my company, but I would be logging into my gdfuego account on AIM.

Jabber XCP has a plugin (for additional per user cost), which allows you to actually use the same myname@company.com address within AIM itself. Unfortuantely MSN and YIM aren't an option at this time.

Right now I'm getting close to turning our new server live, and I've been hitting a number of snags, but that'll be a story for another day.

Even less spam

2008-05-03T21:48:00.001-04:00

Due to my frustration with my personal e-mail, I recently implemented some additional spam filtering which was amazingly effective.

In the past 24 hours, the system has stopped 775 spams destined to valid users, and has accepted 37 legitimate messages. Previously it would have accepted those messages. Most would end up in my spam folder to be reviewed later, but others would end up in my inbox.

This decrease in spam hitting my inbox is through Greylisting, implemented using Postgrey.

A properly implemented mail server needs to be able to deal with a temporary delivery failures, and try again later. A system implementing Greylisting will keep track of the source IP address as well as the source and destination e-mail addresses. The first time a unique combination of these three things is seen, the message is given a temporary failure for 10 minutes.

Legitimate mail is eventually delivered. Spammers give up and don't come back.

I reject your e-mail

2008-04-11T18:29:00.000-04:00

I've always heard that the correct behavior of a mail server is to reject e-mail for local undeliverable addresses rather than accepting them and then bouncing. I never put too much thought into it though until recently when I took over management of our e-mail infrastructure.

When the systems were handed to me, their queues generally had about 800-1000 e-mails
to be delivered at any given point. As I dug into why, I found that the majority of those e-mails were outgoing bounce messages which were undeliverable for one reason or another.

After a few changes, those systems are now rejecting around 1,000,000 spam messages a day. These are messages that would previously have been accepted and sent back out on the internet as backscatter.

Rejecting instead of bouncing allowed me to significantly cut down on the amount of processing power, bandwith and disk space used on these systems. Not to mention cutting down on the amount of e-mail that the backscatter victims were receiving.

Now if only everyone else who runs mail servers would figure this out.

I read your e-mail

2008-04-07T17:27:00.000-04:00

Well, a few more months go by and I find myself responsible for the e-mail system of a 1000+ employee multi-national company. It only took me about a month or so before I had a better understanding of how mailflow works within the company than anyone had in several years.

Its amazing how much crap can build up in a system that passes hands a few dozen employees who never quite put in the effort to fully understand or attempt any sort of cleanup.

Wow, I suck

2007-12-14T22:02:00.000-05:00

Two posts and I vanish for over a year. Wow I suck.

Well, since my last post I've decided to leave my role as a Security Engineer. Instead I'm moving back to a Systems Administration role supporting Unix servers. Security still interests me, but at the end of the day what I enjoy is using technology to solve problems for people. In my security role I found myself finding problems without being able to assist in solving them.

So as of January 2nd I get to go back to my roots, but with a greater understanding of security. Should be interesting.

SSH Security

2006-06-30T18:14:00.000-04:00

I run a linux server that uses SSH as an authentication method. And my server is under attack. There has been over 1000 login attempts from a handful of systems in the past 2 days alone. Since June 1st, 19 different systems have attacked my system over 10,000 times. Chances are, if you're running a system that allows SSH authentication, your system is under attack as well.

For the past few years there has been a remarkable increase in the number of SSH brute force attacks. There are automated scripts out there that scan networks for SSH daemons on port 22, and when found, attempt to log into them using dictionaries of common usernames and passwords.

If you look in your logs, you'll likely see attempts that look something like this:
Jun 30 09:51:58 localhost sshd[32357]: Failed password for invalid user mark from 211.171.202.87 port 41435 ssh2
Jun 30 09:52:02 localhost sshd[32359]: Failed password for invalid user tomas from 211.171.202.87 port 42307 ssh2
Jun 30 09:52:06 localhost sshd[32388]: Failed password for invalid user rpm from 211.171.202.87 port 43192 ssh2
Jun 30 09:52:10 localhost sshd[32390]: Failed password for invalid user jean from 211.171.202.87 port 44095 ssh2

If one of these common username/password pairs actually work against your system, the attacker will gain access to your machine, and likely will use your computer to continue their search for more systems to attack. Your system may also be used as a jumping point for other types of attacks. Your data may be stolen. And who knows what will be traced back to you.

So is there anything you can do to protect yourself? Sure there is. You have a number of options, with different levels of effectiveness.

1) Switch to key based authentication. If you generate an SSH key and use it to provide all access to your system then you're effectively immune against these sorts of attacks. Brute forcing a password using SSH may take days or weeks. Attempting to brute force an RSA based SSH key would take years. Note that you need to REQUIRE keys for authentication, not just use keys. As long as your server still accepts passwords, you're still vulnerable. Regardless of what your standard method for logging in is.

2) If key based authentication isn't an option for you, your next best option is to use a strong password. The longer your password is, and the more random it is, the less likely a password based attack will work. In this case, it's important to ensure that ALL users of the system have strong passwords. Consider using an application like John the Ripper to test the strength of passwords if you have a number of users on the system.

3) An additional option is to move your SSH daemon to a different tcp port. By default SSH listens on port 22. A daemon listening on a different port would be harder to find. Note that unless you mix this will strong passwords you're depending on Security through Obscurity. Obscuring a system like this isn't a bad thing, but it is not a fix all. Assume that someone will eventually find the daemon, and the attack will continue.

4) Restrict who can talk to your SSH daemon. If you're lucky enough to only have a few sources of logins, you can use either application level IP restrictions or firewall rules to ensure you limit who can attempt to login. This works wonders if you trust the people on the allowed hosts list.

The final item I wanted to mention isn't so much a prevention method as a community good deed. Most IPs you see in your logs represent a system that has been compromised by a similiar attack. In most cases, the owner of the system is not aware of what has happened. Or if they are aware, their ISP might be interested to find out about their activities.

If you have some spare time, try tracking down an attacking system. Use tools like dig and whois to attempt to track down the owner of the system and the hosting ISP, and notify them about the attack. Include the log files showing their attack, and explain the issue in plain language. Treat them as a victim, and talk to them in a non-accusitory way.

In a best case situation you'll get a response from the person and they will work to solve the problem. In the worst case scenario you won't get any response. Maybe they solved the problem without saying anything. Maybe they're ignoring you. In either case, at least you tried.

An Awkward Start

2006-06-28T22:07:00.000-04:00

Well folks, I'm here again on my umpteenth attempt to start a regular blog of some sort. These attempts go back about 6 years ago, and generally ends with my anti-social behavior taking over once again.

A little background on myself. I've spent the last 6-7 years working as a Linux sysadmin for a large company based in the Boston area. Last year I made the transitition from Sysadmin work with Infosec (Information Security).

For those of you who may not be familiar with Information Security, it is a rather large field centered around defending computer systems. And of course in order to protect a system you also need to be familiar with how to attack a system. As they say, know your enemy.

This blog is going to cover some of my life, and some of my work. Unless you're interested in computer security, you'll probably want to move on and read something else more up your alley.