I am new to trading, but I was a geek among computer geeks back in the day. Here is a summary of a few conversations I've had with a few hedge fund developers that I know.
1. You have to take the good with the bad when dealing with automated software. Programs make constantly make bad trades that a good discretionary trader would not make. They overcome this with sophisticated money management tools. A bad trade by a computer is the same as a bad trade by a person, it puts money on the table for someone else.
2. An automated system may backtest really well, but can do terribly when actually implemented. New software is often retired because a new player such as a hedge fund can throw off the dynamics that make the strategy successful and a system doesn't automatically adapt. As the market becomes "more chaotic", as some have argued in earlier posts, automated systems will become harder to implement successfully. Over time, I believe that this will become very expensive to support unless significant sums are involved. I don't know if there is enough evidence to say that cost/benefit of using an automated system will always win out.
3. As has historically been the case, larger sums of money continue to be pushed ino large institutions. If a fund is trying to liquate a $200M position, they have to expect a certain amount of inefficiency in the transaction. That leaves a lot of opportunity for crumbs that the quick and nimble can eat up.
4. OTC products are REALLY, REALLY hard to automate.