Build an exchange optimized for API use.

Although the current exchanges vary considerably in the quality of their APIs, they are all amateurish.

For example, the best of the lot, MtGox, provides a real-time stream, which, risibly, the others do not, but MtGox's real time stream relies on TCP/IP's retransmission protocol to ensure no losses. This means application layer failures will violate the integrity of TCP/IP's retransmission protection. All MtGox provides is a timestamp on its streaming events -- which means you cannot be sure you have received all events. This can cost automated traders serious money.

Professional financial applications need serious integrity.

For example, when SAIC contracted with the city of Orlando to automate their toll road collection, the contract stipulated that any losses due to failure to report transactions would be paid directly out of SAIC's pocket. I wrote the transaction telecommunications protocol to have an application-level serial number combined with a multiply redundant, distributed audit trail of the transactions created in real time so that central accounting could always be assured that they could detect the loss of a transaction in transmission and, upon such detection, be assured that when they queried the network for the point in the stream where the missing transaction occurred, the stream would still be retrievable from that point.

Transaction transmission reliability is just one example of the opportunity.