USADA say they have something like 10 witnesses against him. Since it's not going to be arbitrated, it's not clear how much of that is going to be made public. A number of people have already talked to the press (e.g. Tyler Hamilton).
However, you are partially right that. As far as top cyclists during the Armstrong era, "guilty until proven innocent" is usually a pretty safe assumption, unfortunately.