According to the Dell Dimension E510 service manual:
Solid amber
The Dell Diagnostics is running a test, or a device on the system board may be faulty or incorrectly installed.
Blinking amber
A power supply or system board failure has occurred.
If the light is blinking amber continuously, most of the time this means the PSU.
When it doesn't start, check the four diagnostic lights, what is the light pattern?
The light on the motherboard just means there is power going to the motherboard from the power supply. Its called a standby power light
When the computer is plugged into a wall power outlet, there is +5 Volts of DC power coming from the power supply to the motherboard even when the power switch on the front of the computer is off.
The purple wire (pin 9) on the motherboard's main power connector (P1) supplies +5V of standby power at around 2 to 3 Amps.
This is why it is a good idea to unplug your computer when you work on the inside of it because less than 1 Amp can kill you.
On Dell power supplies it will be listed as +5VFP the FP stands for flea power, it is the same thing as standby power.
You can test the PSU with a PSU tester or by unplugging all the power connectors and on the main power connector (P1) short the green wire (pin 16) to a black wire (pins 3, 5, 7, 15, 17, 18, 19 or 24) if the fan on the PSU comes on then the PSU should be good.
You can see from the picture on this 24 pin connector they are shorting pin 16 to pin 19
Dell use a lot of proprietary designs. I think the Dell E510 uses a proprietary BTX case design.
If the switch is bad you may have to replace the power switch board. Here is some links to boards for sell:
Part X8683
http://www.resaleremarketing.net/servlet/the-2844/Dell-Dimension-5150-fdsh-E510-Front/Detail
Part X8682
http://www.resaleremarketing.net/servlet/the-175/Dell-Dimension-5150-fdsh-E510-Front/Detail
Part X8682
http://compare.ebay.com/future/220570951113?var=svip&sort=BestMatch
http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160437506692
Oh I almost forgot there are three caps on the top of the power switch board. You should check them.