"All the things that Trump told us were going to happen, which was infrastructure spend, deregulation, tax reform, people don’t believe any of those will happen, certainly the magnitude and with the speed the market had originally priced in," said Ernesto Ramos, head of equities at BMO Global Asset Management in Chicago.
"That is why you’ve seen a lot of the move back, specifically banks."
Measures of market volatility are at rock-bottom, hitting trading desks at big banks. The US stock market's main gauge of investor anxiety closed at its lowest level in over two decades on May 8 and has not topped its long-term average of 20 since November. It did, however, hit a seven-day high of 11.30 on Wednesday.
JPMorgan shares lost 2.1 percent while Bank of America was down 1.9 percent as the two biggest weights on the S&P 500. Goldman Sachs fell 3.3 percent, the biggest drag on the Dow.
Defensive plays such as utilities, up 0.46 percent and telecoms, up 0.35 percent, were the bright spots as enthusiasm fades for sectors that would benefit from Trump policies.
Energy stocks, down 0.4 percent, also lost ground. Oil prices touched a three-week low as rising output from Nigeria and Libya fueled concerns that OPEC-led output cuts are being undermined. US crude settled down 2.7 percent at $48.32 a barrel and Brent settled 3 percent lower at $50.1.
The Dow Jones Industrial Average fell 20.82 points, or 0.1 percent, to 21,008.65, the S&P 500 lost 1.1 points, or 0.05 percent, to 2,411.81 and the Nasdaq Composite dropped 4.67 points, or 0.08 percent, to 6,198.52.
Shares of Michael Kors plunged 8.5 percent to $33.18. The luxury fashion retailer gave a bleak full-year forecast and said it would shut more than 100 full-price retail stores in the next two years.
Mallinckrodt closed down 1.2 percent at $43.13 after sources said the drugmaker is exploring a sale of its generic drug unit for as much as $2 billion.
Analog Devices rose 1.1 percent to $85.76 after the chipmaker's quarterly results came in above expectations.
Advancing issues outnumbered declining ones on the NYSE by a 1.01-to-1 ratio; on Nasdaq, a 1.04-to-1 ratio favored decliners.
The S&P 500 posted 28 new 52-week highs and 11 new lows; the Nasdaq Composite recorded 82 new highs and 70 new lows.
About 7.85 billion shares changed hands in US exchanges, above the 6.72 billion daily average over the last 20 sessions.