Nothing wrong with stainless, but you have to be careful what grade you are buying. The OEM hardware looks like steel with a zinc/aluminum. The tensile strength of these fasteners is 170,000 PSI. The typical 18-8 stainless known as A2 that is sold in hardware stores and on the internet has a tensile strength of 70,000 PSI or about 40% of the OEM strength.
It is possible to find high strength stainless fasteners in 17-4 PH stainless. They have a tensile strength of 130,000 PSI or about 75% of the OEM strength. There is a grade of stainless A286 that has a tensile of 160,000 PSI, but the prices are about double of other stainless fasteners.
It can be hard to find high strength fasteners, and even then, can you be sure of what you are buying unless or even when you pay premium prices. Personally, I would only buy from a reputable industrial supply house for any highly loaded hardware on my bikes.
All of the data above is from the McMaster-Carr Industrial Supply catalog.