Shimano dates back to 1921, when Shozaburo Shimano founded Shimano Iron Works in Sakai City, near Osaka. The town was a legendary blacksmithing center known for its swords and gun barrels. Rather than follow his father into farming, Shozaburo had apprenticed at an iron works after high school. Later, he started his own company, and the first product it made was a single-speed bicycle freewheel. In ten years, Shimano was exporting freewheels to China.

The business was incorporated as a limited corporation in January 1940 under the name Shimano Iron Works Co., Ltd. In 1951, it was renamed Shimano Industrial Co., Ltd.

Shimano began making its famous derailleurs in 1956. Also called external speed changers, these were the mechanisms that moved the bicycle chain from gear to gear on ten-speed bikes and the like. The next year, the company began producing an internal, three-speed gearing mechanism that was enclosed in the hub of the rear wheel. This internal speed changer was introduced to the U.S. market a few years later and soon became the standard for three-speed bikes. In 1960, Shimano installed a cold forge that enabled stronger products to be made in a more efficient fashion.

Shozaburo Shimano eventually turned over management of the business to his three sons. Though the company made brakes and other components, Shimano refused to produce complete bicycles. "Our founder said, 'never ever compete with a customer,'" remarked one of Shimano's sons to the Straits Times. A U.S. subsidiary, Shimano American Corporation, was set up in January 1965. Shimano launched into the bike-crazy European market in the same year

Shimano's first manufacturing plant abroad was set up in 1973 in Singapore. Opening a sales office in California in 1974, the company was well placed to ride the booming bike market in the United States during the 1970s.

The range of Shimano's cycling offerings expanded throughout the 1980s and 90s. New shifting systems continued to be developed, such as the Rapidfire Remote (1989) for mountain bikes. The company began selling a line of bike shoes in 1988. Shimano became more visible than ever in the late 90s on the global stage as longtime user Lance Armstrong began his winning streak at the Tour de France in 1999.