The company also said Bill Gates would step aside as chairman of the board but would remain a technology adviser to the company. John Thompson, who has been the lead independent director, will become chairman.
Nadella's appointment ends a longer-than-expected search for a new leader after Steve Ballmer announced his intention to retire in August.
Nadella is only the third CEO in Microsoft's 39-year history, following co-founder Bill Gates and Ballmer.
The company took to Twitter to introduce its choice as CEO to the world.
In an email to Microsoft staff, Nadella acknowledged there was work to be done.
"While we have seen great success, we are hungry to do more. Our industry does not respect tradition — it only respects innovation. This is a critical time for the industry and for Microsoft," he said.
Nadella, most recently, head of Microsoft's cloud and enterprise business, has held various executive positions since joining the company in 1992. He previously led Microsoft's server and tools business.
Before Microsoft, the Indian-born executive worked at Sun Microsystems.
The famously ebullient Ballmer -- who joined Microsoft in 1980 as the company's first business manager and rose quickly through the ranks -- leaves the company after 13 difficult years as CEO. The company was once the most valuable in the world, but Microsoft has lost more than half of its market value over the past decade.
Ballmer's tenure included product flops like the Zune MP3 player, Windows Vista and its Bing search engine. But the biggest overarching miss was Microsoft's failure to anticipate how mobile devices would completely upend the computing world.
Microsoft found itself flat-footed and forced to chase Apple and Google in its mobile software and devices offerings.
Critics largely loved Windows Phone, but customers haven't signed on. And the Surface tablet -- the first PC of Microsoft's own design, rather than partnering with a manufacturer -- sold so poorly that Microsoft was forced this summer to take a $900 million writedown on excess Surface inventory.
Microsoft attempted to shuffle its deck by reorganizing the company several times, in an attempt to better align the different businesses around Ballmer's vision for a new Microsoft. In October 2012 Ballmer said the company would transition to a "devices and services" firm, focusing on making hardware, online services and apps that work together seamlessly across multiple screens and gadgets.
NBC technology reporter Julianne Pepitone contributed to this report.