IE 11 is not supported. For an optimal experience visit our site on another browser.

Peace, love and paychecks

Linux began as a labor of love by hippies and hackers. Now the suits are cutting checks and running the show.
/ Source: Forbes

Trond Myklebust has no experience designing data storage devices and has never studied computer science, and yet, for the past year this affable 35-year-old graduate student in particle physics at the University of Oslo in Norway has been collecting handsome monthly stipends from Network Appliance, a Silicon Valley company that makes data storage boxes.

NetApp, as the company is known, has also flown Myklebust to the States for extended visits, arranged his visas, paid his expenses, rented him an apartment and set him up in an office at the University of Michigan. The firm also flies him to headquarters in Sunnyvale, Calif., where he is treated to sushi dinners and stays in the home of one of NetApp's senior technologists.

Why the VIP treatment? Myklebust is one of the chief lieutenants to Linus Torvalds, creator of the grassroots Linux operating system, and he oversees development of a crucial bit of code that controls how Linux-based computers share files with other machines. For NetApp, whose boxes need that 35,000-line chunk of code to interact with Linux computers, the chance to gain favor with Myklebust is worth dishing out a few perks.

By funding Myklebust and the Michigan computer lab, NetApp can influence Linux development and ensure that its storage boxes, called filers, work smoothly with Linux computers. It's like having the head of props for a movie studio on the payroll of your consumer brands company. "Linus wouldn't let NetApp create features for Linux. So we befriended Trond. He is the vehicle to let our company influence Linux," says Daniel Warmenhoven, chief executive of NetApp.

"What's in it for [NetApp] is sales; it can sell into the Linux market. This is not about philanthropy. There is plenty of mutual benefit going on here," says Peter Honeyman, scientific director of the lab where Myklebust works, which receives $192,000 a year from NetApp.

'Linux has changed'
This pay-for-play arrangement may be standard operating procedure among high-tech companies and academic labs, but it represents a big change for Linux, which first gained favor among hippie-esque programmers who disdained revenue and profit, advocating a "peace, love and software" vision of Linux as a free operating system developed without regard for corporate interests.

Linux is not developed by any single company. Instead, its creator, Torvalds, harnesses the creativity of thousands of programmers around the globe, who contribute their code free.

Though Linux was first used by hobbyists, it now runs in data centers of big companies like Charles Schwab & Co. and Sabre Holdings. And though some Linux zealots still consider themselves part of a quasi-religious movement, these days Linux is looking a lot like any other technology product, even those made by Microsoft, the company Linux wonks hate most.

"Linux has changed," Myklebust says. "Most of the big players in the community are now directly employed or sponsored by businesses who are pushing some agenda. That's more or less accepted these days."

Despite his close ties to NetApp, Myklebust insists he gives NetApp no special favors. He says he still feels free to reject any code NetApp submits, though he also points out that working so closely with NetApp means he can suggest ways it might change its code to make it pass muster. "They bounce ideas off me, to find out how I prefer to have things done," is how he describes the process.

Even Torvalds, the young Finnish programmer who created Linux in his college dorm room in 1991, now collects a paycheck from a Beaverton, Ore. lab funded by Hewlett-Packard, IBM and others. Torvalds' second-in-command, Andrew Morton, works there. Other "maintainers," or developers who have primary responsibility over different parts of Linux, have been scooped up by H-P, IBM and Red Hat. Torvalds says it's like Lance Armstrong being paid to train and race. "Linux developers have picked up sponsors," he says via e-mail.

The rise of the middleman
Big companies pick up the tab for Linux development because the system helps them sell hardware and consulting services. HP claims $2.5 billion in Linux-related revenue in 2003, while IBM claims $2 billion. Red Hat, which distributes a version of the Linux operating system, generated $125 million in revenues last fiscal year and carries a market value of $2.3 billion. Last year sales of Linux servers grew 48% to $3.3 billion, and by 2008 Linux server sales could approach $10 billion, according to market researcher IDC.

NetApp, which does millions of dollars in Linux-related sales, got dragged into doing Linux development because its customers started using Linux on their computers and having problems. Moving files between Linux computers and NetApp filers was slow and difficult. The problem had nothing to do with a flaw in NetApp's boxes; it resided in Linux, which used a poor implementation of a file-moving technology called Network File System, or NFS, which was created in the 1980s by Sun Microsystems.

Customers who needed help didn't complain to Torvalds; they called NetApp. "In 1998 and 1999 we started getting an eruption of support calls," recalls Brian Pawlowski, vice president of engineering at NetApp.

Luckily, Pawlowski knows more about NFS than almost anyone else on the planet, since he was part of the group at Sun that created it 20 years ago. In the late 1990s David Hitz, a NetApp cofounder and chief of engineering, had lunch with Linus Torvalds and told him that NetApp would gladly fix the NFS implementation in Linux. Torvalds turned him down. Says NetApp Chief Warmenhoven: "His answer basically was, 'No. I don't trust companies, I trust people. The person I trust on this is Trond.' Dave came back from that lunch and I said, 'Wow, that's wild, how are we going to deal with this? It seems strange. Why would anyone reject the world's top experts?'"

So Pawlowski and his guys would have to submit their "suggestions" to Myklebust, who would accept or reject their ideas alongside submissions from volunteer hackers from around the world.

Companies, developers both profit
NetApp found a way around the roadblock using a time-tested Silicon Valley method: Start spreading money around. In 1999 NetApp began funding the University of Michigan's 18-year-old Center for Information Technology Integration (CITI), where a lot of Linux NFS development was taking place. In 2001 NetApp hired a developer, Charles Lever, who already worked out of the CITI lab in Ann Arbor. Finally, in 2002 NetApp started paying Myklebust a stipend, set him up with an office in the lab and a company-paid apartment in Ann Arbor. In 2003 Myklebust returned to his studies in Norway and left the payroll for six months but came back to NetApp.

Landing a Linux insider like Myklebust was a coup for NetApp and CITI, Honeyman says. "Trond sits at the right hand of Linus. Nothing gets into Linux unless Trond says so. He's the funnel, the gatekeeper. He has the magic wand. And he's sitting right here with us."

Today, in addition to paying Myklebust's stipend and expenses, NetApp pumps $16,000 a month into the CITI lab, including support for one of Honeyman's graduate students, who is pursuing a doctorate. "In total we're spending several hundred thousand dollars a year," Pawlowski says.

NetApp, which earned $152 million on sales of $1.2 billion in the recent fiscal year, can afford the outlay. The payoff is real. Linux now contains bits of code written by NetApp's programmers. More important, developers at CITI are using NetApp filers as the basis for their software design and optimizing their code for those boxes, which may already be giving NetApp an edge in Linux installations. So far it has won business it wouldn't have otherwise at Oracle, Pixar, Southwest Airlines, ConocoPhillips and Weta Digital, the effects studio behind Lord of the Rings.

Myklebust has abandoned his quest for a doctorate in particle physics. He says he's having too much fun doing Linux development, and he enjoys living in the U.S. What's more, he's found a great job in the computer industry, thanks to an American company whose top-notch immigration lawyers moved mountains to get him an H1-B work visa, overcoming immigration officials' objections that Myklebust lacked any formal computer education. Starting in October Myklebust will become a full-time employee at Network Appliance.