Just a few years ago, meteorologists regularly shared an inside joke about forecasts: Anything beyond three days is a guess.
But on the heels of this week's disastrous tornadoes in Moore, Okla., and last October's Hurricane Sandy, it has never been more critical to be able to forecast long range events. Big data analysis is turning the guesswork of yesterday's meteorology into a more precise and predictive science.
Several big data companies are focusing on long-term weather modeling, and have found that by identifying patterns, comparing them to current conditions and adding in a layer of predictive analysis, they're able to confidently predict what's to come weeks and even months in advance.
"Today's traditional forecasting models tend to break down in a week to 10 days because they can't input every observed data piece from around the world," said John Plavan, co-founder and CEO of EarthRisk Technologies. "There's too much chaos in the system."
(Watch: Inside the World of Big Data's 'Big Brother')
While it's still virtually impossible to predict an event like this week's tornado, the models and forecasts from big data companies can be extremely valuable to a variety of businesses, ranging from retailers to insurers, as they plan ahead.
EarthRisk, for example, focuses on the energy trading market. By focusing in on probability models for extreme heat and extreme cold, it can help investors profit in the futures market.
Plavan points to the winter of 2011-2012, which many traders in the natural gas market expected to be cold, driving up futures prices. EarthRisk's models, though, showed that the atmosphere wasn't setting itself up for a high probability of cold weather—letting clients position themselves to make money when natural gas prices declined.
Big data weather modeling is still a niche field, but it's one that's growing fast. IBM has even launched a division focusing on it—dubbed Deep Thunder.
That program tends to take less of a long-term view, working instead with clients (which range from the utility, transportation and agriculture industries to municipal governments) to give highly focused predictions of near-term events.
(Read More: Retail Goes Shopping Through Big Data)
"We start with a client's business problem," said Lloyd Treinish, chief scientist of the program for IBM Research. "Clients indicate to us what kind of lead time they need to be more proactive. ... For a utility company, for instance, it's all about how quickly the power can be restored. And that requires resources. If we understand the nature of not just the impending weather events, but the outages caused by the damage they cause— and the kinds of people and equipment needed to restore that power, then [clients] can be prepared."
Deep Thunder is also heading a project in Rio de Janeiro to better anticipate flooding, and predict where mudslides might be triggered by severe storms.
What big data weather modeling can't do is stop extreme weather from happening.
Kinectic Analysis works with clients to create financial models that examine the impact of weather events. Coming up with those numbers is harder to do long term, but the closer a weather event is to happening, the easier it is to predict how bad it will be.
"The bottom line is we can talk about probability for the long term, but when you get to specific numbers, generally within five to six days out, we can come up with a range of loses from a storm—and by the time we're one day out, we can get within 10 percent or so," said Chuck Watson, director of research and development at Kinetic Analysis.
(Watch: Big Data Giveaway's Big Payoff)
Each firm has different methods in how they analyze the data, but it often comes down to gathering as much information as they have and fitting into their models. Deep Thunder, for instance, gathers data not only from the National Oceanic and Atmospheric Administration, NASA and the U.S. Geological Survey, but also from private sources, such as Earth Networks, owner of WeatherBug, which has a network of more than 8,500 weather stations.
And while major weather events tend to get the most attention, it's everyday weather that many big data weather service clients are interested in. Retailers, for instance, can use them to help determine shipping schedules—or better plan promotional events.
Even with big data and the most advanced physics to support their efforts, companies specializing in weather simulation models admit it's still an inexact science—not even the most powerful algorithms in the world can beat Mother Nature. It's a chaotic fact of life, and business risk, that not all clients of these big data companies can appreciate.
"We can do a pretty decent job of coming up with what long term probabilities are," said Watson. "The big issue with big data analysts is the answers are very much tied to probabilities and statistics, and most business managers aren't really adept at handling that sort of thing. ... What tends to happen is economics drives stuff as much as the hazard risk."
_ Chris Morris, Special to CNBC.com