How Many Big Data Scientists Does it Take to Screw in a Lightbulb?

Light Bulb

  • Start with a Question
  • Stories of Big Data Success
  • True 100% HA Coud

Question: How many big data scientists does it take to screw in a lightbulb?

Answer: Just a minute. Let me run the algorithm.

Digital data doubles every two years. Just think about how far we’ve come: a typical laptop now contains far more data than was used to send a mission to the moon in 1969.

“The rate at which we’re generating data is rapidly outpacing our ability to analyze it,” said University College of London data scientist Dr. Patrick Wolfe. “The trick here is to turn these massive data streams from a liability into a strength.”

Okay, so we aren’t making the most of the data that we are creating. Big whoop. Actually, the extent to which we are missing extraordinarily valuable data analytic opportunities is incredible: right now, only 0.5% of our information is analyzed. That means that 199 out of every 200 pieces of data is dust in the wind, so it turns out that Kansas’s ballad is still relevant even if their hairdos aren’t. In fact, the proportion of data that gets analyzed isn’t going up but down. We have more data, but it’s not always more intelligent data.

The promise of big data is exciting. Even today, it can read DNA instructions and forecast the growth of disease. It can even suggest what movies we might like in our Netflix account.

Big data improves sustainability by reducing power use, and less use of resources also means savings: $200 billion per year, according to one estimate. Chicago and New York City are now being called “smart cities” in the press for integrating Internet of Things sensors with analytics to streamline spending and improve infrastructural efficiency.

Part of the problem with big data is that it’s not valuable until it’s understood. Therefore, you have to clear out what types of data can answer what questions.

“You have to start with a question and not with the data,” said UC Berkeley lecturer Andreas Weigend. “The fact that [the data] gets collected is a good thing,” he added, but what we really need is to figure out what problems we can solve with it.

Stories of Big Data Success

It always helps to hear a few anecdotes to remind us that big data isn’t just a neat idea but something that’s giving businesses serious competitive advantage.

The barbecue restaurant chain Dickey’s has jumped headfirst into the arena. What it’s doing is something of interest to every business: optimizing social media through data analytics, essentially tying together the third platform. By using social to respond to big data findings almost immediately, the restaurant has proven it’s not just a restaurant but an information powerhouse.

Another big enterprise project that hasn’t yet seen the results achieved by Dickey’s is an effort by Carnival Cruises. Their project is focused on data, from every possible reliable source they can access, related to online pricing behavior.

“At Carnival, the number [of passenger cruise days] is 80 million across its fleet of 100 ships and nine brands,” explained the Wall Street Journal (via Forbes). “To CEO Arnold Donald, that means that if every passenger spent just $1 more per day aboard ship, Carnival would see an extra $80 million in revenue for the year.”

The IT climate in Australia has long impressed tech writer Howard Baldwin. He noted that just as with other cutting-edge areas of technology, that it’s booming – not just in the statistics, but on the ground. Baldwin gave the example of Telstra, a telecommunications firm with a predictive model that forecasts possible network problems. In other words, while Dickey’s and Carnival use data to understand their customers, Telstra uses it to improve its own infrastructure. To achieve that, it simply looks at real-time performance vs. standardize baselines.

Avis is pulling together data from various fields that tell how often a person has rented, any problems they’ve had with the car, age and sex, company of employment, comments made by the individual, and any interactions on social media. These figures reveal how much each user is worth. Assumedly, soon on the receipts from Avis, it will let you you know your current dollar value to the company: it might tell you that you are in the 82nd percentile and that your total, real-time worth to the company is $462.13.

Data can even be used in life-or-death situations. Beth Israel Deaconess Medical Center CIO John Halamka takes a forward-thinking approach toward healthcare IT, both professionally and personally. Unrelated specifically to big data, the medical center is the site of the second, psychiatric pilot of the PGHD patient engagement project OpenNotes, which seems to be proving that medical errors are reduced when patients have access to doctor notes.

Back to big data: on the personal side, Halamka used his wife’s genetics and genomics to look at data from several hospitals near Harvard when she was diagnosed with breast cancer.

“Of the last 10,000 Asian females, with a tumor like this, how were they treated, what was the outcome,” said Halamka “Ensure she gets the medicine that seems to provide the best outcome for people like her.”

His wife survived and is now completely recovered from the cancer.

True 100% HA Cloud

It’s clear that big data can help businesses in a wide variety of ways. The most reasonable way to run this type of project is (of course) cloud computing.

The best cloud providers achieve true 100% high availability, impressively outperforming (often 300% better than) AWS, SoftLayer, and others, by:

  • Distributing their architecture rather than using mainframe-era centralized storage
  • Optimizing their network with InfiniBand (IB) rather than Ethernet

Plus, they have the PassMark benchmark ratings to prove their claims.

With Superb, you can create any big data project you need. If nothing else, you can create a model to answer the question at the top of this page.

By Kent Roberts

Loading Facebook Comments ...
Loading Disqus Comments ...

Leave a Reply