Technology News

latest updates from easySERVICE™

Firms Struggle To Implement Big Data In Production

Big-Data1

Introduction:

Enterprise practitioners believe the potential value of Big Data is significant, but many are struggling to derive maximum value from their investments in related technology. While a majority a Fortune 500 companies have Big Data deployments in production, and a significant percentage of mid-sized enterprises have proof-of-concept and pilot projects underway, as per expert’s close to half have not realized the level of value anticipated at their onset.

In a recent survey, 46% of Big Data practitioners report that they have only realized partial value from their Big Data deployments. An unfortunate 2% declared their Big Data deployments total failures, with no value achieved.

Firms Struggle To Implement Big Data In Production

Implementing production-ready big data solutions is easier said than done. Firms face a hydra of challenges that make it difficult to go from a tidy proof-of-concept (POC) to production. The number one challenge: integrating big data solutions in a complex, heterogeneous data management environment. This is not surprising since most large enterprises have an enormously diverse set of technologies, tools, architectures, and information security policies.

Taking data science into action requires deploying statistical models into production environments, usually with real-time processing requirements. Every company that relies on predictive models to drive their applications and operations has a different process for model deployment, but by working with many such companies has seen a common pattern emerge. The real-time model deployment process can be broken down into these five stages:

  • Data distillation
  • Model development
  • Model validation and deployment
  • Model refresh
  • Real-time model scoring

In a recent survey, 46% of Big Data practitioners report that they have only realized partial value from their Big Data deployments. An unfortunate 2% declared their Big Data deployments total failures, with no value achieved.

These findings are all the more striking when considering the level of value Big Data practitioners expected to achieve. Even in cases where initial production uses are successful, subsequent projects often don’t achieve similar levels of value as the lack of skilled practitioners and/or performance issues begin to take a their toll.

Conclusion:

Enterprises that have achieved significant value from Big Data are those that address both these issues at the onset of new Big Data projects. These projects are generally not initiated by IT but driven by line-of-business departments, often marketing, and focus on small but strategic use cases.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: