Your Big Data challenges: Got the budget, got the skills – what’s the problem?

How well are you facing up to the big data challenges that are thrown up as part of your attempts at becoming a data driven organisation?

Whether the budget for your business’ big data initiatives comes from within IT, or the business, or a bit of both,  if it’s Big Data, it probably going to have a material influence on your business. Whether that be driving critical strategic, tactical or operational decisions, eliminating waste, fuelling new innovations through to identifying new markets and business opportunities.

All good so far.  You’re on a roll and ready to kick-start your project. Feel confident that you’re across the big data challenges ahead ?

Got the necessary budget and skills?   Now for the hard part.

Great – you’ve managed to marshal a team with the necessary skills to tackle your Big Data initiative.   Now for the really hard part – to ensure that they not only play nicely together as a team, but have the capability of engaging with the organisation’s key stakeholders as needed.

Having a team of highly talented professionals from a range of backgrounds requires effective leadership in order to ensure the appropriate culture exists.  Process, governance models and procedures alone are no substitute for truly effective leadership – not management, but leadership.

Given that your team member’s skills and competencies are proven, your Big Data initiative’s standout success will rely on a team that cuts across organisational silos, are highly aligned to the defined outcomes, are responsive, adaptable and engaged. Whether the various team members are drawn from your business, your vendor or a consulting firm is less important than their ability to operate as a collaborative, integrated unit.

Data science skills?  No need, we’ll rely on our vendor.

One of the current big data challenges relates to skill shortage.

The demand for individuals with the deep skills needed to underpin Big Data initiatives is on the rise.  Depending on your organisation’s specific Big Data initiatives, these skills could include statistics, mathematics, operations research, information architecture, industrial engineering, not to mention having those with deep industry and organisation specific knowledge actively involved. It also may impact your budget as these skills don’t come cheap.

The aggregated demand for talent from government and national security agencies, large IT providers, consulting firms and large corporations can challenge your organisation’s Big Data initiatives by starving it of those all important specialized skills.

If you cannot get the necessary skills, having an absolute reliance on a vendor to magically deliver your Big Data initiatives as a ‘black box’ is not where you want to be if your business is dependent on these systems and technologies.

Taking an aviation analogy, good pilots really understand the intricacies of how their aircraft works. The same concept applies to your Big Data initiatives.  Understand what’s ‘under the hood’.

Getting the balance right on the proportion of ‘in-house’ skills to those that exist in your vendor’s ecosystem should be deliberately and carefully considered.

More importantly, unless your Big Data initiative is large enough to keep specialists fully utilized, the challenges you face will be in trying to secure expertise on an intermittent basis.  Relying on a revolving door of experts in developing, supporting and maintaining your complex Big Data initiative may present a additional challenge to developing and maintaining the optimal team culture. The impact on your budget could also be substantial.

The key factor to consider here is to assess which skills and knowledge need to be developed or retained in-house to protect your competitive advantage. Where do the short, medium and longer term risks lie, if any exist?

Has your organisation got what it takes to handle Big Data challenges ?

To ingrain Big Data capabilities within your organisation will most likely require a fundamental rethink on the business’ strategy, structure, workflows, business processes and staff structures, not to mention successfully managing the necessary cultural change. Expecting your enterprise-wide, mission critical Big Data initiative to flourish in a highly siloed structure is a risky assumption.

More importantly, if the senior leadership team in your organisation can successfully make the necessary organizational changes in support of your Big Data initiatives in a coordinated manner, a significant barrier to sustained success will be removed.

Otherwise, will your organization’s Big Data initiative be merely another case of “old wine in new bottles”?