The industry-wide trend of digitalisation means many companies now find themselves overwhelmed with large amounts of data. When the data begins streaming in, there naturally arises the question as to what is to be done with all this data.
Data naturally brings with it new set of issues to tackle, the chief among them being accuracy, storage, security, transfer, sharing, and filtering. Most companies do not have the expertise to deal with large amounts of data. Just dealing with the data in terms of storing it securely is useless unless there is somebody competent to analyse it and use it for refining operations, cutting costs, recognizing behaviour, studying trends, identify mistakes, finding areas for improvement, etc.
Companies have therefore identified a few different ways to deal with the data depending on what suits their appetite in handling data:
Outsource: When the company knows they have no expertise to deal with the data and are not interested to embark on it either, they prefer to outsource that task. They leave it to the experts, typically companies that are specialised in data handling. The data handling company deals with all data related issues and provides periodic updates and reports on insights and trends.
IT Department: The moment there is anything to do with data, it is usually brought under the wings of the IT department. The department deals with the data end to end and are exclusively responsible irrespective of where the data originates from or where it is intended to be used. At times an analyst or two are recruited to be part of the IT department.
Data Manager: Companies realise that data generated from various sources is intended for departments with differing expertise and functions. The data manager’s role is usually cross functional and has end to end responsibility for data. The data is collected and distributed depending on who needs it. The data manager is autonomous and usually independent of the IT department. Sometimes the data manager has a cross functional data team to help with the task.
War Room: When data is streaming in real time, companies have realised that it gives them a competitive advantage to take decisions early using the data. Such companies have an entire department, based on the military war room concept. The department is usually dedicated to dealing with real time operational data and includes data experts who deal with collection, filtering, mining, storing, analysis, reporting, etc.
Hybrid: Some companies also operate a hybrid system of handling their data. A typical example is where data collection is taken care of in-house. They however do not employ analysts or indulge in any sort of detailed data scrutiny, analysis or study. Depending on which area of the business demands further analysis, the relevant data set is shared with a specialised data consultancy company for specific analysis. Companies with ‘Intellectual Property’ issues choose this approach, as it allows them to retain the secrecy without revealing crucial confidential information. They usually engage another competent data consulting firm for analysis into another area.
With digitalisation and data collection becoming the norm, managing big data has become a new area that companies certainly need to invest in to remain competitive. Whether they do it all themselves or leave it to the data experts is a trend that will be closely watched.
Is your experience of managing data any different? Have you seen other ways in which data is being dealt with?
Let me know your thoughts.