The trend in Big Data analytics among companies shows no sign in abating, with companies covetously collecting vast amounts of data with the hopes of harvesting market differentiators. A study by open-source research firm Wikibon, for instance, forecasts an annual Big Data software growth rate of 45% through 2017. But what tools are companies using to implement Big Data solutions? For purposes of this article, let's set aside for a moment the intended outcome of whatever Big Data project your company has planned in the coming year (whether it be predicting the outcome of Supreme Court cases or helping a baffled spouse pick out the right lingerie set), and instead let's focus on the tools available in the industry (and some of the associated pitfalls) in getting your company from concept to solution.
First, consider how you are going store and analyze the data. For companies with significant internal resources and focus on Big Data, it may make sense to hire an in-house analytics team and invest in the requisite infrastructure and tools. However, there are many options in the marketplace that require less investment in order to gain actionable insights:
- Database Marketing Outsourcing: An end to end service often used by retailers in which a supplier licenses data and provides data mining analytics, marketing campaign sales management and analysis, and other ancillary functions.
- Analytics-as-a-Service: A "software-as-a-service" offering through which a supplier can quickly deploy data analytics resources without an upfront investment from the customer. AaaS offerings often draw data from external data sources as part of the services.
- Data Warehouse: A central location to store copies of data from multiple sources. Data warehouses vary in complexity from providing a relatively simple datamart to performing more complex functions such as online transaction processing. Generally, data is cleansed, organized and categorized in a manner to facilitate a customer performing its own analysis and reporting with the data.
- Public/Private Cloud: A private cloud provides for easily scalable solutions that can be customized by the customer on a cost effective basis. The public cloud is generally the most low cost option, but perceived risks in security and privacy prevent many companies from utilizing this option.
As the lines blur in these services offerings, we are seeing more analytics and cloud services bundled into a single offering within the industry.
Once your company has determined a solution for implementing your Big Data project, what are a few pitfalls to watch out for?
- Beware of the Supplier Form Contracts: It may seem obvious, but supplier contracts are almost always going to be very one-sided in favor of the supplier and negotiating is unlikely to give your company the same protections you will get when starting with your own form. If possible, advocate for using an alternative, customer friendly form. If you don't have the leverage to use an alternate form, then just focus on the key terms (see below for a starting list of them).
- Identify the Data "Pedigree": What data is going to be used in order to implement your solution? What is the source of data? Will external data be combined with your company's internally sourced data? Key questions for you to ask your supplier are : (1) where did the data come from, (2) how will the data be used as part of the solution, and (3) does the intended use of the data match the scope of the consent for which it was given? Ensure that the supplier has the right to use the data and that the use of the data matches the original scope of consent given by the individual that gave it.
- Define Your Rights to Supplier Data: If you anticipate using any supplier furnished data as part of your Big Data solution, then you need to ensure that you have clearly defined license rights to the data. Typically, a supplier will license its data for specified terms that expire at the end of the agreement. However, if data licensed from the supplier is integrated into the customer's own data, then such data cannot readily be removed and may prove to be expensive to accomplish. In order to protect your company, try to secure unlimited perpetual licenses to any data that is integrated with your own data. As an alternative, if you cannot obtain a perpetual license, then the supplier should bear the expense of removing the data from your data at the end of the relationship. For example, if you are in the business of creating aggregated customer records or scorecards, where supplier data is merged with your data, then extracting the supplier's data will be an expensive and difficult thing to accomplish, and may be detrimental to your business.
- Limited Supplier Termination Rights: Suppliers often ask for a right to terminate an agreement for convenience, or at minimum, for the right to not renew an agreement at the end of an initial fixed term. Generally, it is acceptable for a customer to push back on these terms and argue that the supplier should only be able to terminate for material breach in limited circumstances. However, it is not unrealistic that a supplier may have sound reasons for not wanting to renew an agreement (e.g., lack of predictability in the market, material changes in the service). In any circumstances, you should ensure that you have sufficient notice and time to transition your data back from the supplier so that service is not impacted by the termination. The contract should impose an obligation on the supplier to provide an actual plan on how the supplier will complete the transfer activities.
- Protect Your Customer Relationships and Data: Data analytics companies often rely on data you provide to improve their databases and enhance the services they offer all their customers. They may also use the data you provide about your customers to establish their own contractual relationships and/or market other services directly to your customers. While these arrangements may be acceptable in some contexts, make sure that they are clearly defined and that you have considered the implications of the data analytics provider's business model on your business and customer relationships both during and after the term of your contract.
- Data Security: If the data analytics provider will store or process your customer or other proprietary data in a cloud environment, the contract should impose clear data security obligations on the provider, including defining standards of care, SSAE 16 or other security audit requirements, and notification obligations following any unauthorized access or disclosure of your data.
- Allocation of Risk: Form contracts will often allocate most or all risks of using a data analytics solution onto the customer, even for claims that may arise through no fault of the customer. Likewise, the limitations of liability in form contracts will often cap the provider's liability at a negligible amount while exposing your company to unlimited liability. In most cases, it will be appropriate to negotiate a more balanced allocation of risk between the parties.
Keep these issues in mind whenever you are considering your next Big Data solution, and taking the first steps toward minimizing some of the inherent risks with data analytics.