Tuesday, March 20, 2018

Salesforce Buys MuleSoft and Offers It as a Data Unification Solutions

The Customer Data Platform industry is doing very well, thank you, with new reports out recently from both Gartner  and Forrester  and the CDP Institute launching its European branch.  But the great question hovering over the industry has been why the giant marketing cloud vendors haven’t brought out their own products and what will happen when they do. Oracle sometimes acts as if their BlueKai Data Management Platform fills the CDP role, while Adobe has made clear they don’t intend to do more than create a shared ID that can link data stored in its separate marketing applications. Salesforce has generally claimed its Marketing Cloud product (formerly ExactTarget) is a CDP, a claim that anyone with experience using the Marketing Cloud finds laughable.

The flaws in all these approaches have been so obvious that the question among people who understand the issues has been why the companies haven’t addressed them: after all, the problems must be as obvious to their product strategists as everyone else and the attention gained by CDP makes the gaps in their product offerings even more glaring. My general conclusion has been that the effort needed to rework the existing components of their clouds is too great for the vendors to consider. Remember that the big cloud vendors built their suites by purchasing separate products.  The effort to rebuild those products would be massive and would discard technology those companies spent many billions of dollars to acquire. So rationalization of their existing architectures, along with some strategic obfuscation of their weaknesses, seems the lesser evil.

We got a slightly clearer answer to the question on Tuesday when Salesforce announced a $6.5 billion purchase of Mulesoft, a data integration vendor that provides connectors between separate systems. In essence, Salesforce has adopted the Adobe approach of not pulling all customer data into a single repository, but rather connecting data that sits in its system of origin. In Salesforce’s own words, “MuleSoft will power the new Salesforce Integration Cloud, which will enable all enterprises to surface any data—regardless of where it resides—to drive deep and intelligent customer experiences throughout a personalized 1:1 journey.”

This is a distinct contrast with the CDP approach, which is to load data from source systems into a separate, unified, persistent database. The separate database has some disadvantages – in particular, it can involve replicating a lot of data – but it also has major benefits. These include storing history that may be lost in source systems, using that history to build derived data elements such as trends and aggregates, and formatting the data in ways that optimized for quick access by marketing systems and other customer-focused applications.

Although the difference between these two approaches is clear, some practical compromises can narrow the distance between them. Most CDPs can access external data in place, reducing the amount of data to be moved and allowing the system to use current versions of highly volatile information such as weather, stock prices, or product inventories. Conversely, a system like Mulesoft can push data into a persistent database as easily as it can push it to any other destination, so it can build some version of a persistent database. In fact, many CDPs that started out as tag managers have taken this approach.

But pushing data into a persistent database isn’t enough. Mulesoft and similar products work with well-defined inputs and outputs, while CDPs often can accept and store data that hasn’t been mapped to a specific schema. Even more important, I’m unaware of any meaningful ability in Mulesoft to unify customer identities, even using relatively basic approaches such as identity stitching. It’s possible to build workarounds, such as calls to external identity management systems or custom-built matching processes. Again, these are solutions employed by some CDP vendors that also lack advanced identity management. But such solutions can be costly, complex, and incomplete. From a buyer’s perspective, they are compromises at best. No one – except a salesperson – would argue they’re the ideal approach.

In short, Salesforce’s purchase of Mulesoft offers a partial solution to the needs that have driven the growth of CDPs. It’s probably the best that Salesforce could do without making the impractical investment needed to rebuild its existing marketing cloud components.  Get ready for a lot more confusion about the best way to build unified customer data. To avoid getting distracted, focus on what marketers really need and let that, not theory or vendor hype, drive your evaluation of the alternatives.

Monday, March 19, 2018

Picking the Right First Project for Your Customer Data Platform

For the past year, the most common question about Customer Data Platforms has been how they differ from Data Management Platforms. Recently that seems to have changed.  Today, the question everyone seems to be asking is what project they should pick as their first CDP use case.

That’s certainly progress but it’s a much harder question to answer than the one about DMPs. Like any good consultant, I can only answer that question with “it depends” and by then asking more questions. Here are some of the factors that go into a final answer.
  • What resources do you have available? The goal for your initial use case is to get something done quickly that returns substantial value. Getting something done quickly means you want a project that uses existing resources to the greatest degree possible. Ideally, the only new element would be the CDP itself, and even the CDP deployment would use a small number of data sources. So, an ideal first project would use data in existing systems that is well understood, known to be of high quality, and can easily be extracted to feed the CDP. Alternately, the first project might involve new data collected by the CDP itself, such as Web site behaviors captured by the CDP's own page tag. If the first project is purely analytical, such as customer profiling or journey analysis, then you don’t need to worry about connecting to execution systems, although you do need staff resources to properly interpret the data and possibly some analytical or reporting systems. But if you happen to have good execution systems in place, it may make sense for the first project to connect with them. Or, you may pick a CDP that provides its own execution capabilities or can feed lists or offer recommendations to external delivery systems.
  • What use case will provide value? This is where good delivery resources can be helpful: it’s much easier to deliver value with a use case that involves direct customer interaction and, thus, the opportunity to increase revenue or reduce costs. Often this can still be quite simple, such as a change in Web site personalization (involving just one channel for both source and delivery), an event-triggered email, or a prioritized contact list for sales teams. If execution isn’t an option, an analytical project can still be valuable if it presents information that wasn’t previously available. This may mean combining data that was previously kept separate, reformatting data couldn’t be analyzed in its original form, or simply pulling data from an inaccessible system into an accessible database. The trick here is for the analysis to generate insights that themselves can be the basis for action, even if the CDP isn’t part of the execution process.
  • How much organizational change will be needed? Technical obstacles are often less significant barriers than organizational resistance. In particular, it can be difficult to start with projects that cross lines of authority either within marketing (say, separate Web and email teams) or between marketing and other departments (such as operations or customer support). When considering such changes, take into account the needs to revise business processes, to provide adequate training, to align compensation systems with the new goals, to provide reporting systems that track execution, and to measure the value of results. As a practical matter, the fewer parts of the organization affected by the initial project, the easier it will be to deploy and the higher the likelihood of success.
  • Where’s the pain? It’s tempting to search for an initial project that is primarily easy to deploy. But even an easy project is competing with other demands on company resources in general and on staff and managers’ time in particular. So it’s important to pick a first project that solves a problem that’s recognized as important. If the problem is big enough – and it’s clear the CDP can solve it – then you have a good chance of convincing the company to make a substantial investment from the start. Ultimately, this is the right approach: after all, the CDP isn’t an end in itself, it’s a tool for improving your business. You may see a broad range of applications for your CDP but for those who don’t share that vision, you’ll need to show its value at every step of the way.

Tuesday, March 13, 2018

Eager to Sell Your Personal Data? You'll Have to Wait

Should marketers pay consumers directly to access their personal data? The idea isn’t new but it’s become more popular as people see the huge profits that Google, Facebook, and others make from using that data, as consumers become more aware of the data trade, and as blockchain technology makes low cost micro-payments a possibility.

One result is a crop of new ventures based on the concept has popped up like mushrooms – which, like mushrooms, can be hard to tell apart. I’ve been mentioning these in the CDP Institute newsletter as I spot them but only recently found time to take a closer look. It turns out that these things I’ve been lumping together actually belong to several different species. None seem to be poisonous but it’s worth sharing a field guide to help you tell them apart.

Before we get into the distinguishing features, let’s look at what these all have in common. They’re all positioned as a way for consumers to get value from their data. I’ve also bumped into a number of data marketplaces that serve traditional data owners, such as Web site publishers and compilers. They can often use some of the same technologies, including micro-payments, blockchain, and crypto-currency tokens. Some even sell personal data, especially if they’re selling ads targeted with such data. Some sell other things, such as streams from Internet of Thing devices. Examples of such marketplaces include Sonobi, Kochava, Narrative I/O, Datonics, Rublix and IOTA. Again, the big difference here is the sellers in the traditional marketplaces are data aggregators, not private individuals.

Here’s a look a half-dozen ventures I’ve lumped into the personal data marketplace category (which I suppose needs a three letter acronym of its own).

Dabbl turns out to be a new version of an old idea, which is to pay people for taking surveys. There are dozens of these: here's a list.  Dabbl confused me with a headline that said “Everyone’s profiting from your time online but you.” Payment mechanism is old-school gift cards. On the plus side: unlike most products in this list, Dabble is up and running.

Thrive pays users for sharing their data, but only in the broad sense that they are paid to fill out profiles which are exposed to advertisers when the users visit participating Web sites. The advertisers are paying Thrive; individual users aren’t deciding who sees their data or paid to grant access on a buyer-by-buyer basis. Payments are made via a crypto-token which is on sale as I write this. The ad marketplace is scheduled for launch at the end of 2018. That sequence suggests there’s at least a little cryptocurrency speculation in the mix. (Another hint: they’re based in Malta. Yet another hint: the U.S. Securities Exchange Commission won’t let you buy the tokens.)

Nucleus Vision is also in the midst of its token sale.  But they’re much more interested in discussing a propriety technology that detects mobile phones as they enter a store and shares the owner’s data using blockchain as an exchange, storage, and authorization mechanism. Store owners can then serve appropriate offers to visitors. This sounds like a lot of other products except that Nucleus’ technology does it without a mobile app. (It does apparently need some cooperation from the mobile carrier.) Rewards are paid in tokens which can be earned for store visits, by using coupons or discounts, by making purchases, or by selling data. Each retailer runs its own program, so this isn’t a marketplace where different buyers bid for each consumer’s data.  Sensors are currently running in a handful of stores and the loyalty and couponing systems are under development.

Momentum is an outgrowth of the existing MobileBridge loyalty system.  It rewards customers with yet another crypto-token (on sale in late April) for marketer-selected behaviors. Brands can play as well as retailers but it’s still the same idea: each company defines its own program and each consumer decides which programs to join. The shared token makes it easy to exchange or pool rewards across programs. The published roadmap is ambiguous but it looks like they’re at least a year away from delivering a complete system.

YourBlock gets closer to what I originally had in mind: it stores personal data (in blockchain, of course), uses the data to target offers from different companies, and lets consumers decide which offers to accept. Yep, there’s a crypto-token that will be used to give discounts. Sales started yesterday (March 12) and are set to close by April 23. Development work on the rest of the platform will start after the sale is over, with a live product due this August.

Wibson calls itself a “consumer-controlled personal data marketplace” and, indeed, they fit the archetype: users install a mobile app, grant access to their data, and then entertain offers from potential buyers to read it. Storage and sharing are based on blockchain but payments are made via points rather than a crypto-token. At least that’s how it works at the moment: in fact, Wibson has just completed its initial mobile app and you can’t download it quite yet. During the initial stage, only Wibson will be able to buy users’ data and they’ll just use it for testing. If they’ve published a schedule for further development, I can’t find it.

So, that’s our little stroll through the personal data marketplace. Less here than meets the eye, perhaps – most players offer more or less conventional loyalty programs, although they use blockchain and crypto-tokens to deliver them.  True marketplaces are still in development. But it’s still an interesting field and well worth watching. As with mushrooms, look carefully before you bite.

Sunday, March 04, 2018

State of Customer Data Platforms in Europe

The Customer Data Platform Institute will be launching its European branch later this month with a series of presentations in London, Amsterdam and Hamburg. We’ve seen considerable CDP activity in Europe – nearly one quarter of the CDPs in the Institute's latest industry update are Europe-based, several others with European roots have added a U.S. headquarters, and some of U.S.-based CDPs  have significant European business. A recent analysis of CDP Institute membership also found that one quarter of our individual members are in Europe. So what, exactly, is the state of CDP in Europe?

 It’s long been an article of faith on both sides of the Atlantic that the U.S. market is ahead of Europeans on marketing technology in general and customer data management in particular. That (plus the larger size of the U.S. market) is why so many European vendors have relocated to the U.S. This study from Econsultancy suggests the difference is overstated if it exists at all: 9% of European countries reported a highly integrated tech stack, barely under the 10% figure for North American companies. North American firms were actually more likely to report a fragmented approach (48% vs 42%), although that was only because European countries were more concentrated in the least advanced category (“little or no cloud based technology”) by 20% vs 13%.

 The assumption that cloud-based technology is synonymous with advanced martech is debatable but, then again, the survey was sponsored by Adobe.  What is clear is that European firms have generally lagged the U.S. in cloud adoption -- see, for example, this report from BARC Research.

Lower cloud use probably hasn’t directly impeded CDP deployment: although nearly all CDPs are cloud-based, a substantial number offer an on-premises option. (The ratio was seven out of 24 in the CDP Institute’s recent vendor comparison report, including nearly all of the Europe-based CDPs.) But the slower cloud adoption may be a hint of the generally slower pace of change among European IT departments, which could itself reduce deployment of CDPs.

A Salesforce survey of IT professionals supports this view. Answers to questions about leading digital transformation, being driven by customer expectations, and working closely with business units all found that U.S. IT workers are slightly but distinctly more business-oriented than their European counterparts. Interestingly, there’s a split within the European respondents: UK and Netherlands are more similar to the U.S. answers than France and Germany. I should also point out that I’ve highlighted questions where the U.S. and European answers were significantly different – there were quite a few other questions where the answers were pretty much the same.

Organizational silos outside of IT are another barrier to CDP adoption. A different Salesforce survey, this one of advertising managers, also found that North American firms are generally more integrated than their European counterparts. The critical result from a martech perspective is North American marketing and advertising departments were much more likely to collaborate on buying technology.

Then again, a Marketo survey found that European respondents (from a mix of IT, marketing, sales, and service departments) were generally more satisfied with their tools and performance, even though they lagged North Americas in slightly innovation and more clearly in strategic alignment with corporate objectives. This isn’t necessarily inconsistent with the previous results: being less integrated with other departments may free the Europeans to pursue their departmental goals more effectively, even if they’re less fully aligned with corporate objectives. Other surveys have given similar results: people are generally happier with technology when they buy it for themselves.

Not surprisingly, one area where the Europeans are clearly ahead in preparation for GDPR: a Spiceworks survey at the start of this year found that 56% of European companies had allocated funds for compliance compared with just 31% of U.S. companies. (Almost half the U.S. respondents believe GDPR wouldn’t affect them, even though GDPR applies globally.) While the result clearly relates to the fact that GDPR is a European Union regulation, it may also reflect a generally higher interest in privacy among European consumers: to take one example, ad blocking is much more common in Europe than the U.S. That’s good news for CDP vendors, since GDPR has emerged as one of the primary use cases.

On the other hand, a survey from Aspect found that U.S. consumers are generally more demanding than Europeans about customer service: they care more about having a choice of service channels, are more willing to pay extra for good service and are quicker to stop buying after a poor experience. This is probably bad news for European CDP vendors, since unified customer data is a foundation for modern customer service.

In sum, things really are a bit different in Europe. Integration, the primary CDP use case, is lagging compared to the U.S. So it makes sense that CDP adoption is also lagging.  But GDPR may be changing the equation and consumer attitudes are certainly adding external pressure.  The need for CDP is growing and we hope the CDP Institute’s European operations will make it a little easier for European companies find right solutions.

Friday, February 23, 2018

Will CDP Buyers Consider Private Clouds as On-Premise Deployment?

Most Customer Data Platforms are Software as a Service products, meaning they run on servers managed by the vendor. But some clients prefer to keep their data in-house. So before releasing the CDP Vendor Comparison report – now available here – I added a line for on-premises deployment.

This seemed like a perfect fit: a clear yes/no item that some buyers consider essential. But it turned out to raise several issues:

- on-premises vs on-premise. I originally used “on-premise”, which is how the term is typically rendered. One of the commenters noted this is a common error. A bit of research showed it’s been a topic of discussion but on-premise is now more widely used relating to computer systems.  On-premises actually sounds a bit pedantic to me, but I’m using it to avoid annoying people who care. (Interestingly, no one seems too concerned about whether to use the hyphen. I guess even grammar geeks pick their battles.)

- private clouds. Several vendors argued that on-premises is an old-fashioned concept that’s largely been replaced by private clouds as a solution for companies that want to retain direct control over their systems and data. This resonated: I recalled seeing this survey from 451 Research showing that conventional on-premises [they actually used “on-premise”] deployments now account for just one-quarter of enterprise applications and the share is shrinking.

Percentage of Applications by Venue:
24% Conventional (on-premise, non-cloud)
18% on-premise private cloud
15% hosted private cloud
14% public cloud
13% off-premise non-cloud
Source: 451 Research, Strategy Briefing: Success Factors for Managing Hybrid IT, 2017

My initial interpretation of this was the on-premises private clouds meet the same goals as conventional on-premises deployments, in the sense of giving the company’s IT department complete control. But in discussions with CDP vendors, it turned out that they weren’t necessarily differentiating between on-premises private clouds and off-premise private clouds, which might be running on private servers (think: Rackspace) or as “virtual private servers” on public clouds (think: Amazon Web Services). Clearly there are different degrees of control involved in each of these and companies that want an on-premises solution probably have their limits on how far they’ll go in the private cloud direction.

- public clouds. One vendor speculated that most remaining conventional deployments are old systems that can’t be migrated to the cloud. The implication was that buyers who could run a CDP in the cloud would gladly do this instead of insisting on an on-premises configuration. This survey from Denodo suggested otherwise: while it found that 77% of respondents were using a public cloud and 50% were using a virtual private cloud, it also found that 68% are NOT storing “sensitive data” in the public cloud. Presumably the customer data in a CDP qualifies as sensitive. I don't know whether the respondents would consider a “virtual private cloud” as part of the public cloud.  But I think it’s reasonable to assume that a considerable number of buyers reject external servers of any sort as an option for CDP deployment, and that “on-premises” (including on-premises private clouds) is a reasonable term to describe their preferred configuration.

Monday, February 19, 2018

How Customer Data Platforms Help with Marketing Performance Measurement

John Wanamaker, patron saint of marketing measurement.
If you’ve been following my slow progress towards a set of screening questions for Customer Data Platforms, you may recall that “incremental attribution” was on the list. The original reason was that some of the systems I first identified as CDPs offered incremental attribution as their primary focus. Attribution also seemed like a specific enough feature that it could be meaningfully distinguished from marketing measurement in general, which nearly any CDP could support to some degree.

But as I gathered answers from the two dozen vendors who will be included the CDP Institute’s comparison report, I found that at best one or two provide the type of attribution I had in mind.  This wasn't enough to include in the screening list.  But there was an impressive variety of alternative answers to the question.  Those are worth a look.

- Marketing mix models.  This is the attribution approach I originally intended to cover. It gathers all the marketing touches that reach a customer, including email messages, Web site views, display ad impressions, search marketing headlines, and whatever else can be captured and tied to an individual. Statistical algorithms then look at customers who had a similar set of contacts except for one item and attribute any difference in performance to that.  In practice, this is much more complicated than it sounds because the system needs to deal with different levels of detail and intelligently combine cases that lack enough data to treat separately.  The result is an estimate of the average value generated by incremental spending in each channel. These results are sometimes combined with estimates created using different techniques to cover channels that can’t be tied to individuals, such as broadcast TV. The estimates are used to find the optimal budget allocation across all channels, a.k.a. the marketing mix.

- Next best action and bidding models.  These also estimate the impact of a specific marketing message on results, but work at the individual rather than channel levels. The system uses a history of marketing messages and results to predict the change in revenue (or other target behavior) that will result from sending a particular message to a particular individual. One typical use is deciding how much to bid for a display ad impression; another is to choose products or offers to make during an interaction. They differ from incremental attribution because they create separate predictions for each individual based on their history and the current context. Several CDP systems offer this type of analysis.  But it’s ultimately not different enough from other predictive analytics to treat it as a distinct specialty.

- First/last/fractional touch.  These methods use the individual-level data about marketing contacts and results, but apply fixed rules to allocate credit.  They are usually limited to online advertising channels.  The simplest rules are to attribute all results to either the first or last interaction with a buyer.  Fractional methods divide the credit among several touches but use predefined rules to do the allocation rather than weights derived from actual data.  These methods are widely regarded as inadequate but are by far the most commonly used because alternatives are so much more difficult.  Several CDPs offer these methods. 

- Campaign analysis. This looks at the impact of a particular marketing campaign on results. Again, the fundamental method is to compare performance of individuals who received a particular treatment with those who didn’t. But there’s usually more of an effort to ensure the treated and non-treated groups are comparable, either by setting up a/b test splits in advance or by analyzing results for different segments after the fact. The primary unit of analysis here is the campaign audience, not the specific individuals. The goal is usually to compare results for campaigns in the same channel, not to compare efforts across channels. This is a relatively simple type of analysis to deliver since it doesn’t required advanced statistics or predictive techniques. As a result, it’s fairly common or could be delivered by many systems even without the vendor creating special features to do it.

- Content performance analysis. This is very similar to campaign analysis except that audiences are defined as people who received a particular piece of content, which could be used across several campaigns. Again, there might be formal split tests or more casual comparison of results. Some implementation draw broader conclusions from the data by grouping content with similar characteristics such as product, message, or offer. But unless the groups are identified using artificial intelligence, even this doesn’t add much technical complexity.

- Journey analysis. Truth be told, no vendor in my survey described journey analysis as a type of incremental attribution. But it does come up in some discussions of marketing measurement and optimization. Like marketing mix and next best action methods, journey analysis examines individual-level interactions to find larger patterns and to identify optimal choices for reaching specified goals. But it looks much more closely at the sequence of events, which requires different technical approaches to deal with the higher resulting complexity.

Marketing measurement is one of the primary uses of Customer Data Platforms. Dropping attribution from the list of CDP screening questions shouldn't be interpreted to suggest it’s unimportant. It just means it’s that measurement  is too complicated to embed in a simple screening question. As with other important CDP features, buyers who want their CDP to support marketing measurement will need to define their specific needs in detail and then closely examine individual CDP vendors to see who can meet them.

Sunday, February 18, 2018

Will GDPR Hurt Customer Data Platforms and the Marketers Who Use Them?

Like an imminent hanging, the looming execution of the European Union’s General Data Protection Regulation (GDPR) has concentrated business leaders’ minds on their customer data. This has been a boon for Customer Data Platform vendors, who have been able to offer their systems as solutions to many GDPR requirements. But it raises some issues as well.

First the good news: CDPs are genuinely well suited to help with GDPR. They’re built to solve two of GDPR’s toughest technical challenges: connecting all internal sources of customer data and linking all data related to the same person. In particular, CDPs focus on first party (i.e., company-owned) personally identifiable information and use deterministic matching to ensure accurate linkages. Those are exactly what GDPR needs. Some CDP vendors have added GDPR-specific features such as consent gathering, usage tracking, and data review portals. But those are relatively easy once you’ve assembled and linked the underlying data.

GDPR is also good for CDPs in broader ways. Most obviously, it raises companies’ awareness of customer data management, which is the core CDP use case. It will also raise consumers' awareness of their data and their rights, which should lead to better quality customer information as consumers feel more confident that data they provide will be handled properly. (See this Accenture report that 75% of consumers are willing to share personal data if they can control how it’s used, or this PegaSystems survey in which 45% of EU consumers said they would erase their data from a company that sold or shared it with outsiders.)  Conversely, GDPR-induced constraints on acquiring external data should make a company’s own data that much more valuable.

Collection requirements for GDPR should also make it easier for companies to tailor the degree of personalization to individual preferences.  This Adobe study found that 28% of consumers are not comfortable sharing any information with brands and 26% say that too-creepy personalization is their biggest annoyance with brand content. These results suggest there’s a segment of privacy-focused consumers who would value a privacy-centric marketing approach. (That this approach would itself require sophisticated personalization technology is an irony we marketers can quietly keep to ourselves.)

So, what's not to like?  The downside to GDPR is that greater corporate interest in customer data means that marketers will not be left to manage it on their own.  Marketing departments have been the primary buyers of Customer Data Platforms because corporate IT often lacks the interest and skills needed to meet marketing needs.  GDPR and digital transformation don't give IT new resources but they do mean it will be more involved.  Indeed, this report from data governance vendor Erwin  found that responsibility for meeting data regulations is held by IT alone at 36% of companies and is shared between IT and all business units (not just marketing) at another 55%.  I’ve personally heard many recent stories about corporate IT buying CDPs.

Selling to IT departments isn’t a problem for CDP vendors. Their existing technology should work with little change.  At most, they'll need to retool their sales and marketing. But marketers may suffer more. Corporate IT will have its own priorities and marketing won’t be at the top of the list. For example, this report from master data management vendor Semarchy found that customer experience, service and loyalty applications take priority over sales and marketing applications. More broadly, studies like this one from ComputerWorld consistently show that IT departments prioritize productivity, security and compliance over customer experience and analytics. Putting IT and legal departments in charge of customer data is likely to mean a more conservative approach to how it's used than marketers would apply on their own.  This may prevent some problems but it's also likely to make marketers' jobs harder.

A greater IT role may also reverse the current trend of adding analytical and marketing applications to CDP data management functions. Marketers generally like those applications because it saves them the trouble of buying and integrating separate analytical and marketing systems. IT departments won’t use those features themselves and will probably be more interested in making sure CDP data can be shared by external applications from all departments. Similarly, IT buyers may favor CDP designs that are less tuned specifically to marketing needs and more open to multiple uses. This will favor some technical approaches over others.

The final result is likely to be clearer division of the CDP market into systems that focus on enterprise-wide customer data management and that give marketers integrated data, analytics, and customer engagement. If both types of vendors find enough buyers to survive, the expanded choice means that everyone wins. But the combined data, analytics and execution CDPs could be squeezed between data-only CDPs and the integrated applications of big marketing clouds. If there's not enough room left for them, marketers choices will be reduced.  Should that happen, GDPR will have done CDP vendors and marketers more harm than good.

Friday, February 02, 2018

Celebrus CDP Offers In-Memory Profiles

It’s almost ten years to the day since I first wrote about Celebrus, which then called itself speed-trap (a term that presumably has fewer negative connotations in the U.K. than the U.S.). Back then, they were an easy-to-deploy Web site script that captured detailed visitor behaviors. Today, they gather data from all sources, map it to a client-tailored version of a 100+ table data model, and expose the results to analytics and customer engagement systems as in-memory profiles.

Does that make them a Customer Data Platform? Well, Celebrus calls itself one – in fact, they were an early and enthusiastic adopter of the label. More important, they do what CDPs do: gather, unify, and share customer data. But Celebrus does differ in several ways from most CDP products:

- in-memory data. When Celebrus described their product to me, it sounded like they don’t keep a persistent copy of the detailed data they ingest. But after further discussion, I found they really meant they don’t keep it within those in-memory profiles. They can actually store as much detail as the client chooses and query it to extract information that hasn't been kept in memory.  The queries can run in real time if needed. That’s no different from most other CDPs, which nearly always need to extract and reformat the detailed data to make it available. I’m not sure why Celebrus presents themselves this way; it might be that they have traditionally partnered with companies like Teradata and SAS that themselves provided the data store, or that they partnered with firms like Pega, Salesforce, and Adobe that positioned themselves as the primary repository, or simply to avoid ruffling feathers in IT departments that didn't want another data warehouse or data lake.  In any case, don’t let this confuse you: Celebrus can indeed store all your detailed customer data and will expose whatever parts you need.

- standard data model. Many CDPs load source data without mapping it to a specific schema. This helps to reduce the time and cost of implementation. But mapping is needed later to extract the data in a usable form. In particular, any CDP needs to identify core bits of customer information such as name, address, and identifiers  that connect records related to the same person. Some CDPs do have elaborate data models, especially if they’re loading data from specific source systems or are tailored to a specific industry.  Celebrus does let users add custom fields and tables, so its standard data model doesn’t ultimately restrict what the system can store.

- real-time access.  The in-memory profiles allow external systems to call Celebrus for real-time tasks such as Web site personalization or bidding on impressions..  Celebrus also loads, transforms, and exposes its inputs in real time.  It isn't the only CDP to do this, but it's one of just a few..

Celebrus is also a bit outside the CDP mainstream in other ways. Their clients have been largely concentrated in financial services, while most CDPs have sold primarily to online and offline retailers. While most CDPs run as a cloud-based service, Celebrus supports cloud and on-premise deployments, which are preferred by many financial services companies.  Most CDPs are bought by marketing departments, but Celebrus is often purchased by customer experience, IT, analytics, and digital transformation teams and used for non-marketing applications such as fraud detection and system performance monitoring.

Other Celebrus features are found in some but not most CDPs, so they’re worth noting if they happen to be on your wish list. These include ability to scan for events and issue alerts; handling of offline as well as online identity data; and specialized functions to comply with the European Union’s GDPR privacy rules.

And Celebrus is fairly typical in limiting its focus to data assembly functions, without adding extensive analytics or customer engagement capabilities.  That's particularly common in CDPs that sell to large enterprises, which is  Celebrus' main market.  Similarly, Celebrus is typical in providing only deterministic matching functions to assemble customer data. 

So, yes, Celebrus is a Customer Data Platform.  But, like all CDPs, it has its own particular combination of capabilities that should be understood by buyers who hope to find a system that fits their needs.

As I already mentioned, Celebrus is sold mostly to large enterprises with complex needs.  Pricing reflects this, tending to be "in the six or seven figures" according the company and being based on input volume, types of connected systems, and license model (term or perpetual, SaaS, on-premise, or hybrid).  The company hasn’t released the number of clients but says it gathers data from "tens of thousands" of Web sites, apps, and other digital sources.  Celebrus has been owned since 2011 by D4T4 Solutions  (which looks like the word “data” if you use the right type face), a firm that provides data management services and analytics. 

Saturday, January 27, 2018

Collapse of Civilization Makes Marketers' Jobs Harder

Political situations come and go but trust is the foundation of civilization itself. So I was genuinely shaken to see a report from the Edelman PR agency that trust in U.S. institutions fell last year by a huge margin – 17% for the general public and 34% for the “informed public,” placing us dead last among 27 countries. All four measured institutions (business, media, government, and non-governmental organizations) took similar hits, although government fell the most.

You won’t be surprised to learn that concerns about fake news and social media are especially prominent. What’s less expected is that trust in traditional journalism actually increased in the U.S. The over-all decline in media trust resulted from falling confidence in news from search engines and social media. Similarly, world-wide trust increased in traditional authorities such as technical, academic, and business experts.  So there are rays of hope.

Digging deeper, the sharp fall in U.S. trust levels follows two years when levels were exceptionally high. The current U.S. trust level is roughly the same as the four reports before that. Maybe you shouldn't head for that doomsday cabin quite yet.

Still, other reports also show tremendous doubts about basic questions of truth. A Brand Intelligence study comparing brand attitudes of Democrats vs. Republicans found that eight of top 10 most polarizing brands were news outlets. World-wide, 59% of people told Edelman they were simply not sure what is true and just 36% felt the media were doing a good job of guarding information quality.

The social implications of all this are sadly obvious.  But this blog is about marketing. How can marketers adapt and thrive in a trust-challenged, politically-polarized world?
  • Protect privacy. Consumers can be remarkably cavalier in practice about protecting their data: this McAfee report found 41% don’t immediately change default passwords on new devices and 34% don’t limit access to their home network at all. But they are adamant that companies they do business with be more careful: Accenture studies have found that 92% of U.S. consumers feel it’s extremely important for companies to protect their personal information while 80% won’t do business with companies they don’t trust. Similarly, a Pega survey found that 45% of EU respondents said they would require companies to erase their data if they found it had been sold or shared with other companies. 
  • Personalize wisely. Accenture also found  that 44% of consumers are frustrated when companies don’t deliver relevant, personalized shopping experiences and 41% had switched companies due to lack of personalization or trust. So there’s clearly a price to be paid for not using the data you do collect. Similarly, an Oracle report found 50% of consumers would be attracted to offers based on personal data while just 29% would find them creepy.  In fact, consumers have a remarkably pragmatic attitude toward their data: 24[7] survey found their number one reason for sharing personal information is to receive discounts. This has two implications: ask consumers whether they want personalized messages (or any messages), and be sure the value of your personalization outweighs its inherent creepiness. 
  • Use trusted media. Consumers’ attitudes towards media in general, and social media in particular, are complicated. We’ve already noted that Edelman found growing distrust in online platforms. Other studies by Kantar and Sharethrough found the same. But consumers still spend most of their time on search engines and social media, which GlobalWebindex found remain by far the top research channels. Yet when it comes to building awareness, a different GlobalWebIndex report found that social ranked far behind search engines, TV, and display ads. Further muddying the waters, social and ecommerce companies (Facebook, Amazon, and eBay) topped NetBase’s list of most loved brands while Google ranked just 29th. But love isn’t the same as value: a LivePerson survey of 18-to-34 year olds – presumably the most enthusiastic social media users – found most would delete social apps from their phones before they'd give up practical apps for banking, ride-sharing and shopping. Similarly, The Verge found that Amazon and Google were significantly better liked and trusted than Facebook or Twitter. Taken together, this suggests that marketers need social channels for scale but can’t rely on them for credibility. Indeed, that’s precisely the conclusion of this Trusted Media Brands report about branded video.
  • Consider brand safety. The problems with social and display channels extend beyond general mistrust to actively offensive environments. GumGum found that 68% of brands knew their ads have been placed in objectionable environments, with fake news, divisive politics, and disasters heading the list. A Dun & Bradstreet report on programmatic B2B ads similarly found that 66% of brands have found brand safety increasingly important.  Ad fraud is also a major concern – the two are related because they both reflect brands’ loss of control over their ad placements. This Forrester report on mobile advertising found 69% of marketers felt at least 20% of their budgets were exposed to found mobile ad fraud. Yet all three studies found marketers were plunging ahead with just limited efforts at brand safety and fraud prevention. In a world where consumer trust is tenuous to begin with, this is a very high-stakes gamble.
  • Be careful about politics.  Edelman found that 64% of people want business CEOs to lead change rather than waiting for government to impose it. Sprout Social reported a similar finding:  65% of U.S. consumers felt brands should take a stand on social/political issues and 59% felt CEOs in particular should step in. But Euclid found the opposite: 78% said brands should avoid making political statements. Even more extreme, Bambu found just 2.3% of consumers said posting political content would make them more likely to buy from a salesperson while 34.9% said posting political content was a deal breaker, regardless of whether they agreed.  Still, the real danger is taking a position the customer dislikes: Bambu, Euclid and Sprout all found consumers are likely to boycott firms based on their positions. Sprout noted some compensating gain from people who agree but the net benefit is questionable at best: people are slightly more likely to praise a brand when they agree (28%) than criticize when they disagree (20%). But fewer will recommend it (35%) than warn friends and family (38%) and, most critically, fewer will purchase more (44%) than purchase less (53%).   In short, the data here are wildly conflicting: people want businesses to lead but they’ll punish behaviors they don’t like as often as they’ll reward choices they agree with. Of course, widely popular positions are still safe but many issues today have large numbers of people on both sides.  And remember it’s still possible to annoy everyone: Brand Intelligence found that Democrats, Independents, and Republicans all had Diet Pepsi (Kendall Jenner commercial, presumably) and Diet Mountain Dew (I don’t know why) on their most disliked lists. The ultimate result is probably that business leaders can justify being as active or inactive as they personally prefer.
So there you have it. Assuming we avoid complete social collapse, marketing in today’s polarized, anxiety-ridden world poses unprecedented challenges. Ironically, the loss of trust is happening at the precise moment when physical products are being replaced by trust-based services such as subscriptions and automated recommendations. The stakes couldn’t be higher.  Choose carefully and good luck.

Wednesday, January 24, 2018

Simple Questions to Screen Customer Data Platform Vendors

I’ve been working for months to find a way to help marketers understand the differences between Customer Data Platform vendors. After several trial balloons and with considerable help from industry friends, I recently published a set of criteria that I think will do the job. You can see the full explanation on the CDP Institute blog. But, since this blog has its own readership I figured I’d post the basics here as well.

The primary goal is give marketers a relatively easy way to decide which CDPs are likely to meet their needs. To do this I’ve come up with a a small list of features that relate directly to working with particular data sources and supporting particular applications. The theory is that marketers know what sources and applications they need to support, even if they're not experts in the fine points of CDP technology.

In other words, read these items as meaning: if you want your CDP to support [this data type or application] then it should have [this feature].

Obviously this list covers just a tiny fraction of all possible CDP features. It’s up to marketers to dig into the details of each system to determine how well it supports their specific needs.  We have detailed lists of CDP features in the Evaluation section of the CDP Institute Library.

The final list also includes a few features that are present in all CDPs (or, more precisely, in all systems that I consider a CDP – we can’t control what vendors say about themselves). These are presented since there’s still some confusion about how CDPs differ from other types of systems.

Now that the list is set, the next step is to research which features are actually present in which vendors and publish the results. That will take a while but when it’s done I’ll certainly announce it here.

Here’s the list:

Shared CDP Features: Every CDP does all of these. Non-CDPs may or may not.
  • Retain original detail. The system stores data with all the detail provided when it was loaded. This means all details associated with purchase transactions, promotion history, Web browsing logs, changes to personal data, etc. Inputs might be physically reformatted when they’re loaded into the CDP but can be reconstructed if needed.
  • Persistent data. The system retains the input data as long as the customer chooses. (This is implied by the previous item but is listed separately to simplify comparison with non-CDP systems.)
  • Individual detail. The system can access all detailed data associated with each person. (This is also implied by the first item but is a critical difference from systems that only store and access segment tags on customer records.)
  • Vendor-neutral access. All stored data can be exposed to any external system, not only components of the vendor’s own suite. Exposing particular items might require some set-up and access is not necessarily a real time query.
  • Manage Personally Identifiable Information (PII). The system manages Personally Identifiable Information such as name, address, email, and phone number. PII is subject to privacy and security regulations that vary based on data type, location, permissions, and other factors.
Differentiating CDP Features: A CDP doesn’t have to do any of these although many do some and some do many. These are divided into three subclasses: data management, analytics, and customer engagement.

Data Management. These are features that gather, assemble, and expose the CDP data.

     Base Features. These apply to all types of data.
  • API/query access. External systems can access CDP data via an API or standard query language such as SQL. It’s just barely acceptable for a CDP to not offer this function and instead provide access through data extracts. But API or query access is much preferred and usually available. API or query access often requires some intermediate configuration, reformatting, or indexing to expose items within the CDP’s primarily data store. Those are important details that buyers must explore separately.
  • Persistent ID. The system assigns each person an internal identifier and maintains it over time despite changes or multiple versions of other identifiers, such as email address or phone number. This allows the CDP to maintain individual history over time, even when source systems might discard old identifiers. CDPs that use a persistent ID applied outside of the system do not meet this requirement.
  • Deterministic match (a.k.a. “identity stitching”). The system can store multiple identifiers known to belong to the same person and link them to a shared ID (usually the persistent ID). This enables the system to connect identifiers indirectly: for example, if an email linked to an account is opened on a particular device, subsequent activity on that device can also be linked to the account.
  • Probabilistic match (a.k.a. “cross device match”). The system can apply statistical methods and rules to identify multiple devices used by the same person, such as computers, tablets, smart phones, and home appliances. While many CDPs rely on third party services for this sort of matching, this item refers only to matching done by the CDP itself.
     Unstructured and Semi-Structured Data. This refers to loading data from unstructured or semi-structured sources such as Web logs, social media comments, voice, video, or mages. These are typically managed with “big data” technologies such as Hadoop. Nearly all CDPs use some version of this technology but it’s only essential if clients have unstructured or semi-structured sources and/or very high data volumes. Some CDPs handle very high data volumes in structured databases such as Amazon Redshift.
  • JSON load. The system can accept and store data through JSON feeds without the user specifying in advance the specific attributes that will be included. Additional configuration may later be required to access this data. There are some alternatives to JSON that offer similar capabilities.
  • Schema-free data store. The system uses a data store that does not require advance specification of the elements to be stored. Examples include Hadoop, Cassanda, MongoDB, and Neo4J.
     Web Site. This refers to interactions with the company’s own Web site, whether on a desktop computer or mobile device.
  • Javascript tag. The system provides a Javascript tag that can be loaded into the client’s Web site and used to capture data about customer behaviors. Some CDP vendors provide full tag management systems but this is not a requirement for this item. This item does require that data captured by the Javascript tag can be associated with a customer record in the CDP database. This is usually done with a Web tracking cookie but sometimes through other methods.
  • Cookie management. The system can deploy and maintain Web browser cookies associated with the client’s own Web site. The cookies can be linked to customer records in the CDP database.
     Mobile Apps. This refers to interactions with mobile apps created by the company.
  • SDK load. The system offers a Software Development Kit (SDK) that can load data from a mobile app into the CDP database. It must be able to associate the data with individual customers in the CDP database. This is usually done through an app ID. Other SDK features such as message delivery are not a requirement for this item.
     Display Ads. This refers to interactions through display advertising networks, including social media networks.
  • Audience API. The system has an API that can send customer lists from the CDP to systems that will use them as advertising audiences. The receiving systems might be Data Management Platforms, Demand Side Platforms, advertising exchanges, social media publishers, or others. Ability to receive information back from the advertising systems is not a requirement for this item.
  • Cookie synch. The CDP can match its own cookie IDs with third party cookie IDs to allow the marketer to enrich profiles with external data or reach users through advertising networks.
     Offline. This refers to interactions managed through offline sources such as direct mail and retail stores, where the customer’s primary identifier is name and postal address.
  • Postal Address. The system can clean, standardize, verify, and otherwise work with postal addresses. This processing is reduces inconsistencies and makes matching more effective. Systems meet this requirement so long as the address processing is built into system process flows, even if they rely on third party software. Systems that send records to external systems in a batch process do not meet this requirement.
  • Name/Address Match. The system can find matches between different postal name/address records despite variations in spelling, missing data elements, and similar differences. As with postal processing, systems can meet this requirement with third party matching software so long as the software is embedded in their processing flows.
     Business to Business. This refers to companies that sell to other businesses rather than to consumers.
  • Account-level data. The system can maintain separate customer records for accounts (i.e., businesses) and for individuals within those accounts. This means account information is stored and updated separately from individual information. It also means that selections, campaigns, reports, analyses, and other system activities can combine data from both levels.
  • Lead to Account Match. The system can determine which individuals should be associated with which account records, using information such as company name, address, email domain, and telephone number. This excludes processing done by sending batch files to external vendors.
Analytics. These are applications that use the CDP data but don’t extend to selecting messages, which is the province of customer engagement.
  • Segmentation. The system lets non-technical users define customer segments and automatically send segment member information to external systems on a user-defined schedule. Ideally, all data would be available to use in the segment definitions and to include in the extract files. In practice, some configuration may be needed to expose particular elements. Systems meet this requirement regardless of whether segments are defined manually or discovered by automated processes such as cluster analysis.
  • Incremental attribution. The system has algorithms to estimate the incremental impact of different marketing activities on specified outcomes such as a purchase or conversion. Attribution is a specialized analytical process that relies on the unified customer data assembled by the CDP. Algorithms vary greatly. To qualify for this item, the algorithm must estimate the contribution of different marketing contacts on the final result. That is, fixed approaches such as “first touch” or “U-shaped distribution” are not included.
  • Automated predictive. The system can generate, deploy, and refresh predictive models without involvement of a technical user such as a data scientist or statistician. This usually employs some form of machine learning. There are many different types of automated predictive; systems meet this requirement if they have any of them.
 Engagement. This refers to applications that select messages for individual customers. It does not include content delivery, which is typically handled outside of the CDP.
  • Content selection. The system can select appropriate marketing or editorial content for individual customers in the current situation, based on the data it stores about them, other information, and user instructions. The instructions may employ fixed rules, predictive models, or a combination. Selections may be made as part of a batch process.
  • Multi-step campaigns. The system can select a series of marketing messages for individual customers over time, based on data and user instructions. The message sequence is defined in advance but may change or be terminated depending on customer behaviors as the sequence is executed.
  • Real-time interactions. The system can select appropriate marketing or editorial content for individual customers during a real-time interaction. This requires accepting input about the customer from a customer-facing system, finding that customer’s data within the CDP, selecting appropriate content, and sending the results back to the customer-facing system for delivery. The results might include the actual message or instructions that enable the customer-facing system to generate the message.

Friday, January 12, 2018

The Light Bulbs Have Ears: Why Listening Is Voice-Activated Devices' Most Important Skill

If one picture’s worth a thousand words, why is everyone rushing to replace graphical interfaces with voice-activated systems?

The question has an answer, which we’ll get to below. But even though the phrasing is a bit silly, it truly is worth asking. Anyone who’s ever tried to give written driving directions and quickly switched to drawing a map knows how hard it is to accurately describe any process in words. That’s why research like this study from Invoca shows consumers only want to engage with chatbots on simple tasks and quickly revert to speaking to a human for anything complicated. And it’s why human customer support agents are increasingly equipped with screen sharing tools that let them see what their customer is seeing instead of just talking about it.

Or to put it another way: imagine a voice-activated car that uses spoken commands to replace the steering wheel, gear shifter, gas and brake pedals. It’s a strong candidate for Worst Idea Ever. Speaking the required movements is much harder than making them movements directly.

By contrast, the idea of a self-driving car is hugely appealing. That car would also be voice-activated, in the sense that you get in and tell it where to go. The difference between the two scenarios isn’t vocal instructions or even the ability of the system to engage in a human-like conversation. Some people might like their car to engage in witty banter, but those with human friends would probably rather talk with them by phone or spend their ride quietly. A brisk “yes, ma'am” and confirmation that the car understood the instructions correctly should usually suffice.

What makes the self-driving car appealing isn’t that it can listen or speak, but that it can act autonomously. And what makes that autonomy possible is situational awareness – the car's ability to understand its surrounding environment, including its occupant’s intentions, and to respond appropriately.

The same is ultimately true of other voice-activated devices. If Alexa and her cousins could only do exactly what we told them, they’d be useful in limited situations – say, to turn on the kitchen lights when your hands are full of groceries. But their exciting potential is to do much more complicated things on their own, like ordering those groceries in the first place (and, eventually, coordinating with other devices to receive the grocery delivery, put the groceries in the right cabinets, prepare a delicious dinner, and clean the dishes).

This autonomy only happens if the devices really understand what we want and how to make it happen. Some of that understanding comes from artificial intelligence but the real limit is the what data the AI has available to process. So I’d argue that the most important skill of the voice-activated devices is really listening.  That’s how they collect the data they need to act appropriately. And the larger vision is for all these devices to pool the information they gather, allowing each device to do a better job by itself and in cooperation with the others.

Whether you want to live in a world where the walls, cars, refrigerators, thermostats, doorknobs, and light bulbs all have ears is debatable. But that’s where we’re headed, barring some improbable-but-not-impossible Black Swan event that changes everything. (Like, say, a devastating security flaw in nearly every microprocessor on the planet that goes undetected for years…wait, that just happened.)

Still, in the context of this blog, what really matters is how it all affects marketers. From that perspective, voice interfaces are highly problematic because they make advertising much harder: instead of passively lurking in the corners of a computer screen, appearing alongside search results,  larded into social media feeds, or popping up unbidden during TV shows, voice ads are either front-and-center or nowhere. Chances are consumers will be highly selective about which ads they agree to hear, so marketers will need to gain their permission through incentives such as discounts and coupons. Gaining the consent required by privacy regulations such as GDPR* will be good practice for this but it will soon seem like child’s play compared with what marketers need to do on voice devices. So one change is marketers will need a new set of skills around creating aural ads and convincing consumers to agree to listen to them.

A related skill will be making those ads effective. Remember that people are vastly better at processing visual images than words.  That’s why we have the 1000:1 word:picture cliché. That efficiency is why visual ads can be effective even if people don’t focus on them – they are still being registered on some level and people will pay closer attention to those that look interesting at a glance. Aural ads will transfer much less information per moment of attention and chances are most of that information will be forgotten more quickly. We’re in early days here and there’s much to learn. But if you can buy stock in a jingle-writing company, do it.

Another obvious change will be that the device vendors themselves have more control than ever over the messages their customers receive. This gatekeeper function is already at the center of the business models for Amazon, Facebook, Google, Apple and others (increasingly including non-net-neutral broadband operators). But as fewer channels become available to reach consumers and as the channels themselves deliver fewer messages per minute, the value of those messages will increase dramatically. Insofar as separately-controlled devices compete for consumer attention, the device vendors will have even more reason to deliver experiences that consumers find pleasant rather than annoying. Of course, as I’ve argued extensively elsewhere, “personal network effects” make it likely that most consumers will find themselves dealing primarily with a single vendor, so actual competition may be limited.**

The gatekeepers’ control over their customers’ experience means that marketers will increasingly need to sell to the gatekeepers to earn the opportunity to reach consumers. What’s different in a voice-driven world is the scarcity of contact opportunities, which means that gatekeepers don’t have enough inventory (e.g., ad impressions) to sell to all would-be buyers. This isn’t entirely new: even today, impressions for Web display, paid search, and paid social are auctioned to a considerable degree. But a huge reduction in inventory (and the impact of serving ads that lead consumers to opt out, assuming they really have that option) will make the gatekeepers much more selective and, no doubt, raise prices. The gatekeepers will also have more conflicts with potential advertisers as they sell more services of their own, adding yet another level of complexity and more opportunities for deal making.

Finally, let’s come back to the sensors themselves. Assuming that the gatekeepers are willing to share what they gather, marketers will finally be able to understand exactly how consumers are responding to their messages. It’s not just that they’ll be able to know exactly who saw which messages and what the subsequently purchased. The new systems will be collecting things like heart and respiration rates, creating the potential to measure immediate physical reaction to each advertisement.  It almost seems unnecessary to point out that listening devices will also capture conversations where consumers discuss specific products, not to mention their needs and intentions. The grand mysteries of marketing impact will suddenly be exposed with thoroughness, precision. and clarity. The change will be as revolutionary as X-rays, ultra sounds, and CAT scans becoming available to doctors. As with radiology in medicine, these new information streams will require new skills that form the basis of entirely new specialties.

In short, voice-activated devices will change the world in ways that have nothing to do with the interaction skills of chatbots or ease of placing orders on Alexa. Marketers' jobs will change radically, demanding new skills and creating new power relationships. Visual devices won’t really go away – people are too good at image processing to waste the opportunity. But presenting information to consumers will ultimately be less important than gathering information about them, something that will use all the sensors that devices can deploy.

Who knew the sentient housewares in Disney's Brave Little Toaster were really a product roadmap?

*General Data Protection Regulation.  Have we reached the stage yet where I no longer need to spell it out?

** The essence of personal network effects is the value of pooling data to create the most complete information about each customer. In many ways, situational awareness is another way of describing the same thing.

Tuesday, January 02, 2018

What's Next for Customer Data Platforms? New Report Offers Some Clues.

The Customer Data Platform Institute released its semi-annual Industry Update today. (Download it here).  It’s the third edition of this report, which means we now can look at trends over time. The two dozen vendors in the original report have grown about 25% when measured by employee counts in LinkedIn, which is certainly healthy although not the sort of hyper growth expected from an early stage industry. On the other hand, the report has added two dozen more vendors, which means the measured industry size has doubled. Total employee counts have doubled too. Since many of the new vendors were outside the U.S., LinkedIn probably misses a good portion of their employees, meaning actual growth was higher still.

The tricky thing about this report is that the added vendors aren’t necessarily new companies. Only half were founded in 2014 or later, which might mean they’ve just launched their products after several years of development. The rest are older. Some of these have always been CDPs but just recently came to our attention. This is especially true of companies from outside the U.S. But most of the older firms started as something else and reinvented themselves as CDPs, either through product enhancements or simply by adopting the CDP label.

Ultimately it’s up to the report author (that would be me) to decide which firms qualify for inclusion.   I’ve done my best to list only products that actually meet the CDP definition.*   But I do  give the benefit of the doubt to companies that adopted the label. After all, there’s some value in letting the market itself decide what’s included in the category.

What’s most striking about the newly-listed firms is they are much more weighted towards customer engagement systems than the original set of vendors. Of the original two dozen vendors, eleven focused primarily on building the CDP database, while another six combined database building with analytics such as attribution or segmentation. Only the remaining seven offered customer engagement functions such as personalization, message selection, or campaign management. That’s 29%.**

By contrast, 18 of the 28 added vendors offer customer engagement – that’s 64%. It’s a huge switch. The added firms aren’t noticeably younger than the original vendors, so this doesn’t mean there’s a new generation of engagement-oriented CDPs crowding out older, data-oriented systems. But it does mean that more engagement-oriented firms are identifying themselves as CDPs and adding CDP features as needed to support their positioning. So I think we can legitimately view this as validation that CDPs offer something that marketers recognize they need.

What we don’t know is whether engagement-oriented CDPs will ultimately come to dominate the industry. Certainly they occupy a growing share. But the data- and analysis-oriented firms still account for more than half of the listed vendors (52%) and even higher proportions of employees (57%), new funding (61%) and total funding (74%).  So it’s far from clear that the majority of marketers will pick a CDP that includes engagement functions.

So far, my general observation has been that engagement-oriented CDPs appeal more to mid-size firms while data and analysis oriented CDPs appeal most to large enterprises. I think the reason is that large enterprises already have good engagement systems or prefer to buy such systems separately. Smaller firms are more likely to want to replace their engagement systems at the same time they add a CDP and want to tie the CDP directly to profit-generating engagement functions. Smaller firms are also more sensitive to integration costs, although those should be fairly small when CDPs are concerned.

There’s nothing in the report to support or refute this view, since it doesn’t tell us anything about the numbers or sizes of CDP clients. But assuming it’s correct, we can expect engagement-oriented vendors to increase their share as more mid-size companies buy CDPs. We can also expect engagement-oriented systems to be more common outside the U.S., where companies are generally smaller. For what it’s worth, the report does confirm that’s already the case.

If the market does move towards engagement-oriented systems, will the current data and analytics CDPs add those features? That’s another unknown. There’s already been some movement: four of the original eleven data-only CDPs have added analytics features over the past year.  But it’s a much bigger jump to add customer engagement features, and sophisticated clients won’t accept a stripped-down engagement system. We might see some acquisitions if the large data and analytics vendors want to add those features quickly. But those firms must also be careful about competing with the engagement vendors they currently connect with. Nor are they necessarily eager to lose their differentiation from the big marketing clouds.  Nor is there much attraction to entering the most crowded segment of the market with a me-too product.

So most data and analytics vendors may well limit their themselves to their current scope and invest instead in improving their data and analytics functions. That will limit them to the upper end of the market but it's where they sell now and offers plenty of room for growth.  Certainly there’s a great deal of room for improved machine learning, attribution, scalability, speed, and automated data management. If I had to bet, I’d expect most data and analytics vendors to focus on those areas.

But I don’t have to bet and neither do you. So we’ll just wait to see what comes next. It will surely be interesting.

*CDP is defined as a marketer-controlled system that builds a persistent, unified customer database that is accessible by other systems.

**To further clarify, customer engagement systems select messages for individuals or segments.  Analytics systems may create segments but don't decide which messages go to which segment.  And execution systems, such as email engines, Web content management, or mobile app platforms, deliver the selected messages.