Where did my cookies go??

Yoni Epstein

Yoni Epstein

Yoni Epstein is a senior product manager at taboola, overseeing taboola user data products. Yoni's mission is to put taboola's massive data assets to work for the benefit of its advertisers success and he is responsible for product areas such as audience targeting, identity and privacy. Yoni is passionate about connecting data, technology and business together for the purpose of building great products.

Yoni Epstein | 25 Jan 2019 | Web Development

Tags: cookies, cross_device, data, identity, identity_graph, tracking, user_data

If you are using web cookies to operate your online business you probably know already that just like in real life, cookies do not last long. This is an especially known fact to whoever uses online cookies to store unique user IDs. Most online marketing companies rely on cookies for that purpose, but when cookies disappear – it makes it harder for them get persistent user data.

Interested to know for how long does a cookie really last? in this post I’ll try to provide some answers.

Who is eating web cookies?

Cookies can disappear for various reasons, such as:

  • Clearing the browser historical data by the user
  • Setting the browser to reject third-party cookies
  • Using tools that clean up your device and free up storage space
  • Use of VPNs, Ad Blockers and more.

One very common reason cookies disappear is the use of private browsing modes such as Incognito in chrome or private window in Safari. In most browsers browsing in a private browsing mode will cause the cookie to be deleted when the session ends – effectively turning cookies into ‘session cookies’. In other words, those cookies last for only a few hours. The next time a user will browse in private browsing mode he or she will be assigned a new cookie.

Internet users going undercover

Data we’ve gathered from billions of events around the world suggests that around 17% of global internet traffic comes from users that are only seen for one session and never seen again – much of this phenomena can be explained by usage of private browsing modes. This number seems to be inline with other independent research. This number varies a bit between countries. In UK only 14% of traffic comes from cookies that are a session long while in Germany it is 24%!

However, is private browsing mode the main reason for cookie churn?

Whose party is this cookie?!

An interesting way to look at cookie churn is by drawing its survival curve and calculating its half life time. This is a common method used in nuclear physics to describe radioactive decay. Half lifetime of a cookie represents the median lifetime of a cookie – 50% of all cookies have shorter lifespan and 50% have longer lifespan. We conducted a research in taboola measuring survival rate of third-party cookies and first-party cookies.

Third-party and first-party cookie are terms often used in ambiguous ways, but in general they are regarded as follows:

  • Third-party cookie – a cookie placed by a domain the user has not interacted directly with
  • First-party cookie – a cookie placed by a domain the user has interacted directly with

The survival of a cookie

In the below analysis we looked only at cookies that had a lifespan of more than 1 day –  meaning we excluded sessions cookies, including private browsing modes. Our research suggests that third-party cookies are very short lived with a half life time of only 14 days! First-party cookies, on the other hand, are much more persistent; 30 days after they are created almost three-quarters still survive!   

The two cookie problems

The fact cookies don’t last long is not their only shortcoming. The second problem with cookies is that they represent only a shell of the user.

Most people today are using at least two devices, usually mobile phone and laptop or desktop. On each device there is a different cookie with a different user ID. Not only that, on Safari browser for example, that blocks third-party cookie by default,  you might find dozens of first-party cookies with different user IDs all belonging to one person!

Cookies give a fragmented representation of the user digital interactions.

A new way to think about cookies

For companies that rely mainly on web cookies to track users, I suggest the following framework to think about the persistency of their user data: longevity vs. completeness.

  • Longevity is how long a user identifier lasts
  • Completeness is how much of the user interactions a user identifier captures

Logged-in users using email for log-in, for example, offers high longevity, since a user’s email lasts for years, and also a high level of completeness as the user logs in across all devices with the same email. On the other hand session cookies have very low longevity  since they only lasts for few hours, and they are usually only relevant for one site so they are low on completeness.

First-party cookies, as we’ve seen in the chart above, have higher longevity than third-party cookies but they do not allow the cross domain tracking third-party cookies allow, so they get lower score on completeness. Lastly there are also mobile ad IDs in the app space. They are hardly ever changed by the user so their longevity is usually the lifetime of the device, however they only capture the user interactions in apps of a single device so they are not complete like an email.

What about YOUR cookies?

Cookies pose a challenge to any company that wishes to achieve persistent user data in the web space. There are various strategies to deal with this challenge which I will save discussing for a future post. However the first step in facing this challenge is understanding its magnitude.

so… do you know how long your cookies last??