Making the EU Transparency Register more functional

The EU Transparency Register is becoming an important part of Brussels life, with access to officials increasingly conditioned on adhesion. As a result, the Register now counts more than 8500 entries.

While the Commission provides a basic search capability, it is rather ‘clunky’, does not facilitate comparison of entries, and has no deep linking to promote discussion of the information present. As the Commission makes the raw data freely available for reuse, I decided to have a go at addressing these deficiencies.

The result can be seen on We can now for example:

  • compare the social partners;
  • look at the relative lobby firepower of GAFA  (Google, Apple, Facebook, Amazon), or GAFAM (with Microsoft too)
  • assess the larger Brussels lobbying consultants;
  • or see a selection of NGOs.

What the new functionality also illustrates is that many registrants do not understand the questions they were asked to complete. Dial up the budget filter and you can find a number of organisations that declared their total budget, not that allocated to influencing the EU policy making process. An even larger number seem to misunderstand the question about Full Time Equivalents (FTEs) devoted to lobbying.

If you have feedback, feel free to add a comment below. (Source code)


The case for repealing the EU’s telecommunications legislation

(This article builds on my previous post about telecoms deregulation, and was first published in Info, January 2015)


The telecoms liberalisation process is a true European success. It eliminated borders for traditional wireline telephony, drove rapid roll out of broadband, and the sophisticated incorporation of antitrust concepts is a genuine technocratic triumph (one need only look at the cast-in-law definitions that still haunt the FCC to feel justifiably proud!). But the only true mark of success is whether policy makers feel they can repeal the legislation.

This article argues that that this is now possible, and on the basis of three arguments:

  1. The telecoms market is ready to become sustainably competitive. There are now hundreds of market players in Europe, and market shares are reasonably widely dispersed. Consolidation is expected, and will lead to competitors strong enough to operate in a market policed by competition law.
  2. Many telecom market dynamics are driven by players outside of the scope of the telecoms rules, such that regulation is now applied unevenly across the value chain. Internet technologies are now some of the most powerful substitutes for electronic communications services, even for seemingly intractable problems like roaming. Contracts for audiovisual content can determine the success of new services every bit as much as wholesale telecoms rules.
  3. Regulation is being capitalised into the political economy, distorting incentives to the detriment of European consumers. This is most clear in the area of spectrum allocation.

Sustainable competition

The most effective fixed line competition comes from cable TV. Spectrum policy matters because it is the best means to ensure additional infrastructure competition not reliant upon regulation.

The EU telecoms framework is premised on ensuring effective wholesale regulation, so that retail competition makes additional rules redundant. The ‘ladder of investment’ may have stalled, but it has done its job. In most member states, market shares are reasonably widely distributed among several players; competing brands are well established and some product differentiation is evident. The European Commission recently removed more retail markets from its Relevant Markets recommendation.

Nonetheless, many challengers remain dependent on the availability of wholesale products as the basis for their competing retail solutions. The question is whether this regulation continues to require oversight by a dedicated agency, or whether competition law is sufficient. Historically, entrants argued that competition law remedies were too slow to deliver them respite, but this needs to be re-evaluated.

Several observations apply:

  • Consolidation is necessarily bringing competition authorities more firmly into the telecoms world. Cross-border consolidation should be encouraged, but competition authorities are right to be sceptical about incumbents simply buying out their domestic competitors. Larger players will have deeper pockets, such that case-by-case assessments need not bankrupt a complainee. (And injunctions exist where such a risk does exist.)
  • Incumbent-led consolidation brings in the possibility of a new market dynamic: if incumbent A seeks to give a subsidiary of incumbent B a tough time in A’s home market, B may well be in a position to threaten retaliation against A’s subsidiary in its local market. Both national regulators might consider their incumbents’ behaviour abusive, while a holistic view might welcome the new market dynamics.
  • The presence of regulation is a source of uncertainty for challengers too. Where margins are low, consumer interests are being served and there should be market exit, even if regulators are uncomfortable. More importantly, where margins are high, investment and entry should be expected, but the presence of the regulatory framework seems to have made it easier for policy makers quickly to confiscate rents before market forces had been brought to bear. This is particularly clear with the waves of retail regulation in the mobile market.
  • Two of the four remaining designated ‘relevant markets’ exist primarily because of the calling-party-pays interconnection pricing model. Other models are possible, and jurisdictions where these models prevail simply do not have these artificial bottlenecks in their markets, and regulatory intervention may in itself be barrier to a switchover.

In other words, if the European Commission announced that it would bring forward proposals next year to repeal the telecoms package within, say, a period of five years, this in itself would focus corporate attention on the sort of consolidation necessary for a competition law regulated market.

Extra-market influences on the telecoms sector


Competition drives the strongest consumer benefits when competitors have dramatically different business models and cost bases.

In 1998, market opening enabled consumers to exercise a choice for a specific service – voice telephony. Now the telecoms sector provides consumers with a general purpose technology – the internet – and services such as Whatsapp and Skype are disrupting the rents on SMS and international telephony far more aggressively than anything happening within the telco sector. Looking to the future, another emerging internet phenomena, the ‘sharing economy’, is enabling FON to become a competitor for expensive mobile data roaming.

As a result, vast swathes of traditional telco producer surplus have turned into consumer surplus. The declines in incumbent profitability reflect this huge transfer and deserve celebration, not hand-wringing. (We need to consider better metrics than GDP to capture these sorts of policy success.)

Interconnection of telcos and online players

More recently a debate has emerged around the connections between the major ISPs with the largest online content services. If regulators were once mediating between David and Goliath within the telecoms sector, this new debate is between evenly matched giants. While some telcos have asked regulators to intervene (a policy reversal that is worrying in its own right), this is definitely not the place for telecoms regulators.

The most compelling reason for not jumping in is that the commercial links do not begin and end with the connecting of networks, as was the case between horizontally competing telcos. Google is probably the most targeted by telcos for ‘swamping’ their networks (with traffic requested by telcos’ subscribers!). But the online giant arguably has several relationships with telcos that are far more commercially important than the arrangements it has on the network interconnection level. For example:

  1. Advertising:
    1. Telcos purchase a lot of search advertising from Google to expand their business.
    2. Many telcos receive a substantial share of advertising revenues from Google as they embed a search box and advertisements on their consumer facing portals.
    3. Some telcos resell Google advertisements and other cloud products to their corporate customers.
  2. Marketing: use of the YouTube brand is a powerful way to demonstrate the benefits of upgrading a broadband connection.
  3. Android: what would mobile data subscription numbers look like if Apple and Blackberry dominated the market?

Secondly, telecoms regulators will struggle to give justice to the difference in risk profiles of telcos relative to online players. All businesses operate in environments of risk, but the parameters of that risk can vary a lot. The telecoms world has been conditioned to think of it in terms of the payback periods on massive investments in physical infrastructure. These are unquestionably substantial risks, but in the digital economy they must share the rhetorical limelight with others.

Online players do make investments in infrastructure, notably in data centres, but these are not substantial by comparison. Instead, online players take R&D risks, not dissimilar to pharmaceutical companies or film studios. These range in financial magnitude, but the fact is that huge upfront research costs are well able to result in a complete consumer ‘flop’ that simply needs to be written off. There is no simple way to compare these risks with those of a telco and sector regulators should not seek to do so.

Net neutrality

While extending regulation to the online sector makes no sense, the continued presence of telecoms regulation is a distortion. When telcos sit down with online players, bargaining power is fundamentally affected by the number of outside options each side has. The fact that the number of telcos in the market is directly influenced by regulation means there can be no sense of a level negotiating field.

Some will argue that leaving giants to negotiate commercially would jeopardise ‘net neutrality’ and, with that, consumer interests. Advocates argue that the internet has a tradition of being neutral, but I believe its more important tradition is that it has been lightly regulated.  Mandating net neutrality is not light touch and proponents should be careful about the precedent for intervention that they create.

The best argument for intervention is that today’s big online players can afford to pay for access, and in doing so can create bigger barriers to competition. But it is surely a confusion to address this risk by regulating telcos rather than instructing competition authorities to continue being vigilant about abusive behaviour by online players.

Rights holders and content

Regulated wholesale products are a prerequisite for some to offer a triple/quadruple play to consumers, but a deal with an incumbent is only a part of the picture. Often more important is the ability to secure audiovisual content.

The complexity of relationships between the content makers, rights managers and rights users are legion. Sometimes long term contracts continue to exclude new players, while in other cases exclusivity has been newly sought by an internet provider or an online service provider. A ‘must-have’ broadcaster can also exercise a hold-up power akin to a patent holder in a standards setting process.

Content right holders can often look out of touch with consumers, but competition authorities have only occasionally found reason to intervene. Telecoms regulators may feel that their efforts need to be complemented by additional interventions, but it would generally be wrong to do so.

Distorted political-economic incentives

Regulators’ decisions have changed the face of the market since 1998, to the benefit of consumers. But regulation has also – inevitably – embedded itself in its stakeholders’ incentives, and that is now a threat to consumers’ long-term interests.

The upswing in pro-regulatory rhetoric from several European incumbents is worrying in its own right, and I have made the case above that regulation is no longer a one-way bet for the strongest challengers. Here I look at some additional ‘stakeholders’.

Mobile operators

The defining issue in the mobile market has been roaming, or rather the absence of Single Market behaviour by wireless operators. My contention is that a rush to (roaming) regulation was made possible because it could be built upon the existing rules. Meanwhile, the Commission ignored the root causes of the problem:

  • Demand: why did mobile operators never perceive substantial demand for pan-European services, especially from Small and Medium sized Enterprises? I suggest the answer lies primarily in, for example, the failure of services sector liberalisation. With more cross-border business, a market with three or more mobile operators would surely have responded with Single Market services.
  • Supply: Single Market services needed someone to go first, and the candidate for that was Vodafone. Yet when they acquired Mannesman, the European Commission’s DG Competition bizarrely decided to limit the scope for ‘one Europe’ services. Even so, the merger precipitated a wave of pan-European industry alliances which might in turn have led to the consolidation now recognised as essential. And yet, with the industry in positive flux, the publication of socially regressive, price setting, roaming legislation quickly overwhelmed other market dynamics.

Effective implementation of the Services Directive remains one of the new Commission’s great structural reform challenges. Its impact will be felt widely of course, and that includes in the demands on mobile operators. This is something that the Vice-President for the Digital Single Market will be able to pick up.

Technology could also come into the picture. The emergence of ‘soft SIMs’, for example, offers consumers new opportunities to switch operators when they are travelling. It is a concern that the operators appear to be using their trade association to discuss a common response, but that could be an issue competition authorities are best placed to police.

Incumbent spectrum holders

Spectrum is key to long term infrastructure competition, and the Commission has rightly pushed for more rational management of this truly scarce resource. Leaving aside whether it needs EU or national level administration, the real issue is ensuring that enough spectrum is made available for the internet, as the key general purpose technology of the 21st century.

This is a debate that cuts across the interests of the ministries of defence, the interior, culture and others. Are the representatives of the communications sector in government well armed to win for the telecoms sector?

An unwillingness among policy makers to consider pricing mechanisms, even in private inter-ministerial discussions, means that the debate has an additional level of subjectivity. That is where the presence of regulators could be problematic. While infrastructure competition is strategically vital, ‘hoarders’ within government feel little pressure to cede spectrum to the public interest because regulation is protecting short-term consumer interests.

In other words, the opportunity from repealing the telco rules is that it will enable champions of the sector within government to make a stronger case for refarming.


Structural reform applies to government too. Regulators could no doubt fine tune the application of today’s rules for decades to come, but the best ones’ have kept their eye on the original goal and have strived to put themselves out of work, knowing that demonstrating success is the best way to long-term employability. The Dutch seem to have gone furthest in providing officials with a clear perspective of life-after-telecoms, and their path could be emulated in other member states.

Job done

The telecoms market has transformed beyond recognition. Consumers have more choices, and services are better. Regulation created the conditions for internet access services to take off, and online services now provide some of the most effective competition to traditional telco services, causing huge shifts in economic surplus to consumers.

The telco market today is inseparable from the digital economy being built upon its fibre and copper. The commercial links along the value chain are multiple, and ever changing. They influence the industry every bit as much as sector regulation, but the issues that do emerge in this broader context fall far outside the ambit of the regulators and, indeed, are too fast moving to justify other forms of ex ante regulation.

Competition law was always the long term goal of telecoms liberalisation, and its practitioners will be hugely involved in the predicted wave of mergers. Repealing the telco rules would accelerate the much desired market consolidation and, as the dust settles, competition authorities can be left to police the resultant playing field.

Asia and the Internet of Things

The Chinese Internet economy has grown to staggering proportions, but it is still hard to see from within Europe. Yes, our phones, tablets and computers are stuffed full with Chinese (and other Asian countries’) components, but the services we use still seem to be dominated by Silicon Valley. In the US, although consumers now enjoy the option of buying from Alibaba, it remains to be seen whether the company will be able to make a dent in the sales of Walmart and Amazon.

My guess is that Chinese companies will need to await a paradigm change to get a real foothold. The Internet of Things is, I suspect, that opportunity.

At the recommendation of a friend, Ray, I have recently started listening to the excellent Exponent podcast about the technology industry. I’ve be working through the back catalogue and was listening this week to the programme about Xiaomi. This young Beijing-based, smartphone company has attracted a lot attention for the rapidity with which it has amassed a devoted following, somewhat akin to Apple.

XiaomiThe presenters, Ben and James, discussed at length Xiaomi’s business model which starts with beautiful devices (like Apple) but sold at rock-bottom prices, meaning that services will be vital to achieving real revenue (like Google). And that service layer could well constitute a smartphone-based unification of connected consumer devices.  Although Xiaomi today only makes phones, the presenters thoughtfully speculate that Xiaomi is well placed to extend its brand rapidly into connected consumer electronics, as millions of young Chinese fans move out from their parents’ houses to set up their own homes. By contrast, Apple only knows computers, and Google doesn’t have a compelling device strategy, so Xiaomi also has the potential to conquer new markets with what it learns in China. More on the original Stratechery blog.

Not all of the evidence is yet on the table, but we in Europe need to plan for Asia being a far greater force in the technology world. And the Internet of Things is a more than big enough change to open the door for new giants. Of course best of all would be to create an innovation climate that permits European players to scale up rapidly in a truly single market.

(The Xiaomi podcast then goes on to discuss patents, including the powerful observation that in tech / ‘winner-takes-all’ markets there is already often enough incentive to innovate fast, and patents don’t play a societally useful additional role. Definitely worth a listen!)

A ‘Digital’ narrative for ‘structural reforms’

Gutenberg’s printing press took over a century before its true impact began to be felt. At a hearing yesterday organised by the Industry Committee of the European Parliament, my message was that, despite upheaval in the entertainment and news industries, the true contours of the ‘Digital Economy’ are still to emerge. And, as we think about future policy my advice to policy makers was: “think less about digital, and more about the economy”.

The Internet of Things will be a real game-change, as it takes ‘communication’ beyond the realm of humans into a vast array of items and sensors. That said, we can speculate about its impact, but we’ll probably be wrong.

A more immediate example of the scale of transformation technology enables can be seen in the “sharing economy”, and that is where I focused my contribution. There are many definitions of the sharing economy, but for me it means the aggregation (through technology) of micro-supplies to deliver a service seen as comparable to those provided by the firm-centric, industrial model.

This covers pure sharing services, but also includes commercial couchsurfing (AirBnB) and ride sharing (Uber), and characterises the shifts underway in Germany in the name of Energiewende. All are changing the competitive landscape for sectors that may never have imagined they would be affected by ‘digital’ and creating new choices for consumers.

The conclusion from such an analysis is that while sharing economy services are affected by the traditional elements of digital policy, such as privacy, net neutrality and copyright, the key breaks on potential European champions lie in other parts of our regulatory framework. This can mean sectoral rules (e.g. of taxis), but also more general regulation of small businesses and freelance workers.

There is no silver bullet for unleashing growth, but there is a political opportunity for Brussels policy makers. ‘Structural reforms’, which have hitherto been perceived by citizens as painful austerity, can genuinely be presented as preparing the continent for the digital era.

If Europe is play host to the next generation of global internet champions, we need policy to let out start ups grow quickly in whatever sector they choose to disrupt. You can find signs of this, for example, in the way Portugal has opened up its tourism industry. But the structural reforms agenda does not just apply to ‘programme member states’ alone, and the ongoing reluctance of other member states to open markets (such as the unenthusiastic implementation of the Services Directive) is now the real digital agenda.

Data mining: recommendation systems

During my recent sabbatical I had the privilege of joining some of the lectures from the Masters in Artificial Intelligence at the University of Leuven. They say that ‘you haven’t understood something until you teach it’, so I thought I would try to share, via this blog, some of what I learnt in the ‘Data Mining’ course.

Netflix recommendation algorithm

The first of these occasional posts is about recommendation systems, which is timely as I’ve recently started to enjoy the results of one of the most famous such systems, Netflix. In 2006 Netflix created a $1m prize for entrants to devise an algorithm that predicted recommendations for users based on a training data set of 100,480,507 ratings that 480,189 users gave to 17,770 movies (wikipedia). Competitors used the data to devise an algorithm, and this was tested against an additional set of data retained by Netflix. The winning entrant improved accuracy over Netflix’s existing model by 10%, and presumably has been the cause of much ‘consumer surplus’ ever since.

There are two basic ways to approach the data:

  • Content based filtering: Find patterns in an individual user’s data. For example, if someone reports that they like a series of films all of the ‘romantic comedy’ genre, then you might recommend more of the same.
  • Collaborative filtering: Here we try to find correlations between users that have rated 1 or more of the same items. The calculation returns a value between -1 and 1, where -1 means opposite preferences, 1 means identical preferences and 0 indicates no correlation between the two. A recommendation is made from the (possibly weighted) aggregate scores given by the most similar users to items that the original user has not seen (and this might be augmented by looking the disliked films from anti-correlated users).

Content based filtering excels when an individual has unique tastes, as the ‘similarity’ test in the collaborative approach will find few matches.

Content filters also lead to recommendations that are easy to explain to users (“because you liked several ‘romantic comedies’ and this is another one”), while the results of the collaborative filter cannot readily be interpreted. On the other hand, you have to have the right meta information to be able to find the matches, whereas collaborative filters can work without this extra info.

One criticism of online systems is the so-called ‘filter bubble‘, and the parallel concern here could be that one ends being recommended very similar films. In practise, neither system is terrible, and neither perfect – a content filter will for example recommend a lot of ‘romantic comedies’ but it could include items, for example, in other languages. An aggressive collaborative filter might lead to a narrow set of suggestions, but aggregating the preferences of ‘broadly similar’ users could throw up some wonderful apparent serendipity.

Is it time to repeal telco regulation?

(A more developed version of this post is available on Emerald publishing’s Info periodical (paywall)).

The telecoms liberalisation process is a true EU success. It has eliminated borders for fixed line telephony, driven rapid roll out of broadband, and the sophisticated incorporation of antitrust concepts is a genuine technocratic triumph (one need only look at the cast-in-law definitions that still haunt the FCC to feel justifiably proud!).  But the only true measure of success is when the rules are no longer needed and can be repealed!

Martin Bangemann - architect of an EU success story
Martin Bangemann – architect of an EU success story

Later this year, a new European Commission President will make digital a top priority.  As this blog argues more generally, their plan must mark a switch from supply side thinking (e.g. broadband deployment) to promoting adoption and use of information technology across the entire private and public sector.  There is no more elegant a way of emphasising the raised ambitions than to start the process of repealing the telecoms rules that are too often currently considered to be the digital policy.

The case is not simply political; the rules have developed two fundamental flaws:

  1. the framework promotes horizontal competition within the telecoms sector; yet the impact of vertically related players is now arguably the dominant industry driver; and
  2. regulation is becoming ‘capitalised’ into a political economy, distorting incentives and not serving European consumers, notably with respect to spectrum allocation.

1. From horizontal to vertical

The growing importance of vertical relationships can be seen at both the retail and wholesale level.


Regulators have successfully ensured retail competition and consumer prices have trended steadily downwards.  But it is services like Whatsapp that are truly disrupting the rents on SMS, just as Skype turned international telephony on its head a decade earlier.  And players like Fon have the potential to change the face of mobile data by leveraging the peer economy.


The heart of the regulator’s task was to set prices for electronic communications network providers to buy wholesale capacity from incumbent operators.  That remains a necessity for some challengers (discussed further below), but the defining wholesale relationships now are those of the major ISPs with the largest online content services.  If regulators were once mediating between David and Goliath, the new battle is between giants.

While some of the incumbents have called for regulation of these relationships (a reversal of rhetoric that is worrying in its own right), the fact is that the commercial links do not begin and end with the connecting of networks, as it does between horizontally competing telcos.  There may be marketing, content, advertising, software or other links that are much more significant in practise.

Whatever the breadth of the relationship between a telco and an online player, bargaining power is fundamentally affected by the number of outside options the other party has, and on the telco side of this equation regulation is a major determinant of market structure.

In other words, regulators cannot hope to have a proper overview of these vertical relationships, but their horizontal interventions are distorting the commercial bargaining power of the telcos.

2. Distorted political-economic incentives

Regulators’ decisions have changed the face of the market since 1998, to the benefit of consumers.  But regulation has also – inevitably – embedded itself in all of the stakeholders’ incentives, and that is now a threat to consumers’ long term interests.  The new, pro-regulatory rhetoric of many incumbents is a red flag in its own right, but there are issues for most other stakeholders too.


The most effective competition generally came from e.g. cable TV players which needed little from regulated incumbents.  Spectrum is the best way to help establish more competitors not dependent upon ex ante regulation.

Nonetheless, other challengers are clearly dependent on timely regulation, but as the market consolidates, pockets are deeper and ex post scrutiny is more feasible.  More importantly:

  • can challengers make the sort of long term investment decisions that policy makers want with an unknown a big as regulation to take into account?
  • If the Commission announced the end game for telco regulation, the sector would still have half a decade to prepare, including more cross-border consolidation.

Mobile operators

The defining issue in the mobile market has been roaming, or rather the Single Market.  My contention is that a rush to (roaming) regulation was made too easy by the presence of the 2002 regulatory framework that could be built upon, and the macro policy picture ignored:

  • Demand: why did mobile operators never perceive, and respond to, demand for pan-European services?  I suggest the answer lies primarily in e.g. the failure of services sector liberalisation.  With more cross-border business, a market with 3 or more mobile operators almost certainly have met the sort of demand single market entrepreneurs would have reflected.
  • Supply: Single Market services needed someone to go first, and the candidate for that was Vodafone. Yet when they acquired Mannesman, the competition authorities bizarrely decided to limit the scope for ‘one Europe’ services.  Even so, the merger precipitated a wave of pan-European industry alliances which might in turn have led to the consolidation now recognised as essential.  And yet, with the industry in positive flux, the publication of socially regressive, price setting, roaming legislation quickly overwhelmed other market dynamics.

Incumbent spectrum holders

Spectrum is the big story for the future of telecoms sector, and the Commission has rightly pushed for more rational management of this truly scarce resource.  Leaving aside whether this should EU or national management, there is still far too little hard inter-departmental (defence, cultural, communications … needs) discussion about spectrum allocation.

Two issues exist:

  1. pricing mechanisms are still insufficient for policy makers to compare possible spectrum uses, and to give incentives for incumbent holders (‘hoarders’) to relinquish control.
  2. while spectrum is vital for infrastructure competition, the cost of missing competition is masked by the presence of regulation, and reduces the political pressure on hoarders.  In other words, announcing an end to regulation would improve the hand of telecoms policy makers, relative to other parts of government, when making the case for more spectrum!


Structural reform applies to government too, as I argued recently.  Regulators could no doubt fine tune the application of today’s rules for decades to come, but the best ones’ have kept their eye on the original goal and have strived to put themselves out of work, knowing that their effectiveness and experience will be snapped up.  The Dutch seem to have gone furthest in providing officials with a clear perspective of life-after-telecoms, and this can be emulated in other countries.

Let’s celebrate success

Commissioners Reding and Kroes both recognised that digital policy needed to be about demand as much, if not more, than supply.  When heads of state met to discuss digital last October, their communique recognised the change in the nature of the debate, even while invited to consider a new set of amendments to the telco rule book.

It is time for the new President to declare telco liberalisation successfully completed, and signal that the ‘Digital Agenda’ is no longer simply a sectoral portfolio, but policy for the college as a whole.