One flew over the cuckoo’s nest

The role of aggregation on a decentralized Web

Ruben Verborgh, Ghent Universityimec

EuropeanaTech, 15 May 2018

Watch the video recording of this talk

One flew over the cuckoo’s nest

The role of aggregation on a decentralized Web

Ruben Verborgh

Ghent University – imec

©2011 Bernard Spragg
©2011 Bernard Spragg
©2012 Bernard Spragg
©2018 James West
©1975 United Artists

Centralization and decentralization
both have their merits.

Technology should serve a purpose,
not an ideology.

Cultural heritage starts decentralized.
Why do we centralize via aggregation?

Having access to more data
makes disambiguation easier.

…although it also increases the problem size.

Aggregation makes interlinking
of collections easier.

Centralized infrastructure simplifies access to these innovations.



The case of a small metadata producer:
my scholarly publications.

I have been publishing my own metadata
since before most of these existed.

Why spend time on this while
the aggregators already do?

I want to be the source of truth.
I don’t need to be the only source.

What gets lost in translation
when data is aggregated?

What do data producers get back
in return from aggregators?

What flows back to data producers
as a return from aggregators?

Imagine all sorts of feedback
we are missing out on.

This knowledge lets (only) you improve your data
and the experience of those who eventually use it.

Decentralization can be realized
at very different scales.

Every piece of data in decentralized apps
can come from a different place.

Multiple decentralized Web apps
share access to data stores.

Different app and storage providers
compete independently.

Hard-coded client–server contracts are
unsustainable with multiple sources.

Query-based contracts can make
decentralized Web apps more sustainable.

©2014 tHeDiGiTaLdRoPoUt

The Paradox of Freedom:
you can only be free if you follow rules.

We need to identify those rules
we all need to agree on.

Lessons learned from aggregating hundreds of datasets
are highly useful to inform the discussion.

Decentralization needs replication
for realistic performance.

In addition to technological changes,
we need a shift of mindset.

Current networks are centered
around the aggregator.

We need to create network flows
to and from the aggregator.

The individual network nodes
need to become the source of truth.

Aggregators need to become part
of a larger network.

Aggregators serve as a crucial
but transparent layer in the network.

Aggregators’ main responsibility becomes
fostering a network between nodes.

©2011 Bernard Spragg

The question of identity
becomes one of role.

©2011 Russ

One flew over the cuckoo’s nest

The role of aggregation on a decentralized Web