The Decentralized Web and its challenges for querying

Ruben Verborgh, Ghent University – imec

Aalborg University, 20 November 2019

The Decentralized Web and its challenges for querying

Ruben Verborgh

Ghent University – imec

For several years now,
we have known how to
do Big Data pretty well.

But we are vastly unprepared
for a future that consists of
a large number of small data sets.

The Web strives to be universal
through independence of many factors.

The Web brings freedom of expression
to everyone across the world.

The Web brings permissionless innovation
at a global scale.

Our data has become centralized
in a handful of Web platforms.

Within the walled gardens of social media,
you have to move either data or people.

© David Simonds

The current massive centralization
hurts diversity, innovation, and choice.

Different platforms tackle decentralization
at very different scales.

People choose where they store
every single piece of data they produce.

They can grant apps and people access
to very specific parts of their data.

Separating app and storage competition
drives permissionless innovation.

Solid is not a company or organisation.
Solid is not (just) software.

[the Solid logo]

Anyone can build or host
software for Solid.

The Solid server acts as a data pod
that stores and guards your data.

A typical data pod can contain
any data you create or need online.

Solid clients are browser or native apps
that read from or write to your data pod.

Any app you can envision,
you can build with Solid.

The Solid server and several apps exist
and are usable for developers.

Solid is transitioning from research project
into an ecosystem backed by start-ups.

Interoperability challenges in Solid
are solved through Linked Data in RDF.

With JSON-LD, every piece of data
can link to any other piece of data. 

{
  "@context":  "https://www.w3.org/ns/activitystreams",
  "id":        "#ruben-likes-aau",
  "type":      "Like",
  "actor":     "https://ruben.verborgh.org/profile/#me",
  "object":    "http://dbpedia.org/resource/Aalborg_University",
  "published": "2019-11-20T09:00:00Z"
}

Data shapes and their semantics
enable layered compatibility. 

{
  "@context":  "https://www.w3.org/ns/activitystreams",
  "id":        "#ruben-likes-aau",
  "type":      "Like",
  "actor":     "https://ruben.verborgh.org/profile/#me",
  "object":    "http://dbpedia.org/resource/Aalborg_University",
  "published": "2019-11-20T09:00:00Z"
}

Different source data
can be concatenated. 

{
  "@context":  "https://www.w3.org/ns/activitystreams",
  "@graph": [{
    "type":      "Like",
    "actor":     "https://ruben.verborgh.org/profile/#me",
    "object":    "http://dbpedia.org/resource/Aalborg_University",
    "published": "2019-11-20T09:00:00Z"
  },{
    "type":      "Like",
    "actor":     "https://example.org/people/katja#me",
    "object":    "http://dbpedia.org/resource/Aalborg_University",
    "published": "2019-11-20T09:05:00Z"
  }]
}

Decentralized apps have many back-ends. Back-ends work with many apps.

The current approach to building APIs
does not play well with decentralization.

When clients do not bind to HTTP requests,
APIs can evolve independently of app logic.

The developer experience is the most
crucial factor for Solid success.

Simple tasks should be simple,
complex tasks should be manageable.

People think RDF is a pain
because it is complicated.
The truth is even worse.

RDF is painfully simplistic,
but it allows you to work with real-world data
and problems that are horribly complicated.

Dan Brickley & Libby Miller

The old way of working with RDF
followed the old way of JSON APIs.

The old way of working with RDF
in the browser is hard.

  1. Gather input data into a query.
  2. Send a specific API call the query.
  3. Parse the response as JSON RDF.
  4. Traverse the JSON tree RDF graph structure.
  5. Update the DOM.

React combines all steps into
one component experience. 

<LoggedIn>
  <p>Welcome, <Value src="user.firstName"/></p>
  <Image src="user.image" defaultSrc="profile.svg"/>
  <ul>
    <li><Link href="user.inbox">Your inbox</Link></li>
    <li><Link href="user.homepage">Your homepage</Link></li>
  </ul>
  <h2>Your friends</h2>
  <List src="user.friends.firstName"/>
</LoggedIn>

These components rely on
micro-query expressions. 

<LoggedIn>
  <p>Welcome, <Value src="user.firstName"/></p>
  <Image src="user.image" defaultSrc="profile.svg"/>
  <ul>
    <li><Link href="user.inbox">Your inbox</Link></li>
    <li><Link href="user.homepage">Your homepage</Link></li>
  </ul>
  <h2>Your friends</h2>
  <List src="user.friends.firstName"/>
</LoggedIn>

LDflex exposes Linked Data’s flexibility without RDF’s complexity.

LDflex is a domain-specific language
for JavaScript.

We need research to tackle
large numbers of small datasets.

The Decentralized Web and its challenges for querying

@RubenVerborgh