Web Fundamentals
Web APIs

Ruben Verborgh, Ghent Universityimec

Web Fundamentals
Web APIs

Ruben Verborgh

Ghent University imec IDLab

Creative Commons License Except where otherwise noted, the content of these slides is licensed under a Creative Commons Attribution 4.0 International License.

Unlike many things from the ’90s,
the Web still works and is in active use.

[A photograph of a door; which way does it open?]
©2012 Dave Crosby

An affordance is a property of an object
that explains and aids its usage.

A Web API is a server-side HTTP interface
exposing data and functionality to clients.

Web Fundamentals
Web APIs

Web Fundamentals
Web APIs

Why does the Web scale the way it does?
How should we build for the Web?

The REST architectural style consists of
5 types of mandatory constraints.

  1. client–server constraints
  2. statelessness constraint
  3. cache constraints
  4. layered system constraints
  5. uniform interface constraints

REST architectures inherit
the client–server constraints.

REST architectures constrain
communication to be stateless.

Statelessness applies to application state.
Servers still maintain resource state.

This blog post and this book elaborate on the difference.

Statelessness improves visibility,
reliability, and scalability.

visibility
Intermediaries understand each request
without having to know any others.
reliability
Recovery from partial failures is easier
because no request history is needed.
scalability
No data must be kept between requests.
Each request can be handled by a different,
independent server.

Statelessness has increased bandwidth and
reduced server-side control as trade-offs.

bandwidth
Requests might need to repeat information
that was sent previously but not kept.
control
Clients become responsible to transition
consistently from one state into another.
They cannot necessarily be trusted
to make correct and allowed transitions.

The server does not know any client’s previous state.
Clients might jump to unpermitted states by “guessing”.

The REST architectural style
imposes cache constraints.

The layered system constraints
allow for flexible hierarchical layers.

The uniform interface constraints
are unique to the REST architectural style.

REST introduces 4 uniform interface constraints:

  1. identification of resources
  2. resource manipulation through representations
  3. self-descriptive messages
  4. hypermedia as the engine of application state

resource is the main unit of information.
An identifier points to at most 1 resource.

A resource is a conceptual relationship:
its value might change over time.

Information resources can have
zero or more representations.

Each message in a REST interaction
should be self-descriptive.

The interaction is driven by
hypermedia controls inside of responses.

Hypermedia defines REST interactions,
minimizing the client–server contract.

Web Fundamentals
Web APIs

The Web is one implementation of REST—
and not every website follows the style.

An urban architect does not control individual buildings.

[the Ghent city pavilion]
©2014 City of Ghent

How does the Web implement REST using
its core technologies URL, HTTP & HTML?

  1. client–server constraints
  2. statelessness constraint
  3. cache constraints
  4. layered system constraints
  5. uniform interface constraints

Obviously, the Web consists
of clients and servers.

The HTTP protocol is designed
in a stateless way.

HTTP realizes caching with intermediaries
and explicit caching headers.

HTTP allows transparent insertion
of layers of proxies.

The uniform interface is implemented
by a combination of URL, HTTP, and HTML.

The Web supports all 4 constraints:

  1. identification of resources
  2. resource manipulation through representations
  3. self-descriptive messages
  4. hypermedia as the engine of application state

Web resources are conceptual relations
uniquely identified by HTTP URLs.

Web resources are conceptual relations
uniquely identified by HTTP URLs.

Clients obtain different representations
through HTTP content negotiation.

Clients obtain different representations
through HTTP content negotiation.

Well-defined HTTP methods & headers
and media types enable self-description.

HTML (and other hypermedia) documents
can contain hypermedia controls.

Other media types can also have links,
if an interpretation is specified.

When I say hypertext,
I mean the simultaneous presentation of information and controls such that the information becomes the affordance through which the user (or automaton) obtains choices and selects actions.

Roy Fielding

Web Fundamentals
Web APIs

A REST API is a collection of resources.
Any regular website is in fact a REST API.

A resource should relate
a URL to a concept.

Don’t.

https://​example.org/​songs/​showDetails.php

A resource can have many representations
exposed through a single URL.

Don’t.

https://​example.org/​songs/​3563642 results in HTML.

https://​api.example.org/​getObjectJson.php?id=183113 results in JSON.

Each message should
be self-descriptive.

Don’t.

HTTP/1.1 5027 Incomplete
Custom-Server-Id: song.1.2
Content-Type:
  application/json

The interaction should be
driven by hypermedia.

Don’t.

{
  "title": "Baba O’Riley",
  "artist": {
    "id": 5521

  }
}

Servers offering a hypermedia API
enable the follow your nose principle.

To understand hypermedia’s importance,
imagine using any website without it.

You’d need to read the documentation,
and hard-code it in your implementation.

https://google.com/documentation.html

To use this site,
enter your search term as follows: https://www.google.com/search?q=search+term

In the above, search+term is
the URL-encoded version of your search term.

Web Fundamentals
Web APIs

There exists a large number of Web APIs,
growing at an increasing rate.

number of Web APIs
indexed in ProgrammableWeb

Most Web APIs do not follow REST.
They do not focus on the long term.

Exposing cultural heritage metadata
requires a long-term architectural vision.

Using the Europeana website
as a human is straightforward.

Using the Europeana website
as a machine is possible in two ways.

Which one would be easier?

Retrieving an item through the website
as a machine takes one step.

  1. Use the same URL of the item, but ask for JSON.
curl -H "Accept: application/json" \
https://www.europeana.eu/portal/en/record/9200229/BibliographicResource_3000135601313

Using the dedicated Europeana API
is quite complicated.

All of that wasn’t necessary for the website,
even though it contains the exact same information.

Retrieving an item using the dedicated API
involves at least 5 steps.

The URL you obtain from those steps
is personalized and thus volatile.

https://europeana.eu/api/v2/record/9200229/​BibliographicResource_3000135601313.json?wskey=xxxxxxxxx

The JSON cannot be shared nor cached,
and you need the documentation.

{
  "apikey": "xxxxxxxxx",
  "action": "--deprecated--",
  "success": true,
  "statsDuration": 110,
  "requestNumber": 999,
  "object": {
    "type": "IMAGE",
    "edmDatasetName": [ "9200229_Ag_EU_TEL_a1112_LibGent" ],
    "title": [ "Belfort, Botermarkt, Gent klokkentoren (1853)" ],
    "about": "/9200229/BibliographicResource_3000135601313",
    "europeanaAggregation": {
      "about": "/aggregation/europeana/9200229/BibliographicResource_3000135601313",
      "aggregatedCHO": "/item/9200229/BibliographicResource_3000135601313",
      "edmLandingPage": "https://europeana.eu/portal/record/9200229/BibliographicResource_3000135601313.html",
      "edmCountry": { "def": [ "belgium" ] },
      "edmLanguage": { "def": [ "mul" ] },
      "edmRights": {
        "def": [ "https://creativecommons.org/licenses/by-sa/4.0/" ]
      },
      "edmPreview": "https://europeanastatic.eu/api/image?uri=http%3A%2F%2Fadore.ugent.be%2FOpenURL%2Fresolve%3Fsvc_id%3Dmedium%26url_ver%3DZ39.88-2004%26rft_id%3Darchive.ugent.be%3A3107D15A-BB55-11E3-8B3D-86C4D43445F2%3A1&size=LARGE&type=IMAGE"
    },
    "proxies": [
      {
        "about": "/proxy/provider/9200229/BibliographicResource_3000135601313",
        "dcDescription": {
          "def": [
            "Stempel op keerzijde afbeelding: Copyright A.C.L.",
            "Handgeschreven notitie op keerzijde afbeelding: 102308 B"
          ]
        },
        "dcFormat": { "en": [ "Printed" ] },
        "dcIdentifier": { "def": [ "002075264" ] },
        "dcRights": {
          "def": [
            "Reproductierecht Universiteitsbibliotheek Gent",
            "CC BY-SA (4.0)"
          ]
        },
        "dcTitle": {
          "def": [ "Belfort, Botermarkt, Gent klokkentoren (1853)" ]
        },
        "dcType": { "en": [ "Serial" ] },
        "dctermsExtent": {
          "def": [ "1 fotografische druk : : zwart/wit." ]
        },
        "dctermsIsPartOf": {
          "def": [ "http://data.theeuropeanlibrary.org/Collection/a1112" ]
        },
        "dctermsIssued": {
          "def": [
            "1875? - 1930?",
            "[eind 19e-begin 20e eeuw]."
          ]
        },
        "dctermsSpatial": {
          "def": [ "België, Vlaanderen, Oost-Vlaanderen, Gent (9000), Gent (9000)" ]
        },
        "proxyIn": [
          "/aggregation/provider/9200229/BibliographicResource_3000135601313"
        ],
        "proxyFor": "/item/9200229/BibliographicResource_3000135601313",
        "edmType": "IMAGE",
        "europeanaProxy": false
      },
      {
        "about": "/proxy/europeana/9200229/BibliographicResource_3000135601313",
        "proxyIn": [
          "/aggregation/europeana/9200229/BibliographicResource_3000135601313"
        ],
        "proxyFor": "/item/9200229/BibliographicResource_3000135601313",
        "edmType": "IMAGE",
        "europeanaProxy": true
      }
    ],
    "aggregations": [
      {
        "about": "/aggregation/provider/9200229/BibliographicResource_3000135601313",
        "edmDataProvider": {
          "def": [ "Ghent University Library" ]
        },
        "edmIsShownBy": "https://adore.ugent.be/OpenURL/app?type=carousel&id=archive.ugent.be:3107D15A-BB55-11E3-8B3D-86C4D43445F2",
        "edmObject": "https://adore.ugent.be/OpenURL/resolve?svc_id=medium&url_ver=Z39.88-2004&rft_id=archive.ugent.be:3107D15A-BB55-11E3-8B3D-86C4D43445F2:1",
        "edmProvider": {
          "en": [ "The European Library" ]
        },
        "edmRights": {
          "def": [ "https://creativecommons.org/licenses/by-sa/4.0/" ]
        },
        "aggregatedCHO": "/item/9200229/BibliographicResource_3000135601313",
        "webResources": [
          {
            "webResourceEdmRights": {
              "def": [ "https://creativecommons.org/licenses/by-sa/4.0/" ]
            },
            "about": "https://adore.ugent.be/OpenURL/app?type=carousel&id=archive.ugent.be:3107D15A-BB55-11E3-8B3D-86C4D43445F2",
            "textAttributionSnippet": "Belfort, Botermarkt, Gent klokkentoren (1853) - https://europeana.eu/portal/record/9200229/BibliographicResource_3000135601313.html. Ghent University Library. CC BY-SA - https://creativecommons.org/licenses/by-sa/4.0/",
            "htmlAttributionSnippet": "<span about='https://data.europeana.eu/item/9200229/BibliographicResource_3000135601313'><a href='https://europeana.eu/portal/record/9200229/BibliographicResource_3000135601313.html'><span property='dc:title'>Belfort, Botermarkt, Gent klokkentoren (1853)</span></a>. Ghent University Library. <a href='https://creativecommons.org/licenses/by-sa/4.0/' rel='xhv:license https://www.europeana.eu/schemas/edm/rights'>CC BY-SA</a><span rel='cc:useGuidelines' resource='https://www.europeana.eu/rights/pd-usage-guide/'>.</span></span>"
          },
          {
            "webResourceEdmRights": {
              "def": [ "https://creativecommons.org/licenses/by-sa/4.0/" ]
            },
            "about": "https://adore.ugent.be/OpenURL/resolve?svc_id=medium&url_ver=Z39.88-2004&rft_id=archive.ugent.be:3107D15A-BB55-11E3-8B3D-86C4D43445F2:1",
            "textAttributionSnippet": "Belfort, Botermarkt, Gent klokkentoren (1853) - https://europeana.eu/portal/record/9200229/BibliographicResource_3000135601313.html. Ghent University Library. CC BY-SA - https://creativecommons.org/licenses/by-sa/4.0/",
            "htmlAttributionSnippet": "<span about='https://data.europeana.eu/item/9200229/BibliographicResource_3000135601313'><a href='https://europeana.eu/portal/record/9200229/BibliographicResource_3000135601313.html'><span property='dc:title'>Belfort, Botermarkt, Gent klokkentoren (1853)</span></a>. Ghent University Library. <a href='https://creativecommons.org/licenses/by-sa/4.0/' rel='xhv:license https://www.europeana.eu/schemas/edm/rights'>CC BY-SA</a><span rel='cc:useGuidelines' resource='https://www.europeana.eu/rights/pd-usage-guide/'>.</span></span>"
          }
        ],
        "edmPreviewNoDistribute": false
      }
    ],
    "providedCHOs": [
      {
        "about": "/item/9200229/BibliographicResource_3000135601313"
      }
    ],
    "europeanaCompleteness": 7,
    "europeanaCollectionName": [ "9200229_Ag_EU_TEL_a1112_LibGent" ],
    "language": [ "mul" ],
    "timestamp_created_epoch": 1442493422613,
    "timestamp_update_epoch": 1455546713777,
    "timestamp_created": "2015-09-17T12:37:02.613Z",
    "timestamp_update": "2016-02-15T14:31:53.777Z"
  }
}

API keys for representations
(not resources) are a fallacy.

Europeana’s Web API design choices
have consequences for sustainability.

Europeana’s website design choices
have consequences for sustainability.

Engineer for serendipity.

Roy Fielding

Web Fundamentals
Web APIs

Web APIs started out as Web services.
The difference is not well-defined.

Web services with Remote Procedure Calling
have surprisingly few to do with the Web.

The Simple Object Access Protocol (SOAP)
transfers XML messages on top of HTTP.

A SOAP message consists of 3 main XML elements.

envelope
root with (optional) header and body
header
modular extension mechanism
body
details of a method invocation

The client sends such an XML document with HTTP POST
to a single specific URL that identifies the service.

This SOAP request calls a method
with one input parameter.

<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
  <env:Header>
  </env:Header>
  <env:Body>
    <s:getSongDetails xmlns:m="http://example.org/song-service">
      <s:id>22896</s:id>
    </s:getSongDetails>
  </env:Body>
</env:Envelope>

This SOAP response contains
one output parameter.

<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
  <env:Header>
  </env:Header>
  <env:Body>
    <s:songDetails xmlns:m="http://example.org/song-service">
      <s:title>Baba O’Riley</s:title>
      <s:artistId>5521</s:artistId>
    </s:songDetails>
  </env:Body>
</env:Envelope>

SOAP provides language-independent
programming over a network.

Web Fundamentals
Web APIs

Web API descriptions capture machine-interpretable details about an API.

There exist 3 broad uses for descriptions.

in-band
resource and hypermedia control details
interface
structural properties
functional
effect and/or purpose characterization

Description formats support one or more of them.

The Web Services Description Language
(WSDL) describes SOAP service structure.

OWL-S (Semantic Markup for Web Services)
enriches WSDL with functional aspects.

The Web Application Description Language
(WADL) is the WSDL of non-SOAP Web APIs.

Most recent Web API description formats
focus purely on developer support.

Structural descriptions boost development
but don’t fundamentally change Web APIs.

Hypermedia-based formats place
description inside of the message.

Hydra is an RDF vocabulary to describe
hypermedia controls and API structure.

Web Fundamentals
Web APIs

[a tree with a ladder]
©2006 Project Cartoon

What the customer
thought they needed

[a tree with an unusable swing]
©2006 Project Cartoon

How the project leader
understood it

[a tree with a broken swing]
©2006 Project Cartoon

How the programmer
wrote it

[a roller coaster]
©2006 Project Cartoon

How the customer
was billed

[a tire swing on a tree]
©2006 Project Cartoon

What the customer
really needed

What does the customer
really need?

If a customer wants to make their website accessible
for automated clients, what do you propose?

If they ask for a Web API, what do you propose?

And—most importantly—what do you bill them for? 😉