Overview of API2

LMFDB’s existing API was designed to be a very thin HTTP wrapper over the database core of LMFDB. As such, it requires low level knowledge of the structure of the database, provides no guarantees of stability of the interface and no guarantees or information about what is stored in the database. API2 was designed to provide a middleware layer providing some level of stability without enforcing a rigid design paradigm on the LMFDB developer community.

It works by providing “searcher” objects that describe packaged sets of search operations that are permitted. Each searcher then allows a set of possible query parameters, one or more of which are required to perform a valid search. Human readable descriptions of searchers and search parameters are provided through the API.

API2 versioning

API2 uses a three part version string major.minor.revision. It is expected that breaking changes would only be introduced on a version change. Current version is 1.0.0

API2 resources

API2 consists of a single resource

API2 requests

Since API2 is intended for returning data from the server only all requests should be issued as HTTP(S) GET requests. There are six end points

API2 singleton queries

Singletons are intended to be references to machine readable versions of single objects in the database and are supposed to be the same as the existing human readable versions of the same data. So for example /api2/singletons/EllipticCurve/Q/11/a/2 will give a machine readable version of the same curve as /EllipticCurve/Q/11/a/2

API2 search query strings

The description parts of API2 take all of their parameters through their URL. Searches are performed by passing a query string. The main part of an API2 searcher query is name=value keys representing values for possible search parameters obtained from the description endpoint. These have the simple obvious form for equality, for example /api2/data/elliptic_curves_q?rank=0&conductor=11 is a search for rank=0, conductor = 11 in the elliptic curves over Q. Searching for conditions other than simple equality are performed by prepending an operator immediately after the = symbol. Currently there are 3 operators

For example /api2/data/elliptic_curves_q?rank=0&conductor=>11 is a search for rank=0, conductor > 11 There are also other possible query parameters that are not directly related to searching, but to controlling the data returned. These are

API2 responses

All API2 responses are as JSON documents complying with the ECMA-404 standard. Compliance with RFC 8259 is attempted but not guaranteed. The outer layer of the response is the same for all API2 requests

{"api_version":"1.0.0",
"type":"API_SEARCHERS",
"key": "GLOBAL",
"built_at": "2010-04-20T20:08:21.634121",
"data": [...]}

API_SEARCHERS

If the type field of the API2 response is “API_SEARCHERS” then the key field will be GLOBAL. The data field will consist of a JSON object the keys of which correspond to unique named searchers. The names are used when querying or using the searcher. Each value for these keys has the following format

{"elliptic_curves_q":{
    "human_name":"Elliptic curves over q",
    "desc":"Search elliptic curves defined over the rationals"}
}

API_DESCRIPTIONS

If the type field of the API2 response is “API_DESCRIPTIONS” then the key field will be the name of the searcher requested. The data field will consist of a JSON object with keys corresponding to the names of all available query fields for this searcher. A single key (representing a single query field) has the following structure.

{    "cm": {
      "description": "CM code. 0 (for no CM), or a negative discriminant", 
      "example": "-163", 
      "type": "integer",
      "cname":"CMCODE",
      "codec":"CODEC_TO_COME"
    }
}

It is important to note that a searcher is valid if it contains any subset of these fields, or is also valid if it is null.

API_INVENTORY

If the type field of the API2 response is “API_INVENTORY” then the key field will be the name of the searcher requested. The data field will consist of a JSON object with keys corresponding to the names of all available return fields for this searcher. A single key (representing a single query field) has the following structure.

{    "cm": {
      "description": "CM code. 0 (for no CM), or a negative discriminant", 
      "example": "-163", 
      "type": "integer",
      "cname":"CMCODE",
      "codec":"CODEC_TO_COME"
    }
}

It is important to note that a return field is valid if it contains any subset of these fields, or is also valid if it is null.

You will notice that the returns of API_DESCRIPTIONS and API_INVENTORY are often very similar, but this is not guaranteed. In the simple case the list of items that you can search on and fields that you will be returned will be the same but in general both must be parsed by a client.

API_RECORDS

If the type field of the API2 response is “API_RECORDS” then the key field will be a string containing the “data/” followed by the name of the searcher that generated the records. The data field will be a JSON object containing information about the number of records found in total, the number returned in this request and how to request the next tranche of records. It also contains a list of JSON objects returning the results from the database. The response has the following format

{"record_count":6000,
"view_start":0,
"view_count":100,
"view_next":99,
records:{...}
}

Interpreting the records

The fields returned by an API_RECORDS object have no general form, but the /api2/inventory/ result allows a client to match them to a canonical name and description. For speed, this remapping is not handled by the API but must be handled by the client. See the section on accessing the inventory.

API_ERROR

If the type field of the API2 response is “API_ERROR” then the key field will be a string containing the string “GLOBAL”. The data field will be a string containing additional information about the error. A blank string is valid, but should not normally be returned. Your request has failed.

An example API2 client operation

This is an example of how a valid client would interact with API2

Find searchers

Client would connect to /api2/description/searchers. Result is

[
{"api_version":"1.0.0",
"type":"API_SEARCHERS",
"key": "GLOBAL",
"built_at": "2010-04-20T20:08:21.634121",
"data": [
    {""elliptic_curves_q":{
    "human_name":"Elliptic curves over q",
    "desc":"Search elliptic curves defined over the rationals"}
    },
    {"classical_modular_forms":{
    "human_name":"Classical Modular Forms",
    "desc":"Search classical modular forms"}
    }]

(In a fully working implementation there would be many more than this, but a client should cope with what is available). The client decides that it wishes to search in elliptic_curve_q.

Query searcher

Client connects to /api2/description/elliptic_curves_q. Result is

{"api_version":"1.0.0",
"type":"API_DESCRIPTIONS",
"key": "description/elliptic_curves_q",
"built_at": "2010-04-20T20:08:21.634121",
"data": [
    ...
    ...
    "label":"{
          "description": "Cremona label", 
          "example": "100002a1", 
          "type": "string",
          "cname":"CREMONA_LABEL"
      },
      "lmfdb_label": {
          "description": "LMFDB label", 
          "example": "100.a1", 
          "type": "string"
          "cname":"LMFDB_LABEL"
        },
        ...
        ...
    ]
}

The client parses this response and knows that it seeks to search on the canonical CREMONA_LABEL so it checks every returned search query term to ensure that a searchable field has a matching canonical name. It finds that a field does so and may continue. If searching on multiple fields this check should be performed on each field. Since the server will allow searching on canonical name (which should be unchanging by construction) it is acceptable for a client to skip this step and simply seek to search on a known canonical name but this is not recommended.

(Optional) Get inventory information

If a client is purely recovering data for a human to manage then this step can be skipped, but a typical client will get inventory information about the data returned by a search on the selected searcher.

Client connects to /api2/inventory/elliptic_curves_q. Result is

{"api_version":"1.0.0",
"type":"API_INVENTORY",
"key": "inventory/elliptic_curves_q",
"built_at": "2010-04-20T20:08:21.634121",
"data": [
    "2adic_index": {
      "description": "index in GL(2,Z2) of the 2-adicrepresentation (or 0 for CM curves)", 
      "example": "1", 
      "type": "integer",
      "cname":"2ADIC_INDEX"
    }, 
    "2adic_label": {
      "description": "Rouse label of the associated modular curve (null for CM curves). Based on Rouse, Zureik-Brown classification", 
      "example": "X1", 
      "type": "string",
      "cname":"2ADIC_LABEL"
    }, 
    ...
    ...
    ]
}

client is interested in returning the key with canonical name 2ADIC_INDEX. It can express this as _fields="2ADIC_INDEX" .

Get results

Knowing what it wants to search on and what it wants returned the client connects to /api2/data/elliptical_curve_q?label="100a3"&_fields="2ADIC_INDEX". Result is

{"api_version":"1.0.0",
"type":"API_INVENTORY",
"key": "inventory/elliptic_curves_q",
"built_at": "2010-04-20T20:08:21.634121",
"data": [
  "record_count":1,
  "view_start":0,
  "view_count":1,
  "view_next":-1,
  records:[{2adic_index:12}]]
}

It is important to note that returned database keys are not transformed into canonical name form. The client must use the server supplied mappings from canonical name to database name