Skip to content

Instantly share code, notes, and snippets.

@jb-adams

jb-adams/drs_batch_request_poc.md

Last active June 14, 2021 19:29
Show Gist options
  • Save jb-adams/fbf9ff6079ff531c989be24bab1c3583 to your computer and use it in GitHub Desktop.
Save jb-adams/fbf9ff6079ff531c989be24bab1c3583 to your computer and use it in GitHub Desktop.
Download ZIP
DRS batch request PoC example
Raw
drs_batch_request_poc.md

DRS batch request PoC using GA4GH starter kit

Intro

This gist provides a working, proof-of-concept example of DRS object batch requests. This prototype feature of the DRS specification was developed in the GA4GH starter kit DRS implementation.

The goal is to have a working example for this prototype feature that can be tried out, commented on, and refined as the group works towards a specification for DRS batch requests. Refinements will be incorporated back into the starter kit implementation until a standardized approach is reached.

Setup

The following steps will allow you to perform DRS batch requests on your local machine using the starter kit. The setup only requires docker to be installed locally, and for ports 4500 and 4501 to be free.

Pull the docker image:

docker pull ga4gh/ga4gh-starter-kit-drs:2021-06-14-SNAPSHOT

Run the container:

docker run --rm --name drs-batch-request-test -p 4500:4500 -p 4501:4501 ga4gh/ga4gh-starter-kit-drs:2021-06-14-SNAPSHOT

The DRS service should now be running at http://localhost:4500. In its default configuration, the starter kit DRS service serves a small test set DrsObjects.

Using the GET endpoint for single objects

To start, let's request information about a single DrsObject using the standard GET /objects/{object_id} endpoint. Issue the following GET request: http://localhost:4500/ga4gh/drs/v1/objects/b8cd0667-2c33-4c9f-967b-161b905932c9

The server should respond with the following JSON payload:

{
    "id": "b8cd0667-2c33-4c9f-967b-161b905932c9",
    "description": "Open dataset of 384 phenopackets",
    "created_time": "2021-03-12T20:00:00Z",
    "name": "phenopackets.test.dataset",
    "size": 143601,
    "updated_time": "2021-03-13T12:30:45Z",
    "version": "1.0.0",
    "self_uri": "drs://localhost:4500/b8cd0667-2c33-4c9f-967b-161b905932c9",
    "contents": [
        {
            "name": "phenopackets.mundhofir.family",
            "drs_uri": [
                "drs://localhost:4500/1af5cdcf-898c-4dbc-944e-1ac95e82c0ea"
            ],
            "id": "1af5cdcf-898c-4dbc-944e-1ac95e82c0ea"
        },
        {
            "name": "phenopackets.zhang.family",
            "drs_uri": [
                "drs://localhost:4500/355a74bd-6571-4d4a-8602-a9989936717f"
            ],
            "id": "355a74bd-6571-4d4a-8602-a9989936717f"
        },
        {
            "name": "phenopackets.cao.family",
            "drs_uri": [
                "drs://localhost:4500/a1dd4ae2-8d26-43b0-a199-342b64c7dff6"
            ],
            "id": "a1dd4ae2-8d26-43b0-a199-342b64c7dff6"
        },
        {
            "name": "phenopackets.lalani.family",
            "drs_uri": [
                "drs://localhost:4500/c69a3d6c-4a28-4b7c-b215-0782f8d62429"
            ],
            "id": "c69a3d6c-4a28-4b7c-b215-0782f8d62429"
        }
    ]
}

Using the POST endpoint for multiple objects (batch request)

Now, we will issue the prototype request for retrieving multiple DrsObjects in a single request. Currently, this is a POST request to the /objects endpoint.

Construct and issue the following POST request:

  • URL: http://localhost:4500/ga4gh/drs/v1/objects
  • Headers:
    • Content-type: application/json
  • Body:
{
    "selection": [
        "b8cd0667-2c33-4c9f-967b-161b905932c9",
        "00000000-0000-0000-0000-000000000000"
    ]
}

The server should respond with the following payload:

{
    "summary": {
        "requested": 2,
        "loaded": 1,
        "unloaded": 1
    },
    "drs_objects": {
        "b8cd0667-2c33-4c9f-967b-161b905932c9": {
            "id": "b8cd0667-2c33-4c9f-967b-161b905932c9",
            "description": "Open dataset of 384 phenopackets",
            "created_time": "2021-03-12T20:00:00Z",
            "name": "phenopackets.test.dataset",
            "size": 143601,
            "updated_time": "2021-03-13T12:30:45Z",
            "version": "1.0.0",
            "self_uri": "drs://localhost:4500/b8cd0667-2c33-4c9f-967b-161b905932c9",
            "contents": [
                {
                    "name": "phenopackets.mundhofir.family",
                    "drs_uri": [
                        "drs://localhost:4500/1af5cdcf-898c-4dbc-944e-1ac95e82c0ea"
                    ],
                    "id": "1af5cdcf-898c-4dbc-944e-1ac95e82c0ea"
                },
                {
                    "name": "phenopackets.zhang.family",
                    "drs_uri": [
                        "drs://localhost:4500/355a74bd-6571-4d4a-8602-a9989936717f"
                    ],
                    "id": "355a74bd-6571-4d4a-8602-a9989936717f"
                },
                {
                    "name": "phenopackets.cao.family",
                    "drs_uri": [
                        "drs://localhost:4500/a1dd4ae2-8d26-43b0-a199-342b64c7dff6"
                    ],
                    "id": "a1dd4ae2-8d26-43b0-a199-342b64c7dff6"
                },
                {
                    "name": "phenopackets.lalani.family",
                    "drs_uri": [
                        "drs://localhost:4500/c69a3d6c-4a28-4b7c-b215-0782f8d62429"
                    ],
                    "id": "c69a3d6c-4a28-4b7c-b215-0782f8d62429"
                }
            ]
        }
    },
    "unloaded_drs_objects": {
        "00000000-0000-0000-0000-000000000000": 404
    }
}

Explanation of the batch request endpoint

The above POST request example makes a request for multiple DrsObjects using a provided selection in the request body. The selection is simply an array of multiple requested DRS IDs. In the example, 2 ids were provided. The first id maps to a valid DrsObject in the database, the second id is invalid, in that no DrsObject with the id exists.

The response body is a complex object with 3 main attributes: summary, drs_objects, and unloaded_drs_objects:

  • summary simply indicates how many objects were requested in the selection, how many of those were successfully returned, and how many were not returned.
  • drs_objects is a JSON object/dictionary in which the key is the requested DRS id, and the value is the DrsObject for that id. Each successfully loaded DrsObject is in this dictionary.
  • unloaded_drs_objects is a JSON object/dictionary for requested objects that could not be loaded. The key is the requested DRS id, and the value is a simple explanation of why a DrsObject could not be loaded (in this case, the explanation is an HTTP code, indicating the requested entity was not found). Each requested DRS id for which a DrsObject was not loaded is in this dictionary.

Batch request endpoint: additional examples

Any combination of valid and invalid DRS ids can be submitted to the batch request endpoint

Additional example 1:

  • URL: http://localhost:4500/ga4gh/drs/v1/objects
  • Headers:
    • Content-type: application/json
  • Body:
{
    "selection": [
        "1a570e4e-2489-4218-9333-f65549495872",
        "4d83ba3f-a476-4c7c-868f-3d1fcf77fe29",
        "924901d5-6d31-4c33-b443-7931eadfac4b",
        "0f8abce3-e161-4bdf-981f-86257d505d69",
        "00000000-0000-0000-0000-000000000000",
        "11111111-1111-1111-1111-111111111111"
    ]
}

Additional IDs

The following IDs can be provided to the test DRS service (either GET or POST) to retrieve a valid DrsObject. Please experiment with the endpoint using various combinations of the following ids, as well as non-valid id:

  • b8cd0667-2c33-4c9f-967b-161b905932c9
  • a1dd4ae2-8d26-43b0-a199-342b64c7dff6
  • 1a570e4e-2489-4218-9333-f65549495872
  • 4d83ba3f-a476-4c7c-868f-3d1fcf77fe29
  • 924901d5-6d31-4c33-b443-7931eadfac4b
  • 0f8abce3-e161-4bdf-981f-86257d505d69
  • c69a3d6c-4a28-4b7c-b215-0782f8d62429
  • 456e9ee0-5b60-4f38-82b5-83ba5d338038
  • 1af6b862-7fc8-411a-86c5-d8e280e5b77a
  • c37b37fd-0450-432d-b6aa-9ffdece35ad0
  • 0bb9d297-2710-48f6-ab4d-80d5eb0c9eaa
  • a3bb76d7-76ae-414b-81f7-97e663b02206
  • 1af5cdcf-898c-4dbc-944e-1ac95e82c0ea
  • 2506f0e1-29e4-4132-9b37-f7452dc8a89b
  • c00c264a-8f17-471f-8ded-1a1f10e965ac
  • 355a74bd-6571-4d4a-8602-a9989936717f
  • 697907bf-d5bd-433e-aac2-1747f1faf366
  • 3a45fab2-f350-445d-a137-4b1f946bf184
  • ac53ca59-46f3-4f61-b1e7-bb75a49bfbfe
  • 1275f896-4c8f-47e1-99a1-873a6b2ef5fb
  • 8f40acc0-0c54-45c5-8c85-a6f5fb32a1a7
  • 41898242-62a9-4129-9a2c-5a4e8f5f0afb
  • 6b994f18-6189-4233-bb83-139686490d68
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment