AcousticBrainz Server Documentation

Contents:

Web API

The AcousticBrainz server provides a Web API for getting and submitting data.

Base URL: https://acousticbrainz.org

Endpoint Reference

Core data

GET /api/v1/high-level

Get high-level data for many recordings at once.

Example response:

{"mbid1": {"offset1": {document},
           "offset2": {document}},
 "mbid2": {"offset1": {document}},
 "mbid_mapping": {"MBID1": "mbid1"}
}

MBIDs and offset keys are returned as strings (as per JSON encoding rules). If an offset is not specified in the request for an mbid or is not a valid integer >=0, the offset will be 0.

MBID keys are always returned in a normalised form (all lower-case, separated in groups of 8-4-4-4-12 characters), even if the provided recording MBIDs are not given in this form. In the case that a requested MBID is not given in this normalised form, the value mbid_mapping in the response will be a dictionary mapping user-provided MBIDs to this form.

If the list of MBIDs in the query string has a recording which is not present in the database, then it is silently ignored and will not appear in the returned data.

Query Parameters:
 
  • recording_ids

    Required. A list of recording MBIDs to retrieve

    Takes the form mbid[:offset];mbid[:offset]. Offsets are optional, and should be >= 0

    You can specify up to MAX_ITEMS_PER_BULK_REQUEST MBIDs in a request.

  • map_classesOptional. If set to ‘true’, map class names to human-readable values
Response Headers:
 
GET /api/v1/low-level

Get low-level data for many recordings at once.

Example response:

{"mbid1": {"offset1": {document},
           "offset2": {document}},
 "mbid2": {"offset1": {document}},
 "mbid_mapping": {"MBID1": "mbid1"}
}

MBIDs and offset keys are returned as strings (as per JSON encoding rules). If an offset is not specified in the request for an MBID or is not a valid integer >=0, the offset will be 0.

MBID keys are always returned in a normalised form, even if the provided recording MBIDs are not given in this form. In the case that a requested MBID is not given in this normalised form, the value mbid_mapping in the response will be a dictionary that maps user-provided MBIDs to this form.

If the list of MBIDs in the query string has a recording which is not present in the database, then it is silently ignored and will not appear in the returned data.

If you only need a specific feature and don’t want to download the whole low-level document, you can request specific features using the features parameter. The shape of the returned document will be the same as a full low-level document, and will contain only the specified features as well as the metadata.audio_properties and metadata.version objects.

Query Parameters:
 
  • recording_ids

    Required. A list of recording MBIDs to retrieve

    Takes the form mbid[:offset];mbid[:offset]. Offsets are optional, and should be >= 0

    You can specify up to MAX_ITEMS_PER_BULK_REQUEST MBIDs in a request.

  • features

    Optional. A list of features to be returned for each mbid.

    Takes the form feature1;feature2.

    You can specify the following features in a request: LOWLEVEL_INDIVIDUAL_FEATURES.

Response Headers:
 
GET /api/v1/count

Get low-level count for many recordings at once. MBIDs not found in the database are omitted in the response.

Example response:

{"mbid1": {"count": 3},
 "mbid2": {"count": 1},
 "mbid_mapping": {"MBID1": "mbid1"}
}

MBID keys are always returned in a normalised form (all lower-case, separated in groups of 8-4-4-4-12 characters), even if the provided recording MBIDs are not given in this form. In the case that a requested MBID is not given in this normalised form, the value mbid_mapping in the response will be a dictionary mapping user-provided MBIDs to this form.

Query Parameters:
 
  • recording_ids

    Required. A list of recording MBIDs to retrieve

    Takes the form mbid;mbid.

You can specify up to MAX_ITEMS_PER_BULK_REQUEST MBIDs in a request.

Response Headers:
 
GET /api/v1/(uuid: mbid)/high-level

Get high-level data for recording with a given MBID.

This endpoint returns one document at a time. If there are many submissions for an MBID, you can browse through them by specifying an offset parameter n. Documents are sorted by the submission time of their associated low-level documents.

You can get the total number of low-level submissions using /<mbid>/count endpoint.

Query Parameters:
 
  • nOptional. Integer specifying an offset for a document.
  • map_classesOptional. If set to ‘true’, map class names to human-readable values

TODO: provide a link to what these mappings are

Response Headers:
 
GET /api/v1/(uuid: mbid)/low-level

Get low-level data for a recording with a given MBID.

This endpoint returns one document at a time. If there are many submissions for an MBID, you can browse through them by specifying an offset parameter n. Documents are sorted by their submission time.

You can the get total number of low-level submissions using the /<mbid>/count endpoint.

Query Parameters:
 
  • nOptional. Integer specifying an offset for a document.
Response Headers:
 
POST /api/v1/(uuid: mbid)/low-level

Submit low-level data to AcousticBrainz.

Request Headers:
 
Response Headers:
 
GET /api/v1/(uuid: mbid)/count

Get the number of low-level data submissions for a recording with a given MBID.

Example response:

{
    "mbid": mbid,
    "count": n
}

MBID values are always lower-case, even if the provided recording MBID is upper-case or mixed case.

Response Headers:
 

Datasets

POST /api/v1/datasets/

Create a new dataset.

Example request:

{
    "name": "Mood",
    "description": "Dataset for mood classification.",
    "public": true,
    "classes": [
        {
            "name": "Happy",
            "description": "Recordings that represent happiness.",
            "recordings": ["770cc467-8dde-4d22-bc4c-a42f91e"]
        },
        {
            "name": "Sad"
        }
    ]
}
Request Headers:
 
Request JSON Object:
 
  • name (string) – Required. Name of the dataset.
  • description (string) – Optional. Description of the dataset.
  • public (boolean) – Optional. true to make dataset public, false to make it private. New datasets are public by default.
  • classes (array) –

    Optional. Array of objects containing information about classes to add into new dataset. For example:

    {
        "name": "Happy",
        "description": "Recordings that represent happiness.",
        "recordings": ["770cc467-8dde-4d22-bc4c-a42f91e"]
    }
    
Response Headers:
 
Response JSON Object:
 
  • success (boolean) – True on successful creation.
  • dataset_id (string) – ID (UUID) of newly created dataset.
PUT /api/v1/datasets/(uuid: dataset_id)/recordings

Add recordings to a class in a dataset.

Example request:

{
    "class_name": "Happy",
    "recordings": ["770cc467-8dde-4d22-bc4c-a42f91e"]
}
Request Headers:
 
Request JSON Object:
 
  • class_name (string) – Required. Name of the class.
  • recordings (array) – Required. Array of recoding MBIDs (string) to add into that class.
Response Headers:
 
DELETE /api/v1/datasets/(uuid: dataset_id)/recordings

Delete recordings from a class in a dataset.

Example request:

{
    "class_name": "Happy",
    "recordings": ["770cc467-8dde-4d22-bc4c-a42f91e"]
}
Request Headers:
 
Request JSON Object:
 
  • class_name (string) – Required. Name of the class.
  • recordings (array) – Required. Array of recoding MBIDs (string) that need be deleted from a class.
Response Headers:
 
POST /api/v1/datasets/(uuid: dataset_id)/classes

Add a class to a dataset.

The data can include an optional list of recording ids. If these are included, the recordings are also added to the list. Duplicate recording ids are ignored.

If a class with the given name already exists, the recordings (if provided) will be added to the existing class.

Example request:

{
    "name": "Not Mood",
    "description": "Dataset for mood misclassification.",
    "recordings": ["770cc467-8dde-4d22-bc4c-a42f91e"]
}
Request Headers:
 
Request JSON Object:
 
  • name (string) – Required. Name of the class. Must be unique within a dataset.
  • description (string) – Optional. Description of the class.
  • recordings (array) – Optional. Array of recording MBIDs (string) to add into that class. For example: ["770cc467-8dde-4d22-bc4c-a42f91e"].
Response Headers:
 
PUT /api/v1/datasets/(uuid: dataset_id)/classes

Update class in a dataset.

If one of the fields is not specified, it will not be updated.

Example request:

{
    "name": "Very happy",
    "new_name": "Recordings that represent ultimate happiness."
}
Request Headers:
 
Request JSON Object:
 
  • name (string) – Required. Current name of the class.
  • new_name (string) – Optional. New name of the class. Must be unique within a dataset.
  • description (string) – Optional. Description of the class.
Response Headers:
 
DELETE /api/v1/datasets/(uuid: dataset_id)/classes

Delete class and all of its recordings from a dataset.

Example request:

{
    "name": "Sad"
}
Request Headers:
 
Request JSON Object:
 
  • name (string) – Required. Name of the class.
Response Headers:
 
GET /api/v1/datasets/(uuid: dataset_id)

Retrieve a dataset.

Response Headers:
 
DELETE /api/v1/datasets/(uuid: dataset_id)

Delete a dataset.

PUT /api/v1/datasets/(uuid: dataset_id)

Update dataset details.

If one of the fields is not specified, it will not be updated.

Example request:

{
    "name": "Not Mood",
    "description": "Dataset for mood misclassification.",
    "public": true
}
Request Headers:
 
Request JSON Object:
 
  • name (string) – Optional. Name of the dataset.
  • description (string) – Optional. Description of the dataset.
  • public (boolean) – Optional. true to make dataset public, false to make it private.
Response Headers:
 

Similarity

GET /api/v1/similarity/(metric)/between/

Get the distance measure for similarity between two MBIDs. The distance measure will correspond to the index of the specified metric.

The metrics available are shown here BASE_METRIC_NAMES.

Example response:

{"metric": [<distance_vector>]}

NOTE If an (MBID, offset) combination specified does not exist, or is not present in the index, an empty dictionary will be returned.

Query Parameters:
 
  • recording_ids

    Required. A list of the two recordings for which similarity should be found between.

    Takes the form mbid[:offset]:mbid[:offset]. Offsets are optional, and should be integers >= 0

Response Headers:
 
GET /api/v1/similarity/(metric)/

Get the most similar submissions to multiple (MBID, offset) combinations.

Example response:

{"mbid1": {"offset1": [(most_similar_mbid, offset),
                        ...,
                    (least_similar_mbid, offset)],
          "offset2": [(most_similar_mbid, offset),
                        ...,
                    (least_similar_mbid, offset)]
         },
 ...,

 "mbidN": {"offset1": [(most_similar_mbid, offset),
                        ...,
                       (least_similar_mbid, offset)]
          }
}

MBIDs and offset keys are returned as strings (as per JSON encoding rules). If an offset is not specified in the request for an mbid or is not a valid integer >=0, the offset will be 0.

If the list of MBIDs in the query string has a recording which is not present in the database, then it is silently ignored and will not appear in the returned data.

Query Parameters:
 
  • recording_idsRequired. A list of recording MBIDs to retrieve Takes the form mbid[:offset];mbid[:offset]. Offsets are optional, and should be >= 0
  • n_neighboursOptional. The number of similar recordings that should be returned for each item in recording_ids (1-1000). Default is 200 recordings.
  • metricRequired. String specifying the metric name to be used when finding the most similar recordings. The metrics available are shown here BASE_METRIC_NAMES.
  • thresholdOptional. Only return items whose distance from the query recording is less than this (0-1). This may result in the number of returned items being less than n_neighbours if it is set.
  • remove_dupsOptional. If samescore, remove duplicate submissions that have the same score. If all, remove all duplicate MBIDs even if they have different scores, returning only the submission with the lowest score. This may result in the number of returned items being less than n_neighbours if it is set.
Response Headers:
 

Rate limiting

The AcousticBrainz API is rate limited via the use of rate limiting headers that are sent as part of the HTTP response headers. Each call will include the following headers:

  • X-RateLimit-Limit: Number of requests allowed in given time window
  • X-RateLimit-Remaining: Number of requests remaining in current time window
  • X-RateLimit-Reset-In: Number of seconds when current time window expires (recommended: this header is resilient against clients with incorrect clocks)
  • X-RateLimit-Reset: UNIX epoch number of seconds (without timezone) when current time window expires [1]

We typically set the limit to 10 queries every 10 seconds per IP address, but these values may change. Make sure you check the response headers if you want to know the specific values.

Rate limiting is automatic and the client must use these headers to determine the rate to make API calls. If the client exceeds the number of requests allowed, the server will respond with error code 429: Too Many Requests. Requests that provide the Authorization header with a valid user token may receive higher rate limits than those without valid user tokens.

[1]Provided for compatibility with other APIs, but we still recommend using X-RateLimit-Reset-In wherever possible

Constants

Constants that are relevant to using the API:

webserver.views.api.v1.core.MAX_ITEMS_PER_BULK_REQUEST = 25

The maximum number of items that you can pass as a recording_ids parameter to bulk lookup endpoints

webserver.views.api.v1.core.LOWLEVEL_INDIVIDUAL_FEATURES = ['lowlevel.average_loudness', 'lowlevel.dynamic_complexity', 'metadata.audio_properties.replay_gain', 'metadata.tags', 'rhythm.beats_count', 'rhythm.beats_loudness.mean', 'rhythm.bpm', 'rhythm.bpm_histogram_first_peak_bpm.mean', 'rhythm.bpm_histogram_second_peak_bpm.mean', 'rhythm.danceability', 'rhythm.onset_rate', 'tonal.chords_changes_rate', 'tonal.chords_key', 'tonal.chords_scale', 'tonal.key_key', 'tonal.key_scale', 'tonal.key_strength', 'tonal.tuning_equal_tempered_deviation', 'tonal.tuning_frequency']

Features that can be selected individually from the bulk low-level endpoint

similarity.metrics.BASE_METRIC_NAMES = ['mfccs', 'mfccsw', 'gfccs', 'gfccsw', 'key', 'bpm', 'onsetrate', 'moods', 'instruments', 'dortmund', 'rosamerica', 'tzanetakis']

Available similarity metrics

Recording Similarity

An emerging feature in AcousticBrainz is recording similarity.

We compute similarity between recordings based on a selection of high and low-level features [metrics].

Similarity metrics for each recording should be computed on submission. When the metrics for a recording have been added to static index files, we can query for track-track similarity based on our choice of feature.

This dataset is made available through a set of API endpoints, and distribution of the static index files for more flexibility.

Similarity Metrics

Similarity metrics are methods of applying a vectorized measurement to one, or many features of high and low-level data.

Thus far, the following metrics can be used to assess similarity:

Metric Hybrid Category
MFCCs False Timbre
MFCCs (Weighted) False Timbre
GFCCs False Timbre
GFCCs (Weighted) False Timbre
Key False Rhythm (Key/Scale)
BPM False Rhythm
OnsetRate False Rhythm
Moods False High-Level
Instruments False High-Level
Dortmund False High-Level (Genre)
Rosamerica False High-Level (Genre)
Tzanetakis False High-Level (Genre)

Note: Hybrid metrics are combinations of multiple metrics. In the future, we hope to integrate more metrics that combine low-level features for a more holistic approach to similarity.

Similarity Statistics

Some of our metrics are normalized using the mean and standard deviation of their features from the lowlevel table. We must compute the statistics for such metrics prior to computing the metrics themselves. To do so, we collect a sample of the lowlevel table and use it to approximate the mean and standard deviation. Currently, the metrics that require statistics are the following:

  • MFCCs
  • Weighted MFCCs
  • GFCCs
  • Weighted GFCCs

Indexing

Metric vectors for each recording are added to static indices, created with the Annoy library. An individual index exists for each of the metrics available. An index uses a nearest neighbours approach for queries.

Note that computing an index takes time, and thus it cannot happen each time a recording is submitted. Indices are recomputed, including new submissions, on a time interval.

More can be read about indices and contributing to similarity work in the developer reference.

Evaluation

The similarity engine is an ongoing project, and its results are still largely experimental. While we tune different index parameters (described in the developer reference), we’d like to gather feedback from the community.

As such, we’ve made an evaluation available to the public. When viewing the summary data for a recording, you may access similar recordings organized by metric. Recordings are organized from most similar to least similar. Alongside the list of the most similar recordings, you may provide input:

  • Rate whether the recording should be higher or lower on the list of similarity, or whether this result seems accurate.
  • Provide additional suggestions related to a specific similar recording, or in general.

Feel free to provide as much or as little feedback as you wish when browsing. We appreciate your help in improving similarity at AcousticBrainz!

Deployment Practices

We run two instances of AcousticBrainz on the same data.

The beta instance is used to test new features and bugfixes on the actual dataset before being officially deployed into production.

We follow the general MetaBrainz guidelines to deploy, test and release code into beta and production.

Developing for Similarity

Setting up the environment

NOTE: If there is no submitted data in the lowlevel table, similarity cannot be initialized.

Once your database and server are built, there are a few steps required to initialize the similarity engine. These steps can be run completed altogether using the following cli command:

./develop.sh manage similarity init

Environment subcommands

The init similarity command runs each of these sub-commands in order. They can be run individually to perform a specific step.

Compute required statistics over the lowlevel table:

./develop.sh manage similarity compute-stats

Extract data values for all recordings in the lowlevel table:

./develop.sh manage similarity add-metrics

This command can be run again at any time to extract values for recordings that have been added since the last time that this command was run.

Finally, build the annoy indices:

./develop.sh manage similarity add-indices

This command will rebuild an index from scratch, reading from the metrics that were extracted from the add-metrics command. This should be run each time after add-metrics to rebuild the similarity indexes.

It is possible to alter the index parameters using additional arguments.

Similarity implementation details

Metrics list

We store a list of metrics in the similarity.similarity_metrics database table. A definition of these each of these metrics is stored in the similarity.metrics python module. These definitions describe which database field to load the metric from, and any preprocessing that has to be performed before adding it to the index.

Statistics

Some features (Weighted MFCCs, Weighted GFCCs) require a mean and standard deviation value to perform normalisation. We can’t compute the mean of all items in the database as this would take too long, so instead we take a random sample (by default similarity.manage.NORMALIZATION_SAMPLE_SIZE, 10000) of items and compute the mean and SD on those items. In our experiments this gave a good tradeoff of accuracy vs computation speed.

Speed of database access and indexing

Reading all items directly from the database (lowlevel_json table) is too slow to do every time the annoy index needs to be updated. Because of this, we split the import process into two steps. The add-metrics subcommand extracts data from the lowlevel table into a new summary table (similarity.similarity) that includes only the specific numerical feature necessary for the similarity index.

The add-indices subcommand reads from the similarity.similarity table and builds an Annoy index. We build this index from scratch each time the command is run due to technical requirements in Annoy, and the fact that this operation doesn’t take too long to complete.

Adding a new feature

(todo: fill in with more detail)

  • Add new metric to admin/sql/populate_metrics_table.sql
  • Add new metric class to similarity.metrics
  • Add this class to similarity.metrics.BASE_METRICS
  • Extract new data if necessary in db.similarity.get_batch_data
  • alter similarity.similarity table to include new column for this feature
    • todo: should we be able to fill in just this column, or do we recreate similarity.similarity?

Index parameters

Indices take a number of parameters when being built. We are currently evaluating them ourselves and have developed a set of base indices, but you may want to experiment with different parameters.

Queries to an index become more precise when the index is built with a higher number of trees, but they will also take longer to build. Indices can also vary in the method used to calculate distance between recordings. Limitations of parameter selection can be found in the Annoy documentation and our codebase

Indices and tables