General

Retrieving NFT metadata typically requires reading individual smart contracts, understanding and working with various encodings and edge-cases, crawling external resources stored on public HTTP or IPFS servers, dealing with often unreliable or unresponsive 3rd party servers. Keeping metadata up-to-date is an even more complex task.

Mnemonic indexes on-chain and off-chain NFT metadata in real-time, normalizes it and keeps it up-to-date by constantly re-indexing changed resources on a global scale of an entire NFT universe.

Understanding Metadata

Metadata is possibly one of the most critical pieces of data for builders creating NFT-related experiences. What people don’t see is just how deceptively difficult it is to fetch and build with NFT metadata, especially at scale. Check out our high level overview about the complexities of building with NFT metadata.

Before we jump into technical details, it is important to outline certain basics first to set the baseline.

Disparate standards and formats

While there are certain standard proposals that are commonly accepted by the developer community, it is not guaranteed that a contract implementation follows these standards.

Particularly, because none of these standards are enforceable it makes it impossible to build any deterministic approach that would yield predicable results.

In essence, an NFT metadata can be any text or binary data stored on-chain or off-chain.

Note: It is also important to note that NFT is not guaranteed to have any metadata at all.

After crawling hundreds of millions of NFTs (210 million NFTs were crawled by Mnemonic at the time of writing), we identified the following most common metadata formats:

  • On-chain JSON, HTML, JavaScript or text documents (plain or base64 encoded);
  • On-chain SVG (plain or base64 encoded);
  • Off-chain JSON, HTML, JavaScript or text documents (plain or base64 encoded);
  • Off-chain SVG (plain or base64 encoded);
  • Off-chain PNG, JPEG, MP4, MP3 and other image/video/audio formats;
  • Off-chain binary data (often either encrypted documents or archives)

Aside from that, there are tens of thousands of malformed JSON documents, URLs, and non-existent or invalid resources (such as a domain where the metadata used to be stored no longer exists).

With Mnemonic API you don't have to worry about any of that, we've got you covered.

Missing metadata

As mentioned above, it is not guaranteed that every NFT has metadata, although, in these cases, there is a valid question to be raised as to whether an NFT is valid in the first place if it lacks metadata.

There are other scenarios in which metadata may be missing or not available via Mnemonic API:

  • The domain where the metadata was initially stored no longer exists or is not accessible due to other reasons.
  • The remote server where the metadata was stored is not available (either it was removed, is faulty, or is not able to serve HTTP requests anymore at a provided host).
  • The tokenURI or URI methods are not implemented by the NFT contract (contract is not following the standard).
  • The contract no longer exists (was destroyed or due to some other reasons).

Mnemonic makes its best attempt to crawl such metadata over a certain period of time by automatically revisiting these resources. However, after several unsuccessful attempts the crawler will abondon these URLs.

We flag such collections as not "trust-worthy".

Dynamic metadata

Some contracts implement dynamic metadata, such that under certain conditions on-chain or by external interraction with the contract the metadata changes.

Most commonly:

  • In-game assets obtain a new property based on the game/character progression.
  • A new property or trait is added by the creators of a collection.
  • A new property or trait is obtained by the holder based on certain conditions.
  • Metadata is being revealed after a collection was fully minted.

This creates a challenge, because it requires constant crawling of the metadata to ensure that it is up-to-date.

Mnemonic constantly crawls ALL NFTs across the blockchains at different paces based on various characteristics to provide the most up-to-date metadata to developers.

Mnemonic schema

Mnemonic normalizes metadata from all the disparate formats into a common schema to ensure consistency for its users.

URI resources

The URIs obtained from the contracts (commonly from tokenURI or URI methods) are normalized into the following schema:

Note: The same URI schema is also used for media resources derrived from the image portion of the JSON-encoded metadata.

Copy
Copied
// UriResource holds details about a resource (such as metadata, image, etc).
//
// A `uri` value in this message is not normalized and is returned as is like it was
// defined in the contract.
//
// Clients should follow the `mime_type` in order to properly handle a resource that
// this message represents.
//
// The `mime_type` holds the mime-type of an enclosed content, and not of an envelope type. Such that,
// if the `uri` represents a `base64` resource (either on-chain or remotely stored on IPFS or HTTP) the
// `mime_type` value will hold the mime-type of the content that is encoded in `base64`. For example, if
// a `uri` is a `base64` of a JSON content, the `mime_type` will be `application/json`.
message UriResource {
    // Original URI (may lead to HTTP(s), IPFS, base64 encoded data, etc).
    string uri = 1;

    // Detected MIME-type of the resource.
    string mime_type = 2;
}

URI normalization

Sometimes, retrieved URIs from the contracts (from tokenURI or URI) may be pinned to some public IPFS gateways or represent a malformed URL (i.e. https://https://somedomain.com., yes, we've seen that too and much more).

All such URIs are automatically cleaned and normalized by the Mnemonic crawler, and served in the correct format in the API.

Particularly, all pinned URIs that point to public IPFS gateways such as https://ipfs.io/ipfs/QmeSjSinHpPnmXmspMjwiXyN6zS4E9zccariGR3jxcaWtq/131 will always be provided as ipfs://QmeSjSinHpPnmXmspMjwiXyN6zS4E9zccariGR3jxcaWtq/131 in the API responses.

Full metadata

The full metadata that is obtained on-chain or off-chain is normalized into the following schema:

Copy
Copied
// TokenFullMetadata holds fully decoded, as well as a raw NFT metadata as defined by the
// `tokenURI` of the NFT contract.
//
// The decoded part of the metadata holds only those fields which are defined as part of the
// token metadata specification (such as ERC721 Metadata JSON Schema, ERC-1155 Metadata URI JSON Schema etc.).

// Some tokens may include additional attributes as part of their JSON metadata,
// in which case the full metadata can be unmarshalled from the provided `raw` field
// in this message, which contains full JSON encoded metadata of the token.
//
// Note: It is not guaranteed that the token has a JSON-encoded metadata.
//  In some cases, the `tokenURI` may contain base64-encoded binary data (such as an image) or
// point to an IPFS or HTTP resource that represents such a file. In this case, only the `token_uri`
// will be populated. Clients should pay attention to the mime-type provided in the `token_uri` in this
// message to properly handle the metadata.
message TokenFullMetadata {
    // Details about original metadata resource returned by the `tokenURI` (ERC-721)
    // or `uri` (ERC-1155) methods of the contract that tracks this NFT.
    mnemonic.types.v1beta1.UriResource metadata_uri = 1;

    // Token name.
    string name = 2;

    // Token description.
    string description = 3;

    // Token image.
    mnemonic.types.v1beta1.UriResource image = 4;

    // Raw metadata as returned by `tokenURI` of the contract (in most cases
    // it is JSON data, but overall it is defined by the contract developers).
    string raw = 5;

    // Timestamp when metadata was indexed.
    google.protobuf.Timestamp indexed_at = 6;
}

Note: All non-binary and non-JSON metadata (such as a plain text, HTML or some other type of document) is provided in the raw field above.

Examples

Below are a few examples of the normalized Metadata provided by the Mnemonic API.

Example (ArtBlocks)

Copy
Copied
{
  "metadata": {
    "metadataUri": {
      "uri": "https://api.artblocks.io/token/235000341",
      "mimeType": "application/json"
    },
    "name": "Maps for grief #341",
    "description": "Maps for Grief explores the hyper-connected existence, filled with individuals who share common intents, ideals and feelings, but tend to live their sentiments in isolation. Its compositions oppose areas of fluid movement and immobility, revealing the structure of the piece itself: two interacting force fields that dictate the direction, as well as the strength – or absence – of movement.\n\nOn the keyboard: Press A to have the lines gently animate. Use the 1/2/3 keys to inspect the fields that make up each iteration.",
    "image": {
      "uri": "https://media.artblocks.io/235000341.png",
      "mimeType": "image/png"
    },
    "raw": "{\"name\": \"Maps for grief #341\", \"image\": \"https://media.artblocks.io/235000341.png\", \"artist\": \"Louis-André Labadie\", \"series\": \"N/A\", \"traits\": [{\"value\": \"All Maps for griefs\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"Coping: Spiritual\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"Radial: No\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"No dots: No\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"Palette: IKEA Showroom\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"Tip size: Fat marker\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"Region count: 3\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"Stroke length: Long\", \"trait_type\": \"Maps for grief\"}, {\"value\": \"Persistent lines: No\", \"trait_type\": \"Maps for grief\"}], \"license\": \"CC BY-NC 4.0\", \"tokenID\": \"235000341\", \"website\": \"https://louis-andre.net/journal/maps-for-grief\", \"features\": {\"Coping\": \"Spiritual\", \"Radial\": \"No\", \"No dots\": \"No\", \"Palette\": \"IKEA Showroom\", \"Tip size\": \"Fat marker\", \"Region count\": 3, \"Stroke length\": \"Long\", \"Persistent lines\": \"No\"}, \"platform\": \"Art Blocks Factory\", \"is_static\": false, \"project_id\": \"235\", \"token_hash\": \"0x2de6d983d544e3081b24f19d6e4c77adf7b0034ec6eadd4fb8c8f9198328ded8\", \"description\": \"Maps for Grief explores the hyper-connected existence, filled with individuals who share common intents, ideals and feelings, but tend to live their sentiments in isolation. Its compositions oppose areas of fluid movement and immobility, revealing the structure of the piece itself: two interacting force fields that dictate the direction, as well as the strength – or absence – of movement.\\n\\nOn the keyboard: Press A to have the lines gently animate. Use the 1/2/3 keys to inspect the fields that make up each iteration.\", \"royaltyInfo\": {\"artistAddress\": \"0x9e7101c7741b919a389fc9495d85843fecafea50\", \"royaltyFeeByID\": \"5\", \"additionalPayee\": \"0x0000000000000000000000000000000000000000\", \"additionalPayeePercentage\": \"0\"}, \"script_type\": \"js\", \"aspect_ratio\": 1.41428571, \"external_url\": \"https://artblocks.io/token/235000341\", \"animation_url\": \"https://generator.artblocks.io/235000341\", \"generator_url\": \"https://generator.artblocks.io/235000341\", \"payout_address\": \"0x6C093Fe8bc59e1e0cAe2Ec10F0B717D3D182056B\", \"collection_name\": \"Maps for grief by Louis-André Labadie\", \"curation_status\": \"factory\", \"interactive_nft\": {\"version\": \"0.0.9\", \"code_uri\": \"https://generator.artblocks.io/235000341\"}}",
    "indexedAt": "2022-01-12T00:40:07.572351Z"
  }
}

Example (BAYC)

Original IPFS URIs provided as defined by the token metadata.

Copy
Copied
{
  "metadata": {
    "metadataUri": {
      "uri": "ipfs://QmeSjSinHpPnmXmspMjwiXyN6zS4E9zccariGR3jxcaWtq/2087",
      "mimeType": "application/json"
    },
    "name": "",
    "description": "",
    "image": {
      "uri": "ipfs://QmYhUX5fjigN2HgGmq3AcEtzVjSX3iR4EjKEoWCMTkwb6g",
      "mimeType": "image/png"
    },
    "raw": "{\"image\": \"ipfs://QmYhUX5fjigN2HgGmq3AcEtzVjSX3iR4EjKEoWCMTkwb6g\", \"attributes\": [{\"value\": \"Bored Cigarette\", \"trait_type\": \"Mouth\"}, {\"value\": \"Purple\", \"trait_type\": \"Background\"}, {\"value\": \"Trippy\", \"trait_type\": \"Fur\"}, {\"value\": \"Angry\", \"trait_type\": \"Eyes\"}]}",
    "indexedAt": "2021-11-05T06:10:17.544354Z"
  }
}

Example (CryptoPunks)

In many cases the metadata, including images, is stored on-chain. CryptoPunks, for example, is encoded as base64 string.

Clients should pay attention to the mimeType property in the response. Mnemonic detects the type of encoded media (on-chain or off-chain) to make it easier for the client to decode and display this data.

Copy
Copied
{
  "metadata": {
    "metadataUri": {
      "uri": "",
      "mimeType": ""
    },
    "name": "321",
    "description": "",
    "image": {
      "uri": "ZGF0YTppbWFnZS9zdmcreG1sO3V0ZjgsPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmci...<trimmed>",
      "mimeType": "image/svg+xml;base64"
    },
    "raw": "{\"Name\": \"321\", \"Image\": \"ZGF0YTppbWFnZS9zdmcreG1sO3V0ZjgsPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmci...<trimmed>\", \"Attributes\": [{\"Value\": \"Male 2\", \"TraitType\": \"Head type\"}, {\"Value\": \" Peak Spike\", \"TraitType\": \"Feature\"}], \"Description\": \"\"}",
    "indexedAt": "2021-12-11T02:34:54.639471Z"
  }
}

Media caching

Ensuring NFT media loads quickly and reliably within your product can be tricky because most NFTs are stored and served by IPFS or third party servers. To eliminate slow load times and potential time-outs or other media accessibility issues, Mnemonic caches NFT media and serves it through a CDN so you can provide the best possible experience to your users.

Our endpoints that return NFT media in the response return both cached media URIs (when available) and original URIs

  • Supported media types: .png, .gif, .webp, .jpg, .svg
  • Media sizing: Returned cached media has a default maximum dimension of 1600px.
  • Chains: We currently return cached media on our Ethereum endpoints, but will be releasing media caching for our Polygon endpoints in the near future.

Metadata freshness

Mnemonic's crawler uses various heuristics to identify when metadata needs to be re-indexed. The most common use case is when metadata is revealed only after the entire collection has been minted, or in a game where an object has obtained some new traits or properties.

Mnemonic constantly crawls metadata changes to deliver the most up to date results.

Note: In beta the expected SLA on updated metadata for reveals is up to 24 hr. For revealed metadata, Mnemonic delivers real-time updates.