The indexer is built on a service-based architecture. Services are connected to each other through queues. All services are connected to a Postgres database, which is used to store the indexed data and also acts as the backend of the queuing system. Here is a diagram that visualizes the architecture of the indexer:
The controller service subscribes to the TzKt WebSocket API and gets notified every time a new block is created. When that happens, it adds a new job to the event-producer queue signaling the TezTok system that there is a new block that needs to be processed.
The event producer service handles the jobs from the event-producer queue. There is one job per block. This is how the event producer processes a block:
The purpose of the rebuild service is to update data models. It processes the jobs from the rebuild-queue. For Example: Once a sale of a token happens, the event producer service will first create a TezTok event that reflects that sale. After that, it will create a rebuild job for the token. The rebuild service then fetches all TezTok events related to that token in chronological order and then re-creates any data models associated with that token. This includes the token model itself, listings, tags, royalty receivers, offers, and holdings.
The fetch-metadata service is responsible for fetching token metadata from IPFS. It receives its jobs from the fetch-metadata queue. After fetching metadata, it adds jobs to the rebuild queue that will re-create the tokens where the metadata is used.
The indexer uses Hasura to provide a GraphQL API. Read more about Hasura here.