Skip to main content

Running TezGraph Using Docker-Compose

This guide serves to help a newcomer to deploy a Tezos Indexer and TezGraph API in 4 commands or less. The only requirement for the user would be to have a machine with Docker and Docker-compose installed. The process involves downloading a docker-compose file and a dump of an already indexed database, bringing up the stack with docker-compose, and testing the TezGraph API.

Tezos RPC node​

This guide uses a public Tezos node, but you could target a different Tezos node by modifying the TEZOS_NODE variable in the docker-compose.yml file after it has been downloaded.

Prerequisites​

  • Docker
  • Docker Compose
  • Disk Space - ~1GB for Limanet, ~10GB for Kathmandunet, ~80GB for Ghostnet, ~580GB for Mainnet
  • Tezos RPC node

Steps to run TezGraph locally​

  1. Create a new directory and change to that directory

    mkdir tezgraph-deploy && cd tezgraph-deploy

  2. Download a database dump and a docker-compose.yml file. The docker-compose.yml file will be different based on the Tezos network and whether you want the API to cater to query the database or not. Choose one of the following commands based on your use case.

curl -LJo docker-compose.yml \
https://gitlab.com/tezgraph/tezgraph/-/raw/master/examples/mainnet-docker-compose.yml \
    && curl -LJo indexer-db-dump.tar https://storage.googleapis.com/tezgraph-db-snaps/$(curl https://storage.googleapis.com/tezgraph-db-snaps/indexer-v10-2-4-mainnet.json | jq -r '.latest') \
    && mkdir indexer-db-dump && tar -xvf indexer-db-dump.tar -C indexer-db-dump

  1. Startup the Indexer, Database, and API using docker-compose and follow the progress through the console logs. The database setup may require a significant amount of time. You will see errors in the console that can be ignored—more details can be found in the Pitfalls and Known Issues section below.

    docker-compose up

    A good indication of the stack being ready and waiting for tests is when you see something similar to the following in the console:

    indexer_database           | LOG:  starting PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
    indexer_database           | LOG:  listening on IPv4 address "0.0.0.0", port 5432
    indexer_database           | LOG:  listening on IPv6 address "::", port 5432
    indexer_database           | LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
    indexer_database           | LOG:  database system was shut down at 2020-12-15 05:54:49 UTC
    indexer_database           | LOG:  database system is ready to accept connection

Testing​

Once the stack is up and running you can navigate to https://localhost:3000/graphql. The page will look like this:

TezGraph Local Host 3000

You can then use the TezGraph API to execute GraphQL queries against the database. An example of a test on Mainnet would be:

query {
  accounts(
    first: 1
    filter: { addresses: ["tz1MBidfvWhJ64MuJapKExcP5SV4HQWyiJwS"] }
  ) {
    edges {
      node {
        operations(
          first: 10
          filter: { kind: transaction, relationship_type: source }
        ) {
          edges {
            node {
              hash
              block {
                level
              }
              source {
                address
              }
              kind
            }
          }
        }
      }
    }
  }
}

Security Considerations​

TezGraph exposes a read-only database built using the public data, which is already publicly available on the Tezos blockchain. Therefore, confidentiality is generally not a concern in a TezGraph deployment. That said, information exposed by TezGraph can be used as the basis for financial decisions or evidence of a transaction. Therefore, other aspects of information security are still relevant, namely: integrity and availability.

The general information security practices apply to TezGraph as well. These also depend on your infrastructure, so here is a non-exhaustive list of things to pay extra attention to:

  • Ensure you follow the best practices for secrets in your infrastructure for DATABASE_CONNECTION_STRING, REDIS_CONNECTION_STRING, and if you use it, API_KEY.
  • GraphQL queries can be made prohibitively complex and costly to run. A malicious actor can use this to compromise the availability of your TezGraph instance. If you make your TezGraph instance publicly available, ensure you have predefined limits (GRAPH_QL_DEPTH_LIMIT, GRAPH_QL_COMPLEXITY_LIMIT, PAGE_SIZE_LIMIT). Setting an acceptable value for statement_timeout in PostgreSQL can also be helpful.
  • Closely monitor the last indexed block and how much it lags behind the current height of the blockchain. Out-of-date data (like the current balance of an account) can be misleading to users.

Pitfalls and Known Issues​

  • The most important thing to note is that the database will take time to restore from the backup. The database restore time can differ depending on the Tezos network, CPU, or RAM on the host, but the approximate time is 10 minutes for Kathmandunet, and 6 hours for Mainnet. During this time, you will not be able to execute queries against the database. When you try and execute a query, you will receive a response similar to:

    {
      "errors": [
        {
          "message": "\nInvalid `prisma.queryRaw()` invocation:\n\n\n  Can't reach database server at `indexer_database`:`5432`\n\nPlease make sure your database server is running at `indexer_database`:`5432`.",
          "locations": [
            {
              "line": 2,
              "column": 3
            }
          ],
          "path": [
            "account"
          ],
          "extensions": {
            "code": "INTERNAL_SERVER_ERROR",
            "exception": {
              "clientVersion": "2.12.1",
              "stacktrace": [
                "Error: ",
                "Invalid `prisma.queryRaw()` invocation:",
                "",
                "",
                "  Can't reach database server at `indexer_database`:`5432`",
                "",
                "Please make sure your database server is running at `indexer_database`:`5432`.",
                "    at PrismaClientFetcher.request (/app/node_modules/@prisma/client/runtime/index.js:79361:15)",
                "    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
              ]
            }
          }
        }
      ],
      "data": null
    }

  • Sometimes, after the stack has started, the TezGraph container is still unable to connect to the database container. In that case, you will see an error similar to the one above. The easiest way to remedy this issue is to bring the stack down by typing ctrl + c, waiting for the containers to stop gracefully, and then bringing the stack up again using the docker-compose up command.

  • Here are some of the errors in the console that can safely be ignored while the stack is starting up and the database is being restored.

indexer_database           |  ERROR:  canceling autovacuum task
indexer_database           |  CONTEXT:  while scanning block 190427 of relation "public.balance"
indexer_database           |    automatic vacuum of table "indexer.public.balance"
indexer_database           | LOG:  checkpoints are occurring too frequently (22 seconds apart)
indexer_database           | HINT:  Consider increasing the configuration parameter "max_wal_size".
indexer          | Database connection error: Failed to connect to <postgresql://indexer:_@indexer_database:5432/indexer?sslmode=disable>: Connection failure: could not connect to server: Connection refused
indexer          |  Is the server running on host "indexer_database" (XXX.XXX.XXX.XXX) and accepting
indexer          |  TCP/IP connections on port 5432?
indexer          |
indexer          | To get more information about specifying how to connect to the database, run:
indexer          | /indexer/tezos-indexer --help
indexer exited with code 3

This error will keep repeating until the database becomes available and can be ignored until then.

  • If there are any issues that are not listed here or that you are unable to overcome, you can restart the process by bringing down the stack and deleting the volumes using the following command:
docker-compose down -v