Deploy Ocean Subgraph

About Ocean Subgraph

Ocean subgraph allows querying the datatoken, data NFT, and all event information using GraphQL. Hosting the Ocean subgraph saves the cost and time required in querying the data directly from the blockchain. The steps in this tutorial will explain how to host Ocean subgraph for the EVM-compatible chains supported by Ocean Protocol.

Ocean Subgraph is deployed on top of graph-node, therefore, in this document, we will show first how to deploy graph-node - either using Docker Engine or Kubernetes - and then how to install Ocean Subgraph on the graph-node system.

Deploying Graph-node using Docker Engine and Docker Compose

Prerequisites

A server for hosting Graph-node. See this guide for how to create a server;
Docker Compose and Docker Engine are installed and configured on the server. See this guide for how to install these products.
The RPC URLs and API keys for each of the networks to which Ocean Subgraph will be connected. See this guide for how to obtain the URL and the API key.

Steps

1. Create the /etc/docker/compose/graph-node/docker-compose.yml file

From a terminal console, create the /etc/docker/compose/graph-node/docker-compose.yml file, then copy and paste the following content to it (. Check the comments in the file and replace the fields with the specific values of your implementation.

/etc/docker/compose/graph-node/docker-compose.yml (annotated - example for mumbai network)

version: '3'
services:
  graph-node:
    image: graphprotocol/graph-node:v0.28.2
    container_name: graph-node
    restart: on-failure
    ports:
      - '8000:8000'
      - '8020:8020'
      - '8030:8030'
      - '8040:8040'
    depends_on:
      - ipfs
      - postgres-graph
    environment:
      postgres_host: postgres-graph
      postgres_user: graph-node
      postgres_pass: < password >
      postgres_db: mumbai
      ipfs: 'ipfs:5001'
      ethereum: 'mumbai:https://polygon-mumbai.infura.io/v3/< INFURA ID >'
      GRAPH_LOG: info
  ipfs:
    image: ipfs/go-ipfs:v0.4.23
    container_name: ipfs
    restart: on-failure
    ports:
      - '5001:5001'
    volumes:
      - ipfs-graph-node:/data/ipfs
  postgres-graph:
    image: postgres:15.3
    container_name: postgres
    restart: on-failure
    ports:
      - '5432:5432'
    command: ["postgres", "-cshared_preload_libraries=pg_stat_statements"]
    environment:
      POSTGRES_USER: graph-node
      POSTGRES_PASSWORD: < password >
      POSTGRES_DB: mumbai
    volumes:
      - pgdata-graph-node:/var/lib/postgresql/data
volumes:
  pgdata-graph-node:
    driver: local
  ipfs-graph-node:
    driver: local

2. Create the /etc/systemd/system/[email protected] file

Create the /etc/systemd/system/[email protected] file then copy and paste the following content to it. This example file could be customized if needed.

[Unit]
Description=%i service with docker compose
Requires=docker.service
After=docker.service

[Service]
Type=oneshot
RemainAfterExit=true
Environment="PROJECT=ocean"
WorkingDirectory=/etc/docker/compose/%i
ExecStartPre=/usr/bin/env docker-compose -p $PROJECT pull
ExecStart=/usr/bin/env docker-compose -p $PROJECT up -d
ExecStop=/usr/bin/env docker-compose -p $PROJECT stop
ExecStopPost=/usr/bin/env docker-compose -p $PROJECT down


[Install]
WantedBy=multi-user.target

3. Reload the systemd manager configuration

Run the following command to reload the systemd manager configuration

sudo systemctl daemon-reload

Optionally, you can enable the services to start at boot, using the following command:

sudo systemctl enable [email protected]

4. Start graph-node service

To start the Ocean Subgraph service, run the following command:

sudo systemctl start [email protected]

5. Check the service's status

Check the status of the service by running the following command. The output of the command should be similar to the one presented here.

$ sudo systemctl status [email protected]
● [email protected] - graph-node service with docker compose
     Loaded: loaded (/etc/systemd/system/[email protected]; disabled; vendor preset: enabled)
     Active: active (exited) since Sun 2023-06-25 17:05:25 UTC; 6s ago
    Process: 4878 ExecStartPre=/usr/bin/env docker-compose -p $PROJECT pull (code=exited, status=0/SUCCESS)
    Process: 4887 ExecStart=/usr/bin/env docker-compose -p $PROJECT up -d (code=exited, status=0/SUCCESS)
   Main PID: 4887 (code=exited, status=0/SUCCESS)
        CPU: 123ms

Jun 25 17:05:24 testvm env[4887]:  Container ipfs  Created
Jun 25 17:05:24 testvm env[4887]:  Container graph-node  Creating
Jun 25 17:05:24 testvm env[4887]:  Container graph-node  Created
Jun 25 17:05:24 testvm env[4887]:  Container ipfs  Starting
Jun 25 17:05:24 testvm env[4887]:  Container postgres  Starting
Jun 25 17:05:24 testvm env[4887]:  Container ipfs  Started
Jun 25 17:05:25 testvm env[4887]:  Container postgres  Started
Jun 25 17:05:25 testvm env[4887]:  Container graph-node  Starting
Jun 25 17:05:25 testvm env[4887]:  Container graph-node  Started
Jun 25 17:05:25 testvm systemd[1]: Finished graph-node service with docker compose.

6. Check graph-node service logs

If needed, use docker CLI to check Ocean Subgraph service logs.

First, check the container status

$ docker ps --format "table {{.Image}}\t{{.Ports}}\t{{.Names}}\t{{.Status}}"
IMAGE                              PORTS                                                                                                                                                                                  NAMES        STATUS
graphprotocol/graph-node:v0.28.2   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 0.0.0.0:8020->8020/tcp, :::8020->8020/tcp, 0.0.0.0:8030->8030/tcp, :::8030->8030/tcp, 0.0.0.0:8040->8040/tcp, :::8040->8040/tcp, 8001/tcp   graph-node   Up 55 minutes
ipfs/go-ipfs:v0.4.23               4001/tcp, 8080-8081/tcp, 0.0.0.0:5001->5001/tcp, :::5001->5001/tcp                                                                                                                     ipfs         Up 55 minutes
postgres:15.3                      0.0.0.0:5432->5432/tcp, :::5432->5432/tcp                                                                                                                                              postgres     Up 55 minutes

Then, check the logs of the Ocean Subgraph docker container:

docker logs graph-node  [--follow]

Deploying graph-node using Kubernetes

In this example, we will deploy graph-node as a Kubernetes deployment service. graph-node has the following dependencies: PostgreSQL and IPFS.

Prerequisites:

A server for hosting graph-node. See this guide for how to create a server;
Kubernetes with Docker Engine is installed and configured on the server. See this chapter for information on installing Kubernetes.
The RPC URLs and API keys for each of the networks to which the Provider will be connected. See this guide for how to obtain the URL and the API key.

Steps

1. Deploy PostgreSQL

It is recommended to deploy PostgreSQL as helm chart.

References: https://github.com/bitnami/charts/tree/main/bitnami/postgresql/#installing-the-chart

Once PostgreSQL pods are running, a database must be created: eg. mumbai.

2. Deploy IPFS

The following template can be customized to deploy IPFS statefulset and service.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: ipfs
  name: ipfs
spec:
  podManagementPolicy: OrderedReady
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: ipfs
  serviceName: ipfs
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: ipfs
    spec:
      containers:
      - image: ipfs/go-ipfs:v0.4.22
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /debug/metrics/prometheus
            port: api
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        name: s1-ipfs
        ports:
        - containerPort: 5001
          name: api
          protocol: TCP
        - containerPort: 8080
          name: gateway
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /debug/metrics/prometheus
            port: api
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /data/ipfs
          name: ipfs-storage
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1000
        runAsUser: 1000
      terminationGracePeriodSeconds: 30
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      creationTimestamp: null
      name: ipfs-storage
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 1G
      volumeMode: Filesystem
    status:
      phase: Pending
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: ipfs
  name: ipfs
spec:
  clusterIP:
  clusterIPs:
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: api
    port: 5001
  - name: gateway
    port: 8080
  selector:
    app: ipf

Deploy Graph-node

The following annotated templated can be customized to deploy graph-node deployment and service:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
  labels:
    app: mumbai-graph-node
  name: mumbai-graph-node
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: mumbai-graph-node
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: mumbai-graph-node
    spec:
      containers:
      - env:
        - name: ipfs
          value: ipfs.<namespace>.svc.cluster.local:5001
        - name: postgres_host
          value: postgresql.<namespace>.svc.cluster.local
        - name: postgres_user
          value: < postgresql user >
        - name: postgres_pass
          value: < postgresql database password >
        - name: postgres_db
          value: < postgresql database >
        - name: ethereum
          value: mumbai:https://polygon-mumbai.infura.io/v3/< INFURA ID>
        - name: GRAPH_KILL_IF_UNRESPONSIVE
          value: "true"
        image: graphprotocol/graph-node:v0.28.2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 8000
            scheme: HTTP
          initialDelaySeconds: 20
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: mumbai-graph-node
        ports:
        - containerPort: 8000
          name: graphql
          protocol: TCP
        - containerPort: 8020
          name: jsonrpc
          protocol: TCP
        - containerPort: 8030
          name: indexnode
          protocol: TCP
        - containerPort: 8040
          name: metrics
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 8000
            scheme: HTTP
          initialDelaySeconds: 20
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          limits:
            cpu: "2"
            memory: 1536Mi
          requests:
            cpu: 1500m
            memory: 1536Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: mumbai-graph-node
  name:  mumbai-graph-node
spec:
  clusterIP:
  clusterIPs:
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: graphql
    port: 8000
  - name: jsonrpc
    port: 8020
  - name: indexnode
    port: 8030
  - name: metrics
    port: 8040
  selector:
    app: mumbai-graph-nodeyam