Health checks explained
Every health check the Home page can show, what it means, and what to do about it.
The Admin keeps checking the health of your AVADO while you have it open. When a check finds a problem, it shows up on Home with a short explanation and a fix button. The most urgent problem becomes the headline. The problems for one app also appear on that app's Overview tab.
Each problem has one of three levels:
Critical
Action required (red)
Fix this now. Often your validators are missing rewards.
Warning
Needs attention (amber)
Fix this soon.
Tip
Tip
A suggestion. You can hide most tips with Hide.
If nothing is wrong, Home says Your AVADO is healthy. and the status light is green.
In the titles below, App stands for the name of your app, for example "Nimbus" or "Geth".
Staking setup
App has no execution client (critical)
You run a consensus client, but no execution client for the same network. A consensus client needs one to follow the chain. Until you install one, your validators cannot attest and miss rewards.
Click Install an execution client and install Geth or Nethermind. See Setting up the ETH Clients.
App has no consensus client (warning)
You run an execution client, but no consensus client for the same network. Since the Merge, an execution client cannot follow the chain on its own.
Click Install a consensus client and install Teku, Prysm or Nimbus.
Apps
App is stopped (critical for a client, warning for other apps)
The app is not running. A stopped client does not follow the chain, so your validators miss attestations and rewards.
Click Start it. If it stops again, click See why in the logs.
App keeps restarting (critical)
The app starts, fails and starts again.
Click Open the logs and look at the last error before the restart. If it says the disk is full, free space in System → Storage. If it mentions a setting you changed, switch to Advanced mode and undo the change in the app's Settings tab. Still restarting? Download the diagnostics report in Help and send it to support.
System service App is not running (critical)
One of your AVADO's own system services is down. Your AVADO needs them to install, update and reach apps.
Click Open System (Advanced mode) and restart the stopped service. If that doesn't help, reboot your AVADO from System.
Chain and sync
App is syncing (N%) (tip)
Your client is catching up with the chain. Validators can only attest once it is synced.
Nothing. Wait until it is done. A first sync can take hours to days.
App can't be reached (warning)
Your AVADO couldn't ask this client how far it has synced. If the client isn't answering, your validators may miss attestations.
Open the app and check that it is running. If it was just started or updated, give it a few minutes. Still can't be reached? Restart it and look at the last lines of its logs.
App is N slots behind the chain (warning, needs monitoring)
The client is running but not keeping up with the network, so your validators may miss attestations.
Check that your execution client is running and synced. Check the peer count: fewer than 10 peers makes a client fall behind. Restart the client; if it falls behind again, open its logs.
App has only N peers (warning, needs monitoring)
Your client is connected to fewer than 10 other nodes. It hears about new blocks late and can fall behind or miss attestations.
Click Improve connectivity. Usually you need to open the peer-to-peer ports on your router. See Opening Network Ports.
Attestations and monitoring
N attestations missed in the last hour (warning, critical when more than half are missed)
Each missed attestation costs a small reward. A few are normal. A steady stream means something needs fixing.
Click Find the cause. Check that both clients are synced, that you have enough peers, and that the box clock is right. See Attestation Troubleshooting.
Install monitoring to check missed attestations (tip)
You run a consensus client but no monitoring. Monitoring lets your AVADO warn you about missed attestations, low peers or falling behind.
Click Install monitoring. See Monitoring with Grafana.
Monitoring has stopped (warning)
The monitoring package (Prometheus) isn't running, so missed attestations, peers and sync can't be checked.
Click Restart monitoring.
Can't read your clients' metrics (tip)
Monitoring is installed but the Admin couldn't reach Prometheus. This also happens when you open the Admin through Remote Connect or by IP address instead of http://my.ava.do.
Open the Admin at http://my.ava.do. Otherwise check in System that Prometheus is running, and restart it.
Updates
N app updates are available (warning)
Newer versions of some apps are available. Updates fix bugs and keep clients compatible with network upgrades. A client that misses a hard fork stops following the chain.
Click Review updates and update the apps. See Updates, storage and history.
An AVADO system update is available (warning)
A new version of AVADO's own system software is available.
Click Review the update, then Update system. Keep your AVADO powered on during the update.
Automatic updates are off for App (tip)
Auto-update is switched off for one of your clients, so you have to install updates yourself, in time.
Click Turn on automatic updates and switch on Auto-update for the client in My DApps.
Can't check for updates (tip)
Your AVADO could not reach the AVADO store. Your apps keep running.
Check that the network cable is plugged in and that your router has internet. Restart your router if other devices are offline too. The check runs again every 10 minutes.
Access and network
Your router is not forwarding the ports your clients need (warning)
Without open peer-to-peer ports your clients find fewer peers. They then follow the chain more slowly and can miss attestations.
Turn on UPnP in your router's settings; your AVADO then opens the ports itself. If your router has no UPnP, forward the ports shown on each client's Overview tab to your AVADO's internal IP. See Opening Network Ports.
UPnP is not available on your router (tip)
Your AVADO can't open ports by itself, so you forward them by hand, once.
Click Read how, or see Opening Network Ports.
Use the internal address at home (tip)
Your router does not route its public address back into your home network.
At home, open your AVADO at http://my.ava.do or its internal IP (shown in the box panel at the bottom of the sidebar).
Set up remote access (tip)
Neither Remote Connect nor the VPN is installed, so you can't reach your AVADO when you are away.
Click Install Remote Connect. See Set up Remote Connect + ZeroTier.
Disk space
Your disk is N% full (warning from 80%, critical from 90%)
Clients keep growing. When the disk is full, your clients stop and your validators go offline. The message names the apps that use the most space.
Click Free up space. In System → Storage you can clean up unused images and see which app uses the most. See Updates, storage and history.
App is using N GB (warning)
A consensus client is over 200 GB, which is more than normal. It can fill your disk.
Click Free up space. Only the chain data is deleted; your validator keys stay. See Free up space on your consensus client.
System checks
These come from built-in system checks. When one fails, the Admin shows the check's own message and the steps to fix it.
Session is closed
Your browser lost its connection to your AVADO.
Check your connection (Wi-Fi, home network or Remote Connect) and reload the page. If you use Remote Connect, disconnect and connect again. See Connection Troubleshooting.
DAPPMANAGER is not connected
The AVADO service that installs and manages apps doesn't answer.
Reconnect and reload the page. If the problem stays, reboot your AVADO.
IPFS is not resolving
Your AVADO can't download packages, so installs and updates fail.
In System, check that IPFS is running and restart it. Also make sure the disk is not full.
Still stuck?
Open Help in the sidebar for step-by-step topics. See Help and troubleshooting in the Admin.
Download the diagnostics report from Help and send it with your question.
Priority Care subscribers get a reply within 8 business hours when their box is down, and an email when these checks find a serious problem.
Last updated