Performance monitoring

What to watch on your node, and how to tune it. Uptime and task completion feed your TrustFingerprint score and your share of node rewards.

Rewards are not being paid yet. See Testnet Status.

Recommended tools

• Prometheus: metrics collection and storage
• Grafana: dashboards
• Alertmanager: alert routing
• Node Exporter: system metrics

Targets

What the score measures

1. Uptime (40% weight): historical availability
2. Performance (30% weight): task completion speed and accuracy
3. Reliability (20% weight): consistency over time
4. Participation (10% weight): governance and network engagement

Dashboards

• System: CPU usage by core, memory and swap, disk I/O and space, network traffic, system load average
• Node: uptime, task completion rate, reward earnings, TrustFingerprint score, peer connections, block and transaction processing

Alerts

Set alerts for high CPU or memory usage, low disk space, network connectivity issues, node offline, low TrustFingerprint score and missed tasks.

• Critical, act now: node offline >5 minutes, CPU >95% for >10 minutes, memory >98%, disk space <5%, network disconnected
• Warning, investigate: CPU >80% for >1 hour, memory >85%, disk space <20%, TrustFingerprint score declining, task completion rate <95%
• Info, awareness only: software updates available, reward distribution completed, governance proposals active, network announcements

Tuning

# Check CPU frequency scaling
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
# Set to performance modeecho performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# Adjust swappiness
sudo sysctl vm.swappiness=10
# Clear cache if needed
sudo sync && sudo sysctl -w vm.drop_caches=3

# Increase network buffer sizes
sudo sysctl -w net.core.rmem_max=134217728
sudo sysctl -w net.core.wmem_max=134217728

# Enable TRIM for SSDs
sudo fstrim -v /
# Check disk health
sudo smartctl -a /dev/sda

Fix common problems

• High CPU usage: find the process with top or htop. Check for runaway processes and verify the node configuration. Consider a hardware upgrade.
• Memory leak: watch memory over time. Restart the node if it grows continuously. Check for software updates and report the issue if it persists.
• Network latency: test with ping 8.8.8.8 and speedtest-cli. Verify the router configuration. Consider an ISP upgrade.
• Low TrustFingerprint score: review historical performance, find the periods of downtime, check your task completion rate and improve system reliability.

Routine

• Daily: check the dashboard for alerts, verify the node is online, review overnight performance, check reward earnings.
• Weekly: review performance trends, update software if needed, check disk space, back up configuration.
• Monthly: analyze TrustFingerprint trends, tune the system, review alerts, plan hardware upgrades, rerun the benchmarks below.

Benchmarks

Record a baseline after setup. Run it again monthly to detect degradation.

# CPU benchmark
sysbench cpu run
# Memory benchmark
sysbench memory run
# Disk benchmark
fio --name=random-write --ioengine=libaio --rw=randwrite --bs=4k --size=1G
# Network benchmark
iperf3 -c speedtest.net

Report an issue

Collect diagnostics:

# System info
uname -a
lscpu
free -h
df -h
# Node logs
journalctl -u noderr-node -n 1000 > node-logs.txt
# Metrics export
curl http://localhost:8080/metrics > metrics.txt

Then report in Discord or on GitHub.

Next

• Setup Guide
• Troubleshooting
• Compare the tiers