Troubleshooting
Start here when a user-visible workflow fails. Check health first, then narrow by wallet, cache, task, or provider signals.
First Checks
curl http://127.0.0.1:9090/healthz
synaps3 admin status
synaps3 admin task statsExpected healthy baseline:
{"status":"ok"}If health is not ok, use the error text as the next branch.
Setup Mode
Health may return:
{"status":"setup"}This means SynapS3 started in setup mode because required settings are missing, usually filecoin.private_key.
Fix:
synaps3 wallet generateSet the generated private key in SYNAPS3_FILECOIN_PRIVATE_KEY or filecoin.private_key in the config file, then restart SynapS3.
Expected result: health changes from setup to ok after restart.
Unhealthy Worker
Example:
{"status":"unhealthy","errors":["worker/uploader: not responding"]}Check task pressure:
synaps3 admin task stats
synaps3 admin task list --status running --limit 20If the process recently restarted, stale leases should be released during startup recovery. If the worker stays unhealthy, inspect logs and restart the service after capturing the current task state.
Wallet Funding or Deposit Fails
Check wallet status:
synaps3 admin statusFor Calibration, fund the wallet address again:
synaps3 wallet fund-testnet 0x...Then retry deposit:
synaps3 wallet deposit 2 # 2 USDFCIf faucet funding fails, claim manually from ChainSafe or Plumbline, then rerun synaps3 admin status.
Cache Full
Upload endpoints can fail when cache capacity is exhausted. Check usage:
synaps3 admin status
synaps3 admin settings get cache.max_size_gbRecovery options:
- Increase
cache.max_size_gbif disk capacity allows. - Restore provider or worker connectivity so queued uploads can complete and eviction can run.
- Keep
cache.eviction_policy = "lru"if local cache cleanup should be queued after remote storage succeeds.
Exhausted Tasks
List exhausted work:
synaps3 admin task list --status exhausted --limit 100Retry only after the underlying dependency is fixed. Typical dependencies are RPC connectivity, provider availability, wallet balance, or cache disk capacity.
synaps3 admin task retry 42Provider or RPC Issues
Check provider health and readiness in the dashboard, or inspect the Admin API:
curl http://127.0.0.1:9090/api/v1/filecoin/readiness
curl http://127.0.0.1:9090/api/v1/observability/providersRecovery:
- Restore the configured
filecoin.rpc_url. - Confirm provider URLs are reachable from the SynapS3 host.
- Keep
filecoin.allow_private_networks = falseunless private provider URLs are expected and trusted.
S3 Client Cannot Upload
Check these in order:
- S3 client uses path-style addressing.
- Access key and secret came from
synaps3 admin s3-user create. - Endpoint is
http://localhost:8080or the correct remote S3 address. - Test object is at least 127 bytes.
- Dashboard task view shows whether Filecoin storage is queued, running, or exhausted.