Files
IPFS-portal/ARCHITECTURE.md
T
maikrolf bac719723d docs: ARCHITECTURE.md bijwerken naar de huidige generatie
- Gateway-host op regel 18 en in Configuration: maos.dedyn.io -> ipfs.maos.dedyn.io
  (config.ts gatewayUrl() en de CSP in next.config.ts)
- Upload-flow (quick + crypto): proxyIPFS -> handleLocalUpload() -> .data/files/
  met auto-pin, i.p.v. Kubo /api/v0/add
- Nieuwe sectie 'Lokale opslag (.data/) en de gateway-route': endpoint-tabel voor
  pins/files, opslaglayout, het feit dat een lokale CID geen echte IPFS-CID is,
  en GET /ipfs/<cid> inclusief content-type, Content-Disposition en foutpaden
- src/lib/api.ts (bestaat niet meer) -> src/lib/api/ als module-tabel; het
  achtergebleven api.ts.bak benoemd
- route.ts: ~320 -> 121 regels, met verwijzing naar proxy-ipfs.ts en write-auth.ts
- mapToKuboAPI-lijst: dode mappings /pins en /files/upload weg, met de reden
- Pages-tabel en API-flow noemden pins/files nog als Kubo-afhankelijk
- Schema en het bestaande write-auth-gedeelte ongemoeid
- Warnt expliciet dat .data/ nog niet in .gitignore staat
2026-10-01 21:53:08 +02:00

426 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# IPFS Portal — Architecture
## Overview
IPFS Portal is a **Next.js SPA** (Single Page Application) that provides a browser-based interface for managing an IPFS node, uploading files, and handling crypto payments via a smart contract. All pages are `'use client'` — no server-side rendering.
### Stack
| Layer | Technology |
|---|---|
| Frontend | Next.js 14+ (React, `'use client'` SPA) |
| Styling | Tailwind CSS + custom `glass`/`surface` design tokens |
| Icons | Lucide React |
| Wallet | EIP-6963 + viem |
| Blockchain | zkSync Local (chain 270) |
| IPFS | Kubo RPC API (port 5001) |
| User Mgmt | Python backend (port 8444) |
| IPFS Gateway | `https://ipfs.maos.dedyn.io/ipfs/<CID>` |
---
## System Architecture
```
BROWSER (React SPA)
│
┌─────────────────────┼──────────────────────┐
│ │ │
▼ ▼ ▼
11 Pages (SPA) WalletProvider PaymentService
┌──────────────┐ (EIP-6963) (viem, chain 270)
│ apiFetch() │ │ │
│ wrapper │ │ │
└──────┬───────┘ │ │
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────┐
│ Next.js API Route: [...path]/route.ts │
│ │
│ matchRoute(path) → │
│ 'health' → inline JSON │
│ 'users' → proxyUserAPI() → :8444 (Python) │
│ default → proxyIPFS() → :5001 (Kubo) │
│ pins/files → lokale .data-opslag (nooit Kubo) │
│ │
│ mapToKuboAPI(path) → /api/v0/* translation │
└─────────────────────┬───────────────────────────────┘
│
┌─────────────┴─────────────┐
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Kubo IPFS │ │ Python User API │
│ Daemon │ │ (port 8444) │
│ (port 5001) │ │ │
│ │ │ htpasswd mgmt │
│ /api/v0/id │ │ Auth: X-Admin- │
│ /api/v0/swarm │ │ Key header │
│ /api/v0/pin/* │ │ │
│ /api/v0/add │ └──────────────────┘
│ /api/v0/ls │
│ /api/v0/name/ │
│ /api/v0/repo │ CHAIN 270
│ /api/v0/bw │ ┌──────────────────┐
└─────────────────┘ │ Smart Contract │
│ 0xCBc6... │
│ zkSync Local │
│ RPC :3050 │
│ │
│ payWithETH() │
│ payWithToken() │
│ getPrice() │
│ configureToken()│
└──────────────────┘
```
---
## Pages
| Page | Route | Lines | Backend Dependencies | Description |
|---|---|---|---|---|
| Landing | `/` | ~130 | Kubo (health) | Hero + connect wallet + quick stats |
| Dashboard | `/dashboard` | 253 | Kubo (peers, repo/stats), inline health, lokale pins | Node overview, stats, bandwidth |
| Upload | `/upload` | **590** | Lokale `.data`-opslag + Smart Contract | Quick + crypto file upload |
| Explorer | `/explorer` | 244 | Kubo (ls, cat) + lokale pins | IPFS content browser |
| IPNS | `/ipns` | 285 | Kubo (keys, publish, resolve) | IPNS name management |
| Users | `/users` | 275 | Python API + Smart Contract | htpasswd users + wallet lookup |
| Peers | `/peers` | 63 | Kubo (swarm/peers) | Connected peers list |
| Pins | `/pins` | 167 | Lokale `.data/pins.json` | Pin management |
| Admin/Payment | `/admin/payment` | **536** | Smart Contract | Owner payment dashboard |
| Settings | `/settings` | 319 | localStorage only | App configuration |
| History | `/history` | 393 | localStorage only | Upload history viewer |
**Note**: IPNS page does NOT wrap in `PortalLayout` — appears standalone. All other pages use the shared layout.
---
## Data Flow
### 1. API Flow (all pages except Settings, History)
```
Page Component
→ apiFetch('/api/endpoint')
→ fetch('/api/endpoint', { credentials: 'include' })
→ Next.js [...path]/route.ts
→ matchRoute(path)
→ proxyIPFS() → Kubo RPC :5001 (explorer/*, ipns/*, …)
→ proxyIPFS() → lokale .data-opslag (pins, files)
→ proxyUserAPI() → Python :8444
```
### 2. Upload Flow (Quick)
```
User drags file
→ UploadPage state: FileEntry[]
→ startUpload() — sequential per file
→ uploadFile(file) → apiFetch POST /api/files/upload (FormData)
→ proxyIPFS → handleLocalUpload() → .data/files/<CID>-<naam>
→ daarna auto-pin in .data/pins.json
→ addToHistory({ cid, name, size }) → localStorage
→ Show results (CID, gateway link, copy)
```
### 3. Upload Flow (Crypto)
```
User selects file
→ PaymentPanel shown
→ WalletProvider.connectWallet() → get address
→ PaymentService.getPrice(bytes, addr) → { finalWei, isFree }
→ PaymentService.getSupportedTokens() → [ETH, USDC, MAOS]
→ PaymentService.getMAOSDiscountTiers() → discount %
→ User pays
→ Step 1: uploadFile(file) → lokale .data-opslag → get CID
→ Step 2a (free): onPaid({ method: 'free' })
→ Step 2b (ETH): payWithETH(provider, cid, size, wei)
→ sendTransaction → contract.payWithETH(cid, bytesSize)
→ Step 2c (token): approveAndPayWithToken(...)
→ check allowance → approve if needed → contract.payWithToken(...)
→ addToHistory({ method: 'eth' }) → localStorage
```
### 4. Payment Admin Flow
```
Admin connects wallet
→ Check isOwner (compare connected address to contract.owner)
→ Load: getOwner, getBasePricePerMB, getFreeTierBytes,
getSupportedTokens, getMAOSDiscountTiers, getTotalUploads
→ Per token: getTotalRevenue, getBalance (ETH) / balanceOf (ERC20)
→ Owner can: setPrice, setFreeTier, configureToken,
add/remove discount tiers, withdraw funds
```
---
## Key Modules
### `src/lib/api/` (8 modules, barrel via `index.ts`)
Central API layer, opgesplitst per domein. `apiFetch()` (in `client.ts`) doet de
fetch met `credentials: 'include'`; alle modules daaromheen zijn één regel per
endpoint.
| Module | Exporteert | Endpoint |
|---|---|---|
| `client.ts` | `apiFetch()` + gedeelde types | — |
| `gateway.ts` | `getNodeInfo()`, `getPeers()`, `getRepoStats()`, `getBWStats()`, `checkHealth()` | `/api/node/info`, `/api/peers`, `/api/repo/stats`, `/api/bw/stats`, `/api/health` |
| `pins.ts` | `listPins()` / `addPin()` / `removePin()` | `/api/pins` (lokaal) |
| `files.ts` | `uploadFile()`, `listFiles()` | `/api/files/upload`, `/api/files` (lokaal) |
| `users.ts` | `listUsers()` / `createUser()` / `deleteUser()` | `/api/users` (Python) |
| `explorer.ts` | `explorerLs()` / `explorerCat()` / `explorerStat()` | `/api/explorer/*` (Kubo) |
| `ipns.ts` | `listIPNSKeys()` / `ipnsPublish()` / `ipnsResolve()` / `ipnsGenKey()` | `/api/ipns/*` (Kubo) |
`src/lib/api.ts.bak` is een achtergebleven backup van het oude monolitische
`api.ts` en wordt nergens geïmporteerd.
### `src/lib/wallet.ts` (196 lines)
Wallet connection layer.
- **EIP-6963** event-based provider discovery (`discoverWallets`)
- Legacy `window.ethereum` fallback
- MAOS Wallet preferred (`window.maosv6`)
- Chain 270 (zkSync Local) switch + add
- Helpers: `formatWeiToETH`, `formatBytes`, `signMessage`, `getBalance`
### `src/lib/payment.ts` (539 lines)
Smart contract interaction via viem.
- 28 methods wrapping ABI functions
- Supports ETH and ERC20 token payments
- Read: `getPrice`, `getUserStats`, `getSupportedTokens`, `getMAOSDiscountTiers`
- Write: `payWithETH`, `approveAndPayWithToken`
- Admin: `adminSetPricePerMB`, `adminConfigureToken`, `adminWithdrawETH`, etc.
- Contract: `0xCBc6b8aeea129c206F4836799621C833Bf8B9BDe` on chain 270
### `src/lib/storage.ts` (98 lines)
localStorage persistence.
- `PortalSettings`: gateway URL, API endpoint, storage max, refresh, theme
- `UploadRecord`: history with dedup by CID+name, capped at 500 entries
### `src/app/api/[...path]/route.ts` (121 lines)
Catch-all API proxy. Routes:
| Path | Handler | Backend |
|---|---|---|
| `/api/health` | `handleHealth()` | Inline JSON |
| `/api/users*` | `proxyUserAPI()` | Python :8444 |
| `/api/pins*`, `/api/files*` | `proxyIPFS()` → lokale handlers | `.data` op schijf |
| `/api/explorer/*` / `/api/ipns/*` / default | `proxyIPFS()` | Kubo :5001 |
De proxy zelf is 121 regels; de Kubo-vertaling staat in `proxy-ipfs.ts` en de
write-gate in `write-auth.ts`.
**`mapToKuboAPI(path, method)`** in `proxy-ipfs.ts` translates Portal API paths to Kubo RPC endpoints:
- `/node/info` → `/api/v0/id`
- `/peers` → `/api/v0/swarm/peers`
- `/explorer/ls` → `/api/v0/ls`
- `/ipns/publish` → `/api/v0/name/publish`
- `/repo/stats` → `/api/v0/repo/stat`
- `/bw/stats` → `/api/v0/bw`
`pins`, `pins/<cid>` en `files/upload` hebben **geen** mapping: `proxyIPFS()`
onderschept die paden vóór de Kubo-vertaling en serveert ze lokaal (zie onder).
Voeg hier dus geen `pin/*`- of `add`-tak toe zonder `proxyIPFS()` en
`isMutatingRequest()` mee te wijzigen.
---
## Lokale opslag (`.data/`) en de gateway-route
### Pins en files gaan niet naar Kubo
`proxyIPFS()` (`src/app/api/[...path]/proxy-ipfs.ts`) behandelt `pins`,
`pins/<cid>` en `files/*` volledig zelf. Deze paden bereiken de Kubo RPC nooit,
ongeacht of `KUBO_API_URL` wel of niet gezet is — de lokale opslag werkt dus
ook zonder Kubo.
| Portal-endpoint | Handler | Effect |
|---|---|---|
| `GET /api/pins` | `handleLocalPins` | Leest `.data/pins.json` → Kubo-vorm `{ Keys: { <cid>: { Type } } }` |
| `POST /api/pins` | `handleLocalPins` | `localAddPin(cid, name, size)` |
| `DELETE /api/pins/<cid>` | `handleLocalPins` | `localRemovePin(cid)` |
| `POST /api/files/upload` | `handleLocalUpload` | `localSaveFile()` + auto-pin |
| `GET /api/files` | `handleLocalListFiles` | `localListFiles()` |
| `GET /api/files/<cid>` | `handleLocalGetFile` | Serveert `.data/files/<cid>-<naam>` als attachment |
| `DELETE /api/files/<cid>` | `handleLocalDeleteFile` | `localDeleteFile(cid)` |
### Opslaglayout
| Pad | Inhoud |
|---|---|
| `.data/pins.json` | `Record<cid, { cid, name, size, created }>` — `src/lib/local-pins.ts` |
| `.data/files/<cid>-<naam>` | De ruwe bestandsbytes — `src/lib/local-files.ts` |
| `.data/files.json` | `Record<cid, { cid, name, size, modified, path }>` |
De "CID" van een lokaal opgeslagen bestand is geen echte IPFS-CID maar
`"Qm" + sha256(inhoud).slice(0, 44)` (`generateCid()` in `local-files.ts`).
Content is daarmee adresseerbaar binnen deze portal, maar niet op een
ander IPFS-netwerk. `.data/` is runtime-state: het ontstaat bij het eerst
uploaden en moet in een deploy een volume zijn, want een rebuild wist hem anders
leeg. ⚠️ `.data/` staat op dit moment **niet** in `.gitignore` — voeg die regel
toe voordat er per ongeluk runtime-state gecommit wordt.
### `GET /ipfs/<cid>` — de gateway-route
`src/app/ipfs/[cid]/route.ts` is de gateway die deze share-links bedient.
`gatewayLink()` in `src/lib/helpers.ts` bouwt alle deel-links als
`<NEXT_PUBLIC_GATEWAY_URL>/ipfs/<cid>`, wat precies dit pad volgt.
| Request | Gedrag |
|---|---|
| `GET /ipfs/<cid>` | Valideert het CID-prefix (`Qm`/`bafy`/`bafk`), daarna `POST <KUBO_API_URL>/api/v0/cat?arg=<cid>`, met Basic auth als `KUBO_BASIC_AUTH` gezet is |
| `GET /ipfs/<cid>?filename=…` | Zet `Content-Disposition: inline; filename="…"` |
De response draagt het `Content-Type` dat Kubo terugstuurt, met
`application/octet-stream` als fallback als upstream niets bruikbaars stuurt.
De body wordt volledig in geheugen geladen (`arrayBuffer()`) — voor grote
bestanden is dat nog geen streaming. Foutpaden: `400` (ongeldig CID), `501`
(zonder `KUBO_API_URL`), de Kubo-status zelf, of `502` bij een netwerkfout.
---
## Write authentication
Muterende `/api/*`-endpoints eisen een token; lees-endpoints blijven publiek.
Ingang: `authorizeWrite()` in `src/app/api/[...path]/write-auth.ts`, aangeroepen
vanuit `POST` en `DELETE` in `route.ts`. De waarde komt uit
`ipfsWriteToken()` in `src/lib/config.ts` (env `IPFS_WRITE_TOKEN`).
| Endpoint | Methode | Write-gate |
|---|---|---|
| `/api/add` | POST | ✅ |
| `/api/files/upload` | POST | ✅ |
| `/api/files/<cid>` | DELETE | ✅ |
| `/api/pins` | POST | ✅ |
| `/api/pins/<cid>` | DELETE | ✅ |
| `/api/ipns/publish`, `/api/ipns/keys/gen/<naam>` | POST | ✅ |
| `/api/users/<naam>` | DELETE | ✅ |
| `/api/explorer/{ls,cat,stat,resolve}/<cid>` | POST | ❌ lees-only |
| `/api/pins`, `/api/files`, `/api/explorer/*`, `/api/health` | GET | ❌ publiek |
| `/ipfs/<cid>` (gateway) | GET | ❌ publiek |
| `/api/users` (registratie) | POST | ❌ open — eigen auth van Python-backend |
**Toegang tot een mutatie — één van beide volstaat:**
1. `Authorization: Bearer $IPFS_WRITE_TOKEN` (MAOS → portal, server-to-server).
Vergelijking is constant-time via SHA-256 digests (`crypto.timingSafeEqual`).
2. De bestaande portalsessie-JWT cookie `ipfs-portal-session` — identieke
verificatie als `src/middleware.ts`. Zónder dit pad zouden de eigen
upload-/pin-pagina's van de portal kapotgaan.
Geen van beide → `401` (geen `Authorization`-header) of `403` (header aanwezig,
verkeerde waarde), met `WWW-Authenticate: Bearer`.
**`IPFS_WRITE_TOKEN` niet geconfigureerd** → writes blijven **open** met een
eenmalige luide waarschuwing in de log. `src/lib/config.ts` heeft geen
"required in production"-patroon (alleen defaults, in tegenstelling tot
`JWT_SECRET` in `auth-server.ts`), en een harde fail zou elke nog niet
geprovisioneerde deploy direct breken — vandaar bewust fail-open + waarschuwing.
**Besluit — publieke lees-endpoints**: `GET /api/pins` en `GET /api/files`
blijven anoniem toegankelijk. Ze lekken alleen metadata (CID's, namen, maten);
content is sowieso publiek via de gateway. Ze achter de sessie zetten zou de
publieke explorer (pin-badges) én MAOS' pin-status/inventory-reads breken — die
sturen vandaag geen `Authorization`-header op GET. Zodra MAOS ook GET's met
Bearer verstuurt kan dit alsnog achter de gate.
**Rest-risico**: `POST /api/users` (registratie én admin-aanmaak, zelfde
endpoint) blijft open — een registrant heeft noch token noch sessie. De
Python-backend is daar de autoriteit (`X-Admin-Key` + doorgestuurd
`X-Session-Token`).
---
## Smart Contract (Chain 270 — zkSync Local)
**Address**: `0xCBc6b8aeea129c206F4836799621C833Bf8B9BDe`
**RPC**: `http://tom1687.no-ip.org:3050`
### Functions
| Category | Functions |
|---|---|
| **Payment** | `payWithETH(cid, bytesSize)` (payable), `payWithToken(cid, bytes, symbol)` |
| **Pricing** | `getPrice(bytes, user)` → `(totalWei, discountBps, finalWei, isFree)` |
| **Free Tier** | `freeTierBytes()`, `userTotalBytes(user)` |
| **Token Mgmt** | `configureToken(symbol, addr, decimals, weiPerToken, enabled)` |
| **Discounts** | `getMAOSDiscountTiers()`, `setMAOSDiscountTier(minBal, bps)`, `removeMAOSDiscountTier(index)` |
| **Admin** | `setPricePerMB(wei)`, `setFreeTierBytes(bytes)`, `withdrawETH(to)`, `withdrawToken(symbol, to)`, `transferOwnership(to)` |
| **Stats** | `totalRevenue(symbol)`, `totalUploads()`, `getUserStats(user)` |
| **Events** | `UploadPaidETH`, `UploadPaidToken`, `FreeUploadUsed` |
### Known Bug
Contract returns duplicate token entries — admin page deduplicates on symbol.
---
## Graph Analysis
### Static (AST) View
- **251 nodes**, **401 edges**, **17 communities**
- Graphify knowledge graph at `graphify-out/graph.json`
- **79 isolated nodes** (type definitions, interfaces, scripts)
### Bridge Nodes
| Node | Edges | Role |
|---|---|---|
| `PaymentService` | 31 | Bridges Smart Contract ↔ UI (Payments) |
| `WalletProvider` | 15 | Bridges Wallet ↔ PaymentService |
### Graph Gap
The AST parser cannot trace URL-routed connections. The runtime flow:
```
apiFetch('/api/node/info') → fetch('/api/node/info')
→ [...path]/route.ts → proxyIPFS() → Kubo :5001
```
Appears as 3 disconnected communities in the graph but is one continuous flow at runtime.
---
## Code Quality Summary
### Issues Found (24 total)
| Severity | Count | Key Items |
|---|---|---|
| 🔴 Critical | 4 | XSS, unhandled promises, silent catch blocks, oversized files |
| 🟡 Medium | 5 | 15× `@ts-expect-error`, 8× `as any`, duplicate helpers, precision loss |
| 🟢 Low | 3 | Unused imports, console.* logging, IPNS layout inconsistency |
### Priority Fixes
1. **Window type augmentation** — Declare `Window` interface in `wallet.ts` to eliminate 15 `@ts-expect-error`
2. **Extract duplicate patterns** — Consolidate `formatBytes`, `gatewayLink`, `truncateCid` into `lib/helpers.ts`
3. **Add error logging** — Every empty catch block gets `console.error` or structured logging
4. **Split oversized files** — `upload/page.tsx` (590→2 components), `admin/payment/page.tsx` (536→4 views), `payment.ts` (539→abi/types/service)
5. **Fix XSS** — Add DOMPurify or replace `dangerouslySetInnerHTML` in `FilePreview.tsx`
6. **Fix unhandled promises** — Add `.catch()` to `isDeployed().then()`
7. **Replace `as any` casts** — Use proper viem `parseAbi` types instead of `as unknown as X`
---
## Configuration
- **API_BASE**: `''` (empty — same origin; nginx routes `/api/*` to backend)
- **Kubo RPC**: `http://127.0.0.1:5001`
- **Python API**: `http://192.168.1.176:8444` (htpasswd + admin key)
- **Smart Contract**: `0xCBc6...` on chain 270 (zkSync Local)
- **IPFS Gateway**: `https://ipfs.maos.dedyn.io/ipfs/`
- **Refresh Interval**: 15s (Dashboard)
- **Max Uploads**: 20 files, 100MB each
- **Storage Max**: 30 GB (localStorage setting)
- **Admin Key**: `maos-admin-2024` (X-Admin-Key header)