abrban/cloud-host
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: abrban/cloud-host:a58142cc4ad09f20b4dfa23a14728195a02ac2b1
Branches Tags
abrban/cloud-host:main abrban/cloud-host:ci/kaniko-fix abrban/cloud-host:ci/egress-proxy-test
..
compare: abrban/cloud-host:5ed2ef0958a336deb22c0aa0e00ee84df8856483
Branches Tags
abrban/cloud-host:main abrban/cloud-host:ci/kaniko-fix abrban/cloud-host:ci/egress-proxy-test

10 Commits
a58142cc4a .. 5ed2ef0958

Author SHA1 Message Date
keyhan 5ed2ef0958 Add GitOps stack for abrban.com with Gitea Actions CI/CD.
Build and Deploy Platform / build-push-deploy (push) Has been cancelled
Details 
Harbor in-cluster builds via Kaniko, ArgoCD auto-sync, and production Helm values for abrban.com domains.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-02 01:27:20 +03:30
keyhan ee5bd0a291 Add Harbor/Ceph runbooks and align single-node Ceph docs with abr deployment. 
Document registry ingress split, image mirroring, loop OSD setup, and platform integration paths.
 2026-07-01 00:22:06 +03:30
keyhan 8d1855b89c Detect and validate app runtime from uploaded archives. 
Reject zip uploads when the selected runtime does not match archive contents, and re-validate before Kaniko builds to fail fast instead of producing the wrong Dockerfile.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-30 00:27:03 +03:30
keyhan 837f0fa63f Harden platform security, reliability, and CI after full audit. 
Close deployment IDOR and gate stub payment endpoints, add production
secret validation, health probes, Redis-backed build progress, GitHub
Actions CI, expanded tests, billing/k8s refactors, and ops runbooks.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-29 20:59:49 +03:30
keyhan a87bc49393 docs: sync README + ARCHITECTURE to the Kaniko build pipeline 
Update both docs to match the code after reverting the build-pipeline revamp:
Kaniko with per-runtime generated Dockerfiles, disk/PVC + git source, no
Nixpacks/MinIO/Trivy/registry-GC/build-queue. Also refreshes the stack
(Next.js 16, NestJS 11), mobile-OTP auth, the three Helm charts, project
structure, and local + cluster deploy steps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-23 19:27:32 +03:30
keyhan 9c16b462f4 revert(build): remove app build pipeline revamp (Nixpacks/MinIO/Trivy/registry GC) 
Reverts commits 3eff38f and c379a23 and restores the previous Kaniko-only
build pipeline (runtime detection + per-runtime Dockerfile generation,
disk-based source upload).

Removed: Nixpacks Dockerfile generation, MinIO source storage (common/storage),
Bull build queue + Redis build state (common/redis, deployment.processor),
Trivy image scan (scan.service, deployment.vulnerabilitySummary), and daily
registry garbage collection (registry-gc). Nothing outside the build/deploy
path depended on these. Backend tsc + 105/106 tests green (the pre-existing
helm.service chartPath failure is unrelated); frontend tsc green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-23 19:21:30 +03:30
keyhan bd14eb2daa feat(blog): real bilingual blog with covers, list & article pages 
- Add blog content module (posts.ts): 4 platform-focused articles (deploy,
  managed databases, custom domain + SSL, replicas & cost) in fa/en with
  tag, date, reading time, excerpt and structured body.
- Self-contained gradient cover art (BlogCover) — no external images, themed
  per topic with the brand cloud motif.
- Landing blog section now shows the 3 latest posts with covers, linking to
  article pages; "view all" links to /blog.
- Dedicated /blog list page and /blog/[slug] article page over the shared
  cloud-sky shell (BlogShell), with a not-found state.
- Remove the now-redundant "Learn" header nav item (blog covers getting
  started) and drop its orphan section anchor.
- i18n: blog minRead/notFound/backToBlog strings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-23 13:35:02 +03:30
keyhan f89c3de826 feat(landing): glass header nav with side drawer + blog section 
- Header: full glassmorphism navigation (services, pricing, estimator, learn,
  blog). Desktop shows an inline glass pill; mobile opens a side drawer that
  slides in from the inline-start edge (right in RTL, left in LTR) over a
  blurred backdrop.
- Nav links smooth-scroll via the shared Lenis instance (exposed through
  scroll-store: setLenis/scrollToId), with an 80px offset under the header.
- "Learn" links to the existing how-it-works steps (id="learn"); add a
  lightweight Blog teaser section (id="blog", "coming soon" cards).
- i18n: fa/en strings for nav and blog.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-23 11:36:27 +03:30
keyhan 985a23751e fix(landing): merge the final scene's two suns into one 
The closing sky drew two suns: the 3D sun mesh (corona + GodRays) at a
fixed world point, and a disc/halo painted by the sky shader aimed at a
fixed direction. As the camera flew forward, parallax split them apart —
the mesh rode high while the painted halo stayed low.

Aim the sky shader's uSunDir at the real camera→sun direction each frame
so the halo locks onto the mesh (one sun), and lower SUN_POS.y 34→13 so
that single sun sits at the lower, near-horizon spot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-23 00:48:49 +03:30
keyhan f7974dd382 feat(landing,billing): public pricing, global discount, services bar & estimator 
Backend
- Add platform-wide global discount (platform_settings: global_discount_percent),
  applied centrally in PricingCatalogService.computeTotalsFromDb so it reaches
  every real charge (previews, deploys, renewals, upgrades, invoices). Admin
  GET/PATCH /billing/settings/global-discount.
- Add unauthenticated PublicPricingController (catalog + calculate) for the
  public landing page, returning gross/net and the discount percentage.
- Bill application replicas by the user-selected footprint: app CPU/RAM/storage
  now all scale by replica count; the single-replica database stays unscaled.

Frontend
- Landing: Services bar (PaaS active, DBaaS, KaaS/LaaS "coming soon" with
  expandable runtime/database menus), transparent Pricing section (per-resource
  rate cards with cycle toggle + discount strikethrough), and a cost Estimator
  ("estimate your package").
- Optional services and the database are priced like runtimes: the estimator
  lets users pick their CPU/RAM/storage (and DB type) so the cost scales by need.
- Admin billing: global-discount editor.
- i18n: fa/en strings for services, pricing, estimator and global discount.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-22 23:44:30 +03:30
177 changed files with 11960 additions and 3457 deletions
Expand all files Collapse all files
.gitea/workflows/build-deploy.yaml
+61
View File
@@ -0,0 +1,61 @@
name: Build and Deploy Platform
on:
push:
branches: [main, master]
workflow_dispatch:
env:
# Push via internal Harbor registry (no creds needed from runner pod)
REGISTRY_INTERNAL: harbor-registry.cloudhost.svc.cluster.local:5000
REGISTRY: registry.abrban.com
BACKEND_IMAGE: abrban/cloudhost-backend
FRONTEND_IMAGE: abrban/cloudhost-frontend
jobs:
build-push-deploy:
runs-on: abrban-kaniko
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set image tag
shell: bash
run: |
SHA="${GITHUB_SHA:-$(git rev-parse HEAD)}"
echo "IMAGE_TAG=$(date +%Y%m%d-%H%M)-${SHA:0:8}" >> "$GITHUB_ENV"
- name: Build backend (Kaniko)
run: |
/kaniko/executor \
--context=./backend \
--dockerfile=./backend/Dockerfile \
--destination="${REGISTRY_INTERNAL}/${BACKEND_IMAGE}:${IMAGE_TAG}" \
--insecure \
--skip-tls-verify
- name: Build frontend (Kaniko)
run: |
/kaniko/executor \
--context=./frontend \
--dockerfile=./frontend/Dockerfile \
--build-arg=NEXT_PUBLIC_API_URL=https://api.abrban.com \
--destination="${REGISTRY_INTERNAL}/${FRONTEND_IMAGE}:${IMAGE_TAG}" \
--insecure \
--skip-tls-verify
- name: Update GitOps values
shell: bash
run: |
sed -i "s|tag: \"[^\"]*\"|tag: \"${IMAGE_TAG}\"|g" gitops/platform/values-abrban.yaml
git config user.email "ci@abrban.com"
git config user.name "Gitea Actions"
git add gitops/platform/values-abrban.yaml
git diff --cached --quiet || git commit -m "ci: deploy platform ${IMAGE_TAG}"
- name: Push GitOps update
shell: bash
env:
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git remote set-url origin "https://oauth2:${GITEA_TOKEN}@git.abrban.com/abrban/cloud-host.git"
git push origin HEAD:main
.github/workflows/ci.yml
+60
View File
@@ -0,0 +1,60 @@
name: CI
on:
push:
branches: [main, master]
pull_request:
branches: [main, master]
jobs:
backend:
name: Backend
runs-on: ubuntu-latest
defaults:
run:
working-directory: backend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '24'
cache: npm
cache-dependency-path: backend/package-lock.json
- run: npm ci
- run: npm run typecheck
- run: npm run lint:check
- run: npm test -- --passWithNoTests
- run: npm run test:e2e
frontend:
name: Frontend
runs-on: ubuntu-latest
defaults:
run:
working-directory: frontend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '24'
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: npm ci
- run: npm run typecheck
- run: npm run lint
- run: npm test
- run: npm run build
env:
NEXT_PUBLIC_API_URL: http://localhost:4000
helm:
name: Helm Charts
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: azure/setup-helm@v4
with:
version: v3.15.4
- run: helm lint backend/helm/cloudhost-platform
- run: helm lint backend/helm/cloudhost-app
- run: helm lint backend/helm/cloudhost-logging
ARCHITECTURE.md
+187 -175
View File
@@ -2,7 +2,14 @@
## Overview
CloudHost is a self-service PaaS platform that enables users to deploy **Node.js**, **Laravel**, and **WordPress** applications onto Kubernetes clusters managed by a super admin. It includes a full billing/wallet system, automated lifecycle management, and Helm-based deployments.
CloudHost is a self-service PaaS that lets users deploy applications onto Kubernetes
clusters managed by a super admin. Source code (uploaded archive or git repo) is turned
into a container image **inside the cluster** with Kaniko, then rolled out via Helm. It
includes a full billing/wallet system, automated lifecycle management, managed
databases/services, Elasticsearch-backed logging, and a bilingual (Persian/English) panel.
Supported runtimes — each built from a platform-maintained `Dockerfile` template:
**Node.js, Laravel, Go, PHP, Python, Django, .NET**, and **WordPress** (official image).
---
@@ -10,42 +17,34 @@ CloudHost is a self-service PaaS platform that enables users to deploy **Node.js
```
┌─────────────────────────────────────────────────────────────────┐
│ USERS / ADMINS │
│ (Browser / CLI) │
│ USERS / ADMINS (Browser) │
└──────────────────────────┬──────────────────────────────────────┘
│ HTTPS
▼
┌─────────────────────────────────────────────────────────────────┐
│ FRONTEND (Next.js 14) │
│ ┌──────────┐ ┌───────────────┐ ┌──────────┐ ┌───────────┐ │
│ │ Auth UI │ │ Deploy Wizard │ │ Dashboard│ │Admin Panel│ │
│ └──────────┘ └───────────────┘ └──────────┘ └───────────┘ │
│ FRONTEND — Next.js 16 (App Router, bilingual) │
│ Landing │ Auth (OTP) │ Deploy Wizard │ Dashboard │ Admin Panel │
└──────────────────────────┬──────────────────────────────────────┘
│ REST API (JSON)
│ REST /api/v1 (JSON)
▼
┌─────────────────────────────────────────────────────────────────┐
│ BACKEND (NestJS 10) │
│ │
│ ┌──────────┐ ┌──────────────┐ ┌────────────┐ ┌──────────┐ │
│ │Auth │ │Applications │ │Deployments │ │Clusters │ │
│ │Module │ │Module │ │Module │ │Module │ │
│ └──────────┘ └──────────────┘ └────────────┘ └──────────┘ │
│ │
│ ┌──────────┐ ┌──────────────┐ ┌────────────┐ ┌──────────┐ │
│ │Billing │ │Lifecycle │ │Snapshots │ │Tickets │ │
│ │Module │ │Module │ │Module │ │Module │ │
│ └──────────┘ └──────────────┘ └────────────┘ └──────────┘ │
│ │
│ ┌──────────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Kubernetes │ │ Helm │ │ Build │ │
│ │ Service │ │ Service │ │ Service │ │
│ └────────┬─────────┘ └──────┬───────┘ └──────┬───────────┘ │
└───────────┼───────────────────┼──────────────────┼───────────────┘
│ │ │
┌───────▼────────┐ ┌──────▼────────┐ ┌──────▼────────┐
│ Kubernetes │ │ Helm CLI │ │ Kaniko │
│ Cluster(s) │ │ (v3) │ │ (in-cluster) │
└───────────────┘ └───────────────┘ └───────────────┘
│ BACKEND — NestJS 11 │
│ Auth · Users · Admin · Applications · Deployments · Clusters │
│ Build · Billing · Lifecycle · Snapshots · Tickets · Access · │
│ Notifications · Kubernetes/Helm/Registry │
└───┬───────────┬───────────────────┬───────────────┬─────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌────────┐ ┌────────┐ ┌──────────┐ ┌──────────────────┐
│Postgres│ │ Redis │ │ Registry │ │ Kubernetes │
│ 16 │ │(cache/ │ │ (:2) │ │ Cluster(s) │
│ │ │ Bull) │ └──────────┘ │ ┌────────────┐ │
└────────┘ └────────┘ │ │ Build Jobs │ │
│ │ (Kaniko) │ │
│ └────────────┘ │
│ Helm releases │
│ (user apps) │
└──────────────────┘
```
---
@@ -54,110 +53,118 @@ CloudHost is a self-service PaaS platform that enables users to deploy **Node.js
| Module | Purpose |
|--------|---------|
| **Auth** | JWT access/refresh tokens, Passport strategies, role guards |
| **Users** | User CRUD, admin activate/deactivate, profile management |
| **Applications** | App CRUD, code upload (zip), metadata, runtime detection |
| **Build** | Kaniko-based image builds via BullMQ queue; auto-detects Node.js/Laravel/WordPress |
| **Kubernetes** | K8s API interactions — namespace, scale, delete, pod logs, build pods |
| **Helm** | Helm CLI wrapper — install/upgrade, rollback, uninstall, history |
| **Deployments** | Deployment lifecycle orchestration, history, stop/restart |
| **Clusters** | Multi-cluster management, kubeconfig storage, default cluster selection |
| **Billing** | Wallet system (deposit/deduct), transaction ledger, plan cost calculation |
| **Lifecycle** | Cron-based scanner: auto-suspend expired apps, auto-delete after grace period |
| **Snapshots** | Application snapshot/backup management |
| **Tickets** | Support ticket system for users |
| **Auth** | Mobile-number + OTP (SMS) and password login; JWT access/refresh; Passport strategies; role guards |
| **Users** | User CRUD, profile, phone verification |
| **Admin** | Super-admin user-detail dashboard and operations |
| **Applications** | App CRUD, code upload (→ disk), git config, runtime/version metadata |
| **Application-migrations** | Import / migrate existing applications (Bull queue) |
| **Build** | `build.service` — runtime detection + per-runtime Dockerfile generation, Kaniko image builds inside K8s |
| **Deployments** | Deploy orchestration (build → Helm), history, stop/restart |
| **Kubernetes** | K8s API wrapper, Helm CLI wrapper, registry service |
| **Clusters** | Multi-cluster management, kubeconfig storage, default cluster |
| **Billing** | Wallet (deposit/deduct), transaction ledger, invoices, pricing catalog, coupons/discounts |
| **Lifecycle** | Interval scanner: auto-suspend expired apps, auto-delete after grace period |
| **Snapshots** | Application snapshot/backup & restore |
| **Tickets** | Support ticket system (technical/sales departments) |
| **Access** | Time-limited external access to app services via temporary NodePort grants (Bull queue) |
| **Notifications** | User-facing notifications |
| **Common / Config** | Shared enums, guards, decorators; env & TypeORM config |
---
## 🔧 Tech Stack
### Frontend: Next.js 14 (App Router) + Tailwind CSS
### Frontend: Next.js 16 (App Router) + Tailwind CSS v4
| Reason | Detail |
|--------|--------|
| **SSR & SEO** | Server-side rendering for fast initial loads |
| **App Router** | React Server Components, layouts, loading states |
| **Tailwind CSS** | Rapid UI development, consistent design system |
| **TypeScript** | End-to-end type safety with shared types |
| **React Query** | Server state management, caching, polling for live status |
| **Zustand** | Lightweight client state management (auth store) |
| **SSR & SEO** | Server-side rendering for fast initial loads and a public landing/blog |
| **App Router** | React Server Components, layouts, locale routing under `app/[lang]/` |
| **Bilingual** | `fa-IR` (default) + `en-US`; `middleware.ts` also splits landing vs authenticated panel |
| **React Query** | Server state, caching, polling for live build/deploy status |
| **Zustand** | Lightweight client auth store |
| **TypeScript** | End-to-end type safety |
### Backend: NestJS 10 (Node.js)
### Backend: NestJS 11 (Node.js 20)
| Reason | Detail |
|--------|--------|
| **Modular architecture** | Each domain is a self-contained module |
| **TypeScript native** | Full type safety, shared interfaces with frontend |
| **@kubernetes/client-node** | Official K8s client for direct API interaction |
| **Helm CLI** | Shell-out to helm for chart-based deployments |
| **Bull/BullMQ** | Redis-backed job queues for async build pipelines |
| **TypeORM** | PostgreSQL ORM with entity-based schema |
| **Modular** | Each domain is a self-contained module |
| **@kubernetes/client-node** | Direct K8s API interaction (Jobs, Deployments, logs, scale) |
| **Helm CLI** | Shell-out to helm for chart-based app deployments |
| **Bull (Redis)** | Async queues for service-access grants and application migrations |
| **TypeORM** | PostgreSQL ORM. `synchronize` is **development-only**; production schema changes ship as idempotent SQL migrations / `ALTER ... IF NOT EXISTS` |
### Build System: Kaniko (in-cluster)
| Reason | Detail |
|--------|--------|
| **No Docker daemon** | Builds inside K8s pods — no Docker-in-Docker |
| **Runtime detection** | Auto-detects Node.js, Laravel, WordPress from source files |
| **WordPress support** | Custom entrypoint script for wp-content merging |
| **Registry push** | Native push to insecure or authenticated registries |
| **No Docker daemon** | Builds run as unprivileged K8s Jobs in `cloudhost-builds` |
| **Runtime detection** | `detectRuntime()` infers Node.js / Laravel / WordPress from source files; the app may also pin a runtime explicitly |
| **Per-runtime Dockerfiles** | `generateDockerfile()` emits a tailored Dockerfile for Node.js, Laravel, WordPress, Go, PHP, Python, Django, or .NET |
| **WordPress** | Templated Dockerfile + custom entrypoint that merges `wp-content` |
| **Source ingestion** | Uploaded archives saved to disk and streamed into a per-build PVC (helper pod + `kubectl cp`); git repos cloned in-pod |
| **Registry push** | Native push to the in-cluster (insecure) registry |
### Deployment: Helm v3 Charts
| Reason | Detail |
|--------|--------|
| **Templated manifests** | Single chart handles Node.js, Laravel, WordPress |
| **Rollback support** | Built-in revision history and rollback |
| **Resource policies** | PVCs and secrets persist across helm uninstall |
| **Registry pull secrets** | Auto-created per namespace for insecure registries |
| **Templated manifests** | One `cloudhost-app` chart handles all runtimes + attached services |
| **Rollback** | Built-in revision history |
| **Persistence** | DB/app PVCs and secrets use keep policies so they survive helm uninstall |
| **Ingress** | Traefik by default (k3s); `INGRESS_CLASS=nginx` for ingress-nginx |
---
## 🔄 Build & Deploy Flow
```
User triggers deploy (panel)
│
▼
Backend runs the build-and-deploy pipeline, creating a Kubernetes Job in `cloudhost-builds`:
┌─ init: prepare source (uploaded zip → disk → helper pod + `kubectl cp` → build PVC) ─┐
│ …or… │ → /workspace/source
└─ init: git-clone (clone repo; token injected into the URL for private repos) ───┘
│
▼
The platform detects the runtime and generates a Dockerfile for it
(Node.js / Laravel / WordPress / Go / PHP / Python / Django / .NET)
│
▼
container: kaniko → build image (cache per user) → push to in-cluster registry
│
▼
HelmService install/upgrade `cloudhost-app`
│
▼
Helm creates: Namespace, Deployment, Service, Ingress (+TLS), per-app
DB/Redis/RabbitMQ, PVCs, Secrets, registry pull secret, log shipper
│
▼
App live at https://<subdomain>.<PLATFORM_DOMAIN>
```
Notes:
- The build runs **inline** within the deploy request (it is not queued); build
progress/logs are tracked in memory and polled by the frontend. This assumes a single
active backend replica for an in-flight build.
- Bull/Redis queues are used by other subsystems (service-access grants, application
migrations), not by the image build.
---
## 🔐 Security Architecture
- JWT Authentication (access + refresh tokens)
- Role-Based Access Control (User / Admin)
- K8s Namespace Isolation per user
- K8s RBAC — scoped ServiceAccounts
- Network Policies between namespaces
- Resource Quotas & Limit Ranges
- Secrets encryption (K8s Secrets)
- Input validation (class-validator on all DTOs)
- Helmet HTTP security headers
- Bcrypt password hashing (12 rounds)
---
## 🔄 Deployment Flow
```
User uploads code (zip)
│
▼
API stores file + metadata in PostgreSQL
│
▼
BullMQ build job queued
│
▼
detectRuntime() → nodejs | laravel | wordpress
│
▼
Generate Dockerfile per runtime
│
▼
Kaniko Pod builds image → pushes to registry
│
▼
HelmService.installOrUpgrade() with cloudhost-app chart
│
▼
Helm creates: Namespace, Deployment, Service, Ingress,
DB, PVC, Secrets, Registry Pull Secret, TLS cert
│
▼
App live at https://<subdomain>.apps.cloudhost.ir
```
- Mobile-OTP + password authentication; JWT access + refresh
- **Live** role/active-status enforcement — `JwtStrategy` reads the user from the DB each request
- Role-Based Access Control (`user` / `admin` / `technical` / `sales`)
- K8s namespace isolation per user; scoped ServiceAccounts
- Resource quotas & limit ranges; expandable per-app storage
- Secrets stored as K8s Secrets (env vars, DB creds)
- Input validation (class-validator), Helmet headers, Bcrypt password hashing
---
@@ -168,95 +175,100 @@ ACTIVE ──(expires)──► SUSPENDED ──(grace)──► PENDING_DELETIO
▲ │ │
└────── payment ────────┘ │
└────── payment (within grace) ──────────────────┘
DOCKED ── user removed the service; data retained until the plan expires
```
- **Billing Cycles**: HOURLY | MONTHLY | YEARLY
- **Hourly plans**: auto-renew from wallet each hour
- **Grace periods**: admin-configurable via PlatformSettings table
- **Lifecycle Scanner**: runs every 60s (configurable)
- **Billing cycles**: HOURLY | MONTHLY | YEARLY
- **Wallet**: deposits, deductions, refunds, gateway payments; invoices with coupons/discounts
- **Grace periods**: admin-configurable via the PlatformSettings entity (per cycle)
- **Lifecycle scanner**: runs on an interval (default 60s); suspends expired apps (scale to 0,
data retained) and deletes them after the grace period
> ⚠️ The lifecycle scanner and other interval jobs assume a **single backend replica** —
> guard them (e.g. a Redis lock) before scaling the control plane horizontally.
---
## Helm Chart: cloudhost-platform
## 📦 Helm Charts
Chart at `backend/helm/cloudhost-platform/` deploys the **control plane** (NestJS API, Next.js UI, PostgreSQL, Redis) into a dedicated namespace (default `cloudhost`).
### cloudhost-platform — control plane
| Value | Purpose |
|-------|---------|
| `ingress.enabled` | Create Ingress (default `true`) |
| `ingress.tls.enabled` | cert-manager TLS via `clusterIssuer` |
| `ingress.frontend.host` / `ingress.api.host` | Public hostnames |
| `postgres.password` / `secrets.jwtSecret` | Credentials (auto-generated if empty on first install) |
| `migrations.enabled` | Post-install SQL migration Job |
---
## 🚀 Helm Chart: cloudhost-app
Single chart at `backend/helm/cloudhost-app/` handles all runtimes:
Deploys the API, UI, PostgreSQL, and Redis into a namespace (default `cloudhost`).
| Template | Purpose |
|----------|---------|
| `deployment.yaml` | App pod with imagePullSecrets, probes, WordPress volumes |
| `service.yaml` | ClusterIP (port 80 → app port) |
| `ingress.yaml` | Nginx ingress with cert-manager TLS |
| `secret.yaml` | User env vars as K8s Secret |
| `db-deployment.yaml` | PostgreSQL or MySQL with health probes |
| `db-service.yaml` | Database ClusterIP service |
| `db-pvc.yaml` | Database storage (resource-policy: keep) |
| `db-secret.yaml` | Database credentials (resource-policy: keep) |
| `wp-pvc.yaml` | WordPress wp-content PVC (resource-policy: keep) |
| `registry-pull-secret.yaml` | imagePullSecret for insecure registry |
| `backend-deployment.yaml` / `backend-service.yaml` / `backend-pvc.yaml` | NestJS API + uploads PVC |
| `frontend-deployment.yaml` / `frontend-service.yaml` | Next.js UI |
| `postgres-*.yaml` / `redis-*.yaml` | Control-plane database & queue |
| `ingress.yaml` | Frontend / API / panel host rules (+ TLS) |
| `secret.yaml` | JWT, DB, registry, SMS and other platform secrets |
| `migrations-configmap.yaml` / `migrations-job.yaml` | Optional SQL migration Job (`migrations.enabled`) |
| `namespace.yaml` / `_helpers.tpl` / `NOTES.txt` | Namespace + chart helpers |
Key values: `ingress.enabled`, `ingress.tls.*`, `ingress.frontend.host` / `ingress.api.host`,
`postgres.password`, `secrets.jwtSecret`, `migrations.enabled`.
### cloudhost-app — a single user application
| Template | Purpose |
|----------|---------|
| `deployment.yaml` | App pod (imagePullSecret, probes, WordPress volumes, env) |
| `service.yaml` / `ingress.yaml` | ClusterIP + ingress with TLS |
| `secret.yaml` | User env vars as a K8s Secret |
| `db-deployment.yaml` / `db-service.yaml` / `db-pvc.yaml` / `db-secret.yaml` | Optional managed PostgreSQL/MySQL/MariaDB/MongoDB |
| `redis-deployment.yaml` / `rabbitmq-deployment.yaml` | Optional attached services |
| `app-storage-pvc.yaml` | App persistent storage |
| `storageclass.yaml` | Expandable StorageClass (created on demand) |
| `registry-pull-secret.yaml` | imagePullSecret for the in-cluster registry |
| `fluent-bit-configmap.yaml` / `log-shipper-configmap.yaml` / `_log-shipper.tpl` / `elasticsearch-credentials-secret.yaml` | Per-app log shipping to Elasticsearch |
### cloudhost-logging — observability
Elasticsearch / Kibana / Fluent-bit stack for centralized build and runtime logs
(also see `backend/k8s/logging/`).
---
## 📁 Project Structure
```
host/
├── ARCHITECTURE.md
├── README.md
├── CHANGELOG.md
├── CONTRIBUTING.md
├── docker-compose.yml
├── backend/
cloud-host/
├── ARCHITECTURE.md README.md RUNBOOK.fa.md CHANGELOG.md CONTRIBUTING.md
├── UPGRADE.md UPGRADE.en.md docker-compose.yml
├── backend/ # NestJS 11 API
│ ├── Dockerfile
│ ├── package.json
│ ├── helm/cloudhost-platform/ # Helm chart for control plane
│ ├── helm/cloudhost-app/ # Helm chart for user apps
│ ├── src/
│ │ ├── main.ts / app.module.ts
│ │ ├── auth/ # JWT + Passport
│ │ ├── users/ # User management
│ │ ├── applications/ # App CRUD + upload
│ │ ├── deployments/ # Deploy orchestration
│ │ ├── clusters/ # Multi-cluster (admin)
│ │ ├── kubernetes/ # K8s client + Helm service
│ │ ├── build/ # Kaniko builds (BullMQ)
│ │ ├── billing/ # Wallet + transactions
│ │ ├── lifecycle/ # Auto-suspend/delete
│ │ ├── snapshots/ # App snapshots
│ │ └── tickets/ # Support tickets
│ └── templates/ # Legacy Handlebars (deprecated)
├── frontend/
│ ├── Dockerfile
│ ├── package.json
│ ├── helm/{cloudhost-platform, cloudhost-app, cloudhost-logging}/
│ ├── k8s/{logging, mail}/ # standalone manifests
│ ├── migrations/ # SQL migrations (one-off Jobs in prod)
│ └── src/
│ ├── app/dashboard/ # Apps, deploy, admin pages
│ ├── components/
│ ├── lib/ # API client, auth store
│ └── types/ # Shared TS interfaces
└── uploads/ # User-uploaded code archives
│ ├── main.ts / app.module.ts
│ ├── auth/ users/ admin/ # OTP auth, users, super-admin dashboard
│ ├── applications/ application-migrations/
│ ├── deployments/ # orchestration (build → Helm)
│ ├── build/ # build.service: runtime detection + per-runtime Dockerfiles + Kaniko
│ ├── kubernetes/ # K8s client, Helm, registry
│ ├── clusters/ # multi-cluster management
│ ├── billing/ lifecycle/ snapshots/ tickets/ access/ notifications/
│ ├── common/ # enums, guards, decorators
│ └── config/ # env + TypeORM config
└── frontend/ # Next.js 16 (App Router, fa-IR / en-US)
├── Dockerfile # ARG NEXT_PUBLIC_API_URL
└── src/
├── middleware.ts # locale routing + landing/panel split
├── app/[lang]/{page, login, register, blog, dashboard/*}
├── components/ hooks/ lib/ types/
└── i18n/ # dictionaries, provider, switcher
```
---
## 🔮 Future Considerations
1. Custom domains with auto TLS via cert-manager
2. Horizontal Pod Autoscaler based on CPU/memory
3. WebSocket/SSE for real-time build log streaming
4. GitOps integration (ArgoCD)
5. Additional runtimes (Python, Go, Rust)
6. App marketplace with pre-built templates
7. Per-app resource consumption dashboards
1. Per-app horizontal autoscaling (HPA) based on CPU/memory
2. WebSocket/SSE for real-time build log streaming (currently polled)
3. A Redis-backed build queue (so builds survive a replica restart and the control plane can scale out)
4. In-cluster image vulnerability scanning (report-only)
5. GitOps integration (e.g. ArgoCD) and git-push-to-deploy
6. Automated control-plane database backups (scheduled `pg_dump` + retention)
7. App marketplace with pre-built templates
LICENSE
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 CloudHost
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
README.md
+239 -224
View File
@@ -1,303 +1,318 @@
# ☁️ CloudHost — Self-Service PaaS Platform
A self-service Platform-as-a-Service (PaaS) that lets developers deploy **Node.js**, **Laravel**, and **WordPress** applications onto Kubernetes with zero DevOps overhead. Includes wallet-based billing, automated lifecycle management, and Helm-based deployments.
A self-service Platform-as-a-Service (PaaS) that lets developers deploy applications
onto Kubernetes with zero DevOps overhead. Source code is turned into a container
image **inside the cluster** with Kaniko (no Docker daemon), then rolled out with Helm —
complete with managed databases, wallet-based billing, automated lifecycle management,
live logs, and a bilingual (Persian/English) panel.
Supported runtimes — each built from a platform-maintained `Dockerfile` template:
**Node.js, Laravel, Go, PHP, Python, Django, .NET**, and **WordPress** (official image +
custom `wp-content` entrypoint).
> 🇮🇷 Production deployment on the `abrban.com` k3s cluster — including all the
> Iran-network workarounds — is documented step-by-step in **[RUNBOOK.fa.md](RUNBOOK.fa.md)** (Persian).
---
## Architecture Overview
```
┌─────────────┐ ┌─────────────────┐ ┌──────────────┐
│ Next.js 16 │ REST │ NestJS API │ K8s │ Kubernetes │
│ Frontend │◄───────►│ Backend │◄──────►│ Cluster(s) │
└─────────────┘ └────────┬────────┘ └──────────────┘
│
┌──────────┼──────────┐
▼ ▼ ▼
PostgreSQL Redis Container
(Bull) Registry
┌──────────────┐ REST ┌──────────────────┐ K8s API ┌──────────────┐
│ Next.js 16 │ /api/v1 │ NestJS 11 API │ + Helm │ Kubernetes │
│ Frontend │◄─────────►│ Backend │◄───────────►│ Cluster(s) │
└──────────────┘ └────────┬─────────┘ └──────┬───────┘
│ │ build Jobs
┌─────────────────┼─────────────────┐ ▼
▼ ▼ ▼ ┌──────────┐
PostgreSQL Redis Registry │ Kaniko │
16 (cache + Bull) (:2) └──────────┘
```
| Layer | Technology |
| ------------ | ------------------------------------------------------- |
| Frontend | Next.js 16, Tailwind CSS v4, React Query, Zustand |
| ---------------- | ----------------------------------------------------------------- |
| Frontend | Next.js 16 (App Router, SSR), React 19, Tailwind CSS v4, React Query, Zustand |
| Backend API | NestJS 11, TypeORM, Passport JWT, Bull (Redis) |
| Build Engine | Kaniko (in-cluster, daemon-less Docker builds) |
| Deployment | Helm v3 charts, @kubernetes/client-node |
| Database | PostgreSQL 16 |
| Queue | Redis 7 + BullMQ |
| Build engine | **Kaniko** (daemon-less in-cluster builds) with platform-generated per-runtime Dockerfiles |
| Source ingestion | Uploaded archive (zip/tar.gz) streamed into a build PVC, **or** git clone |
| Deployment | Helm v3 charts, `@kubernetes/client-node` |
| Database | PostgreSQL 16 (control plane); per-app MySQL/MariaDB/PostgreSQL/MongoDB |
| Queue / cache | Redis 7 + Bull (service-access grants, app migrations) |
| Registry | In-cluster `registry:2` |
| Auth | Mobile number + **OTP** (SMS) and password, JWT access/refresh |
> 📖 See [ARCHITECTURE.md](ARCHITECTURE.md) for detailed system design.
> 🔼 See [UPGRADE.en.md](UPGRADE.en.md) ([فارسی](UPGRADE.md)) for the latest dependency-upgrade notes (React 19, Next 16, NestJS 11, Tailwind 4, k8s-client v1).
> 📖 See **[ARCHITECTURE.md](ARCHITECTURE.md)** for detailed system design.
> 🔼 See [UPGRADE.en.md](UPGRADE.en.md) ([فارسی](UPGRADE.md)) for dependency-upgrade notes.
> 📋 See [RUNBOOK.en.md](RUNBOOK.en.md) ([فارسی](RUNBOOK.fa.md)) for operations.
---
## Features
### For Developers
- 🚀 **One-click deploys** from uploaded code archive (zip)
- 🟢 **Node.js** — auto-detected via `package.json` (npm build & start)
- 🟣 **Laravel** — PHP 8.x + Nginx + Supervisor (auto-detected via `artisan`)
- 🔵 **WordPress** — official image + custom entrypoint for wp-content merging
- 🗄️ **Managed databases** — PostgreSQL or MySQL provisioned via Helm
- 💰 **Wallet system** — deposit funds, pay for plans (hourly/monthly/yearly)
- 📊 **Live logs** & deployment history with rollback
- 🔒 **Environment variables** managed as Kubernetes Secrets
- ⚙️ **Resource controls** — CPU, memory, replica count
- 📸 **Snapshots** — backup and restore application state
- 🎫 **Support tickets** — in-app support system
- 🚀 **Deploy from a code archive (zip/tar.gz) _or_ a git URL** (public or private via token)
- 🟢 **Multi-runtime** — Node.js, Laravel, Go, PHP, Python, Django, .NET, each built from a maintained Dockerfile template
- 🔵 **WordPress** — official image + custom entrypoint that merges your `wp-content`
- 🗄️ **Managed databases & services** — PostgreSQL, MySQL, MariaDB, MongoDB, Redis, RabbitMQ provisioned via Helm
- 💰 **Wallet system** — deposit funds, pay per plan (hourly / monthly / yearly), coupons & discounts
- 📊 **Live build & runtime logs** (Elasticsearch-backed) + deployment history with rollback
- 🔒 **Environment variables** stored as Kubernetes Secrets
- ⚙️ **Resource controls** — CPU, memory, replicas, expandable disk
- 🌐 **Custom domains** with automatic TLS
- 📸 **Snapshots** — backup & restore application state
- 🎫 **Support tickets** with technical/sales departments
### For Super Admins
- 🖥️ **Multi-cluster management** — register/remove Kubernetes clusters
- 👥 **User management** — activate, deactivate, change roles
- 📈 **Quotas** — per-cluster limits (CPU, memory, max apps)
- 💳 **Billing oversight** — view all transactions, manage wallet deposits
- ⏱️ **Lifecycle settings** — configure grace periods per billing cycle
- 🔐 **RBAC** — role-based guards on every endpoint
- 🖥️ **Multi-cluster management** — register/remove Kubernetes clusters (kubeconfig stored encrypted)
- 👥 **User management** — activate, deactivate, change roles, per-user detail dashboard
- 📈 **Quotas & pricing** — per-cluster limits and a configurable pricing catalog
- 💳 **Billing oversight** — transactions, invoices, wallet deposits, global discount
- ⏱️ **Lifecycle settings** — grace periods per billing cycle
- 🔐 **RBAC** — role-based guards on every endpoint (`user` / `admin` / `technical` / `sales`)
---
## How the Build & Deploy Pipeline Works
When a user triggers a deploy, the backend runs the build-and-deploy pipeline and creates
the build as a **Kubernetes Job** in the `cloudhost-builds` namespace:
```
1. SOURCE
├─ uploaded archive → saved to disk (UPLOAD_DIR) → streamed into a per-build PVC
│ via a short-lived helper pod + `kubectl cp`, then unpacked (init: prepare source)
└─ git URL → cloned in-pod; private repos inject the token into the clone URL (init: git-clone)
2. DOCKERFILE
The platform detects the runtime (or uses the app's selected runtime) and generates a
Dockerfile for it — Node.js, Laravel, WordPress, Go, PHP, Python, Django, or .NET.
3. BUILD (container: kaniko)
Kaniko builds the image (layer cache per user) and pushes it to the in-cluster
registry — no Docker daemon, no privileged pod.
4. DEPLOY
Helm installs/upgrades the `cloudhost-app` chart → Deployment, Service, Ingress,
per-app DB/Redis/RabbitMQ, PVCs, Secrets, log shipper. App goes live at its subdomain.
```
The build runs inline within the deploy request and its progress/logs are tracked in
memory, then polled by the frontend. (Bull/Redis queues are used elsewhere — service-access
grants and application migrations — but not for image builds.)
---
## Project Structure
```
host/
├── ARCHITECTURE.md # Detailed architecture document
cloud-host/
├── README.md # This file
├── CHANGELOG.md # Version history
├── CONTRIBUTING.md # Development workflow & conventions
├── docker-compose.yml # Local dev / production compose
├── ARCHITECTURE.md # Detailed system design
├── RUNBOOK.fa.md # Persian runbook: local dev + abrban/k3s production deploy
├── CHANGELOG.md / CONTRIBUTING.md / UPGRADE.md / UPGRADE.en.md
├── docker-compose.yml # Local dev stack (Postgres + Redis + API + UI)
│
├── backend/ # NestJS API
├── backend/ # NestJS 11 API (REST under /api/v1)
│ ├── Dockerfile
│ ├── package.json
│ ├── helm/
│ │ ├── cloudhost-platform/ # Helm chart (control plane)
│ │ └── cloudhost-app/ # Helm chart (user apps)
│ │ ├── Chart.yaml
│ │ ├── values.yaml
│ │ └── templates/ # K8s manifest templates
│ ├── src/
│ │ ├── main.ts / app.module.ts
│ │ ├── auth/ # JWT auth (register, login, refresh)
│ │ ├── users/ # User CRUD + admin ops
│ │ ├── applications/ # Application CRUD + code upload
│ │ ├── deployments/ # Deployment pipeline orchestration
│ │ ├── clusters/ # Cluster management (admin)
│ │ ├── kubernetes/ # K8s client + Helm service
│ │ ├── build/ # Kaniko build jobs (Bull queue)
│ │ ├── billing/ # Wallet, transactions, plan costs
│ │ ├── lifecycle/ # Auto-suspend/delete scanner
│ │ ├── snapshots/ # App snapshot management
│ │ ├── tickets/ # Support ticket system
│ │ ├── common/ # Enums, decorators, guards
│ │ └── config/ # Env configuration loader
│ └── templates/ # Legacy Handlebars templates (deprecated)
│
├── frontend/ # Next.js 14 App Router
│ ├── Dockerfile
│ ├── package.json
│ │ ├── cloudhost-platform/ # Helm chart — control plane (API, UI, Postgres, Redis)
│ │ ├── cloudhost-app/ # Helm chart — a single user application + its services
│ │ └── cloudhost-logging/ # Helm chart — Elasticsearch / Kibana / Fluent-bit
│ ├── k8s/ # Standalone manifests (logging, mail)
│ ├── migrations/ # SQL migrations (applied via one-off Jobs in prod)
│ └── src/
│ ├── app/
│ │ ├── login/ & register/
│ │ └── dashboard/
│ │ ├── apps/ # App list + detail (lifecycle status)
│ │ ├── deploy/ # Multi-step deploy wizard
│ │ └── admin/ # Admin: users, clusters, billing, apps
│ ├── components/
│ ├── lib/ # API client, auth store
│ ├── hooks/
│ └── types/ # TypeScript interfaces
│ ├── main.ts / app.module.ts
│ ├── auth/ # Mobile-OTP + password login, JWT strategies, role guards
│ ├── users/ # User CRUD, profile, phone verification
│ ├── admin/ # Super-admin user-detail dashboard & ops
│ ├── applications/ # App CRUD, code upload (→ disk), git config
│ ├── application-migrations/ # Import/migrate existing apps (Bull queue)
│ ├── deployments/ # Deploy orchestration, history, stop/restart
│ ├── build/ # Kaniko build (build.service): per-runtime Dockerfile generation
│ ├── kubernetes/ # K8s client, Helm wrapper, registry service
│ ├── clusters/ # Multi-cluster management, kubeconfig storage
│ ├── billing/ # Wallet, transactions, invoices, pricing catalog, coupons
│ ├── lifecycle/ # Scanner: auto-suspend/delete expired apps
│ ├── snapshots/ # App snapshot/restore
│ ├── tickets/ # Support tickets
│ ├── notifications/ # User notifications
│ ├── access/ # Time-limited external service access (NodePort grants, Bull queue)
│ ├── common/ # Enums, guards, decorators
│ └── config/ # Env configuration loader + TypeORM config
│
└── uploads/ # User-uploaded code archives
├── frontend/ # Next.js 16 App Router (bilingual fa-IR / en-US)
│ ├── Dockerfile # ARG NEXT_PUBLIC_API_URL baked at build time
│ └── src/
│ ├── middleware.ts # Locale routing + landing (abrban.com) vs panel split
│ ├── app/[lang]/
│ │ ├── page.tsx # Landing
│ │ ├── login/ register/
│ │ ├── blog/
│ │ └── dashboard/ # apps, deploy, logs, invoices, wallet, services,
│ │ │ # tickets, account, staff, admin
│ │ └── ...
│ ├── components/ hooks/ lib/ (API client, auth store) types/
│ └── i18n/ # Dictionaries, provider, language switcher
```
---
## Quick Start
## Quick Start (Local Development)
### Prerequisites
**Prerequisites:** Node.js ≥ 20, Docker & Docker Compose, and (for actually building/deploying
user apps) a Kubernetes cluster reachable via kubeconfig.
| Tool | Version |
| --------------- | ------- |
| Node.js | ≥ 20 |
| Docker & Compose| ≥ 24 |
| PostgreSQL | 16 |
| Redis | 7 |
| Helm | ≥ 3.12 |
> ℹ️ The API and UI run fine locally against Postgres + Redis. The **build/deploy pipeline
> itself runs as Kubernetes Jobs**, so triggering a real user-app build requires a cluster
> (with the in-cluster registry). For pure UI/API development you don't need one.
### 1. Clone & Install
### 1. Clone & install
```bash
git clone <repo-url> host && cd host
git clone <repo-url> cloud-host && cd cloud-host
cd backend && npm install && cd ..
cd frontend && npm install && cd ..
```
### 2. Environment Variables
### 2. Environment variables
```bash
cp backend/.env.example backend/.env
cp frontend/.env.local.example frontend/.env.local
# Edit both files with your DB, JWT, Redis, and registry settings
# Edit both — at minimum DB, JWT, Redis. See the Configuration table below.
```
### 3. Run with Docker Compose
### 3. Start Postgres + Redis
```bash
docker compose up --build
docker compose up -d postgres redis
```
Backend at port 4000, Frontend at port 3000.
### 4. Deploy Platform on Kubernetes (Helm)
Prerequisites: NGINX Ingress, cert-manager (if TLS enabled), StorageClass for PVCs.
### 4. Run the apps
```bash
# Build images (set API URL to match ingress.api.host when TLS is on)
export REG=your-registry.example.com
docker build -t $REG/cloudhost-backend:latest ./backend
docker build -t $REG/cloudhost-frontend:latest \
--build-arg NEXT_PUBLIC_API_URL=https://api.platform.example.com ./frontend
docker push $REG/cloudhost-backend:latest $REG/cloudhost-frontend:latest
# Install (copy and edit values-production.example.yaml first)
helm upgrade --install cloudhost ./backend/helm/cloudhost-platform \
-n cloudhost --create-namespace \
-f backend/helm/cloudhost-platform/values-production.example.yaml
```
Key values: `ingress.enabled`, `ingress.tls.enabled`, `ingress.frontend.host`, `ingress.api.host`, `postgres.password`, `secrets.jwtSecret`.
See chart defaults in `backend/helm/cloudhost-platform/values.yaml` and post-install notes via `helm get notes cloudhost -n cloudhost`.
### 5. Run Locally (development)
```bash
# Terminal 1 — Backend
# Terminal 1 — Backend (http://localhost:4000, prefix /api/v1, Swagger at /docs)
cd backend && npm run start:dev
# Terminal 2 — Frontend
# Terminal 2 — Frontend (http://localhost:3000)
cd frontend && npm run dev
```
---
In development `NODE_ENV=development`, so TypeORM `synchronize` builds the schema
automatically and the pricing catalog self-seeds. Set `frontend` `NEXT_PUBLIC_API_URL`
to the backend URL.
## API Endpoints
All endpoints prefixed with `/api/v1`. Full Swagger docs at `http://localhost:4000/docs`.
### Auth
| Method | Path | Description |
|--------|------|-------------|
| POST | /auth/register | Create account |
| POST | /auth/login | Get JWT tokens |
| POST | /auth/refresh | Refresh access token |
### Applications
| Method | Path | Description |
|--------|------|-------------|
| POST | /applications | Create app |
| GET | /applications | List user's apps |
| GET | /applications/:id | App details |
| PATCH | /applications/:id | Update app |
| DELETE | /applications/:id | Delete app + K8s resources |
### Deployments
| Method | Path | Description |
|--------|------|-------------|
| POST | /applications/:appId/deployments | Trigger deploy |
| GET | /applications/:appId/deployments | List deployments |
| GET | /deployments/:id | Deployment detail |
| GET | /deployments/:id/logs | Pod logs |
| POST | /deployments/:id/stop | Stop deployment |
| POST | /deployments/:id/restart | Restart deployment |
### Billing
| Method | Path | Description |
|--------|------|-------------|
| GET | /billing/balance | Get wallet balance |
| POST | /billing/deposit | Add funds to wallet |
| GET | /billing/transactions | Transaction history |
| POST | /billing/pay/:appId | Pay for app plan |
### Lifecycle (Admin)
| Method | Path | Description |
|--------|------|-------------|
| GET | /lifecycle/settings | Get retention periods |
| PATCH | /lifecycle/settings | Update retention periods |
### Snapshots
| Method | Path | Description |
|--------|------|-------------|
| POST | /snapshots | Create snapshot |
| GET | /snapshots | List snapshots |
| POST | /snapshots/:id/restore | Restore snapshot |
### Tickets
| Method | Path | Description |
|--------|------|-------------|
| POST | /tickets | Create ticket |
| GET | /tickets | List tickets |
| PATCH | /tickets/:id | Update ticket |
### Users
| Method | Path | Description |
|--------|------|-------------|
| GET | /users/me | Current user |
| PATCH | /users/me | Update profile |
### Admin — Users
| Method | Path | Description |
|--------|------|-------------|
| GET | /users | List all users |
| PATCH | /users/:id/activate | Activate user |
| PATCH | /users/:id/deactivate | Deactivate user |
| PATCH | /users/:id/role | Change role |
### Admin — Clusters
| Method | Path | Description |
|--------|------|-------------|
| POST | /clusters | Add cluster |
| GET | /clusters | List clusters |
| GET | /clusters/:id | Cluster details |
| PATCH | /clusters/:id | Update cluster |
| DELETE | /clusters/:id | Remove cluster |
> To run the **whole** stack (API + UI + Postgres + Redis) in containers instead:
> `docker compose up --build` (backend on `:4000`, frontend on `:3000`).
---
## Configuration
## Deploy on Kubernetes (Helm)
> This is the **generic** path. For the production `abrban.com` k3s cluster — base-image
> mirroring, the Iran-network proxy/npmmirror, the wildcard TLS cert, registry bootstrap,
> and the exact image-build flow — follow **[RUNBOOK.fa.md](RUNBOOK.fa.md)**.
**Prerequisites:** a Kubernetes cluster, an Ingress controller (Traefik on k3s by default,
or set `INGRESS_CLASS=nginx`), a default StorageClass for PVCs, and a container registry
reachable by the cluster.
### 1. Build & push the platform images
```bash
export REG=your-registry.example.com
docker build -t $REG/cloudhost-backend:1.0.0 ./backend
docker build -t $REG/cloudhost-frontend:1.0.0 \
--build-arg NEXT_PUBLIC_API_URL=https://api.platform.example.com ./frontend
docker push $REG/cloudhost-backend:1.0.0
docker push $REG/cloudhost-frontend:1.0.0
```
### 2. Install the control plane
```bash
cp backend/helm/cloudhost-platform/values-production.example.yaml my-values.yaml
# Edit my-values.yaml: image tags, ingress hosts, postgres password, jwtSecret, registry, SMS/OTP
helm upgrade --install cloudhost ./backend/helm/cloudhost-platform \
-n cloudhost --create-namespace \
-f my-values.yaml \
--set images.backend.tag=1.0.0 \
--set images.frontend.tag=1.0.0
```
Key values: `ingress.enabled`, `ingress.tls.*`, `ingress.frontend.host`, `ingress.api.host`,
`postgres.password`, `secrets.jwtSecret`, `migrations.enabled`. Chart defaults live in
`backend/helm/cloudhost-platform/values.yaml`; post-install notes via
`helm get notes cloudhost -n cloudhost`.
### 3. Cluster-side prerequisites for the build pipeline
Ensure the `cloudhost-builds` namespace has:
- the in-cluster **registry** (`registry:2`) reachable at `REGISTRY_URL`,
- a `kaniko-builder` ServiceAccount with an `imagePullSecret` for the registry,
- enough ephemeral storage for the per-build source PVC + helper pod.
### 4. Verify
```bash
kubectl get deploy -n cloudhost # backend & frontend 1/1
helm status cloudhost -n cloudhost # STATUS: deployed
curl -s -o /dev/null -w '%{http_code}\n' https://<frontend.host>
```
---
## Configuration (key env vars)
| Variable | Description | Default |
|----------|-------------|---------|
| `PORT` | Backend port | `4000` |
| `DB_HOST` | PostgreSQL host | `localhost` |
| `DB_PORT` | PostgreSQL port | `5432` |
| `DB_USERNAME` | Database user | `cloudhost` |
| `DB_PASSWORD` | Database password | — |
| `DB_NAME` | Database name | `cloudhost` |
| `JWT_SECRET` | JWT signing secret | — |
| `JWT_EXPIRES_IN` | Access token TTL | `15m` |
| `REDIS_HOST` | Redis host | `localhost` |
| `REDIS_PORT` | Redis port | `6379` |
| `REGISTRY_URL` | Container registry URL | `localhost:30500` |
| `PLATFORM_DOMAIN` | Base domain for app subdomains | `apps.cloudhost.ir` |
| `LIFECYCLE_SCAN_INTERVAL_MS` | Lifecycle scanner interval | `60000` |
| `LIFECYCLE_HOURLY_DELETE_AFTER_MS` | Hourly plan grace period | `3600000` (1h) |
| `LIFECYCLE_MONTHLY_DELETE_AFTER_MS` | Monthly plan grace period | `259200000` (3d) |
| `LIFECYCLE_YEARLY_DELETE_AFTER_MS` | Yearly plan grace period | `604800000` (7d) |
| `DB_HOST` / `DB_PORT` / `DB_USERNAME` / `DB_PASSWORD` / `DB_DATABASE` | PostgreSQL connection | `localhost` / `5432` / `cloudhost` / — / `cloudhost` |
| `JWT_SECRET` / `JWT_EXPIRES_IN` | Access token secret + TTL | — / `1h` |
| `JWT_REFRESH_SECRET` / `JWT_REFRESH_EXPIRES_IN` | Refresh token secret + TTL | — / `7d` |
| `REDIS_HOST` / `REDIS_PORT` | Redis (cache + Bull queues) | `localhost` / `6379` |
| `SMS_PROVIDER` | OTP provider (`mizbansms` \| `kavenegar`) | `mizbansms` |
| `MIZBANSMS_USERNAME` / `MIZBANSMS_PASSWORD` / `MIZBANSMS_FROM` | OTP SMS credentials (required or OTP send 503s) | — |
| `REGISTRY_URL` / `REGISTRY_PULL_URL` | In-cluster registry (push / pull) | `registry.cloudhost-builds.svc.cluster.local:5000` |
| `BUILD_NAMESPACE` / `BUILD_SERVICE_ACCOUNT` | Build Jobs namespace + SA | `cloudhost-builds` / `kaniko-builder` |
| `KANIKO_IMAGE` | Kaniko executor image | `gcr.io/kaniko-project/executor:v1.23.2` |
| `UPLOAD_DIR` | Disk path for uploaded source archives | `./uploads` |
| `INGRESS_CLASS` | Ingress controller for app Ingress objects | `traefik` |
| `PLATFORM_DOMAIN` / `PREVIEW_BASE_DOMAIN` | Base domain for app subdomains / previews | `apps.cloudhost.local` / — |
| `PLATFORM_STORAGE_CLASS` | StorageClass for new PVCs (needs volume expansion) | `cloudhost-expandable` |
| `ELASTICSEARCH_HOST` / `ELASTICSEARCH_PORT` | Log search backend | cluster DNS / `9200` |
| `LIFECYCLE_SCAN_INTERVAL_MS` | Lifecycle scanner tick | `60000` |
---
## Authentication
Login is **mobile-number based**: the user receives a one-time SMS code (OTP) and can also
set a password. On every request `JwtStrategy` re-reads the user's **role and active status
from the database** (not from the token), so promotions/deactivations take effect immediately.
Tokens: JWT access (`JWT_EXPIRES_IN`, default 1h) + refresh (7d).
---
## API
All endpoints are prefixed with `/api/v1`. Interactive Swagger docs at
`http://localhost:4000/api/docs`. Major route groups: `auth` (OTP request/verify, login,
refresh), `applications`, `deployments`, `clusters`, `billing` (wallet, invoices,
transactions, pricing), `snapshots`, `tickets`, `users`, `admin`, `notifications`.
---
## Security
- **JWT** access + refresh tokens with configurable expiry
- **Bcrypt** password hashing (12 rounds)
- **Helmet** HTTP security headers
- **RBAC** role-based route guards (`@Roles(UserRole.ADMIN)`)
- **Namespace isolation** — each user deploys to their own K8s namespace
- **Secrets** — env vars stored as K8s Secrets, never in plain manifests
- **Input validation** — `class-validator` on all DTOs
- **JWT** access + refresh tokens; live role/active-status enforcement from DB
- **Bcrypt** password hashing
- **Helmet** HTTP security headers, **class-validator** on all DTOs
- **RBAC** role-based route guards (`@Roles(...)`)
- **Namespace isolation** — each user deploys to their own Kubernetes namespace
- **Secrets** — env vars stored as K8s Secrets
---
RUNBOOK-CEPH.fa.md
+346
View File
@@ -0,0 +1,346 @@
# راهنمای Ceph برای ابربان (Rook)
این سند نصب، معماری واقعی روی کلاستر **abr**، مدیریت روزمره و عیب‌یابی **Rook-Ceph** را پوشش می‌دهد.
- چارت و اسکریپت‌ها: [`backend/helm/cloudhost-ceph/`](backend/helm/cloudhost-ceph/)
- README انگلیسی: [`backend/helm/cloudhost-ceph/README.md`](backend/helm/cloudhost-ceph/README.md)
- رجیستری: [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md)
---
## چرا Ceph؟
| نیاز | راه‌حل Ceph |
|------|-------------|
| PVC اپ/دیتابیس با **resize** | Block pool → StorageClass `rook-ceph-block` |
| آپلود **zip** سورس کاربر | Object store (RGW) → StorageClass `rook-ceph-bucket` |
یک کلاستر Ceph هر دو را پوشش می‌دهد؛ zip را روی PVC نگه ندارید — از **bucket** استفاده کنید.
---
## معماری روی abr (single-node)
```
registry.abrban.com
├── rook/ceph:v1.20.1 → Rook operator
└── proxy-dockerhub/ceph/ceph:v19.2 → Ceph daemon (Squid)
Node abr
├── /dev/loop6 (15Gi) → OSD (bluestore raw)
├── mon-a, mgr-a, osd-0, rgw → rook-ceph namespace
└── RGW: rook-ceph-rgw-ceph-objectstore.rook-ceph.svc:80
```
| محدودیت | توضیح |
|---------|--------|
| **۱ OSD** | replication=1؛ بدون HA |
| **loop device** | دیسک خام نداریم؛ `/dev/loop6` از فایل `osd-loopback.img` |
| **HEALTH_WARN** | طبیعی: `OSD count 1 < default size 3`، mon low space |
| **ایمیج‌ها** | باید از قبل در Harbor mirror شده باشند (kubelet به docker.io دسترسی ندارد) |
---
## پیش‌نیازها
### پروفایل `single-node` (کلاستر فعلی abr)
- Kubernetes 1.28+ (k3s + Traefik)
- Harbor بالا و ایمیج‌های `rook/ceph` + `ceph/ceph` mirror شده
- حداقل **۱۵ گیگ** فضا برای loop OSD (`/var/lib/rook/osd-loopback.img`)
- `helm` 3.x و `kubectl` با دسترسی cluster-admin
- Secret `registry-pull-secret` در `rook-ceph` با `harbor_registry_user`
### پروفایل `multi-node` (production)
- حداقل **۳ نود** + دیسک خام (raw)
- فایل values: `values-rook-cluster-multi-node.yaml`
---
## نصب (اولین بار — abr)
### ۱. آماده‌سازی loop device برای OSD
روی نود تک‌دیسک، Rook به دیسک خام نیاز دارد. یک loop device بسازید:
```bash
# روی نود (یا Job privileged — یک‌بار)
truncate -s 15G /var/lib/rook/osd-loopback.img
losetup --find --show /var/lib/rook/osd-loopback.img # → /dev/loop6
```
### ۲. فعال‌سازی loop در Rook operator
```bash
kubectl -n rook-ceph patch cm rook-ceph-operator-config --type merge \
-p '{"data":{"ROOK_CEPH_ALLOW_LOOP_DEVICES":"true"}}'
kubectl -n rook-ceph rollout restart deploy/rook-ceph-operator
```
### ۳. mirror ایمیج‌ها (قبل از نصب cluster)
| ایمیج | مسیر pull |
|-------|-----------|
| `rook/ceph:v1.20.1` | `registry.abrban.com/rook/ceph:v1.20.1` |
| `quay.io/ceph/ceph:v19.2` | `registry.abrban.com/proxy-dockerhub/ceph/ceph:v19.2` |
جزئیات mirror: [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md)
### ۴. نصب operator
```bash
helm repo add rook-release https://charts.rook.io/release
helm repo update rook-release
helm upgrade --install rook-ceph rook-release/rook-ceph \
-n rook-ceph --create-namespace \
--set image.repository=registry.abrban.com/rook/ceph \
--set image.tag=v1.20.1 \
--set imagePullSecrets[0].name=registry-pull-secret
```
### ۵. نصب cluster
```bash
cd backend/helm/cloudhost-ceph
helm upgrade --install rook-ceph-cluster rook-release/rook-ceph-cluster \
-n rook-ceph \
-f values-rook-cluster-single-node.yaml \
--set cephClusterSpec.cephVersion.image=registry.abrban.com/proxy-dockerhub/ceph/ceph:v19.2
```
> **توجه:** `values-rook-cluster-single-node.yaml` از `devices: [{name: "/dev/loop6"}]` استفاده می‌کند (نه directory — در Rook v1.20 حذف شده).
### ۶. extras (bucket + secret)
```bash
kubectl create namespace cloudhost-builds 2>/dev/null || true
helm upgrade --install cloudhost-ceph . \
-n cloudhost-builds -f values.yaml --no-hooks
```
اگر Job `bucket-sync` به‌خاطر `bitnami/kubectl` گیر کرد، secret را دستی بسازید:
```bash
kubectl -n cloudhost-builds get secret app-sources -o yaml
kubectl -n cloudhost-builds get cm app-sources -o yaml # BUCKET_NAME
# → secret ceph-app-sources-credentials (کلیدهای SOURCE_STORAGE_*)
```
### ۷. یکپارچه‌سازی backend
```bash
# کپی secret به namespace پلتفرم (یک‌بار)
kubectl -n cloudhost-builds get secret ceph-app-sources-credentials -o yaml \
| sed 's/namespace: cloudhost-builds/namespace: cloudhost/' \
| kubectl apply -f -
# یا با Helm (پیشنهادی):
helm upgrade cloudhost ./backend/helm/cloudhost-platform -n cloudhost \
--set backend.sourceStorage.enabled=true \
--set backend.env.PLATFORM_STORAGE_CLASS=rook-ceph-block \
--set backend.env.PLATFORM_CREATE_STORAGE_CLASS=false \
--set backend.env.PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com
```
بدون Helm می‌توانید دستی patch کنید:
```bash
kubectl -n cloudhost set env deploy/cloudhost-backend \
PLATFORM_STORAGE_CLASS=rook-ceph-block \
PLATFORM_CREATE_STORAGE_CLASS=false \
PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com
kubectl -n cloudhost patch deploy cloudhost-backend --type=json \
-p '[{"op":"add","path":"/spec/template/spec/containers/0/envFrom","value":[{"secretRef":{"name":"ceph-app-sources-credentials"}}]}]'
```
---
## اسکریپت `install.sh` (نصب تمیز)
برای نصب از صفر (بعد از آماده‌سازی loop + mirror):
```bash
cd backend/helm/cloudhost-ceph
./scripts/install.sh single-node
./scripts/verify.sh
```
اسکریپت operator + cluster + extras را نصب می‌کند. روی abr حتماً **قبلش** loop device و mirror ایمیج را انجام دهید.
---
## خروجی‌های مهم
### StorageClassها
| نام | کاربرد |
|-----|--------|
| `rook-ceph-block` | PVC اپ، DB، Redis، … |
| `rook-ceph-bucket` | claim کردن bucket برای zip |
### Secret پلتفرم
```bash
kubectl -n cloudhost-builds get secret ceph-app-sources-credentials -o yaml
kubectl -n cloudhost get secret ceph-app-sources-credentials -o yaml # کپی در cloudhost
```
کلیدها: `SOURCE_STORAGE_ENDPOINT`, `SOURCE_STORAGE_BUCKET`, `SOURCE_STORAGE_ACCESS_KEY`, `SOURCE_STORAGE_SECRET_KEY`
### RGW endpoint
```
http://rook-ceph-rgw-ceph-objectstore.rook-ceph.svc.cluster.local:80
```
---
## مدیریت روزمره
### سلامت کلاستر
```bash
kubectl -n rook-ceph get cephcluster
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph status
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph osd tree
kubectl get sc | grep rook-ceph
kubectl -n rook-ceph get pods
```
### Dashboard
```bash
kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath='{.data.password}' | base64 -d
kubectl -n rook-ceph port-forward svc/rook-ceph-mgr-dashboard 8443:8443
# https://localhost:8443
```
### bucket و OBC
```bash
kubectl -n cloudhost-builds get obc app-sources
kubectl -n cloudhost-builds get cm app-sources
```
### PVC جدید با Ceph
```yaml
storageClassName: rook-ceph-block
```
فقط **اپ‌های جدید** (یا بعد از migration) از این StorageClass استفاده می‌کنند. PVCهای قدیمی روی `local-path` / `cloudhost-expandable` خودکار منتقل نمی‌شوند.
---
## راه‌اندازی مجدد (reinstall)
### ۱. حذف Helm
```bash
cd backend/helm/cloudhost-ceph
./scripts/uninstall.sh
# تایپ: delete-ceph
```
### ۲. پاک‌سازی روی نود
```bash
sudo losetup -d /dev/loop6 2>/dev/null || true
sudo rm -f /var/lib/rook/osd-loopback.img
sudo rm -rf /var/lib/rook
```
### ۳. نصب مجدد
loop device + mirror + `./scripts/install.sh single-node`
---
## عیب‌یابی
### CephCluster در `Progressing` / Detecting version
```bash
kubectl -n rook-ceph get pods | grep detect-version
kubectl -n rook-ceph describe pod -l job-name=rook-ceph-detect-version
```
| خطا | راه‌حل |
|-----|--------|
| `ceph/ceph:v19.2 not found` | mirror از quay.io؛ tag صحیح `v19.2` نه `v19.2.1` |
| pull timeout | اولین pull بزرگ است (~500MB)؛ صبر یا image را از قبل روی نود بکشید |
| Job `detect-version` Terminating گیر کرد | `kubectl -n rook-ceph delete job rook-ceph-detect-version --force --grace-period=0` |
### OSD بالا نمی‌آید (OSD count 0)
```bash
kubectl -n rook-ceph logs -l app=rook-ceph-osd-prepare --tail=50
```
| خطا | راه‌حل |
|-----|--------|
| `unsupported diskType loop` | `ROOK_CEPH_ALLOW_LOOP_DEVICES=true` |
| `not picked by deviceFilter` | از `devices: [{name: "/dev/loop6"}]` استفاده کنید نه `deviceFilter` |
| `no devices matched` | `losetup -a` روی نود؛ loop6 وجود دارد؟ |
| `directories` در values | در Rook v1.20 کار نمی‌کند — loop یا raw disk |
### Volume mount روی rook-ceph-tools
`rook-ceph-mon-endpoints` و `rook-ceph-mon` تا قبل از بالا آمدن mon ساخته نمی‌شوند — طبیعی است؛ بعد از Ready برطرف می‌شود.
### Helm timeout روی apiserver
اگر `failed to download openapi` دیدید، بدون `--wait` نصب کنید و با `kubectl get cephcluster` پیگیری کنید.
### resize PVC
```bash
kubectl get storageclass rook-ceph-block -o yaml | grep allowVolumeExpansion
kubectl patch pvc <name> -n <ns> --type merge \
-p '{"spec":{"resources":{"requests":{"storage":"5Gi"}}}}'
```
---
## ارتقا (upgrade)
```bash
helm repo update rook-release
helm upgrade rook-ceph rook-release/rook-ceph -n rook-ceph \
--reuse-values --timeout 10m
helm upgrade rook-ceph-cluster rook-release/rook-ceph-cluster \
-n rook-ceph \
-f values-rook-cluster-single-node.yaml \
--set cephClusterSpec.cephVersion.image=registry.abrban.com/proxy-dockerhub/ceph/ceph:v19.2
helm upgrade cloudhost-ceph . -n cloudhost-builds -f values.yaml
```
قبل از upgrade در production: [Rook upgrade guide](https://rook.io/docs/rook/latest/Upgrade/ceph-upgrade/) و snapshot.
---
## نکات امنیتی
- RGW داخل کلاستر HTTP است — برای دسترسی خارجی ingress + TLS اضافه کنید.
- Secret `ceph-app-sources-credentials` را فقط به backend بدهید.
- `single-node` + ۱ OSD فقط staging است؛ production نیاز به ۳+ نود و دیسک جدا دارد.
---
## چک‌لیست بعد از نصب
- [ ] `ceph status` — mon/mgr/osd/rgw up
- [ ] `rook-ceph-block` و `rook-ceph-bucket` در `kubectl get sc`
- [ ] `ceph-app-sources-credentials` در `cloudhost-builds` و `cloudhost`
- [ ] env بک‌اند: `PLATFORM_STORAGE_CLASS=rook-ceph-block`
- [x] `SOURCE_STORAGE_*` در backend از secret خوانده می‌شود (`backend.sourceStorage.enabled=true` در Helm)
- [ ] اپ تست: آپلود zip و deploy با bucket فعال
- [ ] اپ تست با PVC جدید deploy شده
- [ ] ایمیج‌های Rook در Harbor موجود و pull تست شده
RUNBOOK-HARBOR.fa.md
+195
View File
@@ -0,0 +1,195 @@
# راهنمای Harbor — `registry.abrban.com`
این سند معماری فعلی، نصب، مدیریت روزمره و عیب‌یابی **Harbor** روی کلاستر abr را پوشش می‌دهد.
- چارت/values: [`backend/helm/cloudhost-harbor/`](backend/helm/cloudhost-harbor/)
- اسکریپت نصب: [`backend/helm/cloudhost-harbor/scripts/install-harbor-registry.sh`](backend/helm/cloudhost-harbor/scripts/install-harbor-registry.sh)
---
## معماری فعلی (خلاصه)
```
registry.abrban.com (Traefik + TLS wildcard)
├── / → harbor-portal (UI)
├── /api/, /service/, /c/ → harbor-core (API + auth)
├── /v2/proxy-dockerhub/ → harbor-registry (ایمیج‌های mirror‌شده Ceph/Rook)
├── /v2/rook/ → harbor-registry
└── /v2/* → registry قدیمی (ایمیج‌های platform: backend, nixpacks, …)
```
| کامپوننت | نقش |
|----------|-----|
| **Harbor** | UI، proxy-cache، ذخیره ایمیج‌های جدید |
| **registry قدیمی** (`Deployment/registry`) | هنوز بالاست؛ ایمیج‌های platform قبل از Harbor اینجاست |
| **registry-egress-proxy** | secret با `HTTP_PROXY` / `HTTPS_PROXY` برای pull از docker.io/quay از داخل کلاستر |
| **registry-pull-secret** | auth kubelet برای pull از `registry.abrban.com` |
> Harbor و registry قدیمی **هم‌زمان** روی یک hostname هستند؛ مسیر `/v2/` با Ingress split می‌شود.
---
## پیش‌نیازها
- Secret `abrban-wildcard-tls` در namespace `cloudhost`
- Secret `registry-egress-proxy` در namespace `cloudhost` (پروکسی egress)
- Helm repo: `helm repo add harbor https://helm.goharbor.io`
---
## نصب / ارتقا
```bash
cd backend/helm/cloudhost-harbor
./scripts/install-harbor-registry.sh
```
اسکریپت:
1. Harbor را با values + proxy از `registry-egress-proxy` نصب می‌کند
2. Ingress قدیمی `registry` را حذف می‌کند (بعد از نصب باید دستی دوباره route شود — بخش Ingress)
3. `Deployment/registry` را scale به 0 می‌کند (برای rollback نگه داشته می‌شود)
### Ingress بعد از نصب (الزامی)
Harbor به **چند مسیر** نیاز دارد. Ingress نهایی باید شبیه این باشد:
| Path | Service | Port |
|------|---------|------|
| `/` | `harbor-portal` | 80 |
| `/api/`, `/service/`, `/c/`, `/chartrepo/` | `harbor-core` | 80 |
| `/v2/proxy-dockerhub/`, `/v2/rook/` | `harbor-registry` | 5000 |
| `/v2/` (بقیه) | `registry` (قدیمی) | 5000 |
بدون split روی `/v2/`، یا UI 404 می‌دهد یا kubelet ایمیج platform را پیدا نمی‌کند.
---
## دسترسی و credentialها
| کاربرد | کاربر | منبع |
|--------|-------|------|
| UI / API مدیریت | `admin` | `kubectl -n cloudhost get secret harbor-core -o jsonpath='{.data.HARBOR_ADMIN_PASSWORD}' \| base64 -d` |
| push/pull داخلی به harbor-registry | `harbor_registry_user` | secret `harbor-core` → `REGISTRY_CREDENTIAL_PASSWORD` |
| pull kubelet (ایمیج‌های platform) | `cloudhost` | secret `registry-pull-secret` (namespace `cloudhost`) |
| pull kubelet (ایمیج‌های Rook/Ceph) | `harbor_registry_user` | secret `registry-pull-secret` (namespace `rook-ceph`) |
URL: https://registry.abrban.com/
---
## پروژه‌های proxy-cache
| پروژه | upstream | کاربرد |
|-------|----------|--------|
| `proxy-dockerhub` | docker.io | Rook، Ceph، bitnami، … |
| `proxy-quay` | quay.io | cephcsi و … |
| `proxy-k8s` | registry.k8s.io | CSI sidecarها |
ایجاد از UI: **Administration → Registries → New Endpoint** سپس **Projects → New Project** با نوع Proxy Cache.
> health check بعضی endpointها (مثلاً quay) از UI timeout می‌خورد؛ از داخل `harbor-core` با curl و proxy ممکن است OK باشد. در صورت نیاز endpoint را با type `docker-registry` بسازید.
### proxy در Harbor
پروکسی از secret `registry-egress-proxy` در ConfigMapهای `harbor-core` و `harbor-jobservice-env` تزریق می‌شود.
**مهم:** Go (harbor-core/jobservice) به `http_proxy` / `https_proxy` **lowercase** هم نیاز دارد. اگر health check upstream `unhealthy` ماند، هر دو حالت uppercase و lowercase را در ConfigMap بگذارید و podها را restart کنید:
```bash
kubectl -n cloudhost rollout restart deploy/harbor-core deploy/harbor-jobservice
```
---
## mirror دستی ایمیج (وقتی proxy-cache کار نمی‌کند)
روی abr، pull مستقیم از docker.io/quay از kubelet ممکن نیست. ایمیج‌های حیاتی را با Job داخل کلاستر mirror کنید:
**مقصد push:** `harbor-registry.cloudhost.svc.cluster.local:5000` (HTTP، با `harbor_registry_user`)
**مثال مسیرها در registry:**
| ایمیج upstream | مسیر در registry |
|----------------|------------------|
| `rook/ceph:v1.20.1` | `rook/ceph:v1.20.1` |
| `quay.io/ceph/ceph:v19.2` | `proxy-dockerhub/ceph/ceph:v19.2` |
```bash
# لیست ایمیج‌های داخل harbor-registry
kubectl -n cloudhost exec deploy/harbor-portal -- \
curl -s -u "harbor_registry_user:$(kubectl -n cloudhost get secret harbor-core -o jsonpath='{.data.REGISTRY_CREDENTIAL_PASSWORD}' | base64 -d)" \
http://harbor-registry:5000/v2/_catalog
```
> push مستقیم به `harbor-registry:5000` metadata در Harbor UI را به‌روز نمی‌کند؛ برای kubelet کافی است چون `/v2/` به harbor-registry route شده.
---
## مدیریت روزمره
### وضعیت
```bash
kubectl -n cloudhost get pods | grep harbor
kubectl -n cloudhost get ingress registry -o yaml | grep -A3 'path:'
curl -sk -o /dev/null -w "%{http_code}\n" https://registry.abrban.com/
curl -sk -u admin:<pass> https://registry.abrban.com/api/v2.0/systeminfo
```
### لاگ‌ها
```bash
kubectl -n cloudhost logs deploy/harbor-core --tail=50
kubectl -n cloudhost logs deploy/harbor-jobservice --tail=50
kubectl -n cloudhost logs deploy/harbor-registry -c registry --tail=50
```
### ارتقا Harbor
```bash
helm repo update harbor
./scripts/install-harbor-registry.sh
# Ingress split را دوباره تأیید کنید
```
### rollback به registry قدیمی
```bash
kubectl -n cloudhost scale deploy/registry --replicas=1
# Ingress را فقط به service registry:5000 برگردانید
helm uninstall harbor -n cloudhost
```
---
## عیب‌یابی
| علامت | علت محتمل | اقدام |
|-------|-----------|--------|
| `https://registry.abrban.com/` → 404 | Ingress فقط به `harbor-core` وصل است | `/` → `harbor-portal` |
| `ImagePullBackOff` برای `cloudhost-backend` | `/v2/` به Harbor رفته، ایمیج platform آنجا نیست | `/v2/` (عمومی) → `registry` قدیمی |
| `not found` برای `proxy-dockerhub/...` | ایمیج mirror نشده | Job skopeo یا proxy-cache |
| push با 499/503 | Traefik timeout | push از داخل کلاستر به `harbor-registry:5000` |
| registry endpoint `unhealthy` | proxy lowercase یا timeout health check | patch ConfigMap + restart؛ یا mirror دستی |
| `authentication required` روی pull | pull secret اشتباه namespace | `cloudhost` vs `harbor_registry_user` در `rook-ceph` |
### تست pull
```bash
# platform (cloudhost user)
curl -sk -u "cloudhost:<pass>" https://registry.abrban.com/v2/cloudhost-backend/tags/list
# rook/ceph (harbor_registry_user)
curl -sk -u "harbor_registry_user:<pass>" https://registry.abrban.com/v2/rook/ceph/tags/list
```
---
## چک‌لیست بعد از نصب
- [ ] Portal روی `/` پاسخ 200
- [ ] `/api/v2.0/systeminfo` پاسخ JSON
- [ ] Ingress split `/v2/` درست است
- [ ] پروژه‌های `proxy-dockerhub`, `proxy-k8s`, `proxy-quay` ساخته شده
- [ ] `registry-pull-secret` در `cloudhost` و `rook-ceph` به‌روز است
- [ ] ایمیج‌های Rook/Ceph mirror شده و pull تست شده
RUNBOOK.en.md
+69
View File
@@ -0,0 +1,69 @@
# CloudHost — Operations Runbook (English)
Short operational guide. For architecture details see [ARCHITECTURE.md](ARCHITECTURE.md). For Persian production deploy steps see [RUNBOOK.fa.md](RUNBOOK.fa.md).
## Stack
- **Frontend:** Next.js 16 (`/fa-IR`, `/en-US` routes)
- **Backend:** NestJS 11 (`/api/v1`, Swagger at `/api/docs`)
- **Data:** PostgreSQL 16, Redis 7
- **Build:** Kaniko in-cluster (`cloudhost-builds` namespace)
- **Deploy:** Helm charts (`cloudhost-platform`, `cloudhost-app`, `cloudhost-logging`)
## Health checks
| Endpoint | Purpose |
|----------|---------|
| `GET /api/v1/health` | Liveness |
| `GET /api/v1/ready` | Readiness (DB ping) |
## Local development
```bash
docker compose up -d postgres redis
cd backend && cp .env.example .env && npm run start:dev
cd frontend && cp .env.local.example .env.local && npm run dev
```
## SQL migrations
1. Add file under `backend/migrations/`
2. Run `cd backend && npm run sync:migrations` (copies into Helm chart)
3. Helm post-install Job applies migrations on upgrade
## Build namespace bootstrap
```bash
kubectl apply -f backend/k8s/builds/cloudhost-builds-bootstrap.yaml
```
Configure `REGISTRY_URL`, `BUILD_NAMESPACE`, and `CLUSTER_KUBECONFIG_KEY` in backend env.
## Backups (optional Helm)
Enable in `values.yaml`:
```yaml
backups:
postgres:
enabled: true
schedule: "0 3 * * *"
storageSize: 10Gi
```
Restore: `gunzip -c backup.sql.gz | psql` against the control-plane database.
## Monitoring (optional)
Enable Prometheus ServiceMonitor:
```yaml
monitoring:
enabled: true
```
Requires `kube-prometheus-stack` or compatible Prometheus Operator in the cluster.
## CI
GitHub Actions runs backend/frontend tests, typecheck, build, and `helm lint` on push/PR.
RUNBOOK.fa.md
+230
View File
@@ -0,0 +1,230 @@
# CloudHost — راهنمای معماری و اجرای وبسایت (Runbook)
این سند دو بخش دارد:
1. **اپلیکیشن چطور کار می‌کند** — معماری و جریان‌ها.
2. **اجرای وبسایت، مرحله‌به‌مرحله** — هم برای توسعه‌ی محلی، هم برای استقرار (deploy) روی کلاستر k3s سرور (abrban) به‌همراه راه‌حل‌های مخصوص شبکه‌ی ایران.
> اصطلاح‌ها: «پنل» = اپ احرازشده (`panel.abrban.com`)، «لندینگ» = صفحه‌ی معرفی (`abrban.com`)، «اپ کاربر» = اپلیکیشنی که مشتری روی CloudHost دیپلوی می‌کند.
> **به‌روزرسانی ۲۰۲۶:** pipeline بیلد فعلی از **Kaniko** + Dockerfileهای نگهداری‌شده توسط پلتفرم استفاده می‌کند (نه Nixpacks/MinIO). آرشیو سورس روی دیسک/PVC آپلود می‌شود. manifest بوت‌استرپ namespace بیلد: [`backend/k8s/builds/cloudhost-builds-bootstrap.yaml`](backend/k8s/builds/cloudhost-builds-bootstrap.yaml).
---
### ۱.۱ CloudHost چیست
یک **PaaS خودسرویس** برای بازار ایران: کاربر کد/ریپوی خودش را می‌دهد و CloudHost آن را build و روی Kubernetes اجرا می‌کند، با مدیریت دامنه، دیتابیس، لاگ، فاکتور و کیف پول.
### ۱.۲ اجزای اصلی
| جزء | تکنولوژی | نقش |
|---|---|---|
| **Frontend** | Next.js (App Router, SSR) | لندینگ + پنل کاربری/ادمین |
| **Backend** | NestJS (REST `/api/v1`) | منطق کسب‌وکار، ساخت اپ، احراز هویت |
| **Postgres** | postgres:16 | دیتابیس اصلی (کاربر، اپ، فاکتور، …) |
| **Redis** | redis:7 | کش، صف Bull (مهاجرت اپ، دسترسی موقت)، پیشرفت بیلد |
| **Registry داخلی** | Harbor + registry:2 (legacy) | ایمیج‌های build و platform؛ جزئیات: [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md) |
| **Storage (Ceph)** | Rook-Ceph | PVC (`rook-ceph-block`) + bucket zip (`rook-ceph-bucket`)؛ جزئیات: [`RUNBOOK-CEPH.fa.md`](RUNBOOK-CEPH.fa.md) |
| **Build pipeline** | Kaniko | تبدیل سورس به ایمیج Docker داخل کلاستر (بدون Docker daemon) |
| **Kubernetes** | k3s (تک‌نود) | اجرای همه‌ی موارد بالا + اپ‌های کاربر |
### ۱.۳ دامنه‌ها (همه روی `78.157.39.52`، HTTPS با wildcard cert)
- `abrban.com` → لندینگ
- `panel.abrban.com` → پنل احرازشده
- `api.abrban.com` → بک‌اند
- `registry.abrban.com` → Harbor (UI + proxy-cache) + registry قدیمی برای ایمیج‌های platform — [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md)
- `apps.abrban.com` → دامنه‌ی پیش‌فرض اپ‌های کاربر
### ۱.۴ جریان احراز هویت
- ورود مبتنی بر **موبایل + OTP** (پیامک از طریق MizbanSMS) یا رمز عبور.
- توکن **JWT** (access ~15m، refresh ~7d).
- `JwtStrategy` در هر درخواست نقش و فعال‌بودن کاربر را **از دیتابیس** می‌خواند (نه از توکن) تا تغییر نقش/غیرفعال‌سازی بلافاصله اثر کند.
### ۱.۵ جریان دیپلویِ «اپ کاربر» (مهم‌ترین بخش)
وقتی کاربر یک اپ را build/redeploy می‌کند:
```
کاربر (پنل)
│ آپلود zip ──────────────► MinIO (bucket: app-sources) ┐
│ یا git URL │ منبع سورس
▼ │
Backend: یک job در صف Bull («app-deploy») می‌گذارد │
▼ │
ساخت یک Kubernetes Job در namespace «cloudhost-builds»: │
1) init: fetch-source (دانلود از MinIO) یا git-clone ◄───────┘
2) init: nixpacks-prepare
• اگر سورس Dockerfile دارد → همان (BYO)
• وگرنه → با Nixpacks یک Dockerfile می‌سازد
3) container: Kaniko → build ایمیج → push به registry داخلی
▼
(غیرمسدودکننده) اسکن Trivy → خلاصه‌ی آسیب‌پذیری
▼
Backend با Helm، اپ را روی کلاستر بالا می‌آورد (Deployment + Service + Ingress)
▼
GC رجیستری: روزانه فقط N نسخه‌ی آخر هر اپ را نگه می‌دارد
```
- پیشرفت بیلد در Redis نگه‌داری می‌شود؛ فرانت هر ۱.۵ ثانیه `build-progress`/`build-logs` را poll می‌کند.
- توکن git در یک Secret موقتِ هر بیلد می‌نشیند و در `finally` پاک می‌شود (در manifest درج نمی‌شود).
---
## بخش ۲ — اجرای محلی (Local Dev)
پیش‌نیاز: Node 20، Docker، Docker Compose.
```bash
# ۱) دیتابیس و Redis
docker compose up -d # از docker-compose.yml ریشه‌ی پروژه
# ۲) بک‌اند
cd backend
cp .env.example .env # مقادیر را پر کن (DB، JWT، SMS، …)
npm install
npm run start:dev # روی http://localhost:4000 (پیشوند /api/v1)
# ۳) فرانت
cd ../frontend
npm install
npm run dev # روی http://localhost:3000
```
- در حالت dev، TypeORM `synchronize` روشن است و جدول‌ها خودکار ساخته می‌شوند.
- `NEXT_PUBLIC_API_URL` فرانت باید به آدرس بک‌اند اشاره کند.
---
## بخش ۳ — استقرار روی کلاستر (abrban / k3s)
> این بخش فرض می‌کند کلاستر k3s و wildcard cert از قبل آماده‌اند. context کوبه: `default`.
### ۳.۰ ثابت‌های محیط
| | مقدار |
|---|---|
| namespace اپ | `cloudhost` |
| namespace بیلد | `cloudhost-builds` |
| Helm release | `cloudhost` |
| چارت | `backend/helm/cloudhost-platform` |
| values | `/tmp/abrban/values-abrban.yaml` |
| رجیستری (push داخلی) | `registry.cloudhost.svc.cluster.local:5000` (HTTP, insecure) |
| رجیستری (pull توسط kubelet) | `registry.abrban.com` (HTTPS, wildcard cert) — همان storage |
| پروکسی build | `http://builder:<pw>@45.129.38.203:9911` |
### ۳.۱ نکات شبکه‌ی ایران (چرا کارها این‌شکلی‌اند)
- کلاستر به **Let's Encrypt، github، docker.io، ghcr.io، gcr.io** مستقیم نمی‌رسد (یا خیلی کند).
- **gTLS/cert**: دستی، secret `abrban-wildcard-tls` (نه cert-manager).
- **npm**: از `registry.npmmirror.com` مستقیم (نه پروکسی).
- **ایمیج‌های پایه**: اول داخل رجیستری داخلی **mirror** می‌شوند، بعد استفاده.
- **دانلودهای build (apk/nix/pip/…)**: از طریق پروکسی بالا.
### ۳.۲ گام‌های استقرار
**گام ۱ — بررسی دسترسی کلاستر**
```bash
kubectl config current-context # باید default باشد
kubectl get nodes
```
**گام ۲ — mirror کردن ایمیج‌های پایه به رجیستری داخلی**
ایمیج‌هایی که کلاستر مستقیم نمی‌تواند pull کند را با یک Job داخل کلاستر کپی کن.
- برای ایمیج‌های **کوچک**: `crane copy <src> registry.cloudhost.svc.cluster.local:5000/<dst> --insecure`
- برای ایمیج‌های **بزرگ** (مثل nixpacks): از **skopeo** استفاده کن — چون بلاب را با PUT یکجا آپلود می‌کند و گیر `PROTOCOL_ERROR` آپلود تکه‌ای crane را ندارد:
```
skopeo copy --override-os linux --override-arch amd64 --dest-tls-verify=false \
docker://<src> docker://registry.cloudhost.svc.cluster.local:5000/<dst>
```
(با `REGISTRY_AUTH_FILE` از secret `kaniko-docker-config` و env پروکسی.)
ایمیج‌های لازم: `minio/minio`, `railwayapp/nixpacks`, `library/alpine:3.19`, `library/postgres`, `library/redis`, و base ای که Nixpacks تولید می‌کند.
**گام ۳ — MinIO (ذخیره‌ی سورس اپ‌ها)**
به‌طور خودکار فقط هنگام **ثبت کلاستر جدید** ساخته می‌شود؛ روی کلاستر موجود دستی بساز: secret `minio-credentials` (`accesskey`/`secretkey`) + PVC ۲۰Gi + Deployment (`registry.abrban.com/minio/minio:latest`) + Service، همه در `cloudhost-builds`. کردنشال پیش‌فرض با config بک‌اند می‌خواند.
**گام ۴ — pull-secret برای namespace بیلد**
```bash
# کپی pull-secret رجیستری به ns بیلد
kubectl get secret registry-pull-secret -n cloudhost -o json \
| jq '.metadata.namespace="cloudhost-builds" | del(.metadata.uid,.metadata.resourceVersion,.metadata.creationTimestamp)' \
| kubectl apply -f -
# وصل به هر دو ServiceAccount که pod بیلد ممکن است از آن‌ها استفاده کند
kubectl patch sa default -n cloudhost-builds -p '{"imagePullSecrets":[{"name":"registry-pull-secret"}]}'
kubectl patch sa kaniko-builder -n cloudhost-builds -p '{"imagePullSecrets":[{"name":"registry-pull-secret"}]}'
```
> ⚠️ `kaniko-builder` حتماً لازم است: pod بیلد با همین SA اجرا می‌شود و init container نیکس‌پکس ایمیجش را از `registry.abrban.com` می‌کشد.
**گام ۵ — build ایمیج‌های frontend/backend (Kaniko)**
سورس را در PVC بیلد (`build-src`) از طریق pod `srcsync` قرار بده، سپس Job‌های Kaniko را اجرا کن.
```bash
# سینک سورس (از working tree؛ tsbuildinfo و dist را حذف کن!)
cd <repo>
tar czf - --exclude=node_modules --exclude=.next --exclude=.git frontend \
| kubectl exec -i srcsync -n cloudhost -- sh -c 'rm -rf /workspace/frontend && tar xzf - -C /workspace'
rm -f backend/tsconfig.tsbuildinfo # ← مهم
tar czf - --exclude=node_modules --exclude=dist --exclude=.git backend \
| kubectl exec -i srcsync -n cloudhost -- sh -c 'rm -rf /workspace/backend && tar xzf - -C /workspace'
# build (manifestهای kaniko-*.yaml: registry-mirror + proxy + npmmirror)
kubectl apply -f /tmp/abrban/kaniko-frontend-<tag>.yaml
kubectl apply -f /tmp/abrban/kaniko-backend-<tag>.yaml
```
**گام ۶ — استقرار با Helm**
```bash
helm upgrade cloudhost backend/helm/cloudhost-platform \
-n cloudhost -f /tmp/abrban/values-abrban.yaml \
--set images.backend.tag=<tag> \
--set images.frontend.tag=<tag> \
--set migrations.enabled=false # ← مهاجرت‌ها روی DB زنده تداخل دارند
```
> بدون `--wait` اجرا کن (وگرنه به‌خاطر کندیِ pull، status اشتباهاً `failed` می‌شود درحالی‌که rollout موفق است).
**گام ۷ — bootstrap اسکیمای دیتابیس (فقط روی DB تازه)**
در پروداکشن `synchronize` خاموش است. روی DB کاملاً تازه: موقتاً `NODE_ENV=development` کن تا synchronize جدول‌ها را بسازد و pricing خودش seed شود، بعد به `production` برگردان. روی DB موجود، فقط مهاجرت‌های idempotent جدید را با یک Job جدا اعمال کن (نه helm hook).
**گام ۸ — تأیید**
```bash
kubectl get deploy -n cloudhost # backend/frontend 1/1
helm status cloudhost -n cloudhost # STATUS: deployed
curl -s -o /dev/null -w '%{http_code}\n' https://panel.abrban.com # 307
curl -s -X POST https://api.abrban.com/api/v1/auth/otp/request \
-H 'Content-Type: application/json' -d '{"phone":"09xxxxxxxxx"}' # {"sent":true}
```
---
## بخش ۴ — env‌های کلیدی build pipeline (روی بک‌اند)
این‌ها در `backend.env` فایل values ست می‌شوند:
| env | مقدار/توضیح |
|---|---|
| `MINIO_SECRET_KEY` | کلید MinIO (هماهنگ با secret) |
| `NIXPACKS_IMAGE` | `registry.abrban.com/railwayapp/nixpacks:latest` (mirror) |
| `NIXPACKS_BUILD_ENV` | `NPM_CONFIG_REGISTRY=https://registry.npmmirror.com` |
| `BUILD_HTTP_PROXY` | پروکسی build (تزریق به Kaniko + init containerها) |
| `BUILD_REGISTRY_MIRROR` | `registry.cloudhost.svc.cluster.local:5000` (pull پایه از mirror) |
| `BUILD_SCAN_ENABLED` | `false` تا وقتی ایمیج Trivy mirror شود |
| `SMS_PROVIDER` + `MIZBANSMS_*` | بدون این‌ها ارسال OTP خطای 503 می‌دهد |
---
## بخش ۵ — عیب‌یابی رایج
| نشانه | علت | راه‌حل |
|---|---|---|
| backend کرش: `Cannot find module '/app/dist/main.js'` | `tsconfig.tsbuildinfo` کهنه در سورس → tsc فایل‌ها را دوباره emit نمی‌کند | قبل از build، `tsconfig.tsbuildinfo` را حذف کن |
| init container بیلد: `no basic auth credentials` | SA `kaniko-builder` بدون pull-secret | گام ۴ را اجرا کن |
| pull ایمیج: `not found` با اینکه push شده | crane در آپلود تکه‌ایِ بلاب بزرگ شکست خورده (tag ناقص) | با **skopeo** دوباره mirror کن |
| pull از docker.io: `TLS handshake timeout` | docker.io از کلاستر بسته است | ایمیج را mirror کن |
| `nixpacks: not found` در init | باینری نیکس‌پکس روی PATH پیش‌فرض نیست | command را با مسیر/ENTRYPOINT درست صدا بزن |
| helm: `no template "...namespace"` یا `nil pointer redis.enabled` | فایل‌های چارت (`_helpers.tpl`/`values.yaml`) از `/tmp` پاک شده‌اند | از سورس اصلی بازیابی + ادیت‌های deploy را دوباره اعمال کن |
| OTP خطای 503 | env پیامک ست نیست | `SMS_PROVIDER`/`MIZBANSMS_*` را ست کن |
> ⚠️ پوشه‌ی `/tmp` در macOS فایل‌های قدیمی‌تر از ~۳ روز را پاک می‌کند؛ درخت کاری deploy در `/tmp` ممکن است فایل از دست بدهد — قبل از build بررسی کن.
---
## بخش ۶ — به‌روزرسانی نسخه (خلاصه)
1. تغییرات کد را در سورس اعمال کن (`tsc --noEmit` بگیر).
2. tag جدید انتخاب کن.
3. سورس را در `srcsync` سینک کن (با حذف `tsbuildinfo`).
4. Job Kaniko را با tag جدید بساز.
5. `helm upgrade ... --set images.*.tag=<tag> --set migrations.enabled=false` (بدون `--wait`).
6. تأیید کن: podها 1/1، helm `deployed`، endpoint‌ها سالم.
backend/.env.example
+7
View File
@@ -20,6 +20,13 @@ JWT_REFRESH_EXPIRES_IN=7d
REDIS_HOST=localhost
REDIS_PORT=6379
# Multi-cluster: AES-256-GCM key for encrypting kubeconfigs at rest (required in production).
# Generate with: openssl rand -hex 32
CLUSTER_KUBECONFIG_KEY=
# Stub payment gateway (dev/staging only — disabled in production unless explicitly enabled)
# PAYMENT_GATEWAY_STUB_ENABLED=true
# ─── OTP SMS ────────────────────────────────────────────────────────────────
# Pick the provider. Without valid credentials, OTP codes are logged to the API
# console in development only; in production a missing config makes OTP send fail
backend/eslint.config.mjs
+28
View File
@@ -0,0 +1,28 @@
import tsPlugin from '@typescript-eslint/eslint-plugin';
import tsParser from '@typescript-eslint/parser';
/** @type {import('eslint').Linter.Config[]} */
export default [
{
ignores: ['dist/**', 'node_modules/**', 'coverage/**'],
},
{
files: ['src/**/*.ts', 'test/**/*.ts'],
languageOptions: {
parser: tsParser,
parserOptions: {
ecmaVersion: 'latest',
sourceType: 'module',
},
},
plugins: {
'@typescript-eslint': tsPlugin,
},
rules: {
...tsPlugin.configs.recommended.rules,
'@typescript-eslint/no-explicit-any': 'off',
'@typescript-eslint/no-require-imports': 'off',
'@typescript-eslint/no-unused-vars': ['warn', { argsIgnorePattern: '^_' }],
},
},
];
backend/helm/cloudhost-ceph/.helmignore
+3
View File
@@ -0,0 +1,3 @@
.rook-cluster-values-ref.yaml
*.md
scripts/
backend/helm/cloudhost-ceph/.rook-cluster-values-ref.yaml
+758
View File
@@ -0,0 +1,758 @@
# Default values for a single rook-ceph cluster
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
# -- Namespace of the main rook operator
operatorNamespace: rook-ceph
# -- The metadata.name of the CephCluster CR
# @default -- The same as the namespace
clusterName:
# -- Optional override of the target kubernetes version
kubeVersion:
# -- Cluster ceph.conf override
configOverride:
# configOverride: |
# [global]
# mon_allow_pool_delete = true
# osd_pool_default_size = 3
# osd_pool_default_min_size = 2
# Installs a debugging toolbox deployment
toolbox:
# -- Enable Ceph debugging pod deployment. See [toolbox](../Troubleshooting/ceph-toolbox.md)
enabled: false
# -- Toolbox image, defaults to the image used by the Ceph cluster
image: #quay.io/ceph/ceph:v20.2.1
# -- Toolbox tolerations
tolerations: []
# -- Toolbox affinity
affinity: {}
# -- Toolbox labels
labels: {}
# -- Toolbox container security context
containerSecurityContext:
runAsNonRoot: true
runAsUser: 2016
runAsGroup: 2016
capabilities:
drop: ["ALL"]
# -- Toolbox resources
resources:
limits:
memory: "1Gi"
requests:
cpu: "100m"
memory: "128Mi"
# -- Set the priority class for the toolbox if desired
priorityClassName:
monitoring:
# -- Enable Prometheus integration, will also create necessary RBAC rules to allow Operator to create ServiceMonitors.
# Monitoring requires Prometheus to be pre-installed
enabled: false
# -- Whether to disable the metrics reported by Ceph. If false, the prometheus mgr module and Ceph exporter are enabled
metricsDisabled: false
# -- Whether to create the Prometheus rules for Ceph alerts
createPrometheusRules: false
# -- Edit Prometheus rules for Ceph alerts
prometheusRuleOverrides: {}
# CephHealthWarning:
# disabled: true
# NVMeoFHighWriteLatency:
# for: 3m
# labels:
# severity: critical
# -- The namespace in which to create the prometheus rules, if different from the rook cluster namespace.
# If you have multiple rook-ceph clusters in the same k8s cluster, choose the same namespace (ideally, namespace with prometheus
# deployed) to set rulesNamespaceOverride for all the clusters. Otherwise, you will get duplicate alerts with multiple alert definitions.
rulesNamespaceOverride:
# Monitoring settings for external clusters:
# externalMgrEndpoints: <list of endpoints>
# externalMgrPrometheusPort: <port>
# Scrape interval for prometheus
# interval: 10s
# allow adding custom labels and annotations to the prometheus rule
prometheusRule:
# -- Labels applied to PrometheusRule
labels: {}
# -- Annotations applied to PrometheusRule
annotations: {}
# imagePullSecrets option allow to pull docker images from private docker registry. Option will be passed to all service accounts.
# imagePullSecrets:
# - name: my-registry-secret
# Labels and annotations to add to the CephCluster CR
cephClusterMetadata:
annotations: {}
labels: {}
# Specify these values to override the Ceph image in the cephClusterSpec below.
# If specifying these values, do not include the cephVersion section in the cephClusterSpec.
cephImage:
# The repository from which to pull the ceph image
repository: quay.io/ceph/ceph
# In production, use a specific version tag instead of the general v20 flag, which pulls the latest release and could result in different
# versions running within the cluster. See tags available at https://hub.docker.com/r/ceph/ceph/tags/.
# To be more precise, you can always use a timestamp tag such as quay.io/ceph/ceph:v20.2.1-20260402
tag: v20.2.1
# Whether to allow unsupported versions of Ceph. Currently Squid and Tentacle are supported.
# Future versions would require this to be set to `true`.
# Do not set to true in production.
allowUnsupported: false
# The image pull policy for pulling the ceph image in the ceph daemon pods, defaults to IfNotPresent
# imagePullPolicy: IfNotPresent
# All values below are taken from the CephCluster CRD
# -- Cluster configuration.
# @default -- See [below](#ceph-cluster-spec)
cephClusterSpec:
# This cluster spec example is for a converged cluster where all the Ceph daemons are running locally,
# as in the host-based example (cluster.yaml). For a different configuration such as a
# PVC-based cluster (cluster-on-pvc.yaml), external cluster (cluster-external.yaml),
# or stretch cluster (cluster-stretched.yaml), replace this entire `cephClusterSpec`
# with the specs from those examples.
# For more details, check https://rook.io/docs/rook/v1.10/CRDs/Cluster/ceph-cluster-crd/
# The path on the host where configuration files will be persisted. Must be specified. If there are multiple clusters, the directory must be unique for each cluster.
# Important: if you reinstall the cluster, make sure you delete this directory from each host or else the mons will fail to start on the new cluster.
# In Minikube, the '/data' directory is configured to persist across reboots. Use "/data/rook" in Minikube environment.
dataDirHostPath: /var/lib/rook
# Whether or not upgrade should continue even if a check fails
# This means Ceph's status could be degraded and we don't recommend upgrading but you might decide otherwise
# Use at your OWN risk
# To understand Rook's upgrade process of Ceph, read https://rook.io/docs/rook/v1.10/Upgrade/ceph-upgrade/
skipUpgradeChecks: false
# Whether or not continue if PGs are not clean during an upgrade
continueUpgradeAfterChecksEvenIfNotHealthy: false
# WaitTimeoutForHealthyOSDInMinutes defines the time (in minutes) the operator would wait before an OSD can be stopped for upgrade or restart.
# If the timeout exceeds and OSD is not ok to stop, then the operator would skip upgrade for the current OSD and proceed with the next one
# if `continueUpgradeAfterChecksEvenIfNotHealthy` is `false`. If `continueUpgradeAfterChecksEvenIfNotHealthy` is `true`, then operator would
# continue with the upgrade of an OSD even if its not ok to stop after the timeout. This timeout won't be applied if `skipUpgradeChecks` is `true`.
# The default wait timeout is 10 minutes.
waitTimeoutForHealthyOSDInMinutes: 10
# Whether or not requires PGs are clean before an OSD upgrade. If set to `true` OSD upgrade process won't start until PGs are healthy.
# This configuration will be ignored if `skipUpgradeChecks` is `true`.
# Default is false.
upgradeOSDRequiresHealthyPGs: false
mon:
# Set the number of mons to be started. Generally recommended to be 3.
# For highest availability, an odd number of mons should be specified.
count: 3
# The mons should be on unique nodes. For production, at least 3 nodes are recommended for this reason.
# Mons should only be allowed on the same node for test environments where data loss is acceptable.
allowMultiplePerNode: false
mgr:
# When higher availability of the mgr is needed, increase the count to 2.
# In that case, one mgr will be active and one in standby. When Ceph updates which
# mgr is active, Rook will update the mgr services to match the active mgr.
count: 2
allowMultiplePerNode: false
modules:
# List of modules to optionally enable or disable.
# Note the "dashboard" and "monitoring" modules are already configured by other settings in the cluster CR.
# - name: rook
# enabled: true
# enable the ceph dashboard for viewing cluster status
dashboard:
enabled: true
# serve the dashboard under a subpath (useful when you are accessing the dashboard via a reverse proxy)
# urlPrefix: /ceph-dashboard
# serve the dashboard at the given port.
# port: 8443
# Serve the dashboard using SSL (if using ingress to expose the dashboard and `ssl: true` you need to set
# the corresponding "backend protocol" annotation(s) for your ingress controller of choice)
ssl: true
# Network configuration, see: https://github.com/rook/rook/blob/master/Documentation/CRDs/Cluster/ceph-cluster-crd.md#network-configuration-settings
network:
connections:
# Whether to encrypt the data in transit across the wire to prevent eavesdropping the data on the network.
# The default is false. When encryption is enabled, all communication between clients and Ceph daemons, or between Ceph daemons will be encrypted.
# When encryption is not enabled, clients still establish a strong initial authentication and data integrity is still validated with a crc check.
# IMPORTANT: Encryption requires the 5.11 kernel for the latest nbd and cephfs drivers. Alternatively for testing only,
# you can set the "mounter: rbd-nbd" in the rbd storage class, or "mounter: fuse" in the cephfs storage class.
# The nbd and fuse drivers are *not* recommended in production since restarting the csi driver pod will disconnect the volumes.
encryption:
enabled: false
# Whether to compress the data in transit across the wire. The default is false.
# The kernel requirements above for encryption also apply to compression.
compression:
enabled: false
# Whether to require communication over msgr2. If true, the msgr v1 port (6789) will be disabled
# and clients will be required to connect to the Ceph cluster with the v2 port (3300).
# Requires a kernel that supports msgr v2 (kernel 5.11 or CentOS 8.4 or newer).
requireMsgr2: false
# # enable host networking
# provider: host
# # EXPERIMENTAL: enable the Multus network provider
# provider: multus
# selectors:
# # The selector keys are required to be `public` and `cluster`.
# # Based on the configuration, the operator will do the following:
# # 1. if only the `public` selector key is specified both public_network and cluster_network Ceph settings will listen on that interface
# # 2. if both `public` and `cluster` selector keys are specified the first one will point to 'public_network' flag and the second one to 'cluster_network'
# #
# # In order to work, each selector value must match a NetworkAttachmentDefinition object in Multus
# #
# # public: public-conf --> NetworkAttachmentDefinition object name in Multus
# # cluster: cluster-conf --> NetworkAttachmentDefinition object name in Multus
# # Provide internet protocol version. IPv6, IPv4 or empty string are valid options. Empty string would mean IPv4
# ipFamily: "IPv6"
# # Ceph daemons to listen on both IPv4 and Ipv6 networks
# dualStack: false
# enable the crash collector for ceph daemon crash collection
crashCollector:
disable: false
# Uncomment daysToRetain to prune ceph crash entries older than the
# specified number of days.
# daysToRetain: 30
# enable log collector, daemons will log on files and rotate
logCollector:
enabled: true
periodicity: daily # one of: hourly, daily, weekly, monthly
maxLogSize: 500M # SUFFIX may be 'M' or 'G'. Must be at least 1M.
# automate [data cleanup process](https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/ceph-teardown.md#delete-the-data-on-hosts) in cluster destruction.
cleanupPolicy:
# Since cluster cleanup is destructive to data, confirmation is required.
# To destroy all Rook data on hosts during uninstall, confirmation must be set to "yes-really-destroy-data".
# This value should only be set when the cluster is about to be deleted. After the confirmation is set,
# Rook will immediately stop configuring the cluster and only wait for the delete command.
# If the empty string is set, Rook will not destroy any data on hosts during uninstall.
confirmation: ""
# sanitizeDisks represents settings for sanitizing OSD disks on cluster deletion
sanitizeDisks:
# method indicates if the entire disk should be sanitized or simply ceph's metadata
# in both case, re-install is possible
# possible choices are 'complete' or 'quick' (default)
method: quick
# dataSource indicate where to get random bytes from to write on the disk
# possible choices are 'zero' (default) or 'random'
# using random sources will consume entropy from the system and will take much more time then the zero source
dataSource: zero
# iteration overwrite N times instead of the default (1)
# takes an integer value
iteration: 1
# allowUninstallWithVolumes defines how the uninstall should be performed
# If set to true, cephCluster deletion does not wait for the PVs to be deleted.
allowUninstallWithVolumes: false
# To control where various services will be scheduled by kubernetes, use the placement configuration sections below.
# The example under 'all' would have all services scheduled on kubernetes nodes labeled with 'role=storage-node' and
# tolerate taints with a key of 'storage-node'.
# placement:
# all:
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: role
# operator: In
# values:
# - storage-node
# podAffinity:
# podAntiAffinity:
# topologySpreadConstraints:
# tolerations:
# - key: storage-node
# operator: Exists
# # The above placement information can also be specified for mon, osd, and mgr components
# mon:
# # Monitor deployments may contain an anti-affinity rule for avoiding monitor
# # collocation on the same node. This is a required rule when host network is used
# # or when AllowMultiplePerNode is false. Otherwise this anti-affinity rule is a
# # preferred rule with weight: 50.
# osd:
# mgr:
# cleanup:
# annotations:
# all:
# mon:
# osd:
# cleanup:
# prepareosd:
# # If no mgr annotations are set, prometheus scrape annotations will be set by default.
# mgr:
# dashboard:
# labels:
# all:
# mon:
# osd:
# cleanup:
# mgr:
# prepareosd:
# # monitoring is a list of key-value pairs. It is injected into all the monitoring resources created by operator.
# # These labels can be passed as LabelSelector to Prometheus
# monitoring:
# dashboard:
resources:
mgr:
limits:
memory: "1Gi"
requests:
cpu: "500m"
memory: "512Mi"
mon:
limits:
memory: "2Gi"
requests:
cpu: "1000m"
memory: "1Gi"
osd:
limits:
memory: "4Gi"
requests:
cpu: "1000m"
memory: "4Gi"
prepareosd:
# limits: It is not recommended to set limits on the OSD prepare job
# since it's a one-time burst for memory that must be allowed to
# complete without an OOM kill. Note however that if a k8s
# limitRange guardrail is defined external to Rook, the lack of
# a limit here may result in a sync failure, in which case a
# limit should be added. 1200Mi may suffice for up to 15Ti
# OSDs ; for larger devices 2Gi may be required.
# cf. https://github.com/rook/rook/pull/11103
requests:
cpu: "500m"
memory: "50Mi"
mgr-sidecar:
limits:
memory: "100Mi"
requests:
cpu: "100m"
memory: "40Mi"
crashcollector:
limits:
memory: "60Mi"
requests:
cpu: "100m"
memory: "60Mi"
logcollector:
limits:
memory: "1Gi"
requests:
cpu: "100m"
memory: "100Mi"
cleanup:
limits:
memory: "1Gi"
requests:
cpu: "500m"
memory: "100Mi"
exporter:
limits:
memory: "128Mi"
requests:
cpu: "50m"
memory: "50Mi"
cmd-reporter:
limits:
memory: "1Gi"
requests:
cpu: "500m"
memory: "100Mi"
# The option to automatically remove OSDs that are out and are safe to destroy.
removeOSDsIfOutAndSafeToRemove: false
# priority classes to apply to ceph resources
priorityClassNames:
mon: system-node-critical
osd: system-node-critical
mgr: system-cluster-critical
storage: # cluster level storage configuration and selection
useAllNodes: true
useAllDevices: true
# deviceFilter:
# config:
# crushRoot: "custom-root" # specify a non-default root label for the CRUSH map
# metadataDevice: "md0" # specify a non-rotational storage so ceph-volume will use it as block db device of bluestore.
# databaseSizeMB: "1024" # uncomment if the disks are smaller than 100 GB
# osdsPerDevice: "1" # this value can be overridden at the node or device level
# encryptedDevice: "true" # the default value for this option is "false"
# # Individual nodes and their config can be specified as well, but 'useAllNodes' above must be set to false. Then, only the named
# # nodes below will be used as storage resources. Each node's 'name' field should match their 'kubernetes.io/hostname' label.
# nodes:
# - name: "172.17.4.201"
# devices: # specific devices to use for storage can be specified for each node
# - name: "sdb"
# - name: "nvme01" # multiple osds can be created on high performance devices
# config:
# osdsPerDevice: "5"
# - name: "/dev/disk/by-id/ata-ST4000DM004-XXXX" # devices can be specified using full udev paths
# config: # configuration can be specified at the node level which overrides the cluster level config
# - name: "172.17.4.301"
# deviceFilter: "^sd."
# The section for configuring management of daemon disruptions during upgrade or fencing.
disruptionManagement:
# If true, the operator will create and manage PodDisruptionBudgets for OSD, Mon, RGW, and MDS daemons. OSD PDBs are managed dynamically
# via the strategy outlined in the [design](https://github.com/rook/rook/blob/master/design/ceph/ceph-managed-disruptionbudgets.md). The operator will
# block eviction of OSDs by default and unblock them safely when drains are detected.
managePodBudgets: true
# A duration in minutes that determines how long an entire failureDomain like `region/zone/host` will be held in `noout` (in addition to the
# default DOWN/OUT interval) when it is draining. This is only relevant when `managePodBudgets` is `true`. The default value is `30` minutes.
osdMaintenanceTimeout: 30
# Configure the healthcheck and liveness probes for ceph pods.
# Valid values for daemons are 'mon', 'osd', 'status'
healthCheck:
daemonHealth:
mon:
disabled: false
interval: 45s
osd:
disabled: false
interval: 60s
status:
disabled: false
interval: 60s
# Change pod liveness probe, it works for all mon, mgr, and osd pods.
livenessProbe:
mon:
disabled: false
mgr:
disabled: false
osd:
disabled: false
ingress:
# -- Enable an ingress for the ceph-dashboard
dashboard: {}
# labels:
# external-dns/private: "true"
# annotations:
# external-dns.alpha.kubernetes.io/hostname: dashboard.example.com
# nginx.ingress.kubernetes.io/rewrite-target: /ceph-dashboard/$2
# If the dashboard has ssl: true the following will make sure the NGINX Ingress controller can expose the dashboard correctly
# nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
# nginx.ingress.kubernetes.io/server-snippet: |
# proxy_ssl_verify off;
# host:
# name: dashboard.example.com
# path: "/ceph-dashboard(/|$)(.*)"
# pathType: Prefix
# tls:
# - hosts:
# - dashboard.example.com
# secretName: testsecret-tls
## Note: Only one of ingress class annotation or the `ingressClassName:` can be used at a time
## to set the ingress class
# ingressClassName: nginx
route:
# -- Enable an HTTPRoute for the ceph-dashboard
dashboard: {}
# labels:
# external-dns/private: "true"
# annotations:
# external-dns.alpha.kubernetes.io/hostname: dashboard.example.com
# nginx.ingress.kubernetes.io/rewrite-target: /ceph-dashboard/$2
# host:
# name: dashboard.example.com
# path: "/"
# pathType: PathPrefix
# parentRefs:
# - name: internal
# namespace: kube-system
# sectionName: https
# -- A list of CephBlockPool configurations to deploy
# @default -- See [below](#ceph-block-pools)
cephBlockPools:
- name: ceph-blockpool
# see https://github.com/rook/rook/blob/master/Documentation/CRDs/Block-Storage/ceph-block-pool-crd.md#spec for available configuration
spec:
failureDomain: host
replicated:
size: 3
# Enables collecting RBD per-image IO statistics by enabling dynamic OSD performance counters. Defaults to false.
# For reference: https://docs.ceph.com/docs/latest/mgr/prometheus/#rbd-io-statistics
# enableRBDStats: true
storageClass:
enabled: true
name: ceph-block
annotations: {}
labels: {}
isDefault: true
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: "Immediate"
mountOptions: []
# see https://kubernetes.io/docs/concepts/storage/storage-classes/#allowed-topologies
allowedTopologies: []
# - matchLabelExpressions:
# - key: rook-ceph-role
# values:
# - storage-node
# see https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/Block-Storage-RBD/block-storage.md#provision-storage for available configuration
parameters:
# (optional) mapOptions is a comma-separated list of map options.
# For krbd options refer
# https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
# For nbd options refer
# https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
# mapOptions: lock_on_read,queue_depth=1024
# (optional) unmapOptions is a comma-separated list of unmap options.
# For krbd options refer
# https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
# For nbd options refer
# https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
# unmapOptions: force
# RBD image format. Defaults to "2".
imageFormat: "2"
# RBD image features, equivalent to OR'd bitfield value: 63
# Available for imageFormat: "2". Older releases of CSI RBD
# support only the `layering` feature. The Linux kernel (KRBD) supports the
# full feature complement as of 5.4
imageFeatures: layering
# These secrets contain Ceph admin credentials.
csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
csi.storage.k8s.io/provisioner-secret-namespace: "{{ .Release.Namespace }}"
csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner
csi.storage.k8s.io/controller-expand-secret-namespace: "{{ .Release.Namespace }}"
csi.storage.k8s.io/controller-publish-secret-name: rook-csi-rbd-provisioner
csi.storage.k8s.io/controller-publish-secret-namespace: "{{ .Release.Namespace }}"
csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
csi.storage.k8s.io/node-stage-secret-namespace: "{{ .Release.Namespace }}"
# Specify the filesystem type of the volume. If not specified, csi-provisioner
# will set default as `ext4`. Note that `xfs` is not recommended due to potential deadlock
# in hyperconverged settings where the volume is mounted on the same node as the osds.
csi.storage.k8s.io/fstype: ext4
# -- A list of CephFileSystem configurations to deploy
# @default -- See [below](#ceph-file-systems)
cephFileSystems:
- name: ceph-filesystem
# see https://github.com/rook/rook/blob/master/Documentation/CRDs/Shared-Filesystem/ceph-filesystem-crd.md#filesystem-settings for available configuration
spec:
metadataPool:
replicated:
size: 3
dataPools:
- failureDomain: host
replicated:
size: 3
# Optional and highly recommended, 'data0' by default, see https://github.com/rook/rook/blob/master/Documentation/CRDs/Shared-Filesystem/ceph-filesystem-crd.md#pools
name: data0
metadataServer:
activeCount: 1
activeStandby: true
resources:
limits:
memory: "4Gi"
requests:
cpu: "1000m"
memory: "4Gi"
priorityClassName: system-cluster-critical
storageClass:
enabled: true
isDefault: false
name: ceph-filesystem
# (Optional) specify a data pool to use, must be the name of one of the data pools above, 'data0' by default
pool: data0
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: "Immediate"
annotations: {}
labels: {}
mountOptions: []
# see https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/Shared-Filesystem-CephFS/filesystem-storage.md#provision-storage for available configuration
parameters:
# The secrets contain Ceph admin credentials.
csi.storage.k8s.io/provisioner-secret-name: rook-csi-cephfs-provisioner
csi.storage.k8s.io/provisioner-secret-namespace: "{{ .Release.Namespace }}"
csi.storage.k8s.io/controller-expand-secret-name: rook-csi-cephfs-provisioner
csi.storage.k8s.io/controller-expand-secret-namespace: "{{ .Release.Namespace }}"
csi.storage.k8s.io/controller-publish-secret-name: rook-csi-cephfs-provisioner
csi.storage.k8s.io/controller-publish-secret-namespace: "{{ .Release.Namespace }}"
csi.storage.k8s.io/node-stage-secret-name: rook-csi-cephfs-node
csi.storage.k8s.io/node-stage-secret-namespace: "{{ .Release.Namespace }}"
# Specify the filesystem type of the volume. If not specified, csi-provisioner
# will set default as `ext4`. Note that `xfs` is not recommended due to potential deadlock
# in hyperconverged settings where the volume is mounted on the same node as the osds.
csi.storage.k8s.io/fstype: ext4
# -- Settings for the filesystem snapshot class
# @default -- See [CephFS Snapshots](../Storage-Configuration/Ceph-CSI/ceph-csi-snapshot.md#cephfs-snapshots)
cephFileSystemVolumeSnapshotClass:
enabled: false
name: ceph-filesystem
isDefault: true
deletionPolicy: Delete
annotations: {}
labels: {}
# see https://rook.io/docs/rook/v1.10/Storage-Configuration/Ceph-CSI/ceph-csi-snapshot/#cephfs-snapshots for available configuration
parameters: {}
# -- Settings for the block pool snapshot class
# @default -- See [RBD Snapshots](../Storage-Configuration/Ceph-CSI/ceph-csi-snapshot.md#rbd-snapshots)
cephBlockPoolsVolumeSnapshotClass:
enabled: false
name: ceph-block
isDefault: false
deletionPolicy: Delete
annotations: {}
labels: {}
# see https://rook.io/docs/rook/v1.10/Storage-Configuration/Ceph-CSI/ceph-csi-snapshot/#rbd-snapshots for available configuration
parameters: {}
# -- A list of CephObjectStore configurations to deploy
# @default -- See [below](#ceph-object-stores)
cephObjectStores:
- name: ceph-objectstore
# see https://github.com/rook/rook/blob/master/Documentation/CRDs/Object-Storage/ceph-object-store-crd.md#object-store-settings for available configuration
spec:
metadataPool:
failureDomain: host
replicated:
size: 3
dataPool:
failureDomain: host
erasureCoded:
dataChunks: 2
codingChunks: 1
parameters:
bulk: "true"
preservePoolsOnDelete: true
gateway:
port: 80
resources:
limits:
memory: "2Gi"
requests:
cpu: "1000m"
memory: "1Gi"
# securePort: 443
# sslCertificateRef:
instances: 1
priorityClassName: system-cluster-critical
# opsLogSidecar:
# resources:
# limits:
# memory: "100Mi"
# requests:
# cpu: "100m"
# memory: "40Mi"
storageClass:
enabled: true
name: ceph-bucket
reclaimPolicy: Delete
volumeBindingMode: "Immediate"
annotations: {}
labels: {}
# see https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/Object-Storage-RGW/ceph-object-bucket-claim.md#storageclass for available configuration
parameters:
# note: objectStoreNamespace and objectStoreName are configured by the chart
region: us-east-1
ingress:
# Enable an ingress for the ceph-objectstore
enabled: false
# The ingress port by default will be the object store's "securePort" (if set), or the gateway "port".
# To override those defaults, set this ingress port to the desired port.
# port: 80
# annotations: {}
# host:
# name: objectstore.example.com
# path: /
# pathType: Prefix
# tls:
# - hosts:
# - objectstore.example.com
# secretName: ceph-objectstore-tls
# ingressClassName: nginx
route:
# Enable an ingress for the ceph-objectstore
enabled: false
# The ingress port by default will be the object store's "securePort" (if set), or the gateway "port".
# To override those defaults, set this ingress port to the desired port.
# port: 80
# annotations: {}
# host:
# name: objectstore.example.com
# path: /
# pathType: PathPrefix
# parentRefs:
# - name: internal
# namespace: kube-system
# sectionName: https
## cephECBlockPools are disabled by default, please remove the comments and set desired values to enable it
## For erasure coded a replicated metadata pool is required.
## https://rook.io/docs/rook/latest/CRDs/Shared-Filesystem/ceph-filesystem-crd/#erasure-coded
#cephECBlockPools:
# - name: ec-pool
# spec:
# metadataPool:
# replicated:
# size: 2
# dataPool:
# failureDomain: osd
# erasureCoded:
# dataChunks: 2
# codingChunks: 1
# deviceClass: hdd
#
# parameters:
# # clusterID is the namespace where the rook cluster is running
# # If you change this namespace, also change the namespace below where the secret namespaces are defined
# clusterID: rook-ceph # namespace:cluster
# # (optional) mapOptions is a comma-separated list of map options.
# # For krbd options refer
# # https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
# # For nbd options refer
# # https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
# # mapOptions: lock_on_read,queue_depth=1024
#
# # (optional) unmapOptions is a comma-separated list of unmap options.
# # For krbd options refer
# # https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
# # For nbd options refer
# # https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
# # unmapOptions: force
#
# # RBD image format. Defaults to "2".
# imageFormat: "2"
#
# # RBD image features, equivalent to OR'd bitfield value: 63
# # Available for imageFormat: "2". Older releases of CSI RBD
# # support only the `layering` feature. The Linux kernel (KRBD) supports the
# # full feature complement as of 5.4
# # imageFeatures: layering,fast-diff,object-map,deep-flatten,exclusive-lock
# imageFeatures: layering
#
# storageClass:
# provisioner: rook-ceph.rbd.csi.ceph.com # csi-provisioner-name
# enabled: true
# name: rook-ceph-block
# isDefault: false
# annotations: { }
# labels: { }
# allowVolumeExpansion: true
# reclaimPolicy: Delete
# -- CSI driver name prefix for cephfs, rbd and nfs.
# @default -- `namespace name where rook-ceph operator is deployed`
csiDriverNamePrefix:
backend/helm/cloudhost-ceph/Chart.yaml
+13
View File
@@ -0,0 +1,13 @@
apiVersion: v2
name: cloudhost-ceph
description: CloudHost extras on top of Rook-Ceph (app source bucket, platform integration secrets)
type: application
version: 0.1.0
appVersion: "1.0.0"
keywords:
- ceph
- rook
- storage
- s3
maintainers:
- name: CloudHost
backend/helm/cloudhost-ceph/README.md
+140
View File
@@ -0,0 +1,140 @@
# CloudHost Ceph (Rook)
Helm chart and install scripts for **Rook-Ceph** on CloudHost clusters:
| Layer | Purpose |
|-------|---------|
| **rook-ceph-block** | Expandable PVCs for apps, databases, registry |
| **rook-ceph-bucket** | S3-compatible storage for uploaded source zip archives |
The chart does **not** vendor Rook itself — it installs the official [`rook-release`](https://charts.rook.io/release) charts and adds CloudHost-specific **ObjectBucketClaim** + credential sync.
---
## Quick install
```bash
cd backend/helm/cloudhost-ceph
./scripts/install.sh single-node # one-node k3s (current abr cluster)
# or
./scripts/install.sh multi-node # production, 3+ nodes + raw disks
```
Verify:
```bash
./scripts/verify.sh
```
---
## Profiles
### `single-node`
- OSD on **loop device** `/dev/loop6` (15Gi file at `/var/lib/rook/osd-loopback.img`) — no spare raw disk required
- Requires `ROOK_CEPH_ALLOW_LOOP_DEVICES=true` on the operator
- Replication **size: 1** (no HA)
- Suitable for **staging / single k3s node**
- Images must be pre-mirrored to `registry.abrban.com` (see `RUNBOOK-HARBOR.fa.md`)
### `multi-node`
- OSD on **raw devices** (`useAllDevices: true`)
- Replication **size: 3** for block + object metadata
- Erasure-coded object data pool
- Requires **3+ nodes** and dedicated disks
---
## What gets installed
| Step | Release | Namespace |
|------|---------|-----------|
| 1 | `rook-ceph` (operator) | `rook-ceph` |
| 2 | `rook-ceph-cluster` | `rook-ceph` |
| 3 | `cloudhost-ceph` (OBC + secrets) | `cloudhost-builds` |
### StorageClasses (from Rook)
| Name | Use |
|------|-----|
| `rook-ceph-block` | App PVC, DB PVC, Redis, registry, … |
| `rook-ceph-bucket` | `ObjectBucketClaim` → S3 bucket + credentials |
### CloudHost extras
| Resource | Description |
|----------|-------------|
| `ObjectBucketClaim/app-sources` | Bucket for user zip uploads |
| `Secret/ceph-app-sources-credentials` | Stable S3 credentials for backend |
| `ConfigMap/cloudhost-ceph-integration` | Suggested `PLATFORM_*` env values |
---
## Platform integration
After install, configure the **backend**:
```env
PLATFORM_STORAGE_CLASS=rook-ceph-block
PLATFORM_CREATE_STORAGE_CLASS=false
PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com
```
Mount or env-from secret `cloudhost-builds/ceph-app-sources-credentials`:
```env
SOURCE_STORAGE_ENDPOINT=http://rook-ceph-rgw-ceph-objectstore.rook-ceph.svc.cluster.local:80
SOURCE_STORAGE_REGION=us-east-1
SOURCE_STORAGE_BUCKET=<from secret>
SOURCE_STORAGE_ACCESS_KEY=<from secret>
SOURCE_STORAGE_SECRET_KEY=<from secret>
```
> **Note:** Existing PVCs on `local-path` / `cloudhost-expandable` are **not** migrated automatically. New apps use `rook-ceph-block` once the backend env is updated. Plan migration per workload (see `RUNBOOK-CEPH.fa.md`).
---
## Uninstall (destructive)
```bash
./scripts/uninstall.sh
# then on each node:
sudo rm -rf /var/lib/rook /var/lib/rook/osd
```
---
## Troubleshooting
```bash
# Cluster health
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph status
# OSD pods
kubectl -n rook-ceph get pods -l app=rook-ceph-osd
# RGW (object store)
kubectl -n rook-ceph get pods -l app=rook-ceph-rgw
# Bucket sync job
kubectl -n cloudhost-builds logs job -l job-name=cloudhost-ceph-bucket-sync --tail=50
```
Full operational guide (Persian): [`../../../RUNBOOK-CEPH.fa.md`](../../../RUNBOOK-CEPH.fa.md)
Registry / Harbor (Persian): [`../../../RUNBOOK-HARBOR.fa.md`](../../../RUNBOOK-HARBOR.fa.md)
---
## Files
| File | Role |
|------|------|
| `values-rook-cluster-single-node.yaml` | Rook cluster values (1 node) |
| `values-rook-cluster-multi-node.yaml` | Rook cluster values (production) |
| `values.yaml` | CloudHost OBC / secret sync |
| `scripts/install.sh` | Full install |
| `scripts/verify.sh` | Health check |
| `scripts/uninstall.sh` | Tear down |
backend/helm/cloudhost-ceph/scripts/install.sh
Executable
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Install Rook-Ceph operator + cluster + CloudHost bucket extras.
# Usage: ./scripts/install.sh [single-node|multi-node]
set -euo pipefail
PROFILE="${1:-single-node}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CHART_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
ROOK_NS="rook-ceph"
EXTRAS_NS="cloudhost-builds"
CLUSTER_VALUES="${CHART_DIR}/values-rook-cluster-${PROFILE}.yaml"
if [[ ! -f "${CLUSTER_VALUES}" ]]; then
echo "Unknown profile: ${PROFILE} (missing ${CLUSTER_VALUES})" >&2
exit 1
fi
echo "==> Profile: ${PROFILE}"
echo "==> Adding rook-release helm repo"
helm repo add rook-release https://charts.rook.io/release 2>/dev/null || true
helm repo update rook-release
echo "==> [1/4] Installing Rook operator in ${ROOK_NS}"
helm upgrade --install rook-ceph rook-release/rook-ceph \
--namespace "${ROOK_NS}" \
--create-namespace \
--wait \
--timeout 10m
echo "==> [2/4] Waiting for Rook operator deployment"
kubectl -n "${ROOK_NS}" rollout status deploy/rook-ceph-operator --timeout=300s
echo "==> [3/4] Installing Ceph cluster (${CLUSTER_VALUES})"
helm upgrade --install rook-ceph-cluster rook-release/rook-ceph-cluster \
--namespace "${ROOK_NS}" \
-f "${CLUSTER_VALUES}" \
--wait \
--timeout 25m
echo "==> Waiting for CephCluster phase = Ready (up to 20 min)"
"${SCRIPT_DIR}/wait-ceph-ready.sh" 1200
echo "==> [4/4] Installing CloudHost Ceph extras (ObjectBucketClaim) in ${EXTRAS_NS}"
kubectl create namespace "${EXTRAS_NS}" 2>/dev/null || true
helm upgrade --install cloudhost-ceph "${CHART_DIR}" \
--namespace "${EXTRAS_NS}" \
-f "${CHART_DIR}/values.yaml" \
--wait \
--timeout 15m
echo ""
echo "==> Done. Run ./scripts/verify.sh to confirm health and print integration hints."
backend/helm/cloudhost-ceph/scripts/uninstall.sh
Executable
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
# Remove CloudHost extras + Rook cluster + operator (DATA LOSS).
set -euo pipefail
read -r -p "This deletes ALL Ceph data. Type 'delete-ceph' to continue: " CONFIRM
if [[ "${CONFIRM}" != "delete-ceph" ]]; then
echo "Aborted."
exit 1
fi
helm uninstall cloudhost-ceph -n cloudhost-builds 2>/dev/null || true
helm uninstall rook-ceph-cluster -n rook-ceph 2>/dev/null || true
helm uninstall rook-ceph -n rook-ceph 2>/dev/null || true
echo "Waiting for Rook resources to terminate..."
sleep 15
kubectl -n rook-ceph get pods 2>/dev/null || true
cat <<'EOF'
IMPORTANT: On each node, wipe Rook state before reinstalling:
sudo rm -rf /var/lib/rook
sudo rm -rf /var/lib/rook/osd
For raw-disk OSDs also zap disks (see RUNBOOK-CEPH.fa.md).
EOF
backend/helm/cloudhost-ceph/scripts/verify.sh
Executable
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
set -euo pipefail
ROOK_NS="rook-ceph"
EXTRAS_NS="cloudhost-builds"
echo "=== StorageClasses ==="
kubectl get storageclass | grep -E 'NAME|rook-ceph' || true
echo ""
echo "=== Ceph status ==="
kubectl -n "${ROOK_NS}" exec deploy/rook-ceph-tools -- ceph status 2>/dev/null || echo "(tools pod not ready yet)"
echo ""
echo "=== OSD / MON pods ==="
kubectl -n "${ROOK_NS}" get pods -l app=rook-ceph-osd 2>/dev/null || kubectl -n "${ROOK_NS}" get pods | grep -E 'osd|mon|mgr|rgw' || true
echo ""
echo "=== Object bucket claim ==="
kubectl -n "${EXTRAS_NS}" get obc,app-sources 2>/dev/null || kubectl -n "${EXTRAS_NS}" get obc 2>/dev/null || true
echo ""
echo "=== Platform credentials secret ==="
if kubectl -n "${EXTRAS_NS}" get secret ceph-app-sources-credentials >/dev/null 2>&1; then
echo "Secret ceph-app-sources-credentials exists"
kubectl -n "${EXTRAS_NS}" get secret ceph-app-sources-credentials -o jsonpath='{.data.SOURCE_STORAGE_BUCKET}' | base64 -d
echo ""
else
echo "Secret ceph-app-sources-credentials not ready — check bucket sync job:"
kubectl -n "${EXTRAS_NS}" get jobs,pods | grep bucket-sync || true
fi
echo ""
echo "=== Suggested backend env ==="
kubectl -n "${EXTRAS_NS}" get configmap cloudhost-ceph-integration -o yaml 2>/dev/null | sed -n '/PLATFORM_/p;/SOURCE_STORAGE_ENDPOINT/p' || true
backend/helm/cloudhost-ceph/scripts/wait-ceph-ready.sh
Executable
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Wait until Ceph reports HEALTH_OK or HEALTH_WARN (single-node often stays WARN).
set -euo pipefail
TIMEOUT="${1:-900}"
ROOK_NS="rook-ceph"
START=$(date +%s)
echo "Waiting for rook-ceph-tools deployment..."
for _ in $(seq 1 60); do
if kubectl -n "${ROOK_NS}" get deploy rook-ceph-tools >/dev/null 2>&1; then
if kubectl -n "${ROOK_NS}" rollout status deploy/rook-ceph-tools --timeout=120s 2>/dev/null; then
break
fi
fi
sleep 5
done
while true; do
NOW=$(date +%s)
if (( NOW - START > TIMEOUT )); then
echo "Timed out after ${TIMEOUT}s waiting for Ceph health" >&2
kubectl -n "${ROOK_NS}" get cephcluster,pod -o wide || true
exit 1
fi
if kubectl -n "${ROOK_NS}" get deploy rook-ceph-tools >/dev/null 2>&1; then
HEALTH=$(kubectl -n "${ROOK_NS}" exec deploy/rook-ceph-tools -- ceph health 2>/dev/null || echo "unknown")
echo "Ceph health: ${HEALTH}"
if [[ "${HEALTH}" == "HEALTH_OK" || "${HEALTH}" == HEALTH_WARN* ]]; then
PHASE=$(kubectl -n "${ROOK_NS}" get cephcluster rook-ceph -o jsonpath='{.status.phase}' 2>/dev/null || echo "")
echo "CephCluster phase: ${PHASE}"
if [[ "${PHASE}" == "Ready" ]]; then
exit 0
fi
fi
fi
sleep 15
done
backend/helm/cloudhost-ceph/templates/NOTES.txt
+23
View File
@@ -0,0 +1,23 @@
CloudHost Ceph storage is ready (or still initializing).
Profiles
single-node Directory OSD on /var/lib/rook/osd — for one-node k3s (no HA)
multi-node Raw disk OSDs with replication=3 — production
StorageClasses created by Rook
rook-ceph-block Block volumes (app PVC, DB, registry, …)
rook-ceph-bucket S3-compatible buckets via ObjectBucketClaim
Verify cluster health
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph status
kubectl get storageclass | grep rook-ceph
kubectl -n cloudhost-builds get obc,secret | grep -E 'app-sources|ceph-app-sources'
Platform backend (after bucket sync Job completes)
PLATFORM_STORAGE_CLASS=rook-ceph-block
PLATFORM_CREATE_STORAGE_CLASS=false
PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com
Mount secret cloudhost-builds/ceph-app-sources-credentials for zip upload S3 settings.
Full guide: backend/helm/cloudhost-ceph/README.md and RUNBOOK-CEPH.fa.md
backend/helm/cloudhost-ceph/templates/_helpers.tpl
+27
View File
@@ -0,0 +1,27 @@
{{/*
CloudHost Ceph chart helpers
*/}}
{{- define "cloudhost-ceph.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- define "cloudhost-ceph.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{- define "cloudhost-ceph.labels" -}}
helm.sh/chart: {{ include "cloudhost-ceph.name" . }}-{{ .Chart.Version }}
app.kubernetes.io/name: {{ include "cloudhost-ceph.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/part-of: cloudhost
{{- end }}
backend/helm/cloudhost-ceph/templates/integration-configmap.yaml
+20
View File
@@ -0,0 +1,20 @@
{{- if .Values.integration.createConfigMap }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.integration.configMapName }}
namespace: {{ .Values.namespace }}
labels:
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
data:
PLATFORM_STORAGE_CLASS: rook-ceph-block
PLATFORM_CREATE_STORAGE_CLASS: "false"
PLATFORM_STORAGE_PROVISIONER: rook-ceph.rbd.csi.ceph.com
SOURCE_STORAGE_ENDPOINT: {{ .Values.platform.endpoint | quote }}
SOURCE_STORAGE_REGION: {{ .Values.platform.region | quote }}
SOURCE_STORAGE_CREDENTIALS_SECRET: {{ .Values.platform.credentialsSecretName | quote }}
README: |
Block PVCs: set PLATFORM_STORAGE_CLASS=rook-ceph-block on the backend.
New app PVCs use rook-ceph-block; existing local-path PVCs are NOT auto-migrated.
Object storage credentials: secret {{ .Values.platform.credentialsSecretName }} in {{ .Values.namespace }}.
{{- end }}
backend/helm/cloudhost-ceph/templates/objectbucketclaim-app-sources.yaml
+15
View File
@@ -0,0 +1,15 @@
{{- if .Values.objectStore.claimName }}
apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
name: {{ .Values.objectStore.claimName }}
namespace: {{ .Values.namespace }}
labels:
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-weight: "5"
spec:
storageClassName: {{ .Values.objectStore.bucketStorageClass | quote }}
generateBucketName: {{ .Values.objectStore.generateBucketName | quote }}
{{- end }}
backend/helm/cloudhost-ceph/templates/platform-bucket-credentials-job.yaml
+113
View File
@@ -0,0 +1,113 @@
{{- if and .Values.platform.createCredentialsSecret .Values.objectStore.claimName }}
# Stable secret name for platform workers. Populated by a post-install Job once the OBC secret exists.
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
namespace: {{ .Values.namespace }}
labels:
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-weight: "1"
helm.sh/hook-delete-policy: before-hook-creation
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
namespace: {{ .Values.namespace }}
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-weight: "1"
helm.sh/hook-delete-policy: before-hook-creation
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "create", "patch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
namespace: {{ .Values.namespace }}
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-weight: "1"
helm.sh/hook-delete-policy: before-hook-creation
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
subjects:
- kind: ServiceAccount
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
namespace: {{ .Values.namespace }}
---
apiVersion: batch/v1
kind: Job
metadata:
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
namespace: {{ .Values.namespace }}
labels:
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-weight: "10"
helm.sh/hook-delete-policy: before-hook-creation
spec:
backoffLimit: 30
template:
spec:
serviceAccountName: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
restartPolicy: OnFailure
containers:
- name: sync
image: registry.abrban.com/proxy-dockerhub/bitnami/kubectl:1.32
imagePullPolicy: IfNotPresent
env:
- name: OBC_SECRET
value: {{ printf "obc-%s-%s" .Values.namespace .Values.objectStore.claimName | quote }}
- name: TARGET_SECRET
value: {{ .Values.platform.credentialsSecretName | quote }}
- name: NAMESPACE
value: {{ .Values.namespace | quote }}
- name: ENDPOINT
value: {{ .Values.platform.endpoint | quote }}
- name: REGION
value: {{ .Values.platform.region | quote }}
command:
- /bin/bash
- -ec
- |
echo "Waiting for OBC secret ${OBC_SECRET} in ${NAMESPACE}..."
for i in $(seq 1 120); do
if kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" >/dev/null 2>&1; then
break
fi
sleep 10
done
kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" >/dev/null
BUCKET=$(kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" -o jsonpath='{.data.BUCKET_NAME}' | base64 -d)
ACCESS=$(kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 -d)
SECRET=$(kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 -d)
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: ${TARGET_SECRET}
namespace: ${NAMESPACE}
labels:
app.kubernetes.io/part-of: cloudhost
type: Opaque
stringData:
SOURCE_STORAGE_ENDPOINT: "${ENDPOINT}"
SOURCE_STORAGE_REGION: "${REGION}"
SOURCE_STORAGE_BUCKET: "${BUCKET}"
SOURCE_STORAGE_ACCESS_KEY: "${ACCESS}"
SOURCE_STORAGE_SECRET_KEY: "${SECRET}"
EOF
echo "Synced bucket credentials to secret ${TARGET_SECRET} (bucket=${BUCKET})"
{{- end }}
backend/helm/cloudhost-ceph/values-rook-cluster-multi-node.yaml
+83
View File
@@ -0,0 +1,83 @@
# Rook-Ceph cluster values — MULTI NODE (production).
# Install: scripts/install.sh multi-node
#
# Prerequisites:
# - At least 3 worker nodes (odd mon count)
# - Raw disks available (useAllDevices) OR dedicated devices per node
# - Taint-free nodes labeled rook-ceph-role=storage-node (optional)
operatorNamespace: rook-ceph
toolbox:
enabled: true
cephClusterSpec:
dataDirHostPath: /var/lib/rook
mon:
count: 3
allowMultiplePerNode: false
mgr:
count: 2
allowMultiplePerNode: false
dashboard:
enabled: true
ssl: true
storage:
useAllNodes: false
useAllDevices: true
# Example: pin OSDs to storage nodes only
# nodes:
# - name: "node-1"
# - name: "node-2"
# - name: "node-3"
cephFileSystems: []
cephBlockPools:
- name: ceph-blockpool
spec:
failureDomain: host
replicated:
size: 3
storageClass:
enabled: true
name: rook-ceph-block
isDefault: false
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
cephObjectStores:
- name: ceph-objectstore
spec:
metadataPool:
failureDomain: host
replicated:
size: 3
dataPool:
failureDomain: host
erasureCoded:
dataChunks: 2
codingChunks: 1
preservePoolsOnDelete: true
gateway:
port: 80
instances: 2
resources:
limits:
memory: "2Gi"
requests:
cpu: "500m"
memory: "1Gi"
storageClass:
enabled: true
name: rook-ceph-bucket
reclaimPolicy: Delete
parameters:
region: us-east-1
ingress:
enabled: false
backend/helm/cloudhost-ceph/values-rook-cluster-single-node.yaml
+102
View File
@@ -0,0 +1,102 @@
# Rook-Ceph cluster values — SINGLE NODE (k3s dev/staging).
# Install: scripts/install.sh single-node
#
# Uses loop device /dev/loop6 (15Gi) on single-node clusters without a spare raw disk.
# Replication factor = 1 (no HA). For production multi-node use values-rook-cluster-multi-node.yaml.
# See RUNBOOK-CEPH.fa.md for loop setup and image mirroring prerequisites.
operatorNamespace: rook-ceph
toolbox:
enabled: true
cephClusterSpec:
dataDirHostPath: /var/lib/rook
skipUpgradeChecks: true
continueUpgradeAfterChecksEvenIfNotHealthy: true
mon:
count: 1
allowMultiplePerNode: true
mgr:
count: 1
allowMultiplePerNode: true
dashboard:
enabled: true
ssl: false
resources:
mon:
limits:
memory: "1Gi"
requests:
cpu: "200m"
memory: "512Mi"
mgr:
limits:
memory: "1Gi"
requests:
cpu: "200m"
memory: "512Mi"
osd:
limits:
memory: "2Gi"
requests:
cpu: "500m"
memory: "1Gi"
storage:
useAllNodes: true
useAllDevices: false
devices:
- name: "/dev/loop6"
# Disable CephFS to save RAM on single-node clusters.
cephFileSystems: []
cephBlockPools:
- name: ceph-blockpool
spec:
failureDomain: osd
replicated:
size: 1
storageClass:
enabled: true
name: rook-ceph-block
isDefault: false
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
cephObjectStores:
- name: ceph-objectstore
spec:
metadataPool:
failureDomain: osd
replicated:
size: 1
dataPool:
failureDomain: osd
replicated:
size: 1
preservePoolsOnDelete: true
gateway:
port: 80
instances: 1
resources:
limits:
memory: "1Gi"
requests:
cpu: "250m"
memory: "512Mi"
storageClass:
enabled: true
name: rook-ceph-bucket
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
region: us-east-1
ingress:
enabled: false
backend/helm/cloudhost-ceph/values.yaml
+25
View File
@@ -0,0 +1,25 @@
# CloudHost Ceph extras (ObjectBucketClaim for zip uploads).
# Rook operator + CephCluster are installed via scripts/install.sh using official rook-release charts.
namespace: cloudhost-builds
objectStore:
# Must match rook-ceph-cluster cephObjectStores[].storageClass.name
bucketStorageClass: rook-ceph-bucket
# Claim name; Rook generates bucket + credentials secret
claimName: app-sources
# Prefix for generated bucket name (Rook appends random suffix)
generateBucketName: cloudhost-app-sources
platform:
# Copy S3 credentials into a stable secret name for backend/workers
createCredentialsSecret: true
credentialsSecretName: ceph-app-sources-credentials
# In-cluster RGW endpoint (adjust if ingress is enabled on object store)
endpoint: http://rook-ceph-rgw-ceph-objectstore.rook-ceph.svc.cluster.local:80
region: us-east-1
integration:
# Emit a ConfigMap with suggested backend env vars (non-secret)
createConfigMap: true
configMapName: cloudhost-ceph-integration
backend/helm/cloudhost-harbor/scripts/install-harbor-registry.sh
Executable
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -euo pipefail
# Replaces registry.abrban.com docker distribution with Harbor.
# WARNING: This will delete the existing `Ingress/registry` routing. The old
# `Deployment/registry` and its PVC are left in place for rollback.
VALUES_FILE="${1:-/Users/keyhan/Documents/keyhan-project/cloud-host/backend/helm/cloudhost-harbor/values-registry.abrban.com.yaml}"
echo "==> Ensuring harbor repo"
helm repo add harbor https://helm.goharbor.io 2>/dev/null || true
helm repo update harbor
echo "==> [0/4] Preflight"
kubectl -n cloudhost get secret abrban-wildcard-tls >/dev/null
kubectl -n cloudhost get secret registry-egress-proxy >/dev/null
echo "==> [1/4] Disabling old registry ingress (host registry.abrban.com)"
kubectl -n cloudhost delete ingress registry --ignore-not-found
echo "==> [2/4] Scaling old registry deployment down (rollback-friendly)"
kubectl -n cloudhost scale deploy/registry --replicas=0 || true
echo "==> [3/4] Installing Harbor"
HTTP_PROXY="$(kubectl -n cloudhost get secret registry-egress-proxy -o jsonpath='{.data.HTTP_PROXY}' | base64 -d)"
HTTPS_PROXY="$(kubectl -n cloudhost get secret registry-egress-proxy -o jsonpath='{.data.HTTPS_PROXY}' | base64 -d)"
NO_PROXY="$(kubectl -n cloudhost get secret registry-egress-proxy -o jsonpath='{.data.NO_PROXY}' | base64 -d)"
TMP_PROXY_VALUES="$(mktemp)"
cat > "${TMP_PROXY_VALUES}" <<EOF
proxy:
httpProxy: "${HTTP_PROXY}"
httpsProxy: "${HTTPS_PROXY}"
noProxy: "${NO_PROXY}"
EOF
helm upgrade --install harbor harbor/harbor \
-n cloudhost \
-f "$VALUES_FILE" \
-f "${TMP_PROXY_VALUES}" \
--wait \
--timeout 20m
rm -f "${TMP_PROXY_VALUES}" || true
echo "==> [4/4] Done"
kubectl -n cloudhost get ingress | grep -n registry || true
backend/helm/cloudhost-harbor/values-registry.abrban.com.yaml
+75
View File
@@ -0,0 +1,75 @@
## Harbor values to REPLACE registry.abrban.com
## Ingress controller on this cluster is Traefik (k3s).
##
## Install:
## helm upgrade --install harbor harbor/harbor -n cloudhost -f backend/helm/cloudhost-harbor/values-registry.abrban.com.yaml
##
externalURL: https://registry.abrban.com
proxy:
# Values are injected by install script from `cloudhost/registry-egress-proxy`.
httpProxy: ""
httpsProxy: ""
noProxy: ""
expose:
type: ingress
tls:
enabled: true
certSource: secret
secret:
secretName: abrban-wildcard-tls
ingress:
className: traefik
hosts:
core: registry.abrban.com
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: websecure
# Increase timeouts for large pushes (skopeo/registry blobs)
traefik.ingress.kubernetes.io/router.tls: "true"
# Disable components we don't need for now to reduce resources
trivy:
enabled: false
notary:
enabled: false
chartmuseum:
enabled: false
# Single-node staging: keep resource usage modest
core:
replicas: 1
jobservice:
replicas: 1
registry:
replicas: 1
persistence:
enabled: true
persistentVolumeClaim:
# Use existing default storage (local-path) until Ceph is ready.
# After Ceph, switch to rook-ceph-block for Harbor's PVCs.
registry:
storageClass: local-path
size: 50Gi
jobservice:
storageClass: local-path
size: 5Gi
database:
storageClass: local-path
size: 10Gi
redis:
storageClass: local-path
size: 5Gi
database:
type: internal
redis:
type: internal
portal:
replicas: 1
# We will create proxy-cache projects after install (todo: configure-proxy-cache)
backend/helm/cloudhost-platform/migrations/018_user_phone_and_verification.sql
+33
View File
@@ -0,0 +1,33 @@
-- Mobile-first auth: phone is the login identifier, email becomes an optional
-- contact field, plus a table of short-lived one-time SMS codes for verifying
-- a phone (registration/login completion and number changes).
-- Email becomes optional (login no longer uses it). Postgres treats NULLs as
-- distinct, so the existing UNIQUE constraint keeps working for users without one.
ALTER TABLE users ALTER COLUMN email DROP NOT NULL;
ALTER TABLE users ADD COLUMN IF NOT EXISTS phone VARCHAR;
ALTER TABLE users ADD COLUMN IF NOT EXISTS "phoneVerified" BOOLEAN NOT NULL DEFAULT FALSE;
-- Unique per non-null phone (NULLs allowed for legacy email-only staff accounts).
CREATE UNIQUE INDEX IF NOT EXISTS users_phone_unique ON users (phone) WHERE phone IS NOT NULL;
-- One-time SMS verification codes (hashed).
DO $$ BEGIN
CREATE TYPE verification_codes_purpose_enum AS ENUM ('login', 'change_phone');
EXCEPTION WHEN duplicate_object THEN null; END $$;
CREATE TABLE IF NOT EXISTS verification_codes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
"userId" UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
purpose verification_codes_purpose_enum NOT NULL,
destination VARCHAR NOT NULL,
"codeHash" VARCHAR NOT NULL,
"expiresAt" TIMESTAMPTZ NOT NULL,
attempts INT NOT NULL DEFAULT 0,
"consumedAt" TIMESTAMPTZ,
"createdAt" TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS verification_codes_user_purpose_idx
ON verification_codes ("userId", purpose);
backend/helm/cloudhost-platform/templates/backend-deployment.yaml
+11 -2
View File
@@ -19,6 +19,10 @@ spec:
labels:
app: {{ include "cloudhost-platform.backend.fullname" . }}
spec:
{{- with .Values.backend.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
initContainers:
{{- if .Values.postgres.enabled }}
- name: wait-postgres
@@ -88,19 +92,24 @@ spec:
- name: {{ $key }}
value: {{ $val | quote }}
{{- end }}
{{- if .Values.backend.sourceStorage.enabled }}
envFrom:
- secretRef:
name: {{ .Values.backend.sourceStorage.existingSecret }}
{{- end }}
volumeMounts:
- name: uploads
mountPath: /app/uploads
livenessProbe:
httpGet:
path: /api/docs
path: /api/v1/health
port: 4000
initialDelaySeconds: 60
periodSeconds: 15
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /api/docs
path: /api/v1/ready
port: 4000
initialDelaySeconds: 20
periodSeconds: 10
backend/helm/cloudhost-platform/templates/frontend-deployment.yaml
+4
View File
@@ -17,6 +17,10 @@ spec:
labels:
app: {{ include "cloudhost-platform.frontend.fullname" . }}
spec:
{{- with .Values.frontend.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: frontend
image: {{ include "cloudhost-platform.frontendImage" . | quote }}
backend/helm/cloudhost-platform/templates/ingress.yaml
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
labels:
{{- include "cloudhost-platform.labels" . | nindent 4 }}
annotations:
{{- if .Values.ingress.tls.enabled }}
{{- if and .Values.ingress.tls.enabled (not .Values.ingress.tls.secretName) }}
cert-manager.io/cluster-issuer: {{ .Values.ingress.tls.clusterIssuer | quote }}
{{- end }}
{{- if and .Values.ingress.singleHost.enabled .Values.ingress.singleHost.apiPath }}
backend/helm/cloudhost-platform/templates/monitoring.yaml
+35
View File
@@ -0,0 +1,35 @@
{{- if .Values.monitoring.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "cloudhost-platform.fullname" . }}-backend-metrics
namespace: {{ include "cloudhost-platform.namespace" . }}
labels:
{{- include "cloudhost-platform.labels" . | nindent 4 }}
app.kubernetes.io/component: backend
spec:
type: ClusterIP
ports:
- name: http
port: 4000
targetPort: 4000
selector:
app: {{ include "cloudhost-platform.backend.fullname" . }}
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "cloudhost-platform.fullname" . }}-backend
namespace: {{ include "cloudhost-platform.namespace" . }}
labels:
{{- include "cloudhost-platform.labels" . | nindent 4 }}
release: prometheus
spec:
selector:
matchLabels:
app: {{ include "cloudhost-platform.backend.fullname" . }}
endpoints:
- port: http
path: /api/v1/health
interval: 30s
{{- end }}
backend/helm/cloudhost-platform/templates/postgres-backup.yaml
+62
View File
@@ -0,0 +1,62 @@
{{- if .Values.backups.postgres.enabled }}
apiVersion: batch/v1
kind: CronJob
metadata:
name: {{ include "cloudhost-platform.fullname" . }}-postgres-backup
namespace: {{ include "cloudhost-platform.namespace" . }}
labels:
{{- include "cloudhost-platform.labels" . | nindent 4 }}
spec:
schedule: {{ .Values.backups.postgres.schedule | quote }}
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
spec:
restartPolicy: OnFailure
containers:
- name: backup
image: {{ .Values.images.postgres | quote }}
env:
- name: PGHOST
value: {{ include "cloudhost-platform.postgres.fullname" . }}
- name: PGPORT
value: "5432"
- name: PGDATABASE
value: {{ .Values.postgres.database | quote }}
- name: PGUSER
value: {{ .Values.postgres.username | quote }}
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: {{ include "cloudhost-platform.secretName" . }}
key: postgres-password
command:
- sh
- -c
- |
set -e
STAMP=$(date +%Y%m%d-%H%M%S)
FILE="/backup/cloudhost-${STAMP}.sql.gz"
pg_dump | gzip > "$FILE"
echo "Backup written to $FILE"
volumeMounts:
- name: backup
mountPath: /backup
volumes:
- name: backup
persistentVolumeClaim:
claimName: {{ include "cloudhost-platform.fullname" . }}-postgres-backup
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ include "cloudhost-platform.fullname" . }}-postgres-backup
namespace: {{ include "cloudhost-platform.namespace" . }}
spec:
accessModes: [ReadWriteOnce]
resources:
requests:
storage: {{ .Values.backups.postgres.storageSize }}
{{- end }}
backend/helm/cloudhost-platform/values-production.example.yaml
+4
View File
@@ -36,6 +36,10 @@ ingress:
clusterIssuer: letsencrypt-prod
backend:
# Enable after copying ceph-app-sources-credentials secret into the cloudhost namespace
sourceStorage:
enabled: false
existingSecret: ceph-app-sources-credentials
env:
PLATFORM_DOMAIN: apps.example.com
REGISTRY_URL: registry.cloudhost-builds.svc.cluster.local:5000
backend/helm/cloudhost-platform/values.yaml
+16
View File
@@ -42,8 +42,13 @@ redis:
backend:
enabled: true
replicas: 1
imagePullSecrets:
- name: registry-pull-secret
uploads:
size: 20Gi
sourceStorage:
enabled: false
existingSecret: ceph-app-sources-credentials
resources: {}
extraEnv: {}
env:
@@ -66,6 +71,8 @@ backend:
frontend:
enabled: true
replicas: 1
imagePullSecrets:
- name: registry-pull-secret
resources: {}
# JWT secrets — set in production (values-production.example.yaml)
@@ -97,3 +104,12 @@ ingress:
migrations:
enabled: true
image: postgres:16-alpine
monitoring:
enabled: false
backups:
postgres:
enabled: false
schedule: "0 3 * * *"
storageSize: 10Gi
backend/k8s/builds/cloudhost-builds-bootstrap.yaml
+77
View File
@@ -0,0 +1,77 @@
apiVersion: v1
kind: Namespace
metadata:
name: cloudhost-builds
labels:
app.kubernetes.io/part-of: cloudhost
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kaniko-builder
namespace: cloudhost-builds
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: kaniko-builder
namespace: cloudhost-builds
rules:
- apiGroups: ['']
resources: ['pods', 'pods/log', 'secrets', 'configmaps', 'persistentvolumeclaims']
verbs: ['create', 'get', 'list', 'watch', 'delete', 'patch', 'update']
- apiGroups: ['batch']
resources: ['jobs']
verbs: ['create', 'get', 'list', 'watch', 'delete']
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kaniko-builder
namespace: cloudhost-builds
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kaniko-builder
subjects:
- kind: ServiceAccount
name: kaniko-builder
namespace: cloudhost-builds
---
# In-cluster registry for Kaniko push + app image pull (HTTP — add TLS in production).
apiVersion: v1
kind: Service
metadata:
name: registry
namespace: cloudhost-builds
spec:
type: ClusterIP
ports:
- port: 5000
targetPort: 5000
selector:
app: registry
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: registry
namespace: cloudhost-builds
spec:
replicas: 1
selector:
matchLabels:
app: registry
template:
metadata:
labels:
app: registry
spec:
containers:
- name: registry
image: registry:2
ports:
- containerPort: 5000
env:
- name: REGISTRY_HTTP_ADDR
value: ':5000'
backend/k8s/mail/maddy.yaml
+248
View File
@@ -0,0 +1,248 @@
# Maddy mail server — lightweight full mail server (SMTP + IMAP + DKIM)
# Namespace: mail | Host: mail.abrban.com | Primary domain: abrban.com
#
# Exposed on node IP 78.157.39.52 via k3s servicelb (klipper).
# TLS uses the *.abrban.com wildcard cert (secret abrban-wildcard-tls, copied into ns mail).
#
# NOTE (Iran/IP reputation): inbound mail (receiving) works; outbound delivery to
# Gmail/Outlook may be blocked or land in spam, and outbound port 25 may be filtered
# by the ISP. Use a smarthost relay if real external delivery is required.
---
apiVersion: v1
kind: ConfigMap
metadata:
name: maddy-config
namespace: mail
data:
maddy.conf: |
## Maddy Mail Server - configuration (mail.abrban.com)
$(hostname) = mail.abrban.com
$(primary_domain) = abrban.com
$(local_domains) = $(primary_domain)
tls file /etc/maddy/tls/tls.crt /etc/maddy/tls/tls.key
# ---- Local storage & authentication ----
storage.imapsql local_mailboxes {
driver sqlite3
dsn imapsql.db
}
auth.pass_table local_authdb {
table sql_table {
driver sqlite3
dsn credentials.db
table_name passwords
}
}
# ---- Routing ----
hostname $(hostname)
table.chain local_rewrites {
optional_step regexp "(.+)\+(.+)@(.+)" "$1@$3"
optional_step static {
entry postmaster postmaster@$(primary_domain)
}
optional_step file /data/aliases
}
msgpipeline local_routing {
destination postmaster $(local_domains) {
modify {
replace_rcpt &local_rewrites
}
deliver_to &local_mailboxes
}
default_destination {
reject 550 5.1.1 "User doesn't exist"
}
}
# ---- Inbound SMTP (port 25) ----
smtp tcp://0.0.0.0:25 {
limits {
all rate 20 1s
all concurrency 10
}
dmarc yes
check {
require_mx_record
dkim
spf
}
source $(local_domains) {
reject 501 5.1.8 "Use Submission for outgoing SMTP"
}
default_source {
destination postmaster $(local_domains) {
deliver_to &local_routing
}
default_destination {
reject 550 5.1.1 "User doesn't exist"
}
}
}
# ---- Submission (ports 465 implicit-TLS, 587 STARTTLS) ----
submission tls://0.0.0.0:465 tcp://0.0.0.0:587 {
limits {
all rate 50 1s
}
auth &local_authdb
source $(local_domains) {
check {
authorize_sender {
prepare_email &local_rewrites
user_to_email identity
}
}
destination postmaster $(local_domains) {
deliver_to &local_routing
}
default_destination {
modify {
dkim $(primary_domain) $(hostname) default
}
deliver_to &remote_queue
}
}
default_source {
reject 501 5.1.8 "Non-local sender domain"
}
}
# ---- Outbound delivery queue ----
target.remote outbound_delivery {
limits {
destination rate 20 1s
destination concurrency 10
}
mx_auth {
dane
mtasts {
cache fs
fs_dir mtasts_cache/
}
local_policy {
min_tls_level encrypted
min_mx_level none
}
}
}
target.queue remote_queue {
target &outbound_delivery
autogenerated_msg_domain $(primary_domain)
bounce {
destination postmaster $(local_domains) {
deliver_to &local_routing
}
default_destination {
reject 550 5.0.0 "Refusing to send DSNs to non-local addresses"
}
}
}
# ---- IMAP (993 implicit-TLS, 143 STARTTLS) ----
imap tls://0.0.0.0:993 tcp://0.0.0.0:143 {
auth &local_authdb
storage &local_mailboxes
}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: maddy-data
namespace: mail
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: local-path
resources:
requests:
storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: maddy
namespace: mail
labels:
app: maddy
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: maddy
template:
metadata:
labels:
app: maddy
spec:
containers:
- name: maddy
image: foxcpp/maddy:0.7
imagePullPolicy: IfNotPresent
env:
- name: MADDY_HOSTNAME
value: mail.abrban.com
- name: MADDY_DOMAIN
value: abrban.com
ports:
- { name: smtp, containerPort: 25 }
- { name: submission, containerPort: 587 }
- { name: smtps, containerPort: 465 }
- { name: imap, containerPort: 143 }
- { name: imaps, containerPort: 993 }
volumeMounts:
- name: data
mountPath: /data
- name: config
mountPath: /data/maddy.conf
subPath: maddy.conf
- name: tls
mountPath: /etc/maddy/tls
readOnly: true
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: "1"
memory: 256Mi
livenessProbe:
tcpSocket: { port: 25 }
initialDelaySeconds: 15
periodSeconds: 30
volumes:
- name: data
persistentVolumeClaim:
claimName: maddy-data
- name: config
configMap:
name: maddy-config
- name: tls
secret:
secretName: abrban-wildcard-tls
---
apiVersion: v1
kind: Service
metadata:
name: maddy
namespace: mail
labels:
app: maddy
spec:
type: LoadBalancer
externalTrafficPolicy: Local # preserve client source IP (needed for SPF/spam checks)
selector:
app: maddy
ports:
- { name: smtp, port: 25, targetPort: 25 }
- { name: submission, port: 587, targetPort: 587 }
- { name: smtps, port: 465, targetPort: 465 }
- { name: imap, port: 143, targetPort: 143 }
- { name: imaps, port: 993, targetPort: 993 }
backend/k8s/nixpacks-spike/nixpacks-spike.yaml
-212
View File
@@ -1,212 +0,0 @@
# ─────────────────────────────────────────────────────────────────────────────
# فاز ۰ — Spike ریسک Nixpacks روی شبکه‌ی ایران (abrban / cloudhost-builds)
#
# هدف: قبل از مهاجرت سیستم بیلد به Nixpacks (فاز ۲)، مطمئن شویم زنجیره‌ی
# nixpacks (تولید Dockerfile) → kaniko (build واقعی + نصب وابستگی‌ها)
# پشت شبکه‌ی ایران کار می‌کند و کشف کنیم چه mirror/proxy لازم است.
#
# چرا این ساختار: `nixpacks build --out` فقط Dockerfile می‌سازد و دانلودی ندارد؛
# دانلود سنگین (nixpkgs + npm/go modules) داخل مرحله‌ی Docker build اتفاق می‌افتد.
# پس برای تست واقعی شبکه باید kaniko همان Dockerfile تولیدی را build کند.
# با --no-push نیازی به رجیستری/کردنشال نیست — فقط build تست می‌شود.
#
# اجرا:
# kubectl apply -f nixpacks-spike.yaml
# kubectl -n cloudhost-builds logs -f job/nixpacks-spike-node
# kubectl -n cloudhost-builds logs -f job/nixpacks-spike-go
# # بعد از اتمام:
# kubectl -n cloudhost-builds delete -f nixpacks-spike.yaml
#
# اگر kaniko سرِ `RUN ... npm install` یا fetch nixpkgs گیر کرد → شبکه‌ی ایران
# مانع است؛ env های mirror را (بخش «نکات mirror» پایین فایل) فعال/تنظیم کنید و
# دوباره اجرا کنید. نتیجه را برای تصمیم فاز ۲ مستند کنید.
# ─────────────────────────────────────────────────────────────────────────────
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nixpacks-spike-node-src
namespace: cloudhost-builds
data:
package.json: |
{
"name": "nixpacks-spike",
"version": "1.0.0",
"private": true,
"scripts": { "start": "node index.js" },
"dependencies": { "express": "^4.18.2" }
}
index.js: |
const express = require('express');
const app = express();
app.get('/', (_req, res) => res.send('nixpacks spike ok'));
app.listen(process.env.PORT || 3000, () => console.log('up'));
---
apiVersion: batch/v1
kind: Job
metadata:
name: nixpacks-spike-node
namespace: cloudhost-builds
spec:
backoffLimit: 0
ttlSecondsAfterFinished: 1800
template:
spec:
restartPolicy: Never
volumes:
- name: workspace
emptyDir: {}
- name: src
configMap:
name: nixpacks-spike-node-src
initContainers:
# 1) staging سورس نمونه از ConfigMap به workspace
- name: stage-source
image: alpine:3.19
imagePullPolicy: IfNotPresent
command:
- sh
- -c
- |
set -e
mkdir -p /workspace/source
cp /src/package.json /workspace/source/package.json
cp /src/index.js /workspace/source/index.js
echo ">>> staged source:" && ls -la /workspace/source
volumeMounts:
- { name: workspace, mountPath: /workspace }
- { name: src, mountPath: /src }
# 2) Nixpacks: تولید Dockerfile در /workspace/source/.nixpacks/Dockerfile
- name: nixpacks-plan
image: ghcr.io/railwayapp/nixpacks:latest
imagePullPolicy: IfNotPresent
# نگاشت همان تنظیماتی که فاز ۲ پاس می‌دهد (نسخه‌ی Node و PORT)
env:
- { name: NIXPACKS_NODE_VERSION, value: "20" }
# - { name: NPM_CONFIG_REGISTRY, value: "https://registry.npmmirror.com" } # ← در صورت نیاز
command:
- nixpacks
- build
- /workspace/source
- --out
- /workspace/source
volumeMounts:
- { name: workspace, mountPath: /workspace }
containers:
# 3) Kaniko: build واقعی Dockerfile تولیدی (تست دانلود وابستگی‌ها). بدون push.
- name: kaniko
image: gcr.io/kaniko-project/executor:v1.23.2
imagePullPolicy: IfNotPresent
args:
- --dockerfile=/workspace/source/.nixpacks/Dockerfile
- --context=dir:///workspace/source
- --no-push
- --verbosity=info
volumeMounts:
- { name: workspace, mountPath: /workspace }
resources:
requests: { cpu: "500m", memory: "1Gi" }
limits: { cpu: "2", memory: "4Gi" }
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nixpacks-spike-go-src
namespace: cloudhost-builds
data:
go.mod: |
module nixpacksspike
go 1.22
main.go: |
package main
import (
"fmt"
"net/http"
"os"
)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, _ *http.Request) {
fmt.Fprintln(w, "nixpacks spike ok")
})
port := os.Getenv("PORT")
if port == "" {
port = "8080"
}
http.ListenAndServe(":"+port, nil)
}
---
apiVersion: batch/v1
kind: Job
metadata:
name: nixpacks-spike-go
namespace: cloudhost-builds
spec:
backoffLimit: 0
ttlSecondsAfterFinished: 1800
template:
spec:
restartPolicy: Never
volumes:
- name: workspace
emptyDir: {}
- name: src
configMap:
name: nixpacks-spike-go-src
initContainers:
- name: stage-source
image: alpine:3.19
imagePullPolicy: IfNotPresent
command:
- sh
- -c
- |
set -e
mkdir -p /workspace/source
cp /src/go.mod /workspace/source/go.mod
cp /src/main.go /workspace/source/main.go
echo ">>> staged source:" && ls -la /workspace/source
volumeMounts:
- { name: workspace, mountPath: /workspace }
- { name: src, mountPath: /src }
- name: nixpacks-plan
image: ghcr.io/railwayapp/nixpacks:latest
imagePullPolicy: IfNotPresent
env:
# - { name: GOPROXY, value: "https://goproxy.cn,direct" } # ← در صورت نیاز (mirror چین)
command:
- nixpacks
- build
- /workspace/source
- --out
- /workspace/source
volumeMounts:
- { name: workspace, mountPath: /workspace }
containers:
- name: kaniko
image: gcr.io/kaniko-project/executor:v1.23.2
imagePullPolicy: IfNotPresent
args:
- --dockerfile=/workspace/source/.nixpacks/Dockerfile
- --context=dir:///workspace/source
- --no-push
- --verbosity=info
volumeMounts:
- { name: workspace, mountPath: /workspace }
resources:
requests: { cpu: "500m", memory: "1Gi" }
limits: { cpu: "2", memory: "4Gi" }
# ─────────────────────────────────────────────────────────────────────────────
# نکات mirror (اگر build گیر کرد، uncomment/تنظیم و دوباره اجرا کنید):
# • npm: NPM_CONFIG_REGISTRY=https://registry.npmmirror.com (روی container kaniko
# اثر ندارد چون Dockerfile تولیدی است؛ بهتر است در فاز ۲ به‌صورت ARG/ENV
# داخل مرحله‌ی نصب تزریق شود — اینجا فقط برای nixpacks-plan گذاشته شده.)
# • nix: اگر دانلود nixpkgs (https://github.com/NixOS/...) شکست خورد، احتمال نیاز به
# HTTP(S)_PROXY روی container kaniko یا آینه‌سازی nixpkgs. در لاگ kaniko دنبال
# خطوط fetch tarball بگردید.
# • go: GOPROXY=https://goproxy.cn,direct یا proxy داخلی.
# • اگر pull از ghcr.io/gcr.io خود مشکل داشت → image ها را به رجیستری داخلی mirror کنید
# (همان الگوی LOGGING_*_IMAGE در configuration.ts).
# ─────────────────────────────────────────────────────────────────────────────
backend/package-lock.json
Generated
+485 -358
View File
File diff suppressed because it is too large Load Diff
backend/package.json
+16 -6
View File
@@ -1,5 +1,5 @@
{
"name": "cloudhost-backend",
"name": "abrban-backend",
"version": "1.0.0",
"description": "CloudHost PaaS Backend API",
"private": true,
@@ -9,7 +9,9 @@
"start:dev": "nest start --watch",
"start:debug": "nest start --debug --watch",
"start:prod": "node dist/main",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"lint": "eslint \"src/**/*.ts\" --fix",
"lint:check": "eslint \"src/**/*.ts\"",
"typecheck": "tsc --noEmit",
"format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
"test": "jest",
"test:watch": "jest --watch",
@@ -18,9 +20,11 @@
"migration:generate": "npm run typeorm -- migration:generate -d src/config/typeorm.config.ts",
"migration:run": "npm run typeorm -- migration:run -d src/config/typeorm.config.ts",
"migration:revert": "npm run typeorm -- migration:revert -d src/config/typeorm.config.ts",
"seed": "ts-node -r tsconfig-paths/register src/seed.ts"
"seed": "ts-node -r tsconfig-paths/register src/seed.ts",
"sync:migrations": "node scripts/sync-helm-migrations.mjs"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.1077.0",
"@kubernetes/client-node": "^1.4.0",
"@nestjs/bull": "^11.0.4",
"@nestjs/common": "^11.1.24",
@@ -30,6 +34,7 @@
"@nestjs/passport": "^11.0.5",
"@nestjs/platform-express": "^11.1.26",
"@nestjs/swagger": "^11.4.4",
"@nestjs/throttler": "^6.5.0",
"@nestjs/typeorm": "^11.0.1",
"bcrypt": "^6.0.0",
"bull": "^4.12.0",
@@ -37,8 +42,8 @@
"class-validator": "^0.15.1",
"handlebars": "^4.7.8",
"helmet": "^8.2.0",
"ioredis": "^5.11.1",
"js-yaml": "^4.2.0",
"minio": "^8.0.7",
"multer": "^2.1.1",
"passport": "^0.7.0",
"passport-jwt": "^4.0.1",
@@ -46,7 +51,8 @@
"reflect-metadata": "^0.2.1",
"rxjs": "^7.8.2",
"typeorm": "^1.0.0",
"uuid": "^14.0.0"
"uuid": "^14.0.0",
"yauzl": "^3.4.0"
},
"devDependencies": {
"@nestjs/cli": "^11.0.23",
@@ -59,6 +65,7 @@
"@types/multer": "^2.1.0",
"@types/node": "^24.0.0",
"@types/passport-jwt": "^4.0.0",
"@types/yauzl": "^3.4.0",
"@typescript-eslint/eslint-plugin": "^8.61.0",
"@typescript-eslint/parser": "^8.61.0",
"eslint": "^9.0.0",
@@ -84,6 +91,9 @@
"**/*.(t|j)s"
],
"coverageDirectory": "../coverage",
"testEnvironment": "node"
"testEnvironment": "node",
"setupFilesAfterEnv": [
"<rootDir>/test-setup.ts"
]
}
}
backend/scripts/sync-helm-migrations.mjs
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env node
/**
* Copy SQL migrations from backend/migrations/ into the Helm chart ConfigMap source.
* Run after adding or editing migration files: npm run sync:migrations
*/
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const sourceDir = path.resolve(__dirname, '../migrations');
const targetDir = path.resolve(__dirname, '../helm/cloudhost-platform/migrations');
if (!fs.existsSync(sourceDir)) {
console.error(`Source not found: ${sourceDir}`);
process.exit(1);
}
fs.mkdirSync(targetDir, { recursive: true });
const files = fs.readdirSync(sourceDir).filter((f) => f.endsWith('.sql')).sort();
for (const file of files) {
fs.copyFileSync(path.join(sourceDir, file), path.join(targetDir, file));
}
// Remove stale SQL files no longer in source
for (const existing of fs.readdirSync(targetDir)) {
if (existing.endsWith('.sql') && !files.includes(existing)) {
fs.unlinkSync(path.join(targetDir, existing));
}
}
console.log(`Synced ${files.length} migration(s) to ${targetDir}`);
backend/src/app.module.ts
+19 -7
View File
@@ -2,6 +2,8 @@ import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { TypeOrmModule } from '@nestjs/typeorm';
import { BullModule } from '@nestjs/bull';
import { ThrottlerModule, ThrottlerGuard } from '@nestjs/throttler';
import { APP_GUARD } from '@nestjs/core';
import { AuthModule } from './auth/auth.module';
import { UsersModule } from './users/users.module';
import { ApplicationsModule } from './applications/applications.module';
@@ -15,8 +17,8 @@ import { SnapshotsModule } from './snapshots/snapshots.module';
import { LifecycleModule } from './lifecycle/lifecycle.module';
import { ApplicationMigrationsModule } from './application-migrations/application-migrations.module';
import { AdminModule } from './admin/admin.module';
import { RedisModule } from './common/redis/redis.module';
import { StorageModule } from './common/storage/storage.module';
import { HealthModule } from './health/health.module';
import { StorageModule } from './storage/storage.module';
import configuration from './config/configuration';
@Module({
@@ -56,13 +58,16 @@ import configuration from './config/configuration';
inject: [ConfigService],
}),
// Shared Redis client (build state across replicas)
RedisModule,
// Shared MinIO storage (application source archives)
StorageModule,
ThrottlerModule.forRoot([
{
name: 'default',
ttl: 60_000,
limit: 120,
},
]),
// Feature modules
StorageModule,
AuthModule,
UsersModule,
ApplicationsModule,
@@ -76,6 +81,13 @@ import configuration from './config/configuration';
LifecycleModule,
ApplicationMigrationsModule,
AdminModule,
HealthModule,
],
providers: [
{
provide: APP_GUARD,
useClass: ThrottlerGuard,
},
],
})
export class AppModule {}
backend/src/applications/applications.controller.ts
+15 -1
View File
@@ -18,7 +18,8 @@ import {
} from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';
import { FileInterceptor } from '@nestjs/platform-express';
import { ApiTags, ApiOperation, ApiBearerAuth, ApiConsumes } from '@nestjs/swagger';
import { ApiTags, ApiOperation, ApiBearerAuth, ApiConsumes, ApiBadRequestResponse } from '@nestjs/swagger';
import { Throttle } from '@nestjs/throttler';
import { ApplicationsService } from './applications.service';
import { DomainService } from './domain.service';
import { CreateApplicationDto, UpdateApplicationDto, ScaleResourcesDto, SetCustomDomainDto, CheckDnsDto } from './dto/application.dto';
@@ -67,7 +68,20 @@ export class ApplicationsController {
}
@Post(':id/upload')
@Throttle({ default: { limit: 10, ttl: 60_000 } })
@ApiOperation({ summary: 'Upload application code (zip file)' })
@ApiBadRequestResponse({
description: 'Runtime mismatch between selected app type and archive contents',
schema: {
example: {
statusCode: 400,
message: 'Selected runtime "nodejs" does not match the uploaded source (detected "go").',
configured: 'nodejs',
detected: 'go',
signals: ['go.mod'],
},
},
})
@ApiConsumes('multipart/form-data')
@UseInterceptors(FileInterceptor('file', {
limits: { fileSize: 10 * 1024 * 1024 * 1024 }, // 10 GiB max application archive
backend/src/applications/applications.service.ts
+39 -18
View File
@@ -17,7 +17,12 @@ import {
} from '../common/enums';
import { ensureAppUrlEnv } from './app-url.util';
import { normalizeCreateApplicationDto } from './managed-service.util';
import { StorageService } from '../common/storage/storage.service';
import {
assertRuntimeMatch,
detectRuntimeFromArchive,
} from '../build/runtime-detector';
import { SourceStorageService } from '../storage/source-storage.service';
import * as os from 'os';
@Injectable()
export class ApplicationsService {
@@ -28,7 +33,7 @@ export class ApplicationsService {
private appsRepository: Repository<Application>,
private clustersService: ClustersService,
private configService: ConfigService,
private storageService: StorageService,
private sourceStorage: SourceStorageService,
) {}
private toDnsLabel(value: string): string {
@@ -206,20 +211,13 @@ export class ApplicationsService {
async delete(id: string, userId: string): Promise<Application> {
const app = await this.findOne(id, userId);
// Delete the uploaded source archive from object storage.
// Delete uploaded source files
if (app.codePath) {
await this.storageService.removeSource(app.codePath);
}
// Remove any legacy on-disk dump/source dir (db dumps are still stored locally).
try {
const uploadDir = this.configService.get<string>('platform.uploadDir') || './uploads';
const appDir = path.join(uploadDir, app.userId, app.id);
if (fs.existsSync(appDir)) {
fs.rmSync(appDir, { recursive: true, force: true });
this.logger.log(`Deleted upload directory: ${appDir}`);
}
await this.sourceStorage.deleteSource(app.userId, app.id, app.codePath);
} catch (e: any) {
this.logger.warn(`Failed to delete upload dir for ${app.name}: ${e.message}`);
this.logger.warn(`Failed to delete source for ${app.name}: ${e.message}`);
}
}
await this.appsRepository.remove(app);
@@ -265,15 +263,38 @@ export class ApplicationsService {
}
const app = await this.findOne(id, userId);
const tempPath = path.join(os.tmpdir(), `upload-${app.id}-${Date.now()}.zip`);
fs.writeFileSync(tempPath, file.buffer);
// Stream the archive to MinIO; codePath stores the object key (build pods
// pull it via a presigned URL — no local disk, no PVC, no kubectl cp).
const key = await this.storageService.putSource(app.userId, app.id, file.buffer);
app.codePath = key;
try {
const detected = await detectRuntimeFromArchive(tempPath);
assertRuntimeMatch(app.runtime, detected);
const storedPath = await this.sourceStorage.putSource(app.userId, app.id, file.buffer);
app.codePath = storedPath;
const saved = await this.appsRepository.save(app);
this.logger.log(`Uploaded code for ${app.name} → ${key} (${(file.size / 1024).toFixed(1)} KB)`);
if (detected.confidence === 'low') {
Object.assign(saved, {
runtimeWarning:
'Could not determine the project type from the archive with high confidence. Build may fail if the selected runtime is wrong.',
});
}
this.logger.log(`Uploaded code for ${app.name} → ${storedPath} (${(file.size / 1024).toFixed(1)} KB)`);
return saved;
} catch (err) {
try {
await this.sourceStorage.deleteSource(app.userId, app.id);
} catch {
// ignore rollback errors
}
throw err;
} finally {
if (fs.existsSync(tempPath)) {
fs.unlinkSync(tempPath);
}
}
}
async uploadDbDump(id: string, userId: string, file: Express.Multer.File): Promise<Application> {
backend/src/auth/auth.controller.ts
+2
View File
@@ -1,5 +1,6 @@
import { Controller, Post, Body, HttpCode, HttpStatus } from '@nestjs/common';
import { ApiTags, ApiOperation, ApiResponse } from '@nestjs/swagger';
import { Throttle } from '@nestjs/throttler';
import { AuthService } from './auth.service';
import { RegisterDto } from './dto/register.dto';
import { LoginDto } from './dto/login.dto';
@@ -7,6 +8,7 @@ import { OtpRequestDto, OtpVerifyDto } from './dto/otp.dto';
import { RefreshTokenDto } from './dto/refresh-token.dto';
@ApiTags('Authentication')
@Throttle({ default: { limit: 20, ttl: 60_000 } })
@Controller('auth')
export class AuthController {
constructor(private readonly authService: AuthService) {}
backend/src/billing/billing-invoices.controller.ts
+149
View File
@@ -0,0 +1,149 @@
import {
Controller,
Get,
Post,
Patch,
Body,
Param,
Query,
UseGuards,
Request,
} from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';
import { ApiTags, ApiOperation, ApiBearerAuth } from '@nestjs/swagger';
import { BillingService } from './billing.service';
import { BillingOpsService } from './billing-ops.service';
import { assertStubGatewayAllowed } from './payment-gateway.util';
import {
ChargeWalletDto,
InitiateInvoicePaymentDto,
VerifyInvoiceGatewayDto,
UpdateInvoiceStatusDto,
} from './dto/billing.dto';
import { RolesGuard } from '../common/guards/roles.guard';
import { Roles } from '../common/decorators/roles.decorator';
import { UserRole, InvoiceStatus, PaymentMethod } from '../common/enums';
@ApiTags('Billing')
@ApiBearerAuth()
@Controller('billing')
@UseGuards(AuthGuard('jwt'), RolesGuard)
export class BillingInvoicesController {
constructor(
private readonly billingService: BillingService,
private readonly billingOpsService: BillingOpsService,
) {}
// ─── Invoices ─────────────────────────────────────────────────────
@Get('invoices')
@ApiOperation({ summary: 'List my invoices' })
async listMyInvoices(
@Request() req: any,
@Query('status') status?: InvoiceStatus,
@Query('applicationId') applicationId?: string,
@Query('limit') limit?: string,
) {
return this.billingService.listInvoices(req.user, {
status,
applicationId,
limit: limit ? parseInt(limit, 10) : undefined,
});
}
@Get('invoices/:id')
@ApiOperation({ summary: 'Get one invoice with line items and transactions' })
async getInvoice(@Request() req: any, @Param('id') id: string) {
return this.billingService.getInvoiceForUser(id, req.user);
}
@Post('invoices/:id/pay/mixed')
@ApiOperation({ summary: 'Pay invoice with wallet first, then gateway for the remaining amount' })
async initiateInvoiceMixed(
@Request() req: any,
@Param('id') id: string,
@Body() dto: InitiateInvoicePaymentDto,
) {
const result = await this.billingService.initiateInvoiceMixedPayment(id, req.user, dto.callbackUrl);
const effect = await this.billingOpsService.completePaidInvoiceEffect(result.invoice);
return { ...result, effect };
}
@Post('invoices/:id/gateway/verify')
@ApiOperation({ summary: 'Verify invoice gateway payment' })
async verifyInvoiceGateway(
@Request() req: any,
@Param('id') id: string,
@Body() dto: VerifyInvoiceGatewayDto,
) {
assertStubGatewayAllowed();
const result = await this.billingService.verifyInvoiceGatewayPayment(
id,
req.user,
dto.trackingCode,
dto.amount,
);
const effect = await this.billingOpsService.completePaidInvoiceEffect(result.invoice);
return { ...result, effect };
}
// ─── Invoice Admin ────────────────────────────────────────────────
@Get('admin/invoices')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'List all invoices (Admin)' })
async listAdminInvoices(
@Request() req: any,
@Query('status') status?: InvoiceStatus,
@Query('userId') userId?: string,
@Query('applicationId') applicationId?: string,
@Query('paymentMethod') paymentMethod?: PaymentMethod,
@Query('search') search?: string,
@Query('limit') limit?: string,
) {
return this.billingService.listInvoices(req.user, {
status,
userId,
applicationId,
paymentMethod,
search,
limit: limit ? parseInt(limit, 10) : undefined,
});
}
@Get('admin/invoices/:id')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'Get invoice details (Admin)' })
async getAdminInvoice(@Request() req: any, @Param('id') id: string) {
return this.billingService.getInvoiceForUser(id, req.user);
}
@Patch('admin/invoices/:id/status')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'Update invoice status with reason (Admin)' })
async updateAdminInvoiceStatus(
@Param('id') id: string,
@Body() dto: UpdateInvoiceStatusDto,
) {
return this.billingService.updateInvoiceStatus(id, dto.status, dto.reason);
}
// ─── Wallet Admin ─────────────────────────────────────────────────
@Get('admin/wallets')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'List all wallets (Admin)' })
async getAllWallets() {
return this.billingService.getAllWallets();
}
@Post('admin/wallets/:userId/charge')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'Charge a user\'s wallet (Admin)' })
async adminChargeWallet(
@Param('userId') userId: string,
@Body() dto: ChargeWalletDto,
) {
return this.billingService.adminChargeWallet(userId, dto.amount, dto.description);
}
}
backend/src/billing/billing-ops.service.ts
+258
View File
@@ -0,0 +1,258 @@
import { BadRequestException, Inject, Injectable, forwardRef } from '@nestjs/common';
import { BillingService } from './billing.service';
import { AppLifecycleService } from '../lifecycle/app-lifecycle.service';
import { ApplicationsService } from '../applications/applications.service';
import { KubernetesService } from '../kubernetes/kubernetes.service';
import { UpgradeResourcesDto } from './dto/billing.dto';
import {
BillingCycle,
InvoiceStatus,
ProductType,
DatabaseType,
UserRole,
} from '../common/enums';
import { Application } from '../applications/entities/application.entity';
@Injectable()
export class BillingOpsService {
constructor(
private readonly billingService: BillingService,
@Inject(forwardRef(() => AppLifecycleService))
private readonly lifecycleService: AppLifecycleService,
@Inject(forwardRef(() => ApplicationsService))
private readonly applicationsService: ApplicationsService,
@Inject(forwardRef(() => KubernetesService))
private readonly kubernetesService: KubernetesService,
) {}
async completePaidInvoiceEffect(invoice: any) {
if (invoice.status !== InvoiceStatus.PAID) return null;
if (invoice.metadata?.completedAt) return invoice.metadata.completionResult || null;
const action = invoice.metadata?.action;
if (!action || !invoice.applicationId) return null;
if (action === 'renew' || action === 'activate') {
const cycle = invoice.metadata?.cycle as BillingCycle;
if (!Object.values(BillingCycle).includes(cycle)) return null;
const activated = await this.lifecycleService.activateApp(invoice.applicationId, cycle);
const result = {
action,
application: {
id: activated.id,
name: activated.name,
lifecycleStatus: activated.lifecycleStatus,
planExpiresAt: activated.planExpiresAt,
billingCycle: activated.billingCycle,
},
};
await this.billingService.markInvoiceEffectCompleted(invoice.id, result);
return result;
}
if (action === 'upgrade') {
const app = await this.applicationsService.findOne(invoice.applicationId);
const resources = (invoice.metadata?.resources || {}) as UpgradeResourcesDto;
const updatedApp = await this.applicationsService.update(
app.id,
app.userId,
this.buildUpgradeEntityPatch(app, resources),
);
try {
await this.applyUpgradeToKubernetes(updatedApp, resources, app);
} catch (e: any) {
console.warn(`K8s resource update failed for ${app.name}: ${e.message}`);
}
const result = {
action,
application: {
id: updatedApp.id,
name: updatedApp.name,
cpuRequest: updatedApp.cpuRequest,
cpuLimit: updatedApp.cpuLimit,
memoryRequest: updatedApp.memoryRequest,
memoryLimit: updatedApp.memoryLimit,
replicas: updatedApp.replicas,
},
};
await this.billingService.markInvoiceEffectCompleted(invoice.id, result);
return result;
}
return null;
}
buildUpgradeEntityPatch(app: Application, dto: UpgradeResourcesDto): Partial<Application> {
const pt = app.productType ?? ProductType.APPLICATION;
if (dto.redisResources) {
return {
optionalServiceResources: {
...app.optionalServiceResources,
redis: {
...app.optionalServiceResources?.redis,
...dto.redisResources,
storageGi:
dto.redisResources.storageGi ?? app.optionalServiceResources?.redis?.storageGi ?? 1,
},
},
};
}
if (dto.rabbitmqResources) {
return {
optionalServiceResources: {
...app.optionalServiceResources,
rabbitmq: {
...app.optionalServiceResources?.rabbitmq,
...dto.rabbitmqResources,
storageGi:
dto.rabbitmqResources.storageGi ??
app.optionalServiceResources?.rabbitmq?.storageGi ??
2,
},
},
};
}
if (pt === ProductType.MANAGED_REDIS || pt === ProductType.MANAGED_RABBITMQ) {
return {};
}
return {
cpuRequest: dto.cpuRequest || app.cpuRequest,
cpuLimit: dto.cpuLimit || app.cpuLimit,
memoryRequest: dto.memoryRequest || app.memoryRequest,
memoryLimit: dto.memoryLimit || app.memoryLimit,
replicas: dto.replicas ?? app.replicas,
dbStorageSize: dto.dbStorageSize || app.dbStorageSize,
appStorageSize: dto.appStorageSize || app.appStorageSize,
};
}
async applyUpgradeToKubernetes(
app: Application,
dto: UpgradeResourcesDto,
previous: Application,
): Promise<void> {
const pt = app.productType ?? ProductType.APPLICATION;
if (pt === ProductType.MANAGED_DATABASE) {
await this.kubernetesService.updateResources(
app,
{
cpuRequest: dto.cpuRequest,
cpuLimit: dto.cpuLimit,
memoryRequest: dto.memoryRequest,
memoryLimit: dto.memoryLimit,
},
'database',
);
if (dto.dbStorageSize && dto.dbStorageSize !== previous.dbStorageSize) {
const resize = await this.kubernetesService.resizeDatabasePvc(app, dto.dbStorageSize);
if (!resize.success) {
throw new BadRequestException(resize.message);
}
}
return;
}
if (pt === ProductType.MANAGED_REDIS) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'redis');
return;
}
if (pt === ProductType.MANAGED_RABBITMQ) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'rabbitmq');
return;
}
if (dto.redisResources && app.enableRedis) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'redis');
}
if (dto.rabbitmqResources && app.enableRabbitmq) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'rabbitmq');
}
const touchesAppWorkload =
dto.cpuRequest !== undefined ||
dto.cpuLimit !== undefined ||
dto.memoryRequest !== undefined ||
dto.memoryLimit !== undefined ||
dto.replicas !== undefined;
if (touchesAppWorkload) {
await this.kubernetesService.updateResources(app, {
cpuRequest: dto.cpuRequest,
cpuLimit: dto.cpuLimit,
memoryRequest: dto.memoryRequest,
memoryLimit: dto.memoryLimit,
replicas: dto.replicas,
});
}
if (dto.appStorageSize && dto.appStorageSize !== previous.appStorageSize) {
await this.kubernetesService.resizeAppStoragePvc(app, dto.appStorageSize);
}
if (
dto.dbStorageSize &&
dto.dbStorageSize !== previous.dbStorageSize &&
previous.databaseType &&
previous.databaseType !== DatabaseType.NONE
) {
const resize = await this.kubernetesService.resizeDatabasePvc(app, dto.dbStorageSize);
if (!resize.success) {
throw new BadRequestException(resize.message);
}
}
}
async applyOptionalServiceUpgrade(
app: Application,
dto: UpgradeResourcesDto,
previous: Application,
service: 'redis' | 'rabbitmq',
): Promise<void> {
const res = app.optionalServiceResources?.[service];
const dtoRes = service === 'redis' ? dto.redisResources : dto.rabbitmqResources;
if (res) {
await this.kubernetesService.updateResources(
app,
{
cpuRequest: res.cpuRequest,
cpuLimit: res.cpuLimit,
memoryRequest: res.memoryRequest,
memoryLimit: res.memoryLimit,
},
service,
);
}
const prevGi =
previous.optionalServiceResources?.[service]?.storageGi ?? (service === 'redis' ? 1 : 2);
const nextGi = dtoRes?.storageGi;
if (nextGi != null && nextGi > prevGi) {
const resize =
service === 'redis'
? await this.kubernetesService.resizeRedisStoragePvc(app, `${nextGi}Gi`)
: await this.kubernetesService.resizeRabbitmqStoragePvc(app, `${nextGi}Gi`);
if (!resize.success) {
throw new BadRequestException(resize.message);
}
}
}
async getAppWithAccess(user: any, applicationId: string) {
const isAdminOrSales = user.role === UserRole.ADMIN || user.role === UserRole.SALES;
if (isAdminOrSales) {
return this.applicationsService.findOne(applicationId);
}
return this.applicationsService.findOne(applicationId, user.id);
}
}
backend/src/billing/billing-wallet.controller.ts
+190
View File
@@ -0,0 +1,190 @@
import {
Controller,
Get,
Post,
Body,
Param,
Query,
UseGuards,
Request,
BadRequestException,
Inject,
forwardRef,
} from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';
import { ApiTags, ApiOperation, ApiBearerAuth } from '@nestjs/swagger';
import { BillingService } from './billing.service';
import { assertStubGatewayAllowed } from './payment-gateway.util';
import { AppLifecycleService } from '../lifecycle/app-lifecycle.service';
import { ApplicationsService } from '../applications/applications.service';
import { ChargeWalletDto, PayApplicationDto } from './dto/billing.dto';
import { RolesGuard } from '../common/guards/roles.guard';
import { BillingCycle, InvoiceReason } from '../common/enums';
@ApiTags('Billing')
@ApiBearerAuth()
@Controller('billing')
@UseGuards(AuthGuard('jwt'), RolesGuard)
export class BillingWalletController {
constructor(
private readonly billingService: BillingService,
@Inject(forwardRef(() => AppLifecycleService))
private readonly lifecycleService: AppLifecycleService,
@Inject(forwardRef(() => ApplicationsService))
private readonly applicationsService: ApplicationsService,
) {}
// ─── Wallet (User) ───────────────────────────────────────────────
@Get('wallet')
@ApiOperation({ summary: 'Get my wallet balance' })
async getBalance(@Request() req: any) {
return this.billingService.getBalance(req.user.id);
}
@Post('wallet/charge')
@ApiOperation({ summary: 'Charge my wallet (self top-up)' })
async chargeMyWallet(@Request() req: any, @Body() dto: ChargeWalletDto) {
return this.billingService.chargeWallet(req.user.id, dto.amount, dto.description || 'Self top-up');
}
@Get('wallet/transactions')
@ApiOperation({ summary: 'Get my wallet transactions' })
async getTransactions(@Request() req: any, @Query('limit') limit?: string) {
return this.billingService.getTransactions(req.user.id, limit ? parseInt(limit, 10) : 50);
}
@Get('resource-credits')
@ApiOperation({ summary: 'List active prepaid resource credits (from deleted apps)' })
async getResourceCredits(@Request() req: any) {
const credits = await this.billingService.getActiveCredits(req.user.id);
return credits.map((c) => this.billingService.formatCreditForApi(c));
}
@Post('wallet/pay/:applicationId')
@ApiOperation({ summary: 'Pay for an application plan from wallet — activates/renews the app' })
async payForApplication(
@Request() req: any,
@Param('applicationId') applicationId: string,
@Body() body: PayApplicationDto,
) {
const cycle = body.cycle as BillingCycle;
if (!Object.values(BillingCycle).includes(cycle)) {
throw new BadRequestException(`Invalid billing cycle: ${body.cycle}`);
}
const app = await this.applicationsService.findOne(applicationId, req.user.id);
const payment = await this.billingService.resolveAppPayment(
req.user.id,
app,
cycle,
);
const coupon = await this.billingService.resolveCoupon(
req.user.id,
body.couponCode,
await this.billingService.getAppChargeBreakdown(app),
cycle,
payment.amountDue,
);
let invoice = null;
if (payment.amountDue > 0) {
invoice = await this.billingService.createInvoice({
userId: req.user.id,
applicationId: app.id,
reason: InvoiceReason.DEPLOY,
lines: [
{
label: `Application payment: ${app.name}`,
description: `Billing cycle: ${cycle}`,
amount: payment.amountDue,
metadata: {
cycle,
waivedAmount: payment.waivedAmount,
creditApplied: payment.creditId || null,
},
},
],
metadata: {
action: 'activate',
cycle,
},
discount: coupon ?? undefined,
});
}
let tx = null;
if (invoice) {
const paid = await this.billingService.payInvoiceWithWallet(invoice.id, req.user);
tx = paid.transaction;
invoice = paid.invoice;
}
const activated = await this.lifecycleService.activateApp(
applicationId,
cycle,
);
return {
transaction: tx,
invoice,
creditApplied: payment.creditId || null,
waivedAmount: payment.waivedAmount,
discountAmount: coupon?.amount ?? 0,
discountCode: coupon?.code ?? null,
paidAmount: invoice ? Number(invoice.total) : 0,
application: {
id: activated.id,
name: activated.name,
lifecycleStatus: activated.lifecycleStatus,
planExpiresAt: activated.planExpiresAt,
},
message: payment.waivedAmount > 0
? payment.amountDue > 0
? `Application "${activated.name}" activated — prepaid credit applied; you paid ${payment.amountDue} Toman for additional services.`
: `Application "${activated.name}" activated using your prepaid resource credit (no charge).`
: payment.amountDue > 0
? `Application "${activated.name}" activated until ${activated.planExpiresAt?.toISOString()}`
: `Application "${activated.name}" activated`,
};
}
// ─── Payment Gateway ─────────────────────────────────────────────
@Post('gateway/initiate')
@ApiOperation({ summary: 'Initiate a payment gateway transaction' })
async initiateGateway(
@Request() req: any,
@Body() body: { amount: number; description?: string; callbackUrl: string },
) {
assertStubGatewayAllowed();
const trackingCode = `PAY-${Date.now()}-${Math.random().toString(36).substring(2, 8).toUpperCase()}`;
return {
success: true,
trackingCode,
gatewayUrl: `${body.callbackUrl}?trackingCode=${trackingCode}&amount=${body.amount}&status=success`,
message: 'Redirect user to gatewayUrl to complete payment',
};
}
@Post('gateway/verify')
@ApiOperation({ summary: 'Verify a payment gateway transaction and charge wallet' })
async verifyGateway(
@Request() req: any,
@Body() body: { trackingCode: string; amount: number },
) {
assertStubGatewayAllowed();
await this.billingService.chargeWallet(
req.user.id,
body.amount,
`Payment gateway: ${body.trackingCode}`,
);
return {
success: true,
message: 'Payment verified and wallet charged',
trackingCode: body.trackingCode,
};
}
}
backend/src/billing/billing.controller.ts
+33 -535
View File
@@ -3,35 +3,27 @@ import {
Get,
Post,
Patch,
Delete,
Body,
Param,
Query,
UseGuards,
Request,
BadRequestException,
Inject,
forwardRef,
ForbiddenException,
} from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';
import { ApiTags, ApiOperation, ApiBearerAuth } from '@nestjs/swagger';
import { BillingService } from './billing.service';
import { BillingOpsService } from './billing-ops.service';
import { AppLifecycleService } from '../lifecycle/app-lifecycle.service';
import { ApplicationsService } from '../applications/applications.service';
import { KubernetesService } from '../kubernetes/kubernetes.service';
import {
ChargeWalletDto,
CalculateCostDto,
CalculateDeployCostDto,
SetOptionalServicesPricingDto,
RenewApplicationDto,
UpgradeResourcesDto,
CalculateUpgradeCostDto,
InitiateInvoicePaymentDto,
VerifyInvoiceGatewayDto,
UpdateInvoiceStatusDto,
PayApplicationDto,
} from './dto/billing.dto';
import { UpdatePricingCatalogDto } from './dto/pricing-catalog.dto';
import { RolesGuard } from '../common/guards/roles.guard';
@@ -41,12 +33,7 @@ import {
BillingCycle,
AppLifecycleStatus,
InvoiceReason,
InvoiceStatus,
PaymentMethod,
ProductType,
DatabaseType,
} from '../common/enums';
import { Application } from '../applications/entities/application.entity';
@ApiTags('Billing')
@ApiBearerAuth()
@@ -55,12 +42,11 @@ import { Application } from '../applications/entities/application.entity';
export class BillingController {
constructor(
private readonly billingService: BillingService,
private readonly billingOpsService: BillingOpsService,
@Inject(forwardRef(() => AppLifecycleService))
private readonly lifecycleService: AppLifecycleService,
@Inject(forwardRef(() => ApplicationsService))
private readonly applicationsService: ApplicationsService,
@Inject(forwardRef(() => KubernetesService))
private readonly kubernetesService: KubernetesService,
) {}
// ─── Pricing catalog (Admin) ──────────────────────────────────────
@@ -98,6 +84,29 @@ export class BillingController {
return this.billingService.calculateDeployPayment(req.user.id, dto, dto.cycle, dto.couponCode);
}
// ─── Global discount (platform-wide) ───────────────────────────
@Get('settings/global-discount')
@ApiOperation({ summary: 'Get the platform-wide discount percentage' })
async getGlobalDiscount() {
return this.billingService.getGlobalDiscount();
}
@Patch('settings/global-discount')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'Set the platform-wide discount percentage (Admin)' })
async setGlobalDiscount(@Body() body: { percentOff: number }) {
if (
body.percentOff === undefined ||
typeof body.percentOff !== 'number' ||
body.percentOff < 0 ||
body.percentOff > 100
) {
throw new BadRequestException('percentOff must be a number between 0 and 100');
}
return this.billingService.setGlobalDiscount(body.percentOff);
}
// ─── Custom Domain Pricing ─────────────────────────────────────
@Get('settings/custom-domain-price')
@@ -129,274 +138,6 @@ export class BillingController {
return this.billingService.setOptionalServicesPricing(dto);
}
// ─── Wallet (User) ───────────────────────────────────────────────
@Get('wallet')
@ApiOperation({ summary: 'Get my wallet balance' })
async getBalance(@Request() req: any) {
return this.billingService.getBalance(req.user.id);
}
@Post('wallet/charge')
@ApiOperation({ summary: 'Charge my wallet (self top-up)' })
async chargeMyWallet(@Request() req: any, @Body() dto: ChargeWalletDto) {
return this.billingService.chargeWallet(req.user.id, dto.amount, dto.description || 'Self top-up');
}
@Get('wallet/transactions')
@ApiOperation({ summary: 'Get my wallet transactions' })
async getTransactions(@Request() req: any, @Query('limit') limit?: string) {
return this.billingService.getTransactions(req.user.id, limit ? parseInt(limit, 10) : 50);
}
@Get('resource-credits')
@ApiOperation({ summary: 'List active prepaid resource credits (from deleted apps)' })
async getResourceCredits(@Request() req: any) {
const credits = await this.billingService.getActiveCredits(req.user.id);
return credits.map((c) => this.billingService.formatCreditForApi(c));
}
// ─── Invoices ─────────────────────────────────────────────────────
@Get('invoices')
@ApiOperation({ summary: 'List my invoices' })
async listMyInvoices(
@Request() req: any,
@Query('status') status?: InvoiceStatus,
@Query('applicationId') applicationId?: string,
@Query('limit') limit?: string,
) {
return this.billingService.listInvoices(req.user, {
status,
applicationId,
limit: limit ? parseInt(limit, 10) : undefined,
});
}
@Get('invoices/:id')
@ApiOperation({ summary: 'Get one invoice with line items and transactions' })
async getInvoice(@Request() req: any, @Param('id') id: string) {
return this.billingService.getInvoiceForUser(id, req.user);
}
@Post('invoices/:id/pay/mixed')
@ApiOperation({ summary: 'Pay invoice with wallet first, then gateway for the remaining amount' })
async initiateInvoiceMixed(
@Request() req: any,
@Param('id') id: string,
@Body() dto: InitiateInvoicePaymentDto,
) {
const result = await this.billingService.initiateInvoiceMixedPayment(id, req.user, dto.callbackUrl);
const effect = await this.completePaidInvoiceEffect(result.invoice);
return { ...result, effect };
}
@Post('invoices/:id/gateway/verify')
@ApiOperation({ summary: 'Verify invoice gateway payment' })
async verifyInvoiceGateway(
@Request() req: any,
@Param('id') id: string,
@Body() dto: VerifyInvoiceGatewayDto,
) {
const result = await this.billingService.verifyInvoiceGatewayPayment(
id,
req.user,
dto.trackingCode,
dto.amount,
);
const effect = await this.completePaidInvoiceEffect(result.invoice);
return { ...result, effect };
}
@Post('wallet/pay/:applicationId')
@ApiOperation({ summary: 'Pay for an application plan from wallet — activates/renews the app' })
async payForApplication(
@Request() req: any,
@Param('applicationId') applicationId: string,
@Body() body: PayApplicationDto,
) {
const cycle = body.cycle as BillingCycle;
if (!Object.values(BillingCycle).includes(cycle)) {
throw new BadRequestException(`Invalid billing cycle: ${body.cycle}`);
}
const app = await this.applicationsService.findOne(applicationId, req.user.id);
const payment = await this.billingService.resolveAppPayment(
req.user.id,
app,
cycle,
);
const coupon = await this.billingService.resolveCoupon(
req.user.id,
body.couponCode,
await this.billingService.getAppChargeBreakdown(app),
cycle,
payment.amountDue,
);
let invoice = null;
if (payment.amountDue > 0) {
invoice = await this.billingService.createInvoice({
userId: req.user.id,
applicationId: app.id,
reason: InvoiceReason.DEPLOY,
lines: [
{
label: `Application payment: ${app.name}`,
description: `Billing cycle: ${cycle}`,
amount: payment.amountDue,
metadata: {
cycle,
waivedAmount: payment.waivedAmount,
creditApplied: payment.creditId || null,
},
},
],
metadata: {
action: 'activate',
cycle,
},
discount: coupon ?? undefined,
});
}
let tx = null;
if (invoice) {
const paid = await this.billingService.payInvoiceWithWallet(invoice.id, req.user);
tx = paid.transaction;
invoice = paid.invoice;
}
const activated = await this.lifecycleService.activateApp(
applicationId,
cycle,
);
return {
transaction: tx,
invoice,
creditApplied: payment.creditId || null,
waivedAmount: payment.waivedAmount,
discountAmount: coupon?.amount ?? 0,
discountCode: coupon?.code ?? null,
paidAmount: invoice ? Number(invoice.total) : 0,
application: {
id: activated.id,
name: activated.name,
lifecycleStatus: activated.lifecycleStatus,
planExpiresAt: activated.planExpiresAt,
},
message: payment.waivedAmount > 0
? payment.amountDue > 0
? `Application "${activated.name}" activated — prepaid credit applied; you paid ${payment.amountDue} Toman for additional services.`
: `Application "${activated.name}" activated using your prepaid resource credit (no charge).`
: payment.amountDue > 0
? `Application "${activated.name}" activated until ${activated.planExpiresAt?.toISOString()}`
: `Application "${activated.name}" activated`,
};
}
// ─── Payment Gateway ─────────────────────────────────────────────
@Post('gateway/initiate')
@ApiOperation({ summary: 'Initiate a payment gateway transaction' })
async initiateGateway(
@Request() req: any,
@Body() body: { amount: number; description?: string; callbackUrl: string },
) {
// In production, integrate with Zarinpal/IDPay/etc.
// For now, simulate a gateway redirect URL.
const trackingCode = `PAY-${Date.now()}-${Math.random().toString(36).substring(2, 8).toUpperCase()}`;
return {
success: true,
trackingCode,
gatewayUrl: `${body.callbackUrl}?trackingCode=${trackingCode}&amount=${body.amount}&status=success`,
message: 'Redirect user to gatewayUrl to complete payment',
};
}
@Post('gateway/verify')
@ApiOperation({ summary: 'Verify a payment gateway transaction and charge wallet' })
async verifyGateway(
@Request() req: any,
@Body() body: { trackingCode: string; amount: number },
) {
// In production, verify with the gateway provider.
// For now, auto-approve and charge the wallet.
await this.billingService.chargeWallet(
req.user.id,
body.amount,
`Payment gateway: ${body.trackingCode}`,
);
return {
success: true,
message: 'Payment verified and wallet charged',
trackingCode: body.trackingCode,
};
}
// ─── Invoice Admin ────────────────────────────────────────────────
@Get('admin/invoices')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'List all invoices (Admin)' })
async listAdminInvoices(
@Request() req: any,
@Query('status') status?: InvoiceStatus,
@Query('userId') userId?: string,
@Query('applicationId') applicationId?: string,
@Query('paymentMethod') paymentMethod?: PaymentMethod,
@Query('search') search?: string,
@Query('limit') limit?: string,
) {
return this.billingService.listInvoices(req.user, {
status,
userId,
applicationId,
paymentMethod,
search,
limit: limit ? parseInt(limit, 10) : undefined,
});
}
@Get('admin/invoices/:id')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'Get invoice details (Admin)' })
async getAdminInvoice(@Request() req: any, @Param('id') id: string) {
return this.billingService.getInvoiceForUser(id, req.user);
}
@Patch('admin/invoices/:id/status')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'Update invoice status with reason (Admin)' })
async updateAdminInvoiceStatus(
@Param('id') id: string,
@Body() dto: UpdateInvoiceStatusDto,
) {
return this.billingService.updateInvoiceStatus(id, dto.status, dto.reason);
}
// ─── Wallet Admin ─────────────────────────────────────────────────
@Get('admin/wallets')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'List all wallets (Admin)' })
async getAllWallets() {
return this.billingService.getAllWallets();
}
@Post('admin/wallets/:userId/charge')
@Roles(UserRole.ADMIN)
@ApiOperation({ summary: 'Charge a user\'s wallet (Admin)' })
async adminChargeWallet(
@Param('userId') userId: string,
@Body() dto: ChargeWalletDto,
) {
return this.billingService.adminChargeWallet(userId, dto.amount, dto.description);
}
// ─── Application Renewal ──────────────────────────────────────────
@Get('applications/:applicationId/renewal-cost')
@@ -405,8 +146,7 @@ export class BillingController {
@Request() req: any,
@Param('applicationId') applicationId: string,
) {
// User can only view their own app, admin/sales can view any
const app = await this.getAppWithAccess(req.user, applicationId);
const app = await this.billingOpsService.getAppWithAccess(req.user, applicationId);
const costs = await this.billingService.calculateRenewalCost(app);
return {
applicationId: app.id,
@@ -425,7 +165,7 @@ export class BillingController {
@Param('applicationId') applicationId: string,
@Body() dto: RenewApplicationDto,
) {
const app = await this.getAppWithAccess(req.user, applicationId);
const app = await this.billingOpsService.getAppWithAccess(req.user, applicationId);
const costs = await this.billingService.calculateRenewalCost(app);
const amount = dto.cycle === BillingCycle.HOURLY ? costs.hourly
: dto.cycle === BillingCycle.MONTHLY ? costs.monthly
@@ -467,10 +207,8 @@ export class BillingController {
@Param('applicationId') applicationId: string,
@Body() dto: RenewApplicationDto,
) {
// User can only renew their own app, admin/sales can renew any
const app = await this.getAppWithAccess(req.user, applicationId);
const app = await this.billingOpsService.getAppWithAccess(req.user, applicationId);
// Calculate cost for the selected cycle
const costs = await this.billingService.calculateRenewalCost(app);
const amount = dto.cycle === BillingCycle.HOURLY ? costs.hourly
: dto.cycle === BillingCycle.MONTHLY ? costs.monthly
@@ -510,7 +248,6 @@ export class BillingController {
const paid = await this.billingService.payInvoiceWithWallet(invoice.id, req.user);
// Activate the application
const renewedApp = await this.lifecycleService.activateApp(app.id, dto.cycle);
return {
@@ -544,7 +281,6 @@ export class BillingController {
}
if (body.bypassPayment) {
// Direct activation without payment (for special cases, support, etc.)
const renewedApp = await this.lifecycleService.activateApp(app.id, cycle);
return {
success: true,
@@ -560,7 +296,6 @@ export class BillingController {
};
}
// Normal renewal - deduct from app owner's wallet
const costs = await this.billingService.calculateRenewalCost(app);
const amount = cycle === BillingCycle.HOURLY ? costs.hourly
: cycle === BillingCycle.MONTHLY ? costs.monthly
@@ -617,7 +352,7 @@ export class BillingController {
@Param('applicationId') applicationId: string,
@Body() dto: CalculateUpgradeCostDto,
) {
const app = await this.getAppWithAccess(req.user, applicationId);
const app = await this.billingOpsService.getAppWithAccess(req.user, applicationId);
const result = await this.billingService.calculateUpgradeCost(app, dto);
return {
@@ -652,7 +387,7 @@ export class BillingController {
@Param('applicationId') applicationId: string,
@Body() dto: UpgradeResourcesDto,
) {
const app = await this.getAppWithAccess(req.user, applicationId);
const app = await this.billingOpsService.getAppWithAccess(req.user, applicationId);
if (app.lifecycleStatus !== AppLifecycleStatus.ACTIVE) {
throw new BadRequestException(
`Cannot upgrade resources for ${app.lifecycleStatus} application. Please renew first.`,
@@ -704,20 +439,17 @@ export class BillingController {
@Param('applicationId') applicationId: string,
@Body() dto: UpgradeResourcesDto,
) {
const app = await this.getAppWithAccess(req.user, applicationId);
const app = await this.billingOpsService.getAppWithAccess(req.user, applicationId);
// Application must be active to upgrade
if (app.lifecycleStatus !== AppLifecycleStatus.ACTIVE) {
throw new BadRequestException(
`Cannot upgrade resources for ${app.lifecycleStatus} application. Please renew first.`
);
}
// Calculate upgrade cost
const costResult = await this.billingService.calculateUpgradeCost(app, dto);
let paidInvoice = null;
// If upgrading (positive difference), require payment
if (costResult.proratedAmount > 0) {
const walletUserId = req.user.role === UserRole.ADMIN || req.user.role === UserRole.SALES
? app.userId
@@ -761,11 +493,11 @@ export class BillingController {
const updatedApp = await this.applicationsService.update(
app.id,
app.userId,
this.buildUpgradeEntityPatch(app, dto),
this.billingOpsService.buildUpgradeEntityPatch(app, dto),
);
try {
await this.applyUpgradeToKubernetes(updatedApp, dto, app);
await this.billingOpsService.applyUpgradeToKubernetes(updatedApp, dto, app);
} catch (e: any) {
console.warn(`K8s resource update failed for ${app.name}: ${e.message}`);
}
@@ -790,238 +522,4 @@ export class BillingController {
: 'Resources updated (downgrade or no cost change).',
};
}
// ─── Helper Methods ───────────────────────────────────────────────
private async completePaidInvoiceEffect(invoice: any) {
if (invoice.status !== InvoiceStatus.PAID) return null;
if (invoice.metadata?.completedAt) return invoice.metadata.completionResult || null;
const action = invoice.metadata?.action;
if (!action || !invoice.applicationId) return null;
if (action === 'renew' || action === 'activate') {
const cycle = invoice.metadata?.cycle as BillingCycle;
if (!Object.values(BillingCycle).includes(cycle)) return null;
const activated = await this.lifecycleService.activateApp(invoice.applicationId, cycle);
const result = {
action,
application: {
id: activated.id,
name: activated.name,
lifecycleStatus: activated.lifecycleStatus,
planExpiresAt: activated.planExpiresAt,
billingCycle: activated.billingCycle,
},
};
await this.billingService.markInvoiceEffectCompleted(invoice.id, result);
return result;
}
if (action === 'upgrade') {
const app = await this.applicationsService.findOne(invoice.applicationId);
const resources = (invoice.metadata?.resources || {}) as UpgradeResourcesDto;
const updatedApp = await this.applicationsService.update(
app.id,
app.userId,
this.buildUpgradeEntityPatch(app, resources),
);
try {
await this.applyUpgradeToKubernetes(updatedApp, resources, app);
} catch (e: any) {
console.warn(`K8s resource update failed for ${app.name}: ${e.message}`);
}
const result = {
action,
application: {
id: updatedApp.id,
name: updatedApp.name,
cpuRequest: updatedApp.cpuRequest,
cpuLimit: updatedApp.cpuLimit,
memoryRequest: updatedApp.memoryRequest,
memoryLimit: updatedApp.memoryLimit,
replicas: updatedApp.replicas,
},
};
await this.billingService.markInvoiceEffectCompleted(invoice.id, result);
return result;
}
return null;
}
private buildUpgradeEntityPatch(app: Application, dto: UpgradeResourcesDto): Partial<Application> {
const pt = app.productType ?? ProductType.APPLICATION;
if (dto.redisResources) {
return {
optionalServiceResources: {
...app.optionalServiceResources,
redis: {
...app.optionalServiceResources?.redis,
...dto.redisResources,
storageGi:
dto.redisResources.storageGi ?? app.optionalServiceResources?.redis?.storageGi ?? 1,
},
},
};
}
if (dto.rabbitmqResources) {
return {
optionalServiceResources: {
...app.optionalServiceResources,
rabbitmq: {
...app.optionalServiceResources?.rabbitmq,
...dto.rabbitmqResources,
storageGi:
dto.rabbitmqResources.storageGi ??
app.optionalServiceResources?.rabbitmq?.storageGi ??
2,
},
},
};
}
if (pt === ProductType.MANAGED_REDIS || pt === ProductType.MANAGED_RABBITMQ) {
return {};
}
return {
cpuRequest: dto.cpuRequest || app.cpuRequest,
cpuLimit: dto.cpuLimit || app.cpuLimit,
memoryRequest: dto.memoryRequest || app.memoryRequest,
memoryLimit: dto.memoryLimit || app.memoryLimit,
replicas: dto.replicas ?? app.replicas,
dbStorageSize: dto.dbStorageSize || app.dbStorageSize,
appStorageSize: dto.appStorageSize || app.appStorageSize,
};
}
private async applyUpgradeToKubernetes(
app: Application,
dto: UpgradeResourcesDto,
previous: Application,
): Promise<void> {
const pt = app.productType ?? ProductType.APPLICATION;
if (pt === ProductType.MANAGED_DATABASE) {
await this.kubernetesService.updateResources(
app,
{
cpuRequest: dto.cpuRequest,
cpuLimit: dto.cpuLimit,
memoryRequest: dto.memoryRequest,
memoryLimit: dto.memoryLimit,
},
'database',
);
if (dto.dbStorageSize && dto.dbStorageSize !== previous.dbStorageSize) {
const resize = await this.kubernetesService.resizeDatabasePvc(app, dto.dbStorageSize);
if (!resize.success) {
throw new BadRequestException(resize.message);
}
}
return;
}
if (pt === ProductType.MANAGED_REDIS) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'redis');
return;
}
if (pt === ProductType.MANAGED_RABBITMQ) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'rabbitmq');
return;
}
if (dto.redisResources && app.enableRedis) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'redis');
}
if (dto.rabbitmqResources && app.enableRabbitmq) {
await this.applyOptionalServiceUpgrade(app, dto, previous, 'rabbitmq');
}
const touchesAppWorkload =
dto.cpuRequest !== undefined ||
dto.cpuLimit !== undefined ||
dto.memoryRequest !== undefined ||
dto.memoryLimit !== undefined ||
dto.replicas !== undefined;
if (touchesAppWorkload) {
await this.kubernetesService.updateResources(app, {
cpuRequest: dto.cpuRequest,
cpuLimit: dto.cpuLimit,
memoryRequest: dto.memoryRequest,
memoryLimit: dto.memoryLimit,
replicas: dto.replicas,
});
}
if (dto.appStorageSize && dto.appStorageSize !== previous.appStorageSize) {
await this.kubernetesService.resizeAppStoragePvc(app, dto.appStorageSize);
}
if (
dto.dbStorageSize &&
dto.dbStorageSize !== previous.dbStorageSize &&
previous.databaseType &&
previous.databaseType !== DatabaseType.NONE
) {
const resize = await this.kubernetesService.resizeDatabasePvc(app, dto.dbStorageSize);
if (!resize.success) {
throw new BadRequestException(resize.message);
}
}
}
private async applyOptionalServiceUpgrade(
app: Application,
dto: UpgradeResourcesDto,
previous: Application,
service: 'redis' | 'rabbitmq',
): Promise<void> {
const res = app.optionalServiceResources?.[service];
const dtoRes = service === 'redis' ? dto.redisResources : dto.rabbitmqResources;
if (res) {
await this.kubernetesService.updateResources(
app,
{
cpuRequest: res.cpuRequest,
cpuLimit: res.cpuLimit,
memoryRequest: res.memoryRequest,
memoryLimit: res.memoryLimit,
},
service,
);
}
const prevGi =
previous.optionalServiceResources?.[service]?.storageGi ?? (service === 'redis' ? 1 : 2);
const nextGi = dtoRes?.storageGi;
if (nextGi != null && nextGi > prevGi) {
const resize =
service === 'redis'
? await this.kubernetesService.resizeRedisStoragePvc(app, `${nextGi}Gi`)
: await this.kubernetesService.resizeRabbitmqStoragePvc(app, `${nextGi}Gi`);
if (!resize.success) {
throw new BadRequestException(resize.message);
}
}
}
private async getAppWithAccess(user: any, applicationId: string) {
const isAdminOrSales = user.role === UserRole.ADMIN || user.role === UserRole.SALES;
if (isAdminOrSales) {
return this.applicationsService.findOne(applicationId);
}
// Regular user - must own the app
return this.applicationsService.findOne(applicationId, user.id);
}
}
backend/src/billing/billing.module.ts
+15 -3
View File
@@ -1,7 +1,11 @@
import { Module, forwardRef } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { BillingService } from './billing.service';
import { BillingOpsService } from './billing-ops.service';
import { BillingController } from './billing.controller';
import { BillingWalletController } from './billing-wallet.controller';
import { BillingInvoicesController } from './billing-invoices.controller';
import { PublicPricingController } from './public-pricing.controller';
import { DiscountController } from './discount.controller';
import { DiscountService } from './discount.service';
import { PricingCatalogService } from './pricing-catalog.service';
@@ -9,6 +13,7 @@ import { PricingRate } from './entities/pricing-rate.entity';
import { AddonRate } from './entities/addon-rate.entity';
import { OptionalServiceProfile } from './entities/optional-service-profile.entity';
import { OptionalServiceRate } from './entities/optional-service-rate.entity';
import { PlatformSetting } from './entities/platform-setting.entity';
import { Wallet } from './entities/wallet.entity';
import { WalletTransaction } from './entities/wallet-transaction.entity';
import { ResourceCredit } from './entities/resource-credit.entity';
@@ -34,13 +39,20 @@ import { KubernetesModule } from '../kubernetes/kubernetes.module';
InvoiceLine,
Discount,
DiscountRedemption,
PlatformSetting,
]),
forwardRef(() => LifecycleModule),
forwardRef(() => ApplicationsModule),
forwardRef(() => KubernetesModule),
],
controllers: [BillingController, DiscountController],
providers: [BillingService, PricingCatalogService, DiscountService],
exports: [BillingService, PricingCatalogService, DiscountService],
controllers: [
BillingController,
BillingWalletController,
BillingInvoicesController,
PublicPricingController,
DiscountController,
],
providers: [BillingService, BillingOpsService, PricingCatalogService, DiscountService],
exports: [BillingService, BillingOpsService, PricingCatalogService, DiscountService],
})
export class BillingModule {}
backend/src/billing/billing.service.ts
+12
View File
@@ -48,6 +48,18 @@ export class BillingService {
return this.pricingCatalog.updateCatalog(dto);
}
// ─── Global (platform-wide) discount ──────────────────────────────
/** Current platform-wide discount percentage (0–100). */
async getGlobalDiscount(): Promise<{ percentOff: number }> {
return { percentOff: await this.pricingCatalog.getGlobalDiscountPercent(true) };
}
/** Set the platform-wide discount percentage (Admin). */
async setGlobalDiscount(percentOff: number): Promise<{ percentOff: number }> {
return { percentOff: await this.pricingCatalog.setGlobalDiscountPercent(percentOff) };
}
// ─── Cost Calculation ─────────────────────────────────────────────
/**
backend/src/billing/payment-gateway.util.spec.ts
+32
View File
@@ -0,0 +1,32 @@
import { ForbiddenException } from '@nestjs/common';
import { assertStubGatewayAllowed } from './payment-gateway.util';
describe('assertStubGatewayAllowed', () => {
const env = process.env;
beforeEach(() => {
process.env = { ...env };
});
afterAll(() => {
process.env = env;
});
it('allows in development', () => {
process.env.NODE_ENV = 'development';
delete process.env.PAYMENT_GATEWAY_STUB_ENABLED;
expect(() => assertStubGatewayAllowed()).not.toThrow();
});
it('blocks in production by default', () => {
process.env.NODE_ENV = 'production';
delete process.env.PAYMENT_GATEWAY_STUB_ENABLED;
expect(() => assertStubGatewayAllowed()).toThrow(ForbiddenException);
});
it('allows in production when explicitly enabled for staging', () => {
process.env.NODE_ENV = 'production';
process.env.PAYMENT_GATEWAY_STUB_ENABLED = 'true';
expect(() => assertStubGatewayAllowed()).not.toThrow();
});
});
backend/src/billing/payment-gateway.util.ts
+14
View File
@@ -0,0 +1,14 @@
import { ForbiddenException } from '@nestjs/common';
/**
* Stub gateway endpoints auto-approve payments without a real provider.
* Disabled in production unless PAYMENT_GATEWAY_STUB_ENABLED=true (staging only).
*/
export function assertStubGatewayAllowed(): void {
if (
process.env.NODE_ENV === 'production' &&
process.env.PAYMENT_GATEWAY_STUB_ENABLED !== 'true'
) {
throw new ForbiddenException('Payment gateway is not configured');
}
}
backend/src/billing/pricing-catalog.service.spec.ts
+9
View File
@@ -5,6 +5,7 @@ import { PricingRate } from './entities/pricing-rate.entity';
import { AddonRate } from './entities/addon-rate.entity';
import { OptionalServiceProfile } from './entities/optional-service-profile.entity';
import { OptionalServiceRate } from './entities/optional-service-rate.entity';
import { PlatformSetting } from './entities/platform-setting.entity';
import {
AppRuntime,
BillingCycle,
@@ -47,6 +48,13 @@ describe('PricingCatalogService', () => {
create: jest.fn().mockImplementation((x) => x),
};
const settingsRepo = {
find: jest.fn().mockResolvedValue([]),
findOne: jest.fn().mockResolvedValue(null),
save: jest.fn().mockImplementation((x) => Promise.resolve(x)),
create: jest.fn().mockImplementation((x) => x),
};
beforeEach(async () => {
jest.clearAllMocks();
const module: TestingModule = await Test.createTestingModule({
@@ -56,6 +64,7 @@ describe('PricingCatalogService', () => {
{ provide: getRepositoryToken(AddonRate), useValue: addonRepo },
{ provide: getRepositoryToken(OptionalServiceProfile), useValue: optionalProfileRepo },
{ provide: getRepositoryToken(OptionalServiceRate), useValue: optionalRateRepo },
{ provide: getRepositoryToken(PlatformSetting), useValue: settingsRepo },
],
}).compile();
backend/src/billing/pricing-catalog.service.ts
+104 -6
View File
@@ -5,6 +5,7 @@ import { PricingRate } from './entities/pricing-rate.entity';
import { AddonRate } from './entities/addon-rate.entity';
import { OptionalServiceProfile } from './entities/optional-service-profile.entity';
import { OptionalServiceRate } from './entities/optional-service-rate.entity';
import { PlatformSetting } from './entities/platform-setting.entity';
import {
AppRuntime,
BillingCycle,
@@ -39,12 +40,22 @@ import {
} from './dto/pricing-catalog.dto';
import { OptionalServiceResourcesDto } from './dto/optional-service-resources.dto';
/** PlatformSetting key holding the platform-wide discount percentage (0–100). */
export const GLOBAL_DISCOUNT_SETTING_KEY = 'global_discount_percent';
export interface CyclePrices {
hourly: number;
monthly: number;
yearly: number;
}
export interface CostTotals {
hourly: number;
monthly: number;
yearly: number;
breakdown: CostBreakdownLine[];
}
export interface PricingRateRow {
resourceType: PricingResourceType;
hourlyPrice: number;
@@ -129,10 +140,18 @@ export class PricingCatalogService implements OnModuleInit {
private readonly optionalProfileRepo: Repository<OptionalServiceProfile>,
@InjectRepository(OptionalServiceRate)
private readonly optionalRateRepo: Repository<OptionalServiceRate>,
@InjectRepository(PlatformSetting)
private readonly settingsRepo: Repository<PlatformSetting>,
) {}
/** In-memory cache of the global discount % (TTL-refreshed; single-replica safe). */
private cachedGlobalDiscountPct = 0;
private cachedGlobalDiscountAt = 0;
private static readonly GLOBAL_DISCOUNT_TTL_MS = 30_000;
async onModuleInit() {
await this.ensureDefaults();
await this.getGlobalDiscountPercent(true);
}
async ensureDefaults() {
@@ -337,13 +356,89 @@ export class PricingCatalogService implements OnModuleInit {
};
}
async computeTotalsFromDb(dto: CalculateCostDto) {
/** Raw totals straight from the catalog, before any platform-wide discount. */
async computeTotalsRawFromDb(dto: CalculateCostDto): Promise<CostTotals> {
const runtime = dto.runtime as AppRuntime;
const rates = await this.getRatesForRuntime(runtime);
const optional = await this.getOptionalBillingContext();
return this.computeTotalsWithRates(dto, rates, optional);
}
/**
* Totals with the platform-wide discount applied. This is the single choke
* point every real charge funnels through (calculateCost → invoices), so the
* discount automatically reaches previews, deploys, renewals and upgrades.
*/
async computeTotalsFromDb(dto: CalculateCostDto): Promise<CostTotals> {
const raw = await this.computeTotalsRawFromDb(dto);
const pct = await this.getGlobalDiscountPercent();
return this.applyGlobalDiscount(raw, pct);
}
/** Scale totals (and each breakdown line) by the platform-wide discount. */
applyGlobalDiscount(totals: CostTotals, percentOff: number): CostTotals {
const pct = Math.min(100, Math.max(0, percentOff || 0));
if (pct <= 0) return totals;
const factor = 1 - pct / 100;
const scale = (n: number) => Math.round(n * factor);
return {
hourly: scale(totals.hourly),
monthly: scale(totals.monthly),
yearly: scale(totals.yearly),
breakdown: totals.breakdown.map((line) => ({
...line,
hourly: scale(line.hourly),
monthly: scale(line.monthly),
yearly: scale(line.yearly),
})),
};
}
/** Platform-wide discount percentage (0–100), cached with a short TTL. */
async getGlobalDiscountPercent(force = false): Promise<number> {
const now = Date.now();
if (
!force &&
now - this.cachedGlobalDiscountAt < PricingCatalogService.GLOBAL_DISCOUNT_TTL_MS
) {
return this.cachedGlobalDiscountPct;
}
try {
const setting = await this.settingsRepo.findOne({
where: { key: GLOBAL_DISCOUNT_SETTING_KEY },
});
const parsed = setting ? parseInt(setting.value, 10) : 0;
this.cachedGlobalDiscountPct = Number.isFinite(parsed)
? Math.min(100, Math.max(0, parsed))
: 0;
this.cachedGlobalDiscountAt = now;
} catch (e: any) {
this.logger.warn(`Failed to read global discount setting: ${e?.message}`);
}
return this.cachedGlobalDiscountPct;
}
/** Persist the platform-wide discount percentage (Admin) and refresh the cache. */
async setGlobalDiscountPercent(percentOff: number): Promise<number> {
const clamped = Math.min(100, Math.max(0, Math.round(percentOff || 0)));
let setting = await this.settingsRepo.findOne({
where: { key: GLOBAL_DISCOUNT_SETTING_KEY },
});
if (!setting) {
setting = this.settingsRepo.create({
key: GLOBAL_DISCOUNT_SETTING_KEY,
value: String(clamped),
description: 'Platform-wide discount percentage applied to all pricing',
});
} else {
setting.value = String(clamped);
}
await this.settingsRepo.save(setting);
this.cachedGlobalDiscountPct = clamped;
this.cachedGlobalDiscountAt = Date.now();
return clamped;
}
computeTotalsWithRates(
dto: CalculateCostDto,
rates: PricingRate[],
@@ -739,13 +834,16 @@ export class PricingCatalogService implements OnModuleInit {
cpuQty += this.parseCpuToCores(dto.databaseResources.cpuLimit);
memoryQty += this.parseMemoryToGb(dto.databaseResources.memoryLimit);
}
const storageQty =
(dto.dbStorageSize
// App resources (CPU/RAM/storage) bill per replica — each replica is a full
// copy of the user-selected footprint. The database is a single-replica
// workload, so its storage is billed once regardless of app replicas.
const dbStorage = dto.dbStorageSize
? parseFloat(String(dto.dbStorageSize).replace(/Gi$/i, '')) || 0
: 0) +
(dto.appStorageSize
: 0;
const appStorage = dto.appStorageSize
? parseFloat(String(dto.appStorageSize).replace(/Gi$/i, '')) || 0
: 0);
: 0;
const storageQty = dbStorage + appStorage * replicas;
const map = new Map<PricingResourceType, number>();
map.set(PricingResourceType.BASE_FEE, 1);
backend/src/billing/public-pricing.controller.ts
+41
View File
@@ -0,0 +1,41 @@
import { Body, Controller, Get, Post } from '@nestjs/common';
import { ApiOperation, ApiTags } from '@nestjs/swagger';
import { PricingCatalogService } from './pricing-catalog.service';
import { CalculateCostDto } from './dto/billing.dto';
/**
* Unauthenticated pricing endpoints for the public landing page. Exposes the
* read-only pricing catalog and a cost estimator, both annotated with the
* platform-wide discount so the marketing site can show before/after prices.
*/
@ApiTags('Public Pricing')
@Controller('public/pricing')
export class PublicPricingController {
constructor(private readonly pricingCatalog: PricingCatalogService) {}
@Get('catalog')
@ApiOperation({ summary: 'Public pricing catalog + platform-wide discount' })
async getCatalog() {
const [catalog, globalDiscountPercent] = await Promise.all([
this.pricingCatalog.getCatalog(),
this.pricingCatalog.getGlobalDiscountPercent(),
]);
return { ...catalog, globalDiscountPercent };
}
@Post('calculate')
@ApiOperation({ summary: 'Estimate cost for a configuration (gross + discounted)' })
async calculate(@Body() dto: CalculateCostDto) {
const [gross, globalDiscountPercent] = await Promise.all([
this.pricingCatalog.computeTotalsRawFromDb(dto),
this.pricingCatalog.getGlobalDiscountPercent(),
]);
const net = this.pricingCatalog.applyGlobalDiscount(gross, globalDiscountPercent);
return {
gross: { hourly: gross.hourly, monthly: gross.monthly, yearly: gross.yearly },
net: { hourly: net.hourly, monthly: net.monthly, yearly: net.yearly },
breakdown: net.breakdown,
globalDiscountPercent,
};
}
}
backend/src/build/build-progress.store.ts
+58
View File
@@ -0,0 +1,58 @@
import { Injectable, OnModuleDestroy } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import Redis from 'ioredis';
import type { BuildProgress } from './build.service';
const KEY_PREFIX = 'build:progress:';
const TTL_SECONDS = 3600;
@Injectable()
export class BuildProgressStore implements OnModuleDestroy {
private readonly redis: Redis;
constructor(private readonly configService: ConfigService) {
this.redis = new Redis({
host: this.configService.get<string>('redis.host'),
port: this.configService.get<number>('redis.port'),
lazyConnect: true,
maxRetriesPerRequest: 1,
});
this.redis.connect().catch(() => {
// Redis may be unavailable in local unit tests — in-memory fallback remains in BuildService.
});
}
async get(deploymentId: string): Promise<BuildProgress | null> {
try {
const raw = await this.redis.get(`${KEY_PREFIX}${deploymentId}`);
return raw ? (JSON.parse(raw) as BuildProgress) : null;
} catch {
return null;
}
}
async set(deploymentId: string, progress: BuildProgress): Promise<void> {
try {
await this.redis.set(
`${KEY_PREFIX}${deploymentId}`,
JSON.stringify(progress),
'EX',
TTL_SECONDS,
);
} catch {
// Best-effort — local map still holds progress for this replica.
}
}
async clear(deploymentId: string): Promise<void> {
try {
await this.redis.del(`${KEY_PREFIX}${deploymentId}`);
} catch {
// ignore
}
}
onModuleDestroy(): void {
this.redis.disconnect();
}
}
backend/src/build/build.module.ts
+3 -3
View File
@@ -1,6 +1,6 @@
import { Module, forwardRef } from '@nestjs/common';
import { BuildService } from './build.service';
import { ScanService } from './scan.service';
import { BuildProgressStore } from './build-progress.store';
import { KubernetesModule } from '../kubernetes/kubernetes.module';
import { ClustersModule } from '../clusters/clusters.module';
@@ -9,7 +9,7 @@ import { ClustersModule } from '../clusters/clusters.module';
forwardRef(() => KubernetesModule),
ClustersModule,
],
providers: [BuildService, ScanService],
exports: [BuildService, ScanService],
providers: [BuildService, BuildProgressStore],
exports: [BuildService],
})
export class BuildModule {}
backend/src/build/build.service.spec.ts
+125 -244
View File
@@ -1,259 +1,140 @@
import { Test, TestingModule } from '@nestjs/testing';
import { ConfigService } from '@nestjs/config';
import * as path from 'path';
import { BuildService } from './build.service';
import { Application } from '../applications/entities/application.entity';
import { AppRuntime } from '../common/enums';
import { ClustersService } from '../clusters/clusters.service';
import { RegistryService } from '../kubernetes/registry.service';
import { BuildProgressStore } from './build-progress.store';
import { SourceStorageService } from '../storage/source-storage.service';
/**
* Tests for build service:
* • Nixpacks build preparation (BYO Dockerfile vs generated) for code runtimes
* • WordPress templated Dockerfile + helper-pod / entrypoint / zip-structure logic
*
* NOTE: like the rest of this file, the Nixpacks tests reproduce the pure logic
* locally instead of importing BuildService — the service pulls in the ESM
* `@kubernetes/client-node`, which this project's Jest config does not transform.
* Keep these copies in sync with nixpacksPrepareInitContainer in build.service.ts.
*/
describe('BuildService', () => {
let service: BuildService;
/**
* Tests for the WordPress build flow — specifically:
* 1. Helper pod PVC race condition (must wait for termination)
* 2. WordPress Dockerfile generation correctness
* 3. Entrypoint should use ENTRYPOINT not CMD to avoid double docker-entrypoint.sh execution
*/
describe('WordPress Dockerfile generation', () => {
// Reproduce the wordpressDockerfile logic from build.service.ts
function wordpressDockerfile(app: {
runtimeVersion?: string;
phpVersion?: string;
codePath?: string;
port?: number;
}): string {
const wpVersion = app.runtimeVersion || '6.7';
const phpVersion = app.phpVersion || '8.3';
const hasUploadedCode = !!app.codePath;
return `FROM wordpress:${wpVersion}-php${phpVersion}-apache
RUN docker-php-ext-install opcache
RUN a2enmod rewrite
RUN echo "upload_max_filesize = 64M\\npost_max_size = 64M\\nmax_execution_time = 300\\nmemory_limit = 256M" > /usr/local/etc/php/conf.d/uploads.ini
${hasUploadedCode ? `COPY . /tmp/user-content
RUN mkdir -p /usr/src/wordpress-user
ENTRYPOINT ["cloudhost-entrypoint.sh"]
CMD []` : `CMD ["apache2-foreground"]`}
EXPOSE 80
`;
}
it('should use ENTRYPOINT (not CMD) when user uploaded code', () => {
const df = wordpressDockerfile({ codePath: '/some/path/source.zip' });
expect(df).toContain('ENTRYPOINT ["cloudhost-entrypoint.sh"]');
expect(df).not.toContain('CMD ["cloudhost-entrypoint.sh"]');
});
it('should use CMD apache2-foreground for fresh install (no code)', () => {
const df = wordpressDockerfile({});
expect(df).toContain('CMD ["apache2-foreground"]');
expect(df).not.toContain('ENTRYPOINT');
});
it('should use correct WordPress and PHP versions', () => {
const df = wordpressDockerfile({ runtimeVersion: '6.4', phpVersion: '8.2' });
expect(df).toContain('FROM wordpress:6.4-php8.2-apache');
});
it('should default to WP 6.7 and PHP 8.3', () => {
const df = wordpressDockerfile({});
expect(df).toContain('FROM wordpress:6.7-php8.3-apache');
});
it('should COPY user content when codePath exists', () => {
const df = wordpressDockerfile({ codePath: '/tmp/source.zip' });
expect(df).toContain('COPY . /tmp/user-content');
});
it('should NOT copy user content for fresh install', () => {
const df = wordpressDockerfile({});
expect(df).not.toContain('COPY . /tmp/user-content');
});
});
describe('Helper pod PVC race condition', () => {
it('should wait for pod deletion (not just fire-and-forget)', () => {
// Simulate the fix: after deleteNamespacedPod, poll readNamespacedPod until 404
const deletionSteps = [
{ exists: true }, // pod still terminating
{ exists: true }, // still terminating
{ exists: false }, // gone (404)
];
let pollCount = 0;
let fullyTerminated = false;
for (const step of deletionSteps) {
pollCount++;
if (!step.exists) {
fullyTerminated = true;
break;
}
}
expect(fullyTerminated).toBe(true);
expect(pollCount).toBe(3);
});
it('should time out if pod never terminates', () => {
const maxPolls = 30; // e.g. 60s / 2s interval
let pollCount = 0;
let timedOut = false;
while (pollCount < maxPolls) {
pollCount++;
// Pod always exists (simulating stuck termination)
const exists = true;
if (!exists) break;
}
if (pollCount >= maxPolls) {
timedOut = true;
}
expect(timedOut).toBe(true);
});
});
describe('WordPress entrypoint script', () => {
const entrypointScript = `#!/bin/bash
set -e
# Merge user wp-content into PVC
if [ -d /usr/src/wordpress-user/wp-content ]; then
mkdir -p /var/www/html/wp-content
cp -a /usr/src/wordpress-user/wp-content/. /var/www/html/wp-content/
chown -R www-data:www-data /var/www/html/wp-content
fi
exec docker-entrypoint.sh apache2-foreground`;
it('should call docker-entrypoint.sh exactly once (via exec)', () => {
const matches = entrypointScript.match(/docker-entrypoint\.sh/g);
expect(matches).toHaveLength(1);
});
it('should use exec to replace process', () => {
expect(entrypointScript).toContain('exec docker-entrypoint.sh apache2-foreground');
});
it('should merge wp-content on every start when staged content exists', () => {
expect(entrypointScript).toContain('/usr/src/wordpress-user/wp-content');
expect(entrypointScript).not.toContain('.user-content-merged');
});
it('should not copy user wp-config.php (credentials come from env vars)', () => {
expect(entrypointScript).not.toContain('wp-config.php');
});
it('should set proper ownership after merging wp-content', () => {
expect(entrypointScript).toContain('chown -R www-data:www-data /var/www/html/wp-content');
});
});
describe('WordPress zip structure handling', () => {
// The unzip init container handles single-subfolder flattening
it('should flatten single subfolder (public_html/) to root', () => {
// Simulate: zip contains only public_html/
const extractedItems = ['public_html'];
const count = extractedItems.length;
const firstItem = extractedItems[0];
let flattenedToRoot = false;
if (count === 1 && firstItem === 'public_html') {
// cp -a /tmp/extract/public_html/. /workspace-out/source/
flattenedToRoot = true;
}
expect(flattenedToRoot).toBe(true);
});
it('should copy as-is when multiple items exist', () => {
// Simulate: zip contains multiple items at root
const extractedItems = ['wp-admin', 'wp-content', 'wp-includes', 'index.php'];
const count = extractedItems.length;
let copiedAsIs = false;
if (count !== 1) {
copiedAsIs = true;
}
expect(copiedAsIs).toBe(true);
});
});
describe('Nixpacks build preparation', () => {
// Local copies of the pure logic in build.service.ts (see NOTE at top of file).
function shellQuote(value: string): string {
return `'${value.replace(/'/g, `'\\''`)}'`;
}
function nixpacksPlanEnv(app: { runtime: AppRuntime; runtimeVersion?: string }): { name: string; value: string }[] {
const env: { name: string; value: string }[] = [];
if (app.runtime === AppRuntime.NODEJS && app.runtimeVersion) {
env.push({ name: 'NIXPACKS_NODE_VERSION', value: String(app.runtimeVersion) });
}
if ((app.runtime === AppRuntime.PYTHON || app.runtime === AppRuntime.DJANGO) && app.runtimeVersion) {
env.push({ name: 'NIXPACKS_PYTHON_VERSION', value: String(app.runtimeVersion) });
}
return env;
}
function nixpacksPrepareInitContainer(
app: { runtime: AppRuntime; runtimeVersion?: string },
config: { nixpacksImage?: string; nixpacksBuildEnv?: string[] } = {},
): any {
const image = config.nixpacksImage || 'ghcr.io/railwayapp/nixpacks:latest';
const buildEnv = config.nixpacksBuildEnv || [];
const envFlags = buildEnv.map((kv) => `--env ${shellQuote(kv)}`).join(' ');
const planEnv = nixpacksPlanEnv(app);
return {
name: 'nixpacks-prepare',
image,
env: planEnv.length ? planEnv : undefined,
command: [
'sh',
'-c',
`if [ -f source/Dockerfile ]; then cp source/Dockerfile /workspace/Dockerfile; ` +
`else nixpacks build source --out source ${envFlags} && cp source/.nixpacks/Dockerfile /workspace/Dockerfile; fi`,
],
volumeMounts: [{ name: 'workspace', mountPath: '/workspace' }],
beforeEach(async () => {
const module: TestingModule = await Test.createTestingModule({
providers: [
BuildService,
{
provide: ConfigService,
useValue: {
get: jest.fn((key: string) => {
const map: Record<string, string> = {
'build.namespace': 'cloudhost-builds',
'build.serviceAccount': 'kaniko-builder',
'registry.url': 'registry.local:5000',
};
}
return map[key];
}),
},
},
{ provide: ClustersService, useValue: {} },
{
provide: BuildProgressStore,
useValue: { get: jest.fn(), set: jest.fn(), clear: jest.fn() },
},
{ provide: RegistryService, useValue: {} },
{
provide: SourceStorageService,
useValue: {
isObjectStorage: () => false,
materializeToTempFile: jest.fn(),
getSize: jest.fn(),
},
},
],
}).compile();
it('prefers a user-provided Dockerfile (BYO), falling back to Nixpacks', () => {
const script = nixpacksPrepareInitContainer({ runtime: AppRuntime.NODEJS }).command[2] as string;
expect(script).toContain('if [ -f source/Dockerfile ]');
expect(script).toContain('cp source/Dockerfile /workspace/Dockerfile');
expect(script).toContain('nixpacks build source --out source');
expect(script).toContain('cp source/.nixpacks/Dockerfile /workspace/Dockerfile');
service = module.get(BuildService);
});
it('uses the configured Nixpacks image (default when unset)', () => {
expect(nixpacksPrepareInitContainer({ runtime: AppRuntime.GO }).image).toBe('ghcr.io/railwayapp/nixpacks:latest');
expect(
nixpacksPrepareInitContainer({ runtime: AppRuntime.GO }, { nixpacksImage: 'registry.local/nixpacks:1.2.3' }).image,
).toBe('registry.local/nixpacks:1.2.3');
describe('generateDockerfile', () => {
it('generates Go Dockerfile with requested runtime version', () => {
const app = {
runtime: AppRuntime.GO,
runtimeVersion: '1.22',
port: 8080,
} as Application;
const dockerfile = (service as any).generateDockerfile(app) as string;
expect(dockerfile).toContain('FROM golang:1.22-alpine');
expect(dockerfile).toContain('EXPOSE 8080');
});
it('bakes build-time mirror env into the build via --env flags', () => {
const script = nixpacksPrepareInitContainer(
{ runtime: AppRuntime.NODEJS },
{ nixpacksBuildEnv: ['NPM_CONFIG_REGISTRY=https://registry.npmmirror.com'] },
).command[2] as string;
expect(script).toContain(`--env 'NPM_CONFIG_REGISTRY=https://registry.npmmirror.com'`);
it('generates Go Dockerfile with cmd package when present in archive entries', () => {
const app = {
runtime: AppRuntime.GO,
runtimeVersion: '1.22',
port: 8080,
} as Application;
const dockerfile = (service as any).generateDockerfile(app, [
'go.mod',
'cmd/server/main.go',
]) as string;
expect(dockerfile).toContain('go build -a -installsuffix cgo -ldflags="-w -s" -o main ./cmd/server');
});
it('maps the selected Node version to NIXPACKS_NODE_VERSION', () => {
const c = nixpacksPrepareInitContainer({ runtime: AppRuntime.NODEJS, runtimeVersion: '20' });
expect(c.env).toContainEqual({ name: 'NIXPACKS_NODE_VERSION', value: '20' });
it('generates Node.js Dockerfile with default port', () => {
const app = {
runtime: AppRuntime.NODEJS,
runtimeVersion: '20',
} as Application;
const dockerfile = (service as any).generateDockerfile(app) as string;
expect(dockerfile).toContain('FROM node:20');
expect(dockerfile).toContain('EXPOSE 3000');
});
it('shellQuote escapes embedded single quotes safely', () => {
expect(shellQuote("a'b")).toBe("'a'\\''b'");
it('generates Laravel Dockerfile with artisan migrate', () => {
const app = {
runtime: AppRuntime.LARAVEL,
phpVersion: '8.3',
} as Application;
const dockerfile = (service as any).generateDockerfile(app) as string;
expect(dockerfile).toContain('php:8.3');
expect(dockerfile).toContain('artisan migrate');
});
it('generates WordPress Dockerfile with official image', () => {
const app = {
runtime: AppRuntime.WORDPRESS,
runtimeVersion: '6.4',
} as Application;
const dockerfile = (service as any).generateDockerfile(app) as string;
expect(dockerfile).toContain('wordpress:6.4');
});
it('generates Django Dockerfile with detected settings module', () => {
const app = {
runtime: AppRuntime.DJANGO,
runtimeVersion: '3.12',
} as Application;
const dockerfile = (service as any).generateDockerfile(app, ['myproject/settings.py']) as string;
expect(dockerfile).toContain('DJANGO_SETTINGS_MODULE=myproject.settings');
expect(dockerfile).toContain('gunicorn');
});
it('generates .NET Dockerfile that restores nested csproj', () => {
const app = {
runtime: AppRuntime.DOTNET,
runtimeVersion: '8.0',
} as Application;
const dockerfile = (service as any).generateDockerfile(app, ['src/App/App.csproj']) as string;
expect(dockerfile).toContain('CSPROJ="src/App/App.csproj"');
expect(dockerfile).toContain('dotnet publish "$CSPROJ"');
});
});
});
backend/src/build/build.service.ts
+913 -359
View File
File diff suppressed because it is too large Load Diff
backend/src/build/fixtures/.gitkeep
View File
backend/src/build/fixtures/django.zip
BIN
View File
Binary file not shown.
backend/src/build/fixtures/go-mod.zip
BIN
View File
Binary file not shown.
backend/src/build/fixtures/go-nested-root.zip
BIN
View File
Binary file not shown.
backend/src/build/fixtures/laravel.zip
BIN
View File
Binary file not shown.
backend/src/build/fixtures/mismatch-node-in-go.zip
BIN
View File
Binary file not shown.
backend/src/build/fixtures/nodejs-nested.zip
BIN
View File
Binary file not shown.
backend/src/build/fixtures/php-composer.zip
BIN
View File
Binary file not shown.
backend/src/build/fixtures/wordpress-wp-content.zip
BIN
View File
Binary file not shown.
backend/src/build/runtime-detector.spec.ts
+141
View File
@@ -0,0 +1,141 @@
import * as path from 'path';
import { BadRequestException } from '@nestjs/common';
import { AppRuntime } from '../common/enums';
import {
assertRuntimeMatch,
detectDjangoSettingsModule,
detectGoBuildTarget,
detectRuntimeFromArchive,
detectRuntimeFromEntries,
listArchiveEntries,
normalizeEntryPath,
stripCommonRootPrefix,
} from './runtime-detector';
const fixturesDir = path.join(__dirname, 'fixtures');
describe('runtime-detector', () => {
describe('normalizeEntryPath / stripCommonRootPrefix', () => {
it('strips a single root folder prefix', () => {
const entries = ['myapp/package.json', 'myapp/src/index.js'];
expect(stripCommonRootPrefix(entries)).toEqual(['package.json', 'src/index.js']);
});
it('normalizes leading ./ segments', () => {
expect(normalizeEntryPath('./package.json')).toBe('package.json');
});
});
describe('detectRuntimeFromEntries', () => {
it('detects nodejs in nested layout', () => {
const result = detectRuntimeFromEntries(['myapp/package.json']);
expect(result).toMatchObject({ runtime: AppRuntime.NODEJS, confidence: 'high' });
});
it('detects go from go.mod', () => {
const result = detectRuntimeFromEntries(['go.mod', 'main.go']);
expect(result).toMatchObject({ runtime: AppRuntime.GO, confidence: 'high' });
});
it('detects laravel from artisan + composer.json', () => {
const result = detectRuntimeFromEntries(['artisan', 'composer.json']);
expect(result).toMatchObject({ runtime: AppRuntime.LARAVEL, confidence: 'high' });
});
it('detects php from composer.json without artisan', () => {
const result = detectRuntimeFromEntries(['composer.json', 'index.php']);
expect(result).toMatchObject({ runtime: AppRuntime.PHP, confidence: 'high' });
});
it('detects django from manage.py', () => {
const result = detectRuntimeFromEntries(['manage.py', 'requirements.txt']);
expect(result).toMatchObject({ runtime: AppRuntime.DJANGO, confidence: 'high' });
});
it('detects python from requirements.txt', () => {
const result = detectRuntimeFromEntries(['requirements.txt', 'app.py']);
expect(result).toMatchObject({ runtime: AppRuntime.PYTHON, confidence: 'high' });
});
it('detects dotnet from shallow csproj', () => {
const result = detectRuntimeFromEntries(['src/App/App.csproj']);
expect(result).toMatchObject({ runtime: AppRuntime.DOTNET, confidence: 'high' });
});
it('detects wordpress wp-content migrate layout', () => {
const result = detectRuntimeFromEntries(['themes/twenty/style.css', 'plugins/hello/hello.php']);
expect(result).toMatchObject({ runtime: AppRuntime.WORDPRESS, confidence: 'high' });
});
it('returns low confidence when package.json and composer.json coexist', () => {
const result = detectRuntimeFromEntries(['package.json', 'composer.json']);
expect(result).toMatchObject({ runtime: null, confidence: 'low' });
});
});
describe('assertRuntimeMatch', () => {
it('passes when configured runtime matches detection', () => {
expect(() =>
assertRuntimeMatch(AppRuntime.GO, {
runtime: AppRuntime.GO,
confidence: 'high',
signals: ['go.mod'],
}),
).not.toThrow();
});
it('throws BadRequestException on high-confidence mismatch', () => {
expect(() =>
assertRuntimeMatch(AppRuntime.NODEJS, {
runtime: AppRuntime.GO,
confidence: 'high',
signals: ['go.mod'],
}),
).toThrow(BadRequestException);
});
it('allows upload when confidence is low', () => {
expect(() =>
assertRuntimeMatch(AppRuntime.NODEJS, {
runtime: null,
confidence: 'low',
signals: [],
}),
).not.toThrow();
});
});
describe('listArchiveEntries + detectRuntimeFromArchive', () => {
it.each([
['nodejs-nested.zip', AppRuntime.NODEJS],
['go-mod.zip', AppRuntime.GO],
['laravel.zip', AppRuntime.LARAVEL],
['php-composer.zip', AppRuntime.PHP],
['django.zip', AppRuntime.DJANGO],
['wordpress-wp-content.zip', AppRuntime.WORDPRESS],
] as const)('reads %s as %s', async (fixture, runtime) => {
const zipPath = path.join(fixturesDir, fixture);
const entries = await listArchiveEntries(zipPath);
expect(entries.length).toBeGreaterThan(0);
const detected = await detectRuntimeFromArchive(zipPath);
expect(detected.runtime).toBe(runtime);
expect(detected.confidence).toBe('high');
});
it('detects go inside a single nested root folder', async () => {
const zipPath = path.join(fixturesDir, 'go-nested-root.zip');
const detected = await detectRuntimeFromArchive(zipPath);
expect(detected.runtime).toBe(AppRuntime.GO);
});
});
describe('dockerfile helpers', () => {
it('prefers cmd/*/main.go for go build target', () => {
expect(detectGoBuildTarget(['go.mod', 'cmd/server/main.go'])).toBe('./cmd/server');
});
it('infers django settings module from project layout', () => {
expect(detectDjangoSettingsModule(['myproject/settings.py'])).toBe('myproject.settings');
});
});
});
backend/src/build/runtime-detector.ts
+288
View File
@@ -0,0 +1,288 @@
import { BadRequestException } from '@nestjs/common';
import { execFile } from 'child_process';
import * as fs from 'fs';
import * as path from 'path';
import { promisify } from 'util';
import * as yauzl from 'yauzl';
import { AppRuntime } from '../common/enums';
const execFileAsync = promisify(execFile);
export interface RuntimeDetectionResult {
runtime: AppRuntime | null;
confidence: 'high' | 'low';
signals: string[];
}
export function normalizeEntryPath(entry: string): string {
return entry.replace(/\\/g, '/').replace(/^\.\//, '').replace(/\/+$/, '');
}
/** Strip a single top-level folder when every entry lives under it. */
export function stripCommonRootPrefix(entries: string[]): string[] {
const normalized = entries.map(normalizeEntryPath).filter(Boolean);
if (normalized.length === 0) return [];
const firstSegments = new Set<string>();
for (const entry of normalized) {
const seg = entry.split('/')[0];
if (seg) firstSegments.add(seg);
}
if (firstSegments.size !== 1) return normalized;
const root = [...firstSegments][0];
const allUnderRoot = normalized.every((entry) => entry === root || entry.startsWith(`${root}/`));
if (!allUnderRoot) return normalized;
const nestedUnderRoot = normalized.filter((entry) => entry.startsWith(`${root}/`));
if (nestedUnderRoot.length === 0) return normalized;
if (normalized.length === 1) {
const parts = normalized[0].split('/').filter(Boolean);
if (parts.length === 2) {
return [parts[1]];
}
return normalized;
}
return normalized.map((entry) => {
if (entry === root) return entry;
return entry.slice(root.length + 1);
});
}
function hasBasename(entries: string[], name: string): boolean {
return entries.some((entry) => path.posix.basename(entry) === name);
}
function hasDirectory(entries: string[], dirName: string): boolean {
return entries.some((entry) => {
const parts = entry.split('/').filter(Boolean);
return parts.includes(dirName);
});
}
function hasShallowCsproj(entries: string[]): string | null {
for (const entry of entries) {
if (!entry.endsWith('.csproj')) continue;
const depth = entry.split('/').filter(Boolean).length;
if (depth <= 3) return entry;
}
return null;
}
export function detectRuntimeFromEntries(rawEntries: string[]): RuntimeDetectionResult {
const entries = stripCommonRootPrefix(rawEntries);
const signals: string[] = [];
const note = (signal: string) => {
if (!signals.includes(signal)) signals.push(signal);
};
const hasWpAdmin = hasDirectory(entries, 'wp-admin');
const hasWpContent = hasDirectory(entries, 'wp-content');
const hasWpConfig =
hasBasename(entries, 'wp-config.php') ||
hasBasename(entries, 'wp-config-sample.php') ||
entries.some((e) => /wp-config[^/]*\.php$/i.test(e));
const hasThemes = hasDirectory(entries, 'themes');
const hasPlugins = hasDirectory(entries, 'plugins');
if (hasWpAdmin || (hasWpContent && hasWpConfig) || (hasThemes && hasPlugins && !hasWpAdmin)) {
if (hasWpAdmin) note('wp-admin');
if (hasWpContent) note('wp-content');
if (hasWpConfig) note('wp-config.php');
if (hasThemes) note('themes');
if (hasPlugins) note('plugins');
return { runtime: AppRuntime.WORDPRESS, confidence: 'high', signals };
}
const hasArtisan = hasBasename(entries, 'artisan');
const hasComposer = hasBasename(entries, 'composer.json');
if (hasArtisan && hasComposer) {
note('artisan');
note('composer.json');
return { runtime: AppRuntime.LARAVEL, confidence: 'high', signals };
}
const hasManagePy = hasBasename(entries, 'manage.py');
if (hasManagePy) {
note('manage.py');
if (hasBasename(entries, 'requirements.txt')) note('requirements.txt');
return { runtime: AppRuntime.DJANGO, confidence: 'high', signals };
}
if (hasBasename(entries, 'go.mod')) {
note('go.mod');
return { runtime: AppRuntime.GO, confidence: 'high', signals };
}
const csproj = hasShallowCsproj(entries);
if (csproj) {
note(csproj);
return { runtime: AppRuntime.DOTNET, confidence: 'high', signals };
}
const hasPackageJson = hasBasename(entries, 'package.json');
if (hasPackageJson && hasComposer) {
note('package.json');
note('composer.json');
return { runtime: null, confidence: 'low', signals };
}
if (hasComposer) {
note('composer.json');
return { runtime: AppRuntime.PHP, confidence: 'high', signals };
}
const hasRequirements = hasBasename(entries, 'requirements.txt');
const hasPyproject = hasBasename(entries, 'pyproject.toml');
if (hasRequirements || hasPyproject) {
if (hasRequirements) note('requirements.txt');
if (hasPyproject) note('pyproject.toml');
return { runtime: AppRuntime.PYTHON, confidence: 'high', signals };
}
if (hasPackageJson) {
note('package.json');
return { runtime: AppRuntime.NODEJS, confidence: 'high', signals };
}
return { runtime: null, confidence: 'low', signals };
}
export function assertRuntimeMatch(configured: AppRuntime, detected: RuntimeDetectionResult): void {
if (detected.confidence !== 'high' || detected.runtime === null) {
return;
}
if (detected.runtime === configured) {
return;
}
throw new BadRequestException({
message: `Selected runtime "${configured}" does not match the uploaded source (detected "${detected.runtime}").`,
configured,
detected: detected.runtime,
signals: detected.signals,
});
}
function listZipEntries(archivePath: string): Promise<string[]> {
return new Promise((resolve, reject) => {
yauzl.open(archivePath, { lazyEntries: true }, (err, zipfile) => {
if (err || !zipfile) {
reject(err ?? new Error(`Failed to open zip: ${archivePath}`));
return;
}
const entries: string[] = [];
zipfile.readEntry();
zipfile.on('entry', (entry) => {
entries.push(entry.fileName);
zipfile.readEntry();
});
zipfile.on('end', () => resolve(entries));
zipfile.on('error', reject);
});
});
}
async function listTarEntries(archivePath: string): Promise<string[]> {
const { stdout } = await execFileAsync('tar', ['-tf', archivePath]);
return stdout.split('\n').filter(Boolean);
}
/** List relative paths up to depth 2 inside a directory (for extracted sources). */
export function listDirectoryEntriesShallow(dirPath: string, maxDepth = 2): string[] {
const results: string[] = [];
const walk = (current: string, depth: number) => {
if (depth > maxDepth) return;
let names: string[];
try {
names = fs.readdirSync(current);
} catch {
return;
}
for (const name of names) {
const full = path.join(current, name);
const rel = path.relative(dirPath, full).replace(/\\/g, '/');
results.push(rel);
let stat: fs.Stats;
try {
stat = fs.statSync(full);
} catch {
continue;
}
if (stat.isDirectory()) {
walk(full, depth + 1);
}
}
};
walk(dirPath, 0);
return results;
}
export async function listArchiveEntries(archivePath: string): Promise<string[]> {
const resolved = path.resolve(archivePath);
if (!fs.existsSync(resolved)) {
return [];
}
const stat = fs.statSync(resolved);
if (stat.isDirectory()) {
return listDirectoryEntriesShallow(resolved);
}
const lower = resolved.toLowerCase();
if (lower.endsWith('.zip')) {
return listZipEntries(resolved);
}
if (lower.endsWith('.tar.gz') || lower.endsWith('.tgz') || lower.endsWith('.tar')) {
return listTarEntries(resolved);
}
return [];
}
export async function detectRuntimeFromArchive(archivePath: string): Promise<RuntimeDetectionResult> {
const entries = await listArchiveEntries(archivePath);
return detectRuntimeFromEntries(entries);
}
export async function validateRuntimeFromArchive(
configured: AppRuntime,
archivePath: string | null | undefined,
): Promise<RuntimeDetectionResult> {
if (!archivePath) {
return { runtime: null, confidence: 'low', signals: [] };
}
const detected = await detectRuntimeFromArchive(archivePath);
assertRuntimeMatch(configured, detected);
return detected;
}
/** Prefer `cmd/<name>/main.go` when present. */
export function detectGoBuildTarget(entries: string[]): string {
const normalized = stripCommonRootPrefix(entries);
const cmdMain = normalized.find((entry) => /^cmd\/[^/]+\/main\.go$/.test(entry));
if (cmdMain) {
return `./${cmdMain.replace(/\/main\.go$/, '')}`;
}
return '.';
}
export function detectShallowCsproj(entries: string[]): string | null {
return hasShallowCsproj(stripCommonRootPrefix(entries));
}
/** Infer DJANGO_SETTINGS_MODULE from settings.py layout. */
export function detectDjangoSettingsModule(entries: string[]): string {
for (const entry of entries.map(normalizeEntryPath).filter(Boolean)) {
const parts = entry.split('/').filter(Boolean);
const settingsIdx = parts.findIndex((part) => part === 'settings.py');
if (settingsIdx === 1) return `${parts[0]}.settings`;
if (settingsIdx === 2) return `${parts[0]}.${parts[1]}.settings`;
}
return 'config.settings';
}
backend/src/build/scan.service.spec.ts
-94
View File
@@ -1,94 +0,0 @@
/**
* Tests for the pure Trivy-report aggregation logic in ScanService.summarize.
*
* NOTE: like the other build specs, this reproduces the pure logic locally rather
* than importing ScanService — the service pulls in the ESM `@kubernetes/client-node`,
* which this project's Jest config does not transform. Keep in sync with scan.service.ts.
*/
interface VulnerabilitySummary {
critical: number;
high: number;
medium: number;
low: number;
unknown: number;
total: number;
scannedAt: string;
}
function parseTrivyJson(output: string): any | null {
if (!output) return null;
try {
return JSON.parse(output);
} catch {
const start = output.indexOf('{');
const end = output.lastIndexOf('}');
if (start >= 0 && end > start) {
try {
return JSON.parse(output.slice(start, end + 1));
} catch {
return null;
}
}
return null;
}
}
function summarize(trivyOutput: string): VulnerabilitySummary {
const counts = { critical: 0, high: 0, medium: 0, low: 0, unknown: 0 };
const parsed = parseTrivyJson(trivyOutput);
const results: any[] = Array.isArray(parsed?.Results) ? parsed.Results : [];
for (const result of results) {
const vulns: any[] = Array.isArray(result?.Vulnerabilities) ? result.Vulnerabilities : [];
for (const v of vulns) {
const sev = String(v?.Severity || 'UNKNOWN').toUpperCase();
if (sev === 'CRITICAL') counts.critical++;
else if (sev === 'HIGH') counts.high++;
else if (sev === 'MEDIUM') counts.medium++;
else if (sev === 'LOW') counts.low++;
else counts.unknown++;
}
}
return {
...counts,
total: counts.critical + counts.high + counts.medium + counts.low + counts.unknown,
scannedAt: new Date().toISOString(),
};
}
describe('ScanService.summarize', () => {
it('counts vulnerabilities per severity across results', () => {
const report = JSON.stringify({
Results: [
{ Vulnerabilities: [{ Severity: 'CRITICAL' }, { Severity: 'HIGH' }, { Severity: 'high' }] },
{ Vulnerabilities: [{ Severity: 'MEDIUM' }, { Severity: 'LOW' }, { Severity: 'WeIrD' }] },
{ Vulnerabilities: null },
{},
],
});
const s = summarize(report);
expect(s).toMatchObject({ critical: 1, high: 2, medium: 1, low: 1, unknown: 1, total: 6 });
expect(typeof s.scannedAt).toBe('string');
});
it('returns all-zero summary for a clean image', () => {
expect(summarize(JSON.stringify({ Results: [{ Target: 'x' }] }))).toMatchObject({
critical: 0,
high: 0,
medium: 0,
low: 0,
unknown: 0,
total: 0,
});
});
it('tolerates leading log noise before the JSON', () => {
const noisy = `2026-06-20 INFO Need to update DB\n{"Results":[{"Vulnerabilities":[{"Severity":"CRITICAL"}]}]}`;
expect(summarize(noisy).critical).toBe(1);
});
it('returns a zero summary on unparseable output', () => {
expect(summarize('not json at all').total).toBe(0);
expect(summarize('').total).toBe(0);
});
});
backend/src/build/scan.service.ts
-169
View File
@@ -1,169 +0,0 @@
import { Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import * as k8s from '@kubernetes/client-node';
import { Application } from '../applications/entities/application.entity';
import { ClustersService } from '../clusters/clusters.service';
import { RegistryService } from '../kubernetes/registry.service';
export interface VulnerabilitySummary {
critical: number;
high: number;
medium: number;
low: number;
unknown: number;
total: number;
scannedAt: string;
}
/**
* Report-only image vulnerability scanning with Trivy. Runs a one-shot K8s Job
* that scans the freshly-pushed image in the in-cluster registry and stores a
* severity summary on the deployment. Never blocks a deployment — any failure is
* logged and ignored.
*/
@Injectable()
export class ScanService {
private readonly logger = new Logger(ScanService.name);
constructor(
private readonly configService: ConfigService,
private readonly clustersService: ClustersService,
private readonly registryService: RegistryService,
) {}
/** Aggregate a Trivy JSON report (raw stdout) into per-severity counts. Pure & testable. */
summarize(trivyOutput: string): VulnerabilitySummary {
const counts = { critical: 0, high: 0, medium: 0, low: 0, unknown: 0 };
const parsed = this.parseTrivyJson(trivyOutput);
const results: any[] = Array.isArray(parsed?.Results) ? parsed.Results : [];
for (const result of results) {
const vulns: any[] = Array.isArray(result?.Vulnerabilities) ? result.Vulnerabilities : [];
for (const v of vulns) {
const sev = String(v?.Severity || 'UNKNOWN').toUpperCase();
if (sev === 'CRITICAL') counts.critical++;
else if (sev === 'HIGH') counts.high++;
else if (sev === 'MEDIUM') counts.medium++;
else if (sev === 'LOW') counts.low++;
else counts.unknown++;
}
}
return {
...counts,
total: counts.critical + counts.high + counts.medium + counts.low + counts.unknown,
scannedAt: new Date().toISOString(),
};
}
/** Trivy prints clean JSON to stdout, but tolerate any leading noise from the log stream. */
private parseTrivyJson(output: string): any | null {
if (!output) return null;
try {
return JSON.parse(output);
} catch {
const start = output.indexOf('{');
const end = output.lastIndexOf('}');
if (start >= 0 && end > start) {
try {
return JSON.parse(output.slice(start, end + 1));
} catch {
return null;
}
}
return null;
}
}
/**
* Scan a pushed image and return a severity summary, or null on any failure.
* Report-only: callers must treat null as "no data", never as a deploy gate.
*/
async scanImage(app: Application, imageUri: string): Promise<VulnerabilitySummary | null> {
if (this.configService.get<boolean>('build.scanEnabled') === false) return null;
const buildNs = this.registryService.getBuildNamespace();
const image = this.configService.get<string>('build.trivyImage') || 'aquasec/trivy:latest';
const dbRepo = this.configService.get<string>('build.trivyDbRepository') || '';
const timeoutSeconds = this.configService.get<number>('build.scanTimeoutSeconds') || 300;
const { username, password } = this.registryService.getRegistryCredentials();
const jobName = `scan-${app.name}-${Date.now()}`.substring(0, 63).replace(/[^a-z0-9-]/g, '');
try {
const cluster = app.clusterId ? await this.clustersService.findOne(app.clusterId) : await this.clustersService.getDefault();
const kc = new k8s.KubeConfig();
kc.loadFromString(cluster.kubeconfig);
const coreApi = kc.makeApiClient(k8s.CoreV1Api);
const batchApi = kc.makeApiClient(k8s.BatchV1Api);
const env: { name: string; value: string }[] = [
{ name: 'TRIVY_INSECURE', value: 'true' }, // in-cluster registry is plain HTTP
{ name: 'TRIVY_NON_SSL', value: 'true' },
];
if (username) env.push({ name: 'TRIVY_USERNAME', value: username });
if (password) env.push({ name: 'TRIVY_PASSWORD', value: password });
if (dbRepo) env.push({ name: 'TRIVY_DB_REPOSITORY', value: dbRepo });
const job: k8s.V1Job = {
apiVersion: 'batch/v1',
kind: 'Job',
metadata: { name: jobName, namespace: buildNs },
spec: {
backoffLimit: 0,
ttlSecondsAfterFinished: 120,
template: {
spec: {
restartPolicy: 'Never',
containers: [
{
name: 'trivy',
image,
imagePullPolicy: 'IfNotPresent',
env,
args: ['image', '--quiet', '--no-progress', '--format', 'json', '--severity', 'CRITICAL,HIGH,MEDIUM,LOW', imageUri],
resources: {
requests: { cpu: '250m', memory: '512Mi' },
limits: { cpu: '1', memory: '1Gi' },
},
},
],
},
},
},
};
await batchApi.createNamespacedJob({ namespace: buildNs, body: job });
await this.waitForJob(batchApi, jobName, buildNs, timeoutSeconds);
const output = await this.getJobPodLogs(coreApi, jobName, buildNs);
const summary = this.summarize(output);
await batchApi
.deleteNamespacedJob({ name: jobName, namespace: buildNs, gracePeriodSeconds: 0, propagationPolicy: 'Foreground' })
.catch(() => undefined);
this.logger.log(`Scan complete for ${imageUri}: ${summary.critical}C/${summary.high}H/${summary.medium}M/${summary.low}L`);
return summary;
} catch (e: any) {
this.logger.warn(`Image scan failed for ${imageUri} (report-only, ignored): ${e.message}`);
return null;
}
}
private async waitForJob(batchApi: k8s.BatchV1Api, jobName: string, namespace: string, timeoutSeconds: number): Promise<void> {
const deadline = Date.now() + timeoutSeconds * 1000;
while (Date.now() < deadline) {
const job = await batchApi.readNamespacedJob({ name: jobName, namespace });
if (job.status?.succeeded) return;
if ((job.status?.failed ?? 0) > 0) return; // Trivy exits non-zero on findings with some flags; read logs anyway
await new Promise((r) => setTimeout(r, 4000));
}
throw new Error(`Scan job ${jobName} timed out after ${timeoutSeconds}s`);
}
private async getJobPodLogs(coreApi: k8s.CoreV1Api, jobName: string, namespace: string): Promise<string> {
const pods = await coreApi.listNamespacedPod({ namespace, labelSelector: `job-name=${jobName}` });
const podName = pods.items[0]?.metadata?.name;
if (!podName) throw new Error(`No pod found for scan job ${jobName}`);
return coreApi.readNamespacedPodLog({ name: podName, namespace, container: 'trivy' });
}
}
backend/src/clusters/clusters.service.spec.ts
+85 -85
View File
@@ -1,105 +1,105 @@
import { Test, TestingModule } from '@nestjs/testing';
import { getRepositoryToken } from '@nestjs/typeorm';
import { NotFoundException } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { ClustersService } from './clusters.service';
import { Cluster } from './entities/cluster.entity';
import { ClusterHealth } from './entities/cluster-health.entity';
import { ClusterPool } from './entities/cluster-pool.entity';
import { ClusterAllocationLog } from './entities/cluster-allocation-log.entity';
import { ClusterStatus } from '../common/enums';
import { RegistryService } from '../kubernetes/registry.service';
import { DataSource } from 'typeorm';
/**
* Tests for ClustersService — getDefault and delete logic.
*/
describe('ClustersService', () => {
let service: ClustersService;
describe('ClustersService getDefault logic', () => {
// Simulate the fixed getDefault behavior
function getDefault(clusters: { id: string; isDefault: boolean; status: string }[]): { id: string } | null {
// Step 1: active + default
let result = clusters.find(c => c.isDefault && c.status === ClusterStatus.ACTIVE);
if (result) return { id: result.id };
const clustersRepository = {
findOne: jest.fn(),
save: jest.fn().mockImplementation((x) => Promise.resolve(x)),
find: jest.fn(),
create: jest.fn(),
delete: jest.fn(),
count: jest.fn(),
};
// Step 2: any active (fallback)
result = clusters.find(c => c.status === ClusterStatus.ACTIVE);
if (result) return { id: result.id };
const healthRepository = { find: jest.fn(), save: jest.fn() };
const poolRepository = { find: jest.fn(), findOne: jest.fn(), save: jest.fn() };
const allocationLogsRepository = { save: jest.fn(), find: jest.fn() };
const dataSource = { transaction: jest.fn() };
const registryService = { ensureRegistryPullSecret: jest.fn() };
return null;
}
beforeEach(async () => {
jest.clearAllMocks();
it('should return active default cluster', () => {
const clusters = [
{ id: '1', isDefault: true, status: ClusterStatus.ACTIVE },
{ id: '2', isDefault: false, status: ClusterStatus.ACTIVE },
];
expect(getDefault(clusters)?.id).toBe('1');
const module: TestingModule = await Test.createTestingModule({
providers: [
ClustersService,
{ provide: getRepositoryToken(Cluster), useValue: clustersRepository },
{ provide: getRepositoryToken(ClusterPool), useValue: poolRepository },
{ provide: getRepositoryToken(ClusterHealth), useValue: healthRepository },
{ provide: getRepositoryToken(ClusterAllocationLog), useValue: allocationLogsRepository },
{ provide: DataSource, useValue: dataSource },
{ provide: RegistryService, useValue: registryService },
{
provide: ConfigService,
useValue: {
get: jest.fn((key: string) => {
if (key === 'CLUSTER_KUBECONFIG_KEY') return '';
if (key === 'cluster.kubeconfigKey') return '';
return undefined;
}),
},
},
],
}).compile();
service = module.get(ClustersService);
});
it('should skip inactive default and return active cluster', () => {
const clusters = [
{ id: '1', isDefault: true, status: ClusterStatus.INACTIVE },
{ id: '2', isDefault: false, status: ClusterStatus.ACTIVE },
];
expect(getDefault(clusters)?.id).toBe('2');
});
describe('getDefault', () => {
it('returns active default cluster', async () => {
const cluster = {
id: 'c-1',
name: 'primary',
isDefault: true,
status: ClusterStatus.ACTIVE,
kubeconfig: 'apiVersion: v1',
} as Cluster;
it('should return null when no active clusters exist', () => {
const clusters = [
{ id: '1', isDefault: true, status: ClusterStatus.INACTIVE },
];
expect(getDefault(clusters)).toBeNull();
});
clustersRepository.findOne.mockResolvedValueOnce(cluster);
it('should handle both clusters being default (picks active one)', () => {
const clusters = [
{ id: 'inactive', isDefault: true, status: ClusterStatus.INACTIVE },
{ id: 'active', isDefault: true, status: ClusterStatus.ACTIVE },
];
expect(getDefault(clusters)?.id).toBe('active');
const result = await service.getDefault();
expect(result.id).toBe('c-1');
expect(clustersRepository.findOne).toHaveBeenCalledWith({
where: { isDefault: true, status: ClusterStatus.ACTIVE },
});
});
describe('ClustersService delete logic', () => {
it('should reassign apps to replacement cluster on delete', () => {
// Simulate: cluster A (being deleted) has 3 apps, cluster B is active
const apps = [
{ id: 'app1', clusterId: 'A' },
{ id: 'app2', clusterId: 'A' },
{ id: 'app3', clusterId: 'B' },
];
const deletedClusterId = 'A';
const replacementId = 'B';
it('falls back to any active cluster when no default is set', async () => {
const fallback = {
id: 'c-2',
name: 'fallback',
isDefault: false,
status: ClusterStatus.ACTIVE,
kubeconfig: 'apiVersion: v1',
} as Cluster;
// Reassign
for (const app of apps) {
if (app.clusterId === deletedClusterId) {
app.clusterId = replacementId;
}
}
clustersRepository.findOne
.mockResolvedValueOnce(null)
.mockResolvedValueOnce(fallback);
expect(apps.filter(a => a.clusterId === 'A')).toHaveLength(0);
expect(apps.filter(a => a.clusterId === 'B')).toHaveLength(3);
const result = await service.getDefault();
expect(result.id).toBe('c-2');
expect(clustersRepository.save).toHaveBeenCalled();
});
it('should promote another cluster to default when default is deleted', () => {
const clusters = [
{ id: 'A', isDefault: true, status: ClusterStatus.ACTIVE },
{ id: 'B', isDefault: false, status: ClusterStatus.ACTIVE },
];
it('throws when no active cluster exists', async () => {
clustersRepository.findOne.mockResolvedValue(null);
// Delete A
const deleted = clusters.splice(0, 1)[0];
expect(deleted.isDefault).toBe(true);
// Promote
const newDefault = clusters.find(c => c.status === ClusterStatus.ACTIVE);
if (newDefault) newDefault.isDefault = true;
expect(clusters[0].isDefault).toBe(true);
expect(clusters[0].id).toBe('B');
});
it('should nullify clusterId when no replacement cluster exists', () => {
const apps = [{ id: 'app1', clusterId: 'A' as string | null }];
const hasReplacement = false;
if (!hasReplacement) {
for (const app of apps) {
app.clusterId = null;
}
}
expect(apps[0].clusterId).toBeNull();
await expect(service.getDefault()).rejects.toThrow(NotFoundException);
});
});
});
backend/src/clusters/clusters.service.ts
-124
View File
@@ -1102,133 +1102,9 @@ export class ClustersService implements OnModuleInit, OnModuleDestroy {
await this.ensureK3sRegistryMirrors(appsApi, registryUrl);
// ── 8. MinIO (S3-compatible) for application source archives ──
await this.ensureMinioInfrastructure(coreApi, appsApi, buildNs);
this.logger.log(`✅ Cluster bootstrap complete — registry: ${registryUrl}`);
}
/**
* Provision in-cluster MinIO (Deployment + PVC + Service + credentials Secret)
* in the build namespace. Application source archives are uploaded here by the
* API and pulled by build pods via presigned URLs.
*/
private async ensureMinioInfrastructure(coreApi: k8s.CoreV1Api, appsApi: k8s.AppsV1Api, buildNs: string): Promise<void> {
const accessKey = this.configService.get<string>('minio.accessKey') || 'cloudhost';
const secretKey = this.configService.get<string>('minio.secretKey') || '';
const pvcName = 'minio-data';
const deployName = 'minio';
const svcName = 'minio';
const secretName = 'minio-credentials';
// 1. Credentials Secret (shared by the MinIO server env and the API client)
const minioSecret: k8s.V1Secret = {
metadata: { name: secretName, namespace: buildNs },
type: 'Opaque',
data: {
accesskey: Buffer.from(accessKey).toString('base64'),
secretkey: Buffer.from(secretKey).toString('base64'),
},
};
try {
await coreApi.readNamespacedSecret({ name: secretName, namespace: buildNs });
await coreApi.replaceNamespacedSecret({ name: secretName, namespace: buildNs, body: minioSecret });
} catch (err: any) {
if (err.code === 404 || err.body?.code === 404) {
await coreApi.createNamespacedSecret({ namespace: buildNs, body: minioSecret });
this.logger.log(`Created ${secretName} Secret`);
} else {
throw err;
}
}
// 2. Data PVC
try {
await coreApi.readNamespacedPersistentVolumeClaim({ name: pvcName, namespace: buildNs });
} catch (err: any) {
if (err.code === 404 || err.body?.code === 404) {
await coreApi.createNamespacedPersistentVolumeClaim({
namespace: buildNs,
body: {
metadata: { name: pvcName, namespace: buildNs },
spec: { accessModes: ['ReadWriteOnce'], resources: { requests: { storage: '20Gi' } } },
},
});
this.logger.log(`Created PVC "${pvcName}" (20Gi)`);
} else {
throw err;
}
}
// 3. Deployment
try {
await appsApi.readNamespacedDeployment({ name: deployName, namespace: buildNs });
} catch (err: any) {
if (err.code === 404 || err.body?.code === 404) {
await appsApi.createNamespacedDeployment({
namespace: buildNs,
body: {
metadata: { name: deployName, namespace: buildNs, labels: { app: 'minio' } },
spec: {
replicas: 1,
selector: { matchLabels: { app: 'minio' } },
template: {
metadata: { labels: { app: 'minio' } },
spec: {
containers: [
{
name: 'minio',
image: process.env.MINIO_IMAGE || 'minio/minio:latest',
args: ['server', '/data', '--console-address', ':9001'],
env: [
{ name: 'MINIO_ROOT_USER', valueFrom: { secretKeyRef: { name: secretName, key: 'accesskey' } } },
{ name: 'MINIO_ROOT_PASSWORD', valueFrom: { secretKeyRef: { name: secretName, key: 'secretkey' } } },
],
ports: [{ containerPort: 9000 }, { containerPort: 9001 }],
volumeMounts: [{ name: 'data', mountPath: '/data' }],
resources: {
requests: { cpu: '100m', memory: '256Mi' },
limits: { cpu: '1', memory: '1Gi' },
},
},
],
volumes: [{ name: 'data', persistentVolumeClaim: { claimName: pvcName } }],
},
},
},
},
});
this.logger.log(`Created MinIO Deployment`);
} else {
throw err;
}
}
// 4. ClusterIP Service (API :9000, console :9001)
try {
await coreApi.readNamespacedService({ name: svcName, namespace: buildNs });
} catch (err: any) {
if (err.code === 404 || err.body?.code === 404) {
await coreApi.createNamespacedService({
namespace: buildNs,
body: {
metadata: { name: svcName, namespace: buildNs, labels: { app: 'minio' } },
spec: {
selector: { app: 'minio' },
ports: [
{ name: 'api', port: 9000, targetPort: 9000 as any, protocol: 'TCP' },
{ name: 'console', port: 9001, targetPort: 9001 as any, protocol: 'TCP' },
],
},
},
});
this.logger.log(`Created MinIO Service`);
} else {
throw err;
}
}
}
/** In-cluster registry mirror for k3s/containerd (HTTP). Removes legacy external-registry DaemonSet if present. */
private async ensureK3sRegistryMirrors(appsApi: k8s.AppsV1Api, registryUrl: string): Promise<void> {
const namespace = 'kube-system';
backend/src/common/redis/redis.module.ts
-51
View File
@@ -1,51 +0,0 @@
import { Global, Module, OnApplicationShutdown, Logger } from '@nestjs/common';
import { ModuleRef } from '@nestjs/core';
import { ConfigService } from '@nestjs/config';
import IORedis, { Redis } from 'ioredis';
/** Injection token for the shared ioredis client. */
export const REDIS_CLIENT = 'REDIS_CLIENT';
/**
* Shared Redis client used for build state (progress / cancel flags / build
* sessions) so the data survives across backend replicas — unlike the previous
* in-memory Maps that only worked with a single instance.
*
* Reuses the same connection details as the Bull queue (`redis.host/port`).
*/
@Global()
@Module({
providers: [
{
provide: REDIS_CLIENT,
inject: [ConfigService],
useFactory: (configService: ConfigService): Redis => {
const client = new IORedis({
host: configService.get<string>('redis.host'),
port: configService.get<number>('redis.port'),
// Build state writes are best-effort telemetry — never let a Redis
// hiccup take down the request that triggered them.
maxRetriesPerRequest: 2,
enableOfflineQueue: true,
});
client.on('error', (err) => {
new Logger('RedisClient').warn(`Redis connection error: ${err.message}`);
});
return client;
},
},
],
exports: [REDIS_CLIENT],
})
export class RedisModule implements OnApplicationShutdown {
constructor(private readonly moduleRef: ModuleRef) {}
async onApplicationShutdown(): Promise<void> {
try {
const client = this.moduleRef.get<Redis>(REDIS_CLIENT, { strict: false });
await client?.quit();
} catch {
/* ignore shutdown errors */
}
}
}
backend/src/common/storage/storage.module.ts
-13
View File
@@ -1,13 +0,0 @@
import { Global, Module } from '@nestjs/common';
import { StorageService } from './storage.service';
/**
* Global module exposing the MinIO-backed {@link StorageService} so both the API
* (source upload) and the build pipeline (presigned download) can inject it.
*/
@Global()
@Module({
providers: [StorageService],
exports: [StorageService],
})
export class StorageModule {}
backend/src/common/storage/storage.service.ts
-64
View File
@@ -1,64 +0,0 @@
import { Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import * as Minio from 'minio';
/**
* Object storage for application source archives, backed by the in-cluster MinIO
* (S3-compatible). Replaces the previous local-disk + PVC + kubectl-cp upload path:
* the API streams the uploaded zip straight to MinIO, and build pods pull it via a
* short-lived presigned URL (no credentials, no kubectl, no helper pod).
*/
@Injectable()
export class StorageService {
private readonly logger = new Logger(StorageService.name);
private readonly client: Minio.Client;
private readonly bucket: string;
private bucketReady = false;
constructor(private readonly configService: ConfigService) {
this.bucket = this.configService.get<string>('minio.bucket') || 'app-sources';
this.client = new Minio.Client({
endPoint: this.configService.get<string>('minio.endpoint') || 'minio.cloudhost-builds.svc.cluster.local',
port: this.configService.get<number>('minio.port') || 9000,
useSSL: this.configService.get<boolean>('minio.useSSL') || false,
accessKey: this.configService.get<string>('minio.accessKey') || 'cloudhost',
secretKey: this.configService.get<string>('minio.secretKey') || '',
});
}
/** Object key for an app's source archive. */
sourceKey(userId: string, appId: string): string {
return `${userId}/${appId}/source.zip`;
}
private async ensureBucket(): Promise<void> {
if (this.bucketReady) return;
const exists = await this.client.bucketExists(this.bucket).catch(() => false);
if (!exists) {
await this.client.makeBucket(this.bucket);
this.logger.log(`Created MinIO bucket "${this.bucket}"`);
}
this.bucketReady = true;
}
/** Upload an app's source archive; returns the stored object key (saved as app.codePath). */
async putSource(userId: string, appId: string, data: Buffer): Promise<string> {
await this.ensureBucket();
const key = this.sourceKey(userId, appId);
await this.client.putObject(this.bucket, key, data, data.length, { 'Content-Type': 'application/zip' });
this.logger.log(`Uploaded source ${(data.length / 1024 / 1024).toFixed(1)}MB → ${this.bucket}/${key}`);
return key;
}
/** Short-lived presigned GET URL the build pod uses to download the source. */
async presignSourceGet(key: string, expirySeconds = 3600): Promise<string> {
return this.client.presignedGetObject(this.bucket, key, expirySeconds);
}
/** Best-effort delete of an app's source object (e.g. on app deletion). */
async removeSource(key: string): Promise<void> {
await this.client.removeObject(this.bucket, key).catch((e) => {
this.logger.warn(`Failed to remove source ${key}: ${e.message}`);
});
}
}
backend/src/config/configuration.ts
+13 -36
View File
@@ -84,6 +84,11 @@ export default () => ({
port: parseInt(process.env.REDIS_PORT || '6379', 10),
},
cluster: {
/** AES-256-GCM key for encrypting stored kubeconfigs. Required in production. */
kubeconfigKey: process.env.CLUSTER_KUBECONFIG_KEY || '',
},
// OTP SMS. Provider selectable via SMS_PROVIDER ('mizbansms' | 'kavenegar').
sms: {
provider: (process.env.SMS_PROVIDER || 'mizbansms').trim().toLowerCase(),
@@ -125,45 +130,9 @@ export default () => ({
password: process.env.REGISTRY_PASSWORD || '',
},
// In-cluster MinIO (S3-compatible) for application source archives.
minio: {
endpoint: process.env.MINIO_ENDPOINT || 'minio.cloudhost-builds.svc.cluster.local',
port: parseInt(process.env.MINIO_PORT || '9000', 10),
useSSL: process.env.MINIO_USE_SSL === 'true',
accessKey: process.env.MINIO_ACCESS_KEY || 'cloudhost',
secretKey: process.env.MINIO_SECRET_KEY || 'CloudHost2024!Minio',
bucket: process.env.MINIO_BUCKET || 'app-sources',
},
build: {
namespace: process.env.BUILD_NAMESPACE || 'cloudhost-builds',
serviceAccount: process.env.BUILD_SERVICE_ACCOUNT || 'kaniko-builder',
/** Max number of build+deploy pipelines processed concurrently across the queue. */
concurrency: parseInt(process.env.BUILD_CONCURRENCY || '3', 10),
/** Per-image-build timeout (Kaniko job) in seconds. */
timeoutSeconds: parseInt(process.env.BUILD_TIMEOUT_SECONDS || '600', 10),
/** Nixpacks builder image used to generate a Dockerfile for code runtimes. */
nixpacksImage: process.env.NIXPACKS_IMAGE || 'ghcr.io/railwayapp/nixpacks:latest',
/**
* Build-time env baked into Nixpacks-generated images (mirrors/proxies for the
* Iran network, e.g. "NPM_CONFIG_REGISTRY=https://registry.npmmirror.com").
* Comma-separated KEY=VALUE pairs — set per the Phase 0 spike findings.
*/
nixpacksBuildEnv: (process.env.NIXPACKS_BUILD_ENV || '')
.split(',')
.map((s) => s.trim())
.filter(Boolean),
/** Report-only Trivy image scan after a successful build. */
scanEnabled: process.env.BUILD_SCAN_ENABLED !== 'false',
trivyImage: process.env.TRIVY_IMAGE || 'aquasec/trivy:latest',
/** Optional mirror for Trivy's vulnerability DB (Iran network); empty = default ghcr.io. */
trivyDbRepository: process.env.TRIVY_DB_REPOSITORY || '',
/** Max seconds to wait for the Trivy scan job. */
scanTimeoutSeconds: parseInt(process.env.BUILD_SCAN_TIMEOUT_SECONDS || '300', 10),
/** Registry garbage collection: keep the N most recent image tags per app repo. */
registryGcEnabled: process.env.REGISTRY_GC_ENABLED !== 'false',
registryKeepVersions: parseInt(process.env.REGISTRY_KEEP_VERSIONS || '3', 10),
registryGcIntervalMs: parseInt(process.env.REGISTRY_GC_INTERVAL_MS || '86400000', 10), // daily
},
elasticsearch: {
@@ -190,6 +159,14 @@ export default () => ({
},
},
sourceStorage: {
endpoint: process.env.SOURCE_STORAGE_ENDPOINT,
region: process.env.SOURCE_STORAGE_REGION || 'us-east-1',
bucket: process.env.SOURCE_STORAGE_BUCKET,
accessKey: process.env.SOURCE_STORAGE_ACCESS_KEY,
secretKey: process.env.SOURCE_STORAGE_SECRET_KEY,
},
platform: {
domain: resolvePlatformDomainFromEnv(),
previewRootDomain: resolvePreviewRootDomainFromEnv(),
backend/src/config/validate-production-config.spec.ts
+40
View File
@@ -0,0 +1,40 @@
import { validateProductionConfig } from './validate-production-config';
describe('validateProductionConfig', () => {
const env = process.env;
beforeEach(() => {
process.env = { ...env };
});
afterAll(() => {
process.env = env;
});
it('does nothing in development', () => {
process.env.NODE_ENV = 'development';
delete process.env.JWT_SECRET;
expect(() => validateProductionConfig()).not.toThrow();
});
it('throws in production when secrets are missing or default', () => {
process.env.NODE_ENV = 'production';
process.env.JWT_SECRET = 'default-jwt-secret';
process.env.JWT_REFRESH_SECRET = 'default-refresh-secret';
process.env.DB_PASSWORD = 'cloudhost_secret';
expect(() => validateProductionConfig()).toThrow(/Production configuration validation failed/);
expect(() => validateProductionConfig()).toThrow(/JWT_SECRET/);
expect(() => validateProductionConfig()).toThrow(/CLUSTER_KUBECONFIG_KEY/);
});
it('passes in production with strong secrets', () => {
process.env.NODE_ENV = 'production';
process.env.JWT_SECRET = 'a-very-long-random-production-secret';
process.env.JWT_REFRESH_SECRET = 'another-very-long-random-refresh-secret';
process.env.DB_PASSWORD = 'strong-db-password-here';
process.env.CLUSTER_KUBECONFIG_KEY = '0123456789abcdef0123456789abcdef';
expect(() => validateProductionConfig()).not.toThrow();
});
});
backend/src/config/validate-production-config.ts
+34
View File
@@ -0,0 +1,34 @@
const DEFAULT_JWT_SECRET = 'default-jwt-secret';
const DEFAULT_REFRESH_SECRET = 'default-refresh-secret';
const DEFAULT_DB_PASSWORD = 'cloudhost_secret';
export function validateProductionConfig(): void {
if (process.env.NODE_ENV !== 'production') {
return;
}
const errors: string[] = [];
const jwtSecret = process.env.JWT_SECRET || DEFAULT_JWT_SECRET;
const refreshSecret = process.env.JWT_REFRESH_SECRET || DEFAULT_REFRESH_SECRET;
const dbPassword = process.env.DB_PASSWORD || DEFAULT_DB_PASSWORD;
if (!process.env.JWT_SECRET || jwtSecret === DEFAULT_JWT_SECRET) {
errors.push('JWT_SECRET must be set to a strong random value in production');
}
if (!process.env.JWT_REFRESH_SECRET || refreshSecret === DEFAULT_REFRESH_SECRET) {
errors.push('JWT_REFRESH_SECRET must be set to a strong random value in production');
}
if (!process.env.DB_PASSWORD || dbPassword === DEFAULT_DB_PASSWORD) {
errors.push('DB_PASSWORD must be changed from the default in production');
}
if (!process.env.CLUSTER_KUBECONFIG_KEY?.trim()) {
errors.push('CLUSTER_KUBECONFIG_KEY must be set in production to encrypt stored kubeconfigs');
}
if (errors.length > 0) {
throw new Error(
`Production configuration validation failed:\n${errors.map((e) => ` - ${e}`).join('\n')}`,
);
}
}
backend/src/deployments/deployment.constants.ts
-2
View File
@@ -1,2 +0,0 @@
/** Bull queue name for build+deploy pipelines. */
export const DEPLOY_QUEUE = 'app-deploy';
backend/src/deployments/deployment.processor.ts
-27
View File
@@ -1,27 +0,0 @@
import { Process, Processor } from '@nestjs/bull';
import { Logger } from '@nestjs/common';
import { Job } from 'bull';
import { DeploymentsService, DeploymentJobData } from './deployments.service';
import { DEPLOY_QUEUE } from './deployment.constants';
/**
* Processes build+deploy pipelines off the `app-deploy` queue with a bounded
* concurrency (BUILD_CONCURRENCY, default 3) so simultaneous user deploys can't
* flood the cluster with Kaniko build jobs (2 CPU / 4Gi each).
*
* Concurrency is read at module-load time from env because Bull's `@Process`
* decorator option must be a constant.
*/
@Processor(DEPLOY_QUEUE)
export class DeploymentProcessor {
private readonly logger = new Logger(DeploymentProcessor.name);
constructor(private readonly deploymentsService: DeploymentsService) {}
@Process({ name: 'run', concurrency: parseInt(process.env.BUILD_CONCURRENCY || '3', 10) })
async handleRun(job: Job<DeploymentJobData>): Promise<void> {
const { deploymentId } = job.data;
this.logger.log(`Processing deployment ${deploymentId} (job ${job.id})`);
await this.deploymentsService.processDeploymentJob(job.data);
}
}
backend/src/deployments/deployments.controller.ts
+11 -4
View File
@@ -10,6 +10,7 @@ import { AuthGuard } from '@nestjs/passport';
import { ApiTags, ApiOperation, ApiBearerAuth } from '@nestjs/swagger';
import { DeploymentsService } from './deployments.service';
import { RolesGuard } from '../common/guards/roles.guard';
import { UserRole } from '../common/enums';
@ApiTags('Deployments')
@ApiBearerAuth()
@@ -18,6 +19,12 @@ import { RolesGuard } from '../common/guards/roles.guard';
export class DeploymentsController {
constructor(private readonly deploymentsService: DeploymentsService) {}
private ownershipUserId(req: { user: { id: string; role: string } }): string | undefined {
const isStaff =
req.user.role === UserRole.ADMIN || req.user.role === UserRole.TECHNICAL;
return isStaff ? undefined : req.user.id;
}
@Post('applications/:appId/deploy')
@ApiOperation({ summary: 'Trigger a new deployment' })
async triggerDeployment(@Param('appId') appId: string, @Request() req: any) {
@@ -26,14 +33,14 @@ export class DeploymentsController {
@Get('applications/:appId')
@ApiOperation({ summary: 'List deployments for an application' })
async findByApplication(@Param('appId') appId: string) {
return this.deploymentsService.findByApplication(appId);
async findByApplication(@Param('appId') appId: string, @Request() req: any) {
return this.deploymentsService.findByApplication(appId, this.ownershipUserId(req));
}
@Get(':id')
@ApiOperation({ summary: 'Get deployment details' })
async findOne(@Param('id') id: string) {
return this.deploymentsService.findOne(id);
async findOne(@Param('id') id: string, @Request() req: any) {
return this.deploymentsService.findOne(id, this.ownershipUserId(req));
}
@Get('applications/:appId/logs')
backend/src/deployments/deployments.module.ts
+1 -5
View File
@@ -1,10 +1,7 @@
import { Module, forwardRef } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { BullModule } from '@nestjs/bull';
import { DeploymentsService } from './deployments.service';
import { DeploymentsController } from './deployments.controller';
import { DeploymentProcessor } from './deployment.processor';
import { DEPLOY_QUEUE } from './deployment.constants';
import { Deployment } from './entities/deployment.entity';
import { ApplicationsModule } from '../applications/applications.module';
import { KubernetesModule } from '../kubernetes/kubernetes.module';
@@ -14,14 +11,13 @@ import { ClustersModule } from '../clusters/clusters.module';
@Module({
imports: [
TypeOrmModule.forFeature([Deployment]),
BullModule.registerQueue({ name: DEPLOY_QUEUE }),
forwardRef(() => ApplicationsModule),
forwardRef(() => ClustersModule),
KubernetesModule,
BuildModule,
],
controllers: [DeploymentsController],
providers: [DeploymentsService, DeploymentProcessor],
providers: [DeploymentsService],
exports: [DeploymentsService],
})
export class DeploymentsModule {}
backend/src/deployments/deployments.service.spec.ts
+88
View File
@@ -0,0 +1,88 @@
import { Test, TestingModule } from '@nestjs/testing';
import { getRepositoryToken } from '@nestjs/typeorm';
import { NotFoundException } from '@nestjs/common';
import { DeploymentsService } from './deployments.service';
import { Deployment } from './entities/deployment.entity';
import { ApplicationsService } from '../applications/applications.service';
import { KubernetesService } from '../kubernetes/kubernetes.service';
import { BuildService } from '../build/build.service';
import { ClustersService } from '../clusters/clusters.service';
describe('DeploymentsService authorization', () => {
let service: DeploymentsService;
const deploymentsRepository = {
find: jest.fn(),
findOne: jest.fn(),
createQueryBuilder: jest.fn(),
};
const applicationsService = {
findOne: jest.fn(),
};
beforeEach(async () => {
jest.clearAllMocks();
const module: TestingModule = await Test.createTestingModule({
providers: [
DeploymentsService,
{ provide: getRepositoryToken(Deployment), useValue: deploymentsRepository },
{ provide: ApplicationsService, useValue: applicationsService },
{ provide: KubernetesService, useValue: {} },
{ provide: BuildService, useValue: {} },
{ provide: ClustersService, useValue: {} },
],
}).compile();
service = module.get(DeploymentsService);
});
describe('findByApplication', () => {
it('verifies application ownership before listing deployments', async () => {
const appId = 'app-1';
const userId = 'user-1';
const deployments = [{ id: 'd-1', applicationId: appId }] as Deployment[];
applicationsService.findOne.mockResolvedValue({ id: appId, userId });
deploymentsRepository.find.mockResolvedValue(deployments);
const result = await service.findByApplication(appId, userId);
expect(applicationsService.findOne).toHaveBeenCalledWith(appId, userId);
expect(result).toEqual(deployments);
});
it('propagates NotFoundException when user does not own the app', async () => {
applicationsService.findOne.mockRejectedValue(new NotFoundException('Application not found'));
await expect(service.findByApplication('app-1', 'other-user')).rejects.toThrow(NotFoundException);
expect(deploymentsRepository.find).not.toHaveBeenCalled();
});
});
describe('findOne', () => {
it('verifies application ownership before returning deployment', async () => {
const deployment = {
id: 'd-1',
applicationId: 'app-1',
application: { id: 'app-1' },
} as Deployment;
deploymentsRepository.findOne.mockResolvedValue(deployment);
applicationsService.findOne.mockResolvedValue({ id: 'app-1', userId: 'user-1' });
const result = await service.findOne('d-1', 'user-1');
expect(applicationsService.findOne).toHaveBeenCalledWith('app-1', 'user-1');
expect(result).toBe(deployment);
});
it('throws when deployment does not exist', async () => {
deploymentsRepository.findOne.mockResolvedValue(null);
await expect(service.findOne('missing', 'user-1')).rejects.toThrow(NotFoundException);
expect(applicationsService.findOne).not.toHaveBeenCalled();
});
});
});
backend/src/deployments/deployments.service.ts
+19 -81
View File
@@ -1,15 +1,11 @@
import { Injectable, NotFoundException, BadRequestException, Logger, Inject, forwardRef, OnModuleInit } from '@nestjs/common';
import { Injectable, NotFoundException, BadRequestException, Logger, Inject, forwardRef } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { InjectQueue } from '@nestjs/bull';
import { Queue } from 'bull';
import { Repository } from 'typeorm';
import * as fs from 'fs';
import { DEPLOY_QUEUE } from './deployment.constants';
import { Deployment } from './entities/deployment.entity';
import { ApplicationsService } from '../applications/applications.service';
import { KubernetesService } from '../kubernetes/kubernetes.service';
import { BuildService, BuildProgress, BuildCancelledError } from '../build/build.service';
import { ScanService } from '../build/scan.service';
import * as crypto from 'crypto';
import {
AppLifecycleStatus,
@@ -19,15 +15,8 @@ import {
} from '../common/enums';
import { ClustersService } from '../clusters/clusters.service';
/** Payload enqueued on the `app-deploy` queue for the build+deploy pipeline. */
export interface DeploymentJobData {
deploymentId: string;
applicationId: string;
previewSubdomain: string | null;
}
@Injectable()
export class DeploymentsService implements OnModuleInit {
export class DeploymentsService {
private readonly logger = new Logger(DeploymentsService.name);
constructor(
@@ -38,41 +27,8 @@ export class DeploymentsService implements OnModuleInit {
private kubernetesService: KubernetesService,
private buildService: BuildService,
private clustersService: ClustersService,
private scanService: ScanService,
@InjectQueue(DEPLOY_QUEUE)
private deployQueue: Queue<DeploymentJobData>,
) {}
async onModuleInit(): Promise<void> {
try {
await this.deploymentsRepository.query(
`ALTER TABLE deployments ADD COLUMN IF NOT EXISTS "vulnerabilitySummary" jsonb`,
);
} catch (e: any) {
this.logger.warn(`Could not ensure deployments.vulnerabilitySummary column: ${e.message}`);
}
}
/**
* Queue worker entrypoint — runs one build+deploy pipeline. Bounded
* concurrency lives on the Bull processor, so this just dispatches to the
* managed (Helm-only) or app (build+deploy) pipeline. Both pipelines handle
* their own errors, so a failure here never triggers a Bull retry.
*/
async processDeploymentJob(data: DeploymentJobData): Promise<void> {
const { deploymentId, applicationId, previewSubdomain } = data;
if (await this.isDeploymentCancelled(deploymentId)) {
this.logger.log(`Deployment ${deploymentId} already cancelled before pickup — skipping`);
return;
}
const app = await this.applicationsService.findOne(applicationId);
if (isManagedProductType(app.productType)) {
await this.executeManagedPipeline(deploymentId, app);
} else {
await this.executePipeline(deploymentId, app, previewSubdomain);
}
}
/**
* Random 7-digit suffix for the preview host: <userId>-<7-digit>.<baseDomain>.
* Generated once per application (see resolvePreviewNumber) and persisted.
@@ -121,13 +77,13 @@ export class DeploymentsService implements OnModuleInit {
saved.previewSubdomain = previewSubdomain;
}
// Enqueue the build+deploy pipeline. The Bull processor runs it with bounded
// concurrency so concurrent user deploys can't flood the cluster.
await this.deployQueue.add(
'run',
{ deploymentId: saved.id, applicationId: app.id, previewSubdomain },
{ removeOnComplete: true, removeOnFail: true },
);
// Trigger async pipeline (Helm-only for managed services, build+deploy for apps)
const run = isManagedProductType(app.productType)
? this.executeManagedPipeline(saved.id, app)
: this.executePipeline(saved.id, app, previewSubdomain);
run.catch((error) => {
this.logger.error(`Pipeline failed for deployment ${saved.id}:`, error);
});
return saved;
}
@@ -231,19 +187,6 @@ export class DeploymentsService implements OnModuleInit {
// Save build log
await this.deploymentsRepository.update(deploymentId, { buildLog: buildResult.buildLog });
// Report-only vulnerability scan — runs alongside the deploy and is
// persisted when it finishes. Never blocks or fails the deployment.
void this.scanService
.scanImage(app, imageUri)
.then((summary) => {
if (summary) {
return this.deploymentsRepository.update(deploymentId, {
vulnerabilitySummary: summary as Record<string, any>,
});
}
})
.catch((e) => this.logger.warn(`Scan persistence failed for ${deploymentId}: ${e.message}`));
// Step 2: Update app with new image tag
await this.applicationsService.updateImageTag(app.id, imageUri);
@@ -520,14 +463,15 @@ export class DeploymentsService implements OnModuleInit {
}
}
async findByApplication(applicationId: string): Promise<Deployment[]> {
async findByApplication(applicationId: string, userId?: string): Promise<Deployment[]> {
await this.applicationsService.findOne(applicationId, userId);
return this.deploymentsRepository.find({
where: { applicationId },
order: { createdAt: 'DESC' },
});
}
async findOne(id: string): Promise<Deployment> {
async findOne(id: string, userId?: string): Promise<Deployment> {
const deployment = await this.deploymentsRepository.findOne({
where: { id },
relations: { application: true },
@@ -535,6 +479,7 @@ export class DeploymentsService implements OnModuleInit {
if (!deployment) {
throw new NotFoundException('Deployment not found');
}
await this.applicationsService.findOne(deployment.applicationId, userId);
return deployment;
}
@@ -543,10 +488,7 @@ export class DeploymentsService implements OnModuleInit {
return this.kubernetesService.getPodLogs(app);
}
async getBuildLogs(
applicationId: string,
userId: string,
): Promise<{ buildLog: string | null; status: string; version: string | null; createdAt: Date; vulnerabilitySummary: Record<string, any> | null }> {
async getBuildLogs(applicationId: string, userId: string): Promise<{ buildLog: string | null; status: string; version: string | null; createdAt: Date }> {
const app = await this.applicationsService.findOne(applicationId, userId);
const latest = await this.deploymentsRepository.findOne({
@@ -555,7 +497,7 @@ export class DeploymentsService implements OnModuleInit {
});
if (!latest) {
return { buildLog: null, status: 'no_deployment', version: null, createdAt: new Date(), vulnerabilitySummary: null };
return { buildLog: null, status: 'no_deployment', version: null, createdAt: new Date() };
}
if (this.isManagedOrHelmOnlyApp(app)) {
@@ -564,7 +506,6 @@ export class DeploymentsService implements OnModuleInit {
status: latest.status,
version: latest.version,
createdAt: latest.createdAt,
vulnerabilitySummary: null,
};
}
@@ -584,7 +525,6 @@ export class DeploymentsService implements OnModuleInit {
status: latest.status,
version: latest.version,
createdAt: latest.createdAt,
vulnerabilitySummary: latest.vulnerabilitySummary ?? null,
};
}
@@ -751,12 +691,10 @@ export class DeploymentsService implements OnModuleInit {
saved.previewSubdomain = previewSubdomain;
}
// Enqueue the build & deploy pipeline (same queue as initial deploy)
await this.deployQueue.add(
'run',
{ deploymentId: saved.id, applicationId: app.id, previewSubdomain },
{ removeOnComplete: true, removeOnFail: true },
);
// Trigger async build & deploy pipeline (same as initial deploy)
this.executePipeline(saved.id, app, previewSubdomain).catch((error) => {
this.logger.error(`Redeploy pipeline failed for deployment ${saved.id}:`, error);
});
this.logger.log(`Redeploy triggered for ${app.name} (${app.gitUrl ? 'git: ' + app.gitUrl : 'zip'})`);
return saved;
backend/src/deployments/entities/deployment.entity.ts
-8
View File
@@ -33,14 +33,6 @@ export class Deployment {
@Column({ type: 'text', nullable: true })
deployLog: string;
/**
* Report-only Trivy vulnerability summary for the built image
* (e.g. { critical, high, medium, low, unknown, total, scannedAt }).
* Non-blocking — never gates a deployment.
*/
@Column({ type: 'jsonb', nullable: true })
vulnerabilitySummary: Record<string, any> | null;
/**
* Per-deployment preview number (derived deterministically from deployment.id).
* Used to build preview ingress host under the main frontend domain.
backend/src/health/health.controller.ts
+25
View File
@@ -0,0 +1,25 @@
import { Controller, Get } from '@nestjs/common';
import { ApiTags, ApiOperation } from '@nestjs/swagger';
import { InjectDataSource } from '@nestjs/typeorm';
import { DataSource } from 'typeorm';
import { SkipThrottle } from '@nestjs/throttler';
@ApiTags('Health')
@Controller()
@SkipThrottle()
export class HealthController {
constructor(@InjectDataSource() private readonly dataSource: DataSource) {}
@Get('health')
@ApiOperation({ summary: 'Liveness probe' })
health() {
return { status: 'ok', timestamp: new Date().toISOString() };
}
@Get('ready')
@ApiOperation({ summary: 'Readiness probe — checks database connectivity' })
async ready() {
await this.dataSource.query('SELECT 1');
return { status: 'ready', timestamp: new Date().toISOString() };
}
}
backend/src/health/health.module.ts
+7
View File
@@ -0,0 +1,7 @@
import { Module } from '@nestjs/common';
import { HealthController } from './health.controller';
@Module({
controllers: [HealthController],
})
export class HealthModule {}
backend/src/kubernetes/helm.service.spec.ts
+2 -2
View File
@@ -26,10 +26,10 @@ describe('HelmService', () => {
expect(service).toBeDefined();
});
describe('chartPath', () => {
describe('resolveChartPath', () => {
it('should resolve to helm/cloudhost-app relative to project root', () => {
const expectedSuffix = path.join('helm', 'cloudhost-app');
expect((service as any).chartPath).toContain(expectedSuffix);
expect((service as any).resolveChartPath('cloudhost-app')).toContain(expectedSuffix);
});
});
backend/src/kubernetes/k8s-client-v1-migration.spec.ts
+13 -6
View File
@@ -9,6 +9,7 @@ jest.mock('@kubernetes/client-node', () => ({
import { RegistryService } from './registry.service';
import { KubernetesService } from './kubernetes.service';
import { K8sLifecycleService } from './k8s-lifecycle.service';
/**
* Regression tests for the @kubernetes/client-node 1.x migration.
@@ -80,19 +81,25 @@ describe('KubernetesService — k8s v1 client shape', () => {
let service: KubernetesService;
const makeService = (clients: { coreApi?: any; appsApi?: any; networkingApi?: any; kc?: any }) => {
const k8sClientService = {
getK8sClient: jest.fn().mockResolvedValue({
coreApi: clients.coreApi,
appsApi: clients.appsApi,
networkingApi: clients.networkingApi,
kc: clients.kc,
}),
getKubeconfig: jest.fn(),
};
const k8sLifecycleService = new K8sLifecycleService(k8sClientService as any);
const svc = new KubernetesService(
configStub,
{} as any, // clustersService
{} as any, // helmService
{} as any, // registryService
k8sClientService as any,
k8sLifecycleService,
{} as any, // deploymentsRepository
);
jest.spyOn(svc as any, 'getK8sClient').mockResolvedValue({
coreApi: clients.coreApi,
appsApi: clients.appsApi,
networkingApi: clients.networkingApi,
kc: clients.kc,
});
return svc;
};
backend/src/kubernetes/k8s-client.service.ts
+38
View File
@@ -0,0 +1,38 @@
import { Injectable } from '@nestjs/common';
import * as k8s from '@kubernetes/client-node';
import { ClustersService } from '../clusters/clusters.service';
import { registerKubeconfigNoProxy } from '../common/kubernetes-proxy.util';
@Injectable()
export class K8sClientService {
constructor(private readonly clustersService: ClustersService) {}
async getK8sClient(clusterId?: string): Promise<{
coreApi: k8s.CoreV1Api;
appsApi: k8s.AppsV1Api;
networkingApi: k8s.NetworkingV1Api;
kc: k8s.KubeConfig;
}> {
const cluster = clusterId
? await this.clustersService.findOne(clusterId)
: await this.clustersService.getDefault();
const kc = new k8s.KubeConfig();
registerKubeconfigNoProxy(cluster.kubeconfig);
kc.loadFromString(cluster.kubeconfig);
return {
coreApi: kc.makeApiClient(k8s.CoreV1Api),
appsApi: kc.makeApiClient(k8s.AppsV1Api),
networkingApi: kc.makeApiClient(k8s.NetworkingV1Api),
kc,
};
}
async getKubeconfig(clusterId?: string): Promise<string> {
const cluster = clusterId
? await this.clustersService.findOne(clusterId)
: await this.clustersService.getDefault();
return cluster.kubeconfig;
}
}
backend/src/kubernetes/k8s-lifecycle.service.ts
+34
View File
@@ -0,0 +1,34 @@
import { Injectable } from '@nestjs/common';
import { Application } from '../applications/entities/application.entity';
import { K8sClientService } from './k8s-client.service';
import { primaryWorkloadLabel, userNamespace } from './k8s-workload.util';
/** Runtime logs and lightweight workload operations extracted from KubernetesService. */
@Injectable()
export class K8sLifecycleService {
constructor(private readonly k8sClient: K8sClientService) {}
async getPodLogs(app: Application): Promise<string> {
const { coreApi } = await this.k8sClient.getK8sClient(app.clusterId);
const namespace = userNamespace(app.userId);
const podLabel = primaryWorkloadLabel(app);
const pods = await coreApi.listNamespacedPod({
namespace,
labelSelector: `app=${podLabel}`,
});
if (pods.items.length === 0) {
return 'No pods found for this application.';
}
const podName = pods.items[0].metadata?.name;
if (!podName) return 'Pod name not found.';
return coreApi.readNamespacedPodLog({
name: podName,
namespace,
tailLines: 200,
});
}
}
backend/src/kubernetes/k8s-workload.util.ts
+42
View File
@@ -0,0 +1,42 @@
import { Application } from '../applications/entities/application.entity';
import { DatabaseType, isManagedProductType } from '../common/enums';
/** Kubernetes namespace for a user's applications. */
export function userNamespace(userId: string): string {
return `user-${userId.split('-')[0]}`;
}
/** Primary pod label selector target for an application workload. */
export function primaryWorkloadLabel(app: Application): string {
if (isManagedProductType(app.productType)) {
if (app.databaseType && app.databaseType !== DatabaseType.NONE) {
return `${app.name}-db`;
}
if (app.enableRedis) return `${app.name}-redis`;
if (app.enableRabbitmq) return `${app.name}-rabbitmq`;
}
return app.name;
}
export function getApplicationWorkloadDeployments(
app: Application,
): { name: string; runningReplicas: number }[] {
const managed = isManagedProductType(app.productType);
const workloads: { name: string; runningReplicas: number }[] = [];
if (!managed) {
workloads.push({ name: app.name, runningReplicas: app.replicas || 1 });
}
if (app.databaseType && app.databaseType !== DatabaseType.NONE) {
workloads.push({ name: `${app.name}-db`, runningReplicas: 1 });
}
if (app.enableRedis) {
workloads.push({ name: `${app.name}-redis`, runningReplicas: 1 });
}
if (app.enableRabbitmq) {
workloads.push({ name: `${app.name}-rabbitmq`, runningReplicas: 1 });
}
return workloads;
}
backend/src/kubernetes/kubernetes.module.ts
+4 -3
View File
@@ -3,7 +3,8 @@ import { TypeOrmModule } from '@nestjs/typeorm';
import { KubernetesService } from './kubernetes.service';
import { HelmService } from './helm.service';
import { RegistryService } from './registry.service';
import { RegistryGcService } from './registry-gc.service';
import { K8sClientService } from './k8s-client.service';
import { K8sLifecycleService } from './k8s-lifecycle.service';
import { ElasticsearchService } from './elasticsearch.service';
import { ElasticsearchController } from './elasticsearch.controller';
import { LogsController } from './logs.controller';
@@ -14,7 +15,7 @@ import { Deployment } from '../deployments/entities/deployment.entity';
@Module({
imports: [forwardRef(() => ClustersModule), TypeOrmModule.forFeature([Application, Deployment])],
controllers: [ElasticsearchController, LogsController],
providers: [KubernetesService, HelmService, RegistryService, RegistryGcService, ElasticsearchService],
exports: [KubernetesService, HelmService, RegistryService, ElasticsearchService],
providers: [KubernetesService, HelmService, RegistryService, ElasticsearchService, K8sClientService, K8sLifecycleService],
exports: [KubernetesService, HelmService, RegistryService, ElasticsearchService, K8sClientService, K8sLifecycleService],
})
export class KubernetesModule {}
backend/src/kubernetes/kubernetes.service.ts
+49 -95
View File
@@ -15,6 +15,8 @@ import { ensureAppUrlEnv } from '../applications/app-url.util';
import { AppRuntime, DatabaseType, CustomDomainStatus, ServiceAccessTarget, ProductType, isManagedProductType } from '../common/enums';
import { HelmService } from './helm.service';
import { RegistryService } from './registry.service';
import { K8sClientService } from './k8s-client.service';
import { K8sLifecycleService } from './k8s-lifecycle.service';
import { registerKubeconfigNoProxy } from '../common/kubernetes-proxy.util';
const execFileAsync = promisify(execFile);
@@ -74,6 +76,8 @@ export class KubernetesService implements OnModuleInit {
private clustersService: ClustersService,
private helmService: HelmService,
private registryService: RegistryService,
private k8sClientService: K8sClientService,
private k8sLifecycleService: K8sLifecycleService,
@InjectRepository(Deployment)
private deploymentsRepository: Repository<Deployment>,
) {}
@@ -99,34 +103,6 @@ export class KubernetesService implements OnModuleInit {
// Helm chart is used for deployments — no local template loading needed
}
private async getK8sClient(clusterId?: string): Promise<{
coreApi: k8s.CoreV1Api;
appsApi: k8s.AppsV1Api;
networkingApi: k8s.NetworkingV1Api;
kc: k8s.KubeConfig;
}> {
const cluster = clusterId ? await this.clustersService.findOne(clusterId) : await this.clustersService.getDefault();
const kc = new k8s.KubeConfig();
registerKubeconfigNoProxy(cluster.kubeconfig);
kc.loadFromString(cluster.kubeconfig);
return {
coreApi: kc.makeApiClient(k8s.CoreV1Api),
appsApi: kc.makeApiClient(k8s.AppsV1Api),
networkingApi: kc.makeApiClient(k8s.NetworkingV1Api),
kc,
};
}
/**
* Get the raw kubeconfig string for a cluster.
*/
private async getKubeconfig(clusterId?: string): Promise<string> {
const cluster = clusterId ? await this.clustersService.findOne(clusterId) : await this.clustersService.getDefault();
return cluster.kubeconfig;
}
/**
* Build Helm values object from an Application entity and image URI.
*/
@@ -412,7 +388,7 @@ export class KubernetesService implements OnModuleInit {
}
async waitForApplicationReady(app: Application, timeoutMs = 600_000, shouldAbort?: () => Promise<boolean>): Promise<void> {
const { coreApi, appsApi } = await this.getK8sClient(app.clusterId);
const { coreApi, appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const managed = isManagedProductType(app.productType);
const workloads = [
@@ -463,14 +439,14 @@ export class KubernetesService implements OnModuleInit {
const previewNumber = app.customDomain ? null : await this.resolvePreviewNumber(app.id);
try {
const kubeconfig = await this.getKubeconfig(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
const imageUri = app.latestImageTag ? this.registryService.normalizeImageReference(app.latestImageTag) : '';
const values = this.buildHelmValues(app, imageUri, previewNumber);
await this.helmService.installOrUpgrade(app.name, namespace, values, kubeconfig);
this.logger.log(`Updated ingress for ${app.name} via Helm (customDomain: ${customDomain || 'none'})`);
} catch (helmError: any) {
this.logger.warn(`Helm ingress update failed for ${app.name}, using direct K8s API: ${helmError.message}`);
const { networkingApi } = await this.getK8sClient(app.clusterId);
const { networkingApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const ctx: ManifestContext = {
appName: app.name,
namespace,
@@ -513,9 +489,9 @@ export class KubernetesService implements OnModuleInit {
// ── Helm-based deployment ─────────────────────────────────────────
private async deployViaHelm(app: Application, imageUri: string, previewNumber?: string | null): Promise<Record<string, any>> {
const kubeconfig = await this.getKubeconfig(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
await this.ensurePlatformStorageClass(kubeconfig);
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const values = this.buildHelmValues(app, imageUri, previewNumber);
const namespace = values.app.namespace as string;
await this.registryService.ensureRegistryPullSecret(coreApi, namespace);
@@ -531,7 +507,7 @@ export class KubernetesService implements OnModuleInit {
}
private async deployManagedViaHelm(app: Application): Promise<Record<string, any>> {
const kubeconfig = await this.getKubeconfig(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
await this.ensurePlatformStorageClass(kubeconfig);
const values = this.buildManagedHelmValues(app);
const namespace = values.app.namespace;
@@ -549,8 +525,8 @@ export class KubernetesService implements OnModuleInit {
// ── Direct K8s API deployment (fallback) ──────────────────────────
private async deployManagedViaK8sApi(app: Application): Promise<Record<string, any>> {
const { coreApi, appsApi } = await this.getK8sClient(app.clusterId);
const kubeconfig = await this.getKubeconfig(app.clusterId);
const { coreApi, appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
await this.ensurePlatformStorageClass(kubeconfig);
const namespace = `user-${app.userId.split('-')[0]}`;
const context: ManifestContext = {
@@ -619,8 +595,8 @@ export class KubernetesService implements OnModuleInit {
if (isManagedProductType(app.productType)) {
return this.deployManagedViaK8sApi(app);
}
const { coreApi, appsApi, networkingApi } = await this.getK8sClient(app.clusterId);
const kubeconfig = await this.getKubeconfig(app.clusterId);
const { coreApi, appsApi, networkingApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
await this.ensurePlatformStorageClass(kubeconfig);
const domain = this.configService.get('platform.domain');
@@ -2247,33 +2223,11 @@ export class KubernetesService implements OnModuleInit {
}
async getPodLogs(app: Application): Promise<string> {
const { coreApi } = await this.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const podLabel = this.primaryWorkloadLabel(app);
const pods = await coreApi.listNamespacedPod({
namespace,
labelSelector: `app=${podLabel}`,
});
if (pods.items.length === 0) {
return 'No pods found for this application.';
}
const podName = pods.items[0].metadata?.name;
if (!podName) return 'Pod name not found.';
const logResponse = await coreApi.readNamespacedPodLog({
name: podName,
namespace,
tailLines: 200,
});
return logResponse;
return this.k8sLifecycleService.getPodLogs(app);
}
async scaleDeployment(app: Application, replicas: number): Promise<void> {
const { appsApi } = await this.getK8sClient(app.clusterId);
const { appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
await appsApi.patchNamespacedDeployment({ name: app.name, namespace, body: { spec: { replicas } } }, k8s.setHeaderOptions('Content-Type', 'application/merge-patch+json'));
@@ -2313,7 +2267,7 @@ export class KubernetesService implements OnModuleInit {
}
async captureWorkloadReplicaSnapshot(app: Application): Promise<Record<string, number>> {
const { appsApi } = await this.getK8sClient(app.clusterId);
const { appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const snapshot: Record<string, number> = {};
@@ -2342,7 +2296,7 @@ export class KubernetesService implements OnModuleInit {
* Returns the replica snapshot captured before scaling.
*/
async suspendApplication(app: Application): Promise<Record<string, number>> {
const { appsApi } = await this.getK8sClient(app.clusterId);
const { appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
this.logger.log(`Suspending application ${app.name} in namespace ${namespace}`);
@@ -2368,7 +2322,7 @@ export class KubernetesService implements OnModuleInit {
* Resume a suspended application using saved replica counts when available.
*/
async resumeApplication(app: Application): Promise<void> {
const { appsApi } = await this.getK8sClient(app.clusterId);
const { appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
this.logger.log(`Resuming application ${app.name} in namespace ${namespace}`);
@@ -2400,7 +2354,7 @@ export class KubernetesService implements OnModuleInit {
}
async restartDeployment(app: Application): Promise<void> {
const { appsApi } = await this.getK8sClient(app.clusterId);
const { appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const deploymentName = isManagedProductType(app.productType) ? this.primaryWorkloadLabel(app) : app.name;
@@ -2595,7 +2549,7 @@ export class KubernetesService implements OnModuleInit {
* Includes application workload, database, and optional Redis / RabbitMQ when enabled.
*/
async getResourceUsage(app: Application): Promise<any> {
const { coreApi, appsApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, appsApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const workloads: any[] = [];
@@ -2685,7 +2639,7 @@ export class KubernetesService implements OnModuleInit {
},
workload: 'app' | 'database' | 'redis' | 'rabbitmq' = 'app',
): Promise<void> {
const { appsApi } = await this.getK8sClient(app.clusterId);
const { appsApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const target = this.workloadDeploymentTarget(app, workload);
@@ -2807,7 +2761,7 @@ export class KubernetesService implements OnModuleInit {
throw new BadRequestException('Application is not assigned to a cluster');
}
const { coreApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = this.getUserNamespace(app.userId);
const { selector, targetPort, portName } = this.resolveAccessTarget(app, target);
const shortId = grantId.split('-')[0];
@@ -2874,7 +2828,7 @@ export class KubernetesService implements OnModuleInit {
const manifestPath = path.join(tmpDir, 'service.json');
try {
fs.writeFileSync(kubeconfigPath, await this.getKubeconfig(clusterId), {
fs.writeFileSync(kubeconfigPath, await this.k8sClientService.getKubeconfig(clusterId), {
mode: 0o600,
});
fs.writeFileSync(manifestPath, JSON.stringify(service), { mode: 0o600 });
@@ -2895,7 +2849,7 @@ export class KubernetesService implements OnModuleInit {
}
async revokeTemporaryAccess(clusterId: string, namespace: string, k8sServiceName: string): Promise<void> {
const { coreApi } = await this.getK8sClient(clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(clusterId);
try {
await coreApi.deleteNamespacedService({
name: k8sServiceName,
@@ -2912,7 +2866,7 @@ export class KubernetesService implements OnModuleInit {
async deleteTemporaryAccessServicesForApp(app: Application): Promise<void> {
if (!app.clusterId) return;
const namespace = this.getUserNamespace(app.userId);
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
try {
const services = await coreApi.listNamespacedService({
@@ -2942,7 +2896,7 @@ export class KubernetesService implements OnModuleInit {
};
case ServiceAccessTarget.REDIS: {
if (!app.clusterId) return {};
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const secret = await coreApi.readNamespacedSecret({
name: `${app.name}-redis-secret`,
namespace,
@@ -2953,7 +2907,7 @@ export class KubernetesService implements OnModuleInit {
case ServiceAccessTarget.RABBITMQ_AMQP:
case ServiceAccessTarget.RABBITMQ_MANAGEMENT: {
if (!app.clusterId) return {};
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const secret = await coreApi.readNamespacedSecret({
name: `${app.name}-rabbitmq-secret`,
namespace,
@@ -2980,7 +2934,7 @@ export class KubernetesService implements OnModuleInit {
host: string;
ingressUrl?: string;
}> {
const { coreApi, networkingApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, networkingApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = this.getUserNamespace(app.userId);
const domain = this.configService.get('platform.domain');
const hostIp = this.getClusterHostIp(kc);
@@ -3044,13 +2998,13 @@ export class KubernetesService implements OnModuleInit {
async deleteApplication(app: Application): Promise<void> {
const namespace = this.getUserNamespace(app.userId);
const { coreApi, appsApi, networkingApi } = await this.getK8sClient(app.clusterId);
const { coreApi, appsApi, networkingApi } = await this.k8sClientService.getK8sClient(app.clusterId);
await this.deleteTemporaryAccessServicesForApp(app);
// Step 1: Try Helm uninstall (handles most resources)
try {
const kubeconfig = await this.getKubeconfig(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
await this.helmService.uninstall(app.name, namespace, kubeconfig);
this.logger.log(`Helm release ${app.name} uninstalled from ${namespace}`);
} catch (error: any) {
@@ -3171,8 +3125,8 @@ export class KubernetesService implements OnModuleInit {
} = {},
): Promise<void> {
const namespace = this.getUserNamespace(app.userId);
const source = await this.getK8sClient(app.clusterId);
const target = await this.getK8sClient(targetClusterId);
const source = await this.k8sClientService.getK8sClient(app.clusterId);
const target = await this.k8sClientService.getK8sClient(targetClusterId);
await this.ensureNamespaceOnCluster(target.coreApi, namespace);
await options.log?.('transfer-secrets-configs', 'Target namespace ensured', { namespace });
@@ -3250,8 +3204,8 @@ export class KubernetesService implements OnModuleInit {
const targetKubeconfig = path.join(tempDir, 'target.kubeconfig');
fs.mkdirSync(tempDir, { recursive: true });
fs.writeFileSync(sourceKubeconfig, await this.getKubeconfig(sourceClusterId), { mode: 0o600 });
fs.writeFileSync(targetKubeconfig, await this.getKubeconfig(targetClusterId), { mode: 0o600 });
fs.writeFileSync(sourceKubeconfig, await this.k8sClientService.getKubeconfig(sourceClusterId), { mode: 0o600 });
fs.writeFileSync(targetKubeconfig, await this.k8sClientService.getKubeconfig(targetClusterId), { mode: 0o600 });
try {
await this.createPvcCopyPod(sourceKubeconfig, namespace, sourcePod, pvcName);
@@ -3407,7 +3361,7 @@ export class KubernetesService implements OnModuleInit {
* Polls pod status with label selector `app=<appName>-db`.
*/
async waitForDatabaseReady(app: Application, timeoutMs = 120_000): Promise<void> {
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const dbLabel = `${app.name}-db`;
const start = Date.now();
@@ -3513,7 +3467,7 @@ export class KubernetesService implements OnModuleInit {
* then runs a restore Job that mounts the PVC and imports the dump.
*/
async restoreDatabaseDump(app: Application, dumpFilePath: string): Promise<{ success: boolean; logs: string }> {
const { coreApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const batchApi = kc.makeApiClient(k8s.BatchV1Api);
const namespace = `user-${app.userId.split('-')[0]}`;
const dbName = `${app.name}-db`;
@@ -3821,7 +3775,7 @@ export class KubernetesService implements OnModuleInit {
* Used when legacy PVCs were created without storageClassName.
*/
private async migrateDatabasePvcToResizableStorage(app: Application, newSize: string, storageClassName: string): Promise<{ success: boolean; message: string }> {
const { coreApi, appsApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, appsApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const batchApi = kc.makeApiClient(k8s.BatchV1Api);
const namespace = `user-${app.userId.split('-')[0]}`;
const oldPvcName = `${app.name}-db`;
@@ -3997,7 +3951,7 @@ export class KubernetesService implements OnModuleInit {
* K8s only supports PVC expansion, not shrinking.
*/
async resizeDatabasePvc(app: Application, newSize: string): Promise<{ success: boolean; message: string }> {
const { coreApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const pvcName = `${app.name}-db`;
@@ -4070,7 +4024,7 @@ export class KubernetesService implements OnModuleInit {
*/
async getDatabasePvcSize(app: Application): Promise<string> {
try {
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const pvcName = `${app.name}-db`;
@@ -4096,7 +4050,7 @@ export class KubernetesService implements OnModuleInit {
totalAllocatedGb: number;
totalUsedGb: number;
}> {
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
const result = {
@@ -4271,7 +4225,7 @@ export class KubernetesService implements OnModuleInit {
* Returns usage in GB.
*/
private async getPvcUsageFromPod(app: Application, deploymentName: string, mountPath: string, namespace: string, containerName: string): Promise<number> {
const { coreApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const pods = await coreApi.listNamespacedPod({
namespace,
@@ -4305,7 +4259,7 @@ export class KubernetesService implements OnModuleInit {
* Expand a named PVC (Redis, RabbitMQ, or other optional service volumes).
*/
async resizeNamedPvc(app: Application, pvcName: string, newSize: string, label: string): Promise<{ success: boolean; message: string }> {
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
try {
@@ -4362,7 +4316,7 @@ export class KubernetesService implements OnModuleInit {
* Resize app storage PVC (all app types).
*/
async resizeAppStoragePvc(app: Application, newSize: string): Promise<{ success: boolean; message: string }> {
const { coreApi } = await this.getK8sClient(app.clusterId);
const { coreApi } = await this.k8sClientService.getK8sClient(app.clusterId);
const namespace = `user-${app.userId.split('-')[0]}`;
// Try new unified name first, then legacy wp-content name
@@ -4427,7 +4381,7 @@ export class KubernetesService implements OnModuleInit {
* Strategy: Run dump command, then sleep for 60s to allow exec retrieval.
*/
async exportDatabaseDump(app: Application, onProgress?: (percent: number) => void): Promise<{ data: Buffer | null; logs: string }> {
const { coreApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const batchApi = kc.makeApiClient(k8s.BatchV1Api);
const namespace = `user-${app.userId.split('-')[0]}`;
const dbName = `${app.name}-db`;
@@ -4608,7 +4562,7 @@ export class KubernetesService implements OnModuleInit {
* Strategy: Create archive, then sleep to allow exec retrieval.
*/
async archiveWpContent(app: Application): Promise<{ data: Buffer | null; logs: string }> {
const { coreApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const batchApi = kc.makeApiClient(k8s.BatchV1Api);
const namespace = `user-${app.userId.split('-')[0]}`;
const pvcName = `${app.name}-storage`;
@@ -4760,7 +4714,7 @@ export class KubernetesService implements OnModuleInit {
* Restore wp-content from a tar.gz archive into the WordPress PVC.
*/
async restoreWpContent(app: Application, archiveBuffer: Buffer): Promise<{ success: boolean; logs: string }> {
const { coreApi, kc } = await this.getK8sClient(app.clusterId);
const { coreApi, kc } = await this.k8sClientService.getK8sClient(app.clusterId);
const batchApi = kc.makeApiClient(k8s.BatchV1Api);
const namespace = `user-${app.userId.split('-')[0]}`;
const pvcName = `${app.name}-storage`;
@@ -4902,7 +4856,7 @@ export class KubernetesService implements OnModuleInit {
const releaseName = app.name;
try {
const kubeconfig = await this.getKubeconfig(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
const helmRevisions = await this.helmService.history(releaseName, namespace, kubeconfig);
if (!helmRevisions || helmRevisions.length === 0) {
@@ -4940,7 +4894,7 @@ export class KubernetesService implements OnModuleInit {
const releaseName = app.name;
try {
const kubeconfig = await this.getKubeconfig(app.clusterId);
const kubeconfig = await this.k8sClientService.getKubeconfig(app.clusterId);
await this.helmService.rollback(releaseName, targetRevision, namespace, kubeconfig);
this.logger.log(`Rolled back ${releaseName} to Helm revision ${targetRevision}`);
return {
backend/src/kubernetes/registry-gc.service.spec.ts
-25
View File
@@ -1,25 +0,0 @@
import { selectTagsToDelete } from './registry-gc.util';
describe('selectTagsToDelete', () => {
it('keeps the N newest numeric (Date.now) tags and deletes the rest', () => {
const tags = ['1000', '3000', '2000', '5000', '4000'];
const toDelete = selectTagsToDelete(tags, 3);
// newest 3 = 5000,4000,3000 → delete 2000,1000
expect(toDelete.sort()).toEqual(['1000', '2000']);
});
it('deletes nothing when tag count is within the keep limit', () => {
expect(selectTagsToDelete(['1000', '2000'], 3)).toEqual([]);
expect(selectTagsToDelete([], 3)).toEqual([]);
});
it('treats non-numeric tags as oldest (eligible for deletion first)', () => {
const tags = ['latest', '2000', '1000'];
// numeric newest kept first: 2000,1000 kept (keep=2) → delete latest
expect(selectTagsToDelete(tags, 2)).toEqual(['latest']);
});
it('keep=0 deletes every tag', () => {
expect(selectTagsToDelete(['1000', '2000'], 0).sort()).toEqual(['1000', '2000']);
});
});

Some files were not shown because too many files have changed in this diff Show More