Files

T

keyhan ee5bd0a291 Add Harbor/Ceph runbooks and align single-node Ceph docs with abr deployment.

Document registry ingress split, image mirroring, loop OSD setup, and platform integration paths.

2026-07-01 00:22:06 +03:30

3.8 KiB

Raw Blame History

CloudHost Ceph (Rook)

Helm chart and install scripts for Rook-Ceph on CloudHost clusters:

Layer	Purpose
rook-ceph-block	Expandable PVCs for apps, databases, registry
rook-ceph-bucket	S3-compatible storage for uploaded source zip archives

The chart does not vendor Rook itself — it installs the official rook-release charts and adds CloudHost-specific ObjectBucketClaim + credential sync.

Quick install

cd backend/helm/cloudhost-ceph
./scripts/install.sh single-node   # one-node k3s (current abr cluster)
# or
./scripts/install.sh multi-node    # production, 3+ nodes + raw disks

Verify:

./scripts/verify.sh

Profiles

`single-node`

OSD on loop device /dev/loop6 (15Gi file at /var/lib/rook/osd-loopback.img) — no spare raw disk required
Requires ROOK_CEPH_ALLOW_LOOP_DEVICES=true on the operator
Replication size: 1 (no HA)
Suitable for staging / single k3s node
Images must be pre-mirrored to registry.abrban.com (see RUNBOOK-HARBOR.fa.md)

`multi-node`

OSD on raw devices (useAllDevices: true)
Replication size: 3 for block + object metadata
Erasure-coded object data pool
Requires 3+ nodes and dedicated disks

What gets installed

Step	Release	Namespace
1	`rook-ceph` (operator)	`rook-ceph`
2	`rook-ceph-cluster`	`rook-ceph`
3	`cloudhost-ceph` (OBC + secrets)	`cloudhost-builds`

StorageClasses (from Rook)

Name	Use
`rook-ceph-block`	App PVC, DB PVC, Redis, registry, …
`rook-ceph-bucket`	`ObjectBucketClaim` → S3 bucket + credentials

CloudHost extras

Resource	Description
`ObjectBucketClaim/app-sources`	Bucket for user zip uploads
`Secret/ceph-app-sources-credentials`	Stable S3 credentials for backend
`ConfigMap/cloudhost-ceph-integration`	Suggested `PLATFORM_*` env values

Platform integration

After install, configure the backend:

PLATFORM_STORAGE_CLASS=rook-ceph-block
PLATFORM_CREATE_STORAGE_CLASS=false
PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com

Mount or env-from secret cloudhost-builds/ceph-app-sources-credentials:

SOURCE_STORAGE_ENDPOINT=http://rook-ceph-rgw-ceph-objectstore.rook-ceph.svc.cluster.local:80
SOURCE_STORAGE_REGION=us-east-1
SOURCE_STORAGE_BUCKET=<from secret>
SOURCE_STORAGE_ACCESS_KEY=<from secret>
SOURCE_STORAGE_SECRET_KEY=<from secret>

Note: Existing PVCs on local-path / cloudhost-expandable are not migrated automatically. New apps use rook-ceph-block once the backend env is updated. Plan migration per workload (see RUNBOOK-CEPH.fa.md).

Uninstall (destructive)

./scripts/uninstall.sh
# then on each node:
sudo rm -rf /var/lib/rook /var/lib/rook/osd

Troubleshooting

# Cluster health
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph status

# OSD pods
kubectl -n rook-ceph get pods -l app=rook-ceph-osd

# RGW (object store)
kubectl -n rook-ceph get pods -l app=rook-ceph-rgw

# Bucket sync job
kubectl -n cloudhost-builds logs job -l job-name=cloudhost-ceph-bucket-sync --tail=50

Full operational guide (Persian): ../../../RUNBOOK-CEPH.fa.md

Registry / Harbor (Persian): ../../../RUNBOOK-HARBOR.fa.md

Files

File	Role
`values-rook-cluster-single-node.yaml`	Rook cluster values (1 node)
`values-rook-cluster-multi-node.yaml`	Rook cluster values (production)
`values.yaml`	CloudHost OBC / secret sync
`scripts/install.sh`	Full install
`scripts/verify.sh`	Health check
`scripts/uninstall.sh`	Tear down

3.8 KiB Raw Blame History