Add Harbor/Ceph runbooks and align single-node Ceph docs with abr deployment.
Document registry ingress split, image mirroring, loop OSD setup, and platform integration paths.
This commit is contained in:
keyhan
@@ -0,0 +1,329 @@
|
|||||||
|
# راهنمای Ceph برای ابربان (Rook)
|
||||||
|
|
||||||
|
این سند نصب، معماری واقعی روی کلاستر **abr**، مدیریت روزمره و عیبیابی **Rook-Ceph** را پوشش میدهد.
|
||||||
|
|
||||||
|
- چارت و اسکریپتها: [`backend/helm/cloudhost-ceph/`](backend/helm/cloudhost-ceph/)
|
||||||
|
- README انگلیسی: [`backend/helm/cloudhost-ceph/README.md`](backend/helm/cloudhost-ceph/README.md)
|
||||||
|
- رجیستری: [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## چرا Ceph؟
|
||||||
|
|
||||||
|
| نیاز | راهحل Ceph |
|
||||||
|
|------|-------------|
|
||||||
|
| PVC اپ/دیتابیس با **resize** | Block pool → StorageClass `rook-ceph-block` |
|
||||||
|
| آپلود **zip** سورس کاربر | Object store (RGW) → StorageClass `rook-ceph-bucket` |
|
||||||
|
|
||||||
|
یک کلاستر Ceph هر دو را پوشش میدهد؛ zip را روی PVC نگه ندارید — از **bucket** استفاده کنید.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## معماری روی abr (single-node)
|
||||||
|
|
||||||
|
```
|
||||||
|
registry.abrban.com
|
||||||
|
├── rook/ceph:v1.20.1 → Rook operator
|
||||||
|
└── proxy-dockerhub/ceph/ceph:v19.2 → Ceph daemon (Squid)
|
||||||
|
|
||||||
|
Node abr
|
||||||
|
├── /dev/loop6 (15Gi) → OSD (bluestore raw)
|
||||||
|
├── mon-a, mgr-a, osd-0, rgw → rook-ceph namespace
|
||||||
|
└── RGW: rook-ceph-rgw-ceph-objectstore.rook-ceph.svc:80
|
||||||
|
```
|
||||||
|
|
||||||
|
| محدودیت | توضیح |
|
||||||
|
|---------|--------|
|
||||||
|
| **۱ OSD** | replication=1؛ بدون HA |
|
||||||
|
| **loop device** | دیسک خام نداریم؛ `/dev/loop6` از فایل `osd-loopback.img` |
|
||||||
|
| **HEALTH_WARN** | طبیعی: `OSD count 1 < default size 3`، mon low space |
|
||||||
|
| **ایمیجها** | باید از قبل در Harbor mirror شده باشند (kubelet به docker.io دسترسی ندارد) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## پیشنیازها
|
||||||
|
|
||||||
|
### پروفایل `single-node` (کلاستر فعلی abr)
|
||||||
|
|
||||||
|
- Kubernetes 1.28+ (k3s + Traefik)
|
||||||
|
- Harbor بالا و ایمیجهای `rook/ceph` + `ceph/ceph` mirror شده
|
||||||
|
- حداقل **۱۵ گیگ** فضا برای loop OSD (`/var/lib/rook/osd-loopback.img`)
|
||||||
|
- `helm` 3.x و `kubectl` با دسترسی cluster-admin
|
||||||
|
- Secret `registry-pull-secret` در `rook-ceph` با `harbor_registry_user`
|
||||||
|
|
||||||
|
### پروفایل `multi-node` (production)
|
||||||
|
|
||||||
|
- حداقل **۳ نود** + دیسک خام (raw)
|
||||||
|
- فایل values: `values-rook-cluster-multi-node.yaml`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## نصب (اولین بار — abr)
|
||||||
|
|
||||||
|
### ۱. آمادهسازی loop device برای OSD
|
||||||
|
|
||||||
|
روی نود تکدیسک، Rook به دیسک خام نیاز دارد. یک loop device بسازید:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# روی نود (یا Job privileged — یکبار)
|
||||||
|
truncate -s 15G /var/lib/rook/osd-loopback.img
|
||||||
|
losetup --find --show /var/lib/rook/osd-loopback.img # → /dev/loop6
|
||||||
|
```
|
||||||
|
|
||||||
|
### ۲. فعالسازی loop در Rook operator
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n rook-ceph patch cm rook-ceph-operator-config --type merge \
|
||||||
|
-p '{"data":{"ROOK_CEPH_ALLOW_LOOP_DEVICES":"true"}}'
|
||||||
|
kubectl -n rook-ceph rollout restart deploy/rook-ceph-operator
|
||||||
|
```
|
||||||
|
|
||||||
|
### ۳. mirror ایمیجها (قبل از نصب cluster)
|
||||||
|
|
||||||
|
| ایمیج | مسیر pull |
|
||||||
|
|-------|-----------|
|
||||||
|
| `rook/ceph:v1.20.1` | `registry.abrban.com/rook/ceph:v1.20.1` |
|
||||||
|
| `quay.io/ceph/ceph:v19.2` | `registry.abrban.com/proxy-dockerhub/ceph/ceph:v19.2` |
|
||||||
|
|
||||||
|
جزئیات mirror: [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md)
|
||||||
|
|
||||||
|
### ۴. نصب operator
|
||||||
|
|
||||||
|
```bash
|
||||||
|
helm repo add rook-release https://charts.rook.io/release
|
||||||
|
helm repo update rook-release
|
||||||
|
|
||||||
|
helm upgrade --install rook-ceph rook-release/rook-ceph \
|
||||||
|
-n rook-ceph --create-namespace \
|
||||||
|
--set image.repository=registry.abrban.com/rook/ceph \
|
||||||
|
--set image.tag=v1.20.1 \
|
||||||
|
--set imagePullSecrets[0].name=registry-pull-secret
|
||||||
|
```
|
||||||
|
|
||||||
|
### ۵. نصب cluster
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend/helm/cloudhost-ceph
|
||||||
|
|
||||||
|
helm upgrade --install rook-ceph-cluster rook-release/rook-ceph-cluster \
|
||||||
|
-n rook-ceph \
|
||||||
|
-f values-rook-cluster-single-node.yaml \
|
||||||
|
--set cephClusterSpec.cephVersion.image=registry.abrban.com/proxy-dockerhub/ceph/ceph:v19.2
|
||||||
|
```
|
||||||
|
|
||||||
|
> **توجه:** `values-rook-cluster-single-node.yaml` از `devices: [{name: "/dev/loop6"}]` استفاده میکند (نه directory — در Rook v1.20 حذف شده).
|
||||||
|
|
||||||
|
### ۶. extras (bucket + secret)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl create namespace cloudhost-builds 2>/dev/null || true
|
||||||
|
helm upgrade --install cloudhost-ceph . \
|
||||||
|
-n cloudhost-builds -f values.yaml --no-hooks
|
||||||
|
```
|
||||||
|
|
||||||
|
اگر Job `bucket-sync` بهخاطر `bitnami/kubectl` گیر کرد، secret را دستی بسازید:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost-builds get secret app-sources -o yaml
|
||||||
|
kubectl -n cloudhost-builds get cm app-sources -o yaml # BUCKET_NAME
|
||||||
|
# → secret ceph-app-sources-credentials (کلیدهای SOURCE_STORAGE_*)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ۷. یکپارچهسازی backend
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost set env deploy/cloudhost-backend \
|
||||||
|
PLATFORM_STORAGE_CLASS=rook-ceph-block \
|
||||||
|
PLATFORM_CREATE_STORAGE_CLASS=false \
|
||||||
|
PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com
|
||||||
|
|
||||||
|
kubectl -n cloudhost patch deploy cloudhost-backend --type=json \
|
||||||
|
-p '[{"op":"add","path":"/spec/template/spec/containers/0/envFrom","value":[{"secretRef":{"name":"ceph-app-sources-credentials"}}]}]'
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## اسکریپت `install.sh` (نصب تمیز)
|
||||||
|
|
||||||
|
برای نصب از صفر (بعد از آمادهسازی loop + mirror):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend/helm/cloudhost-ceph
|
||||||
|
./scripts/install.sh single-node
|
||||||
|
./scripts/verify.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
اسکریپت operator + cluster + extras را نصب میکند. روی abr حتماً **قبلش** loop device و mirror ایمیج را انجام دهید.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## خروجیهای مهم
|
||||||
|
|
||||||
|
### StorageClassها
|
||||||
|
|
||||||
|
| نام | کاربرد |
|
||||||
|
|-----|--------|
|
||||||
|
| `rook-ceph-block` | PVC اپ، DB، Redis، … |
|
||||||
|
| `rook-ceph-bucket` | claim کردن bucket برای zip |
|
||||||
|
|
||||||
|
### Secret پلتفرم
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost-builds get secret ceph-app-sources-credentials -o yaml
|
||||||
|
kubectl -n cloudhost get secret ceph-app-sources-credentials -o yaml # کپی در cloudhost
|
||||||
|
```
|
||||||
|
|
||||||
|
کلیدها: `SOURCE_STORAGE_ENDPOINT`, `SOURCE_STORAGE_BUCKET`, `SOURCE_STORAGE_ACCESS_KEY`, `SOURCE_STORAGE_SECRET_KEY`
|
||||||
|
|
||||||
|
### RGW endpoint
|
||||||
|
|
||||||
|
```
|
||||||
|
http://rook-ceph-rgw-ceph-objectstore.rook-ceph.svc.cluster.local:80
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## مدیریت روزمره
|
||||||
|
|
||||||
|
### سلامت کلاستر
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n rook-ceph get cephcluster
|
||||||
|
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph status
|
||||||
|
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph osd tree
|
||||||
|
kubectl get sc | grep rook-ceph
|
||||||
|
kubectl -n rook-ceph get pods
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dashboard
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath='{.data.password}' | base64 -d
|
||||||
|
kubectl -n rook-ceph port-forward svc/rook-ceph-mgr-dashboard 8443:8443
|
||||||
|
# https://localhost:8443
|
||||||
|
```
|
||||||
|
|
||||||
|
### bucket و OBC
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost-builds get obc app-sources
|
||||||
|
kubectl -n cloudhost-builds get cm app-sources
|
||||||
|
```
|
||||||
|
|
||||||
|
### PVC جدید با Ceph
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
storageClassName: rook-ceph-block
|
||||||
|
```
|
||||||
|
|
||||||
|
فقط **اپهای جدید** (یا بعد از migration) از این StorageClass استفاده میکنند. PVCهای قدیمی روی `local-path` / `cloudhost-expandable` خودکار منتقل نمیشوند.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## راهاندازی مجدد (reinstall)
|
||||||
|
|
||||||
|
### ۱. حذف Helm
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend/helm/cloudhost-ceph
|
||||||
|
./scripts/uninstall.sh
|
||||||
|
# تایپ: delete-ceph
|
||||||
|
```
|
||||||
|
|
||||||
|
### ۲. پاکسازی روی نود
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo losetup -d /dev/loop6 2>/dev/null || true
|
||||||
|
sudo rm -f /var/lib/rook/osd-loopback.img
|
||||||
|
sudo rm -rf /var/lib/rook
|
||||||
|
```
|
||||||
|
|
||||||
|
### ۳. نصب مجدد
|
||||||
|
|
||||||
|
loop device + mirror + `./scripts/install.sh single-node`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## عیبیابی
|
||||||
|
|
||||||
|
### CephCluster در `Progressing` / Detecting version
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n rook-ceph get pods | grep detect-version
|
||||||
|
kubectl -n rook-ceph describe pod -l job-name=rook-ceph-detect-version
|
||||||
|
```
|
||||||
|
|
||||||
|
| خطا | راهحل |
|
||||||
|
|-----|--------|
|
||||||
|
| `ceph/ceph:v19.2 not found` | mirror از quay.io؛ tag صحیح `v19.2` نه `v19.2.1` |
|
||||||
|
| pull timeout | اولین pull بزرگ است (~500MB)؛ صبر یا image را از قبل روی نود بکشید |
|
||||||
|
| Job `detect-version` Terminating گیر کرد | `kubectl -n rook-ceph delete job rook-ceph-detect-version --force --grace-period=0` |
|
||||||
|
|
||||||
|
### OSD بالا نمیآید (OSD count 0)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n rook-ceph logs -l app=rook-ceph-osd-prepare --tail=50
|
||||||
|
```
|
||||||
|
|
||||||
|
| خطا | راهحل |
|
||||||
|
|-----|--------|
|
||||||
|
| `unsupported diskType loop` | `ROOK_CEPH_ALLOW_LOOP_DEVICES=true` |
|
||||||
|
| `not picked by deviceFilter` | از `devices: [{name: "/dev/loop6"}]` استفاده کنید نه `deviceFilter` |
|
||||||
|
| `no devices matched` | `losetup -a` روی نود؛ loop6 وجود دارد؟ |
|
||||||
|
| `directories` در values | در Rook v1.20 کار نمیکند — loop یا raw disk |
|
||||||
|
|
||||||
|
### Volume mount روی rook-ceph-tools
|
||||||
|
|
||||||
|
`rook-ceph-mon-endpoints` و `rook-ceph-mon` تا قبل از بالا آمدن mon ساخته نمیشوند — طبیعی است؛ بعد از Ready برطرف میشود.
|
||||||
|
|
||||||
|
### Helm timeout روی apiserver
|
||||||
|
|
||||||
|
اگر `failed to download openapi` دیدید، بدون `--wait` نصب کنید و با `kubectl get cephcluster` پیگیری کنید.
|
||||||
|
|
||||||
|
### resize PVC
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl get storageclass rook-ceph-block -o yaml | grep allowVolumeExpansion
|
||||||
|
kubectl patch pvc <name> -n <ns> --type merge \
|
||||||
|
-p '{"spec":{"resources":{"requests":{"storage":"5Gi"}}}}'
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ارتقا (upgrade)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
helm repo update rook-release
|
||||||
|
|
||||||
|
helm upgrade rook-ceph rook-release/rook-ceph -n rook-ceph \
|
||||||
|
--reuse-values --timeout 10m
|
||||||
|
|
||||||
|
helm upgrade rook-ceph-cluster rook-release/rook-ceph-cluster \
|
||||||
|
-n rook-ceph \
|
||||||
|
-f values-rook-cluster-single-node.yaml \
|
||||||
|
--set cephClusterSpec.cephVersion.image=registry.abrban.com/proxy-dockerhub/ceph/ceph:v19.2
|
||||||
|
|
||||||
|
helm upgrade cloudhost-ceph . -n cloudhost-builds -f values.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
قبل از upgrade در production: [Rook upgrade guide](https://rook.io/docs/rook/latest/Upgrade/ceph-upgrade/) و snapshot.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## نکات امنیتی
|
||||||
|
|
||||||
|
- RGW داخل کلاستر HTTP است — برای دسترسی خارجی ingress + TLS اضافه کنید.
|
||||||
|
- Secret `ceph-app-sources-credentials` را فقط به backend بدهید.
|
||||||
|
- `single-node` + ۱ OSD فقط staging است؛ production نیاز به ۳+ نود و دیسک جدا دارد.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## چکلیست بعد از نصب
|
||||||
|
|
||||||
|
- [ ] `ceph status` — mon/mgr/osd/rgw up
|
||||||
|
- [ ] `rook-ceph-block` و `rook-ceph-bucket` در `kubectl get sc`
|
||||||
|
- [ ] `ceph-app-sources-credentials` در `cloudhost-builds` و `cloudhost`
|
||||||
|
- [ ] env بکاند: `PLATFORM_STORAGE_CLASS=rook-ceph-block`
|
||||||
|
- [ ] `SOURCE_STORAGE_*` در backend از secret خوانده میشود
|
||||||
|
- [ ] اپ تست با PVC جدید deploy شده
|
||||||
|
- [ ] ایمیجهای Rook در Harbor موجود و pull تست شده
|
||||||
@@ -0,0 +1,195 @@
|
|||||||
|
# راهنمای Harbor — `registry.abrban.com`
|
||||||
|
|
||||||
|
این سند معماری فعلی، نصب، مدیریت روزمره و عیبیابی **Harbor** روی کلاستر abr را پوشش میدهد.
|
||||||
|
|
||||||
|
- چارت/values: [`backend/helm/cloudhost-harbor/`](backend/helm/cloudhost-harbor/)
|
||||||
|
- اسکریپت نصب: [`backend/helm/cloudhost-harbor/scripts/install-harbor-registry.sh`](backend/helm/cloudhost-harbor/scripts/install-harbor-registry.sh)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## معماری فعلی (خلاصه)
|
||||||
|
|
||||||
|
```
|
||||||
|
registry.abrban.com (Traefik + TLS wildcard)
|
||||||
|
├── / → harbor-portal (UI)
|
||||||
|
├── /api/, /service/, /c/ → harbor-core (API + auth)
|
||||||
|
├── /v2/proxy-dockerhub/ → harbor-registry (ایمیجهای mirrorشده Ceph/Rook)
|
||||||
|
├── /v2/rook/ → harbor-registry
|
||||||
|
└── /v2/* → registry قدیمی (ایمیجهای platform: backend, nixpacks, …)
|
||||||
|
```
|
||||||
|
|
||||||
|
| کامپوننت | نقش |
|
||||||
|
|----------|-----|
|
||||||
|
| **Harbor** | UI، proxy-cache، ذخیره ایمیجهای جدید |
|
||||||
|
| **registry قدیمی** (`Deployment/registry`) | هنوز بالاست؛ ایمیجهای platform قبل از Harbor اینجاست |
|
||||||
|
| **registry-egress-proxy** | secret با `HTTP_PROXY` / `HTTPS_PROXY` برای pull از docker.io/quay از داخل کلاستر |
|
||||||
|
| **registry-pull-secret** | auth kubelet برای pull از `registry.abrban.com` |
|
||||||
|
|
||||||
|
> Harbor و registry قدیمی **همزمان** روی یک hostname هستند؛ مسیر `/v2/` با Ingress split میشود.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## پیشنیازها
|
||||||
|
|
||||||
|
- Secret `abrban-wildcard-tls` در namespace `cloudhost`
|
||||||
|
- Secret `registry-egress-proxy` در namespace `cloudhost` (پروکسی egress)
|
||||||
|
- Helm repo: `helm repo add harbor https://helm.goharbor.io`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## نصب / ارتقا
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend/helm/cloudhost-harbor
|
||||||
|
./scripts/install-harbor-registry.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
اسکریپت:
|
||||||
|
1. Harbor را با values + proxy از `registry-egress-proxy` نصب میکند
|
||||||
|
2. Ingress قدیمی `registry` را حذف میکند (بعد از نصب باید دستی دوباره route شود — بخش Ingress)
|
||||||
|
3. `Deployment/registry` را scale به 0 میکند (برای rollback نگه داشته میشود)
|
||||||
|
|
||||||
|
### Ingress بعد از نصب (الزامی)
|
||||||
|
|
||||||
|
Harbor به **چند مسیر** نیاز دارد. Ingress نهایی باید شبیه این باشد:
|
||||||
|
|
||||||
|
| Path | Service | Port |
|
||||||
|
|------|---------|------|
|
||||||
|
| `/` | `harbor-portal` | 80 |
|
||||||
|
| `/api/`, `/service/`, `/c/`, `/chartrepo/` | `harbor-core` | 80 |
|
||||||
|
| `/v2/proxy-dockerhub/`, `/v2/rook/` | `harbor-registry` | 5000 |
|
||||||
|
| `/v2/` (بقیه) | `registry` (قدیمی) | 5000 |
|
||||||
|
|
||||||
|
بدون split روی `/v2/`، یا UI 404 میدهد یا kubelet ایمیج platform را پیدا نمیکند.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## دسترسی و credentialها
|
||||||
|
|
||||||
|
| کاربرد | کاربر | منبع |
|
||||||
|
|--------|-------|------|
|
||||||
|
| UI / API مدیریت | `admin` | `kubectl -n cloudhost get secret harbor-core -o jsonpath='{.data.HARBOR_ADMIN_PASSWORD}' \| base64 -d` |
|
||||||
|
| push/pull داخلی به harbor-registry | `harbor_registry_user` | secret `harbor-core` → `REGISTRY_CREDENTIAL_PASSWORD` |
|
||||||
|
| pull kubelet (ایمیجهای platform) | `cloudhost` | secret `registry-pull-secret` (namespace `cloudhost`) |
|
||||||
|
| pull kubelet (ایمیجهای Rook/Ceph) | `harbor_registry_user` | secret `registry-pull-secret` (namespace `rook-ceph`) |
|
||||||
|
|
||||||
|
URL: https://registry.abrban.com/
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## پروژههای proxy-cache
|
||||||
|
|
||||||
|
| پروژه | upstream | کاربرد |
|
||||||
|
|-------|----------|--------|
|
||||||
|
| `proxy-dockerhub` | docker.io | Rook، Ceph، bitnami، … |
|
||||||
|
| `proxy-quay` | quay.io | cephcsi و … |
|
||||||
|
| `proxy-k8s` | registry.k8s.io | CSI sidecarها |
|
||||||
|
|
||||||
|
ایجاد از UI: **Administration → Registries → New Endpoint** سپس **Projects → New Project** با نوع Proxy Cache.
|
||||||
|
|
||||||
|
> health check بعضی endpointها (مثلاً quay) از UI timeout میخورد؛ از داخل `harbor-core` با curl و proxy ممکن است OK باشد. در صورت نیاز endpoint را با type `docker-registry` بسازید.
|
||||||
|
|
||||||
|
### proxy در Harbor
|
||||||
|
|
||||||
|
پروکسی از secret `registry-egress-proxy` در ConfigMapهای `harbor-core` و `harbor-jobservice-env` تزریق میشود.
|
||||||
|
|
||||||
|
**مهم:** Go (harbor-core/jobservice) به `http_proxy` / `https_proxy` **lowercase** هم نیاز دارد. اگر health check upstream `unhealthy` ماند، هر دو حالت uppercase و lowercase را در ConfigMap بگذارید و podها را restart کنید:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost rollout restart deploy/harbor-core deploy/harbor-jobservice
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## mirror دستی ایمیج (وقتی proxy-cache کار نمیکند)
|
||||||
|
|
||||||
|
روی abr، pull مستقیم از docker.io/quay از kubelet ممکن نیست. ایمیجهای حیاتی را با Job داخل کلاستر mirror کنید:
|
||||||
|
|
||||||
|
**مقصد push:** `harbor-registry.cloudhost.svc.cluster.local:5000` (HTTP، با `harbor_registry_user`)
|
||||||
|
|
||||||
|
**مثال مسیرها در registry:**
|
||||||
|
|
||||||
|
| ایمیج upstream | مسیر در registry |
|
||||||
|
|----------------|------------------|
|
||||||
|
| `rook/ceph:v1.20.1` | `rook/ceph:v1.20.1` |
|
||||||
|
| `quay.io/ceph/ceph:v19.2` | `proxy-dockerhub/ceph/ceph:v19.2` |
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# لیست ایمیجهای داخل harbor-registry
|
||||||
|
kubectl -n cloudhost exec deploy/harbor-portal -- \
|
||||||
|
curl -s -u "harbor_registry_user:$(kubectl -n cloudhost get secret harbor-core -o jsonpath='{.data.REGISTRY_CREDENTIAL_PASSWORD}' | base64 -d)" \
|
||||||
|
http://harbor-registry:5000/v2/_catalog
|
||||||
|
```
|
||||||
|
|
||||||
|
> push مستقیم به `harbor-registry:5000` metadata در Harbor UI را بهروز نمیکند؛ برای kubelet کافی است چون `/v2/` به harbor-registry route شده.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## مدیریت روزمره
|
||||||
|
|
||||||
|
### وضعیت
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost get pods | grep harbor
|
||||||
|
kubectl -n cloudhost get ingress registry -o yaml | grep -A3 'path:'
|
||||||
|
curl -sk -o /dev/null -w "%{http_code}\n" https://registry.abrban.com/
|
||||||
|
curl -sk -u admin:<pass> https://registry.abrban.com/api/v2.0/systeminfo
|
||||||
|
```
|
||||||
|
|
||||||
|
### لاگها
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost logs deploy/harbor-core --tail=50
|
||||||
|
kubectl -n cloudhost logs deploy/harbor-jobservice --tail=50
|
||||||
|
kubectl -n cloudhost logs deploy/harbor-registry -c registry --tail=50
|
||||||
|
```
|
||||||
|
|
||||||
|
### ارتقا Harbor
|
||||||
|
|
||||||
|
```bash
|
||||||
|
helm repo update harbor
|
||||||
|
./scripts/install-harbor-registry.sh
|
||||||
|
# Ingress split را دوباره تأیید کنید
|
||||||
|
```
|
||||||
|
|
||||||
|
### rollback به registry قدیمی
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n cloudhost scale deploy/registry --replicas=1
|
||||||
|
# Ingress را فقط به service registry:5000 برگردانید
|
||||||
|
helm uninstall harbor -n cloudhost
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## عیبیابی
|
||||||
|
|
||||||
|
| علامت | علت محتمل | اقدام |
|
||||||
|
|-------|-----------|--------|
|
||||||
|
| `https://registry.abrban.com/` → 404 | Ingress فقط به `harbor-core` وصل است | `/` → `harbor-portal` |
|
||||||
|
| `ImagePullBackOff` برای `cloudhost-backend` | `/v2/` به Harbor رفته، ایمیج platform آنجا نیست | `/v2/` (عمومی) → `registry` قدیمی |
|
||||||
|
| `not found` برای `proxy-dockerhub/...` | ایمیج mirror نشده | Job skopeo یا proxy-cache |
|
||||||
|
| push با 499/503 | Traefik timeout | push از داخل کلاستر به `harbor-registry:5000` |
|
||||||
|
| registry endpoint `unhealthy` | proxy lowercase یا timeout health check | patch ConfigMap + restart؛ یا mirror دستی |
|
||||||
|
| `authentication required` روی pull | pull secret اشتباه namespace | `cloudhost` vs `harbor_registry_user` در `rook-ceph` |
|
||||||
|
|
||||||
|
### تست pull
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# platform (cloudhost user)
|
||||||
|
curl -sk -u "cloudhost:<pass>" https://registry.abrban.com/v2/cloudhost-backend/tags/list
|
||||||
|
|
||||||
|
# rook/ceph (harbor_registry_user)
|
||||||
|
curl -sk -u "harbor_registry_user:<pass>" https://registry.abrban.com/v2/rook/ceph/tags/list
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## چکلیست بعد از نصب
|
||||||
|
|
||||||
|
- [ ] Portal روی `/` پاسخ 200
|
||||||
|
- [ ] `/api/v2.0/systeminfo` پاسخ JSON
|
||||||
|
- [ ] Ingress split `/v2/` درست است
|
||||||
|
- [ ] پروژههای `proxy-dockerhub`, `proxy-k8s`, `proxy-quay` ساخته شده
|
||||||
|
- [ ] `registry-pull-secret` در `cloudhost` و `rook-ceph` بهروز است
|
||||||
|
- [ ] ایمیجهای Rook/Ceph mirror شده و pull تست شده
|
||||||
+3
-2
@@ -21,7 +21,8 @@
|
|||||||
| **Backend** | NestJS (REST `/api/v1`) | منطق کسبوکار، ساخت اپ، احراز هویت |
|
| **Backend** | NestJS (REST `/api/v1`) | منطق کسبوکار، ساخت اپ، احراز هویت |
|
||||||
| **Postgres** | postgres:16 | دیتابیس اصلی (کاربر، اپ، فاکتور، …) |
|
| **Postgres** | postgres:16 | دیتابیس اصلی (کاربر، اپ، فاکتور، …) |
|
||||||
| **Redis** | redis:7 | کش، صف Bull (مهاجرت اپ، دسترسی موقت)، پیشرفت بیلد |
|
| **Redis** | redis:7 | کش، صف Bull (مهاجرت اپ، دسترسی موقت)، پیشرفت بیلد |
|
||||||
| **Registry داخلی** | registry:2 | ایمیجهای buildشده |
|
| **Registry داخلی** | Harbor + registry:2 (legacy) | ایمیجهای build و platform؛ جزئیات: [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md) |
|
||||||
|
| **Storage (Ceph)** | Rook-Ceph | PVC (`rook-ceph-block`) + bucket zip (`rook-ceph-bucket`)؛ جزئیات: [`RUNBOOK-CEPH.fa.md`](RUNBOOK-CEPH.fa.md) |
|
||||||
| **Build pipeline** | Kaniko | تبدیل سورس به ایمیج Docker داخل کلاستر (بدون Docker daemon) |
|
| **Build pipeline** | Kaniko | تبدیل سورس به ایمیج Docker داخل کلاستر (بدون Docker daemon) |
|
||||||
| **Kubernetes** | k3s (تکنود) | اجرای همهی موارد بالا + اپهای کاربر |
|
| **Kubernetes** | k3s (تکنود) | اجرای همهی موارد بالا + اپهای کاربر |
|
||||||
|
|
||||||
@@ -29,7 +30,7 @@
|
|||||||
- `abrban.com` → لندینگ
|
- `abrban.com` → لندینگ
|
||||||
- `panel.abrban.com` → پنل احرازشده
|
- `panel.abrban.com` → پنل احرازشده
|
||||||
- `api.abrban.com` → بکاند
|
- `api.abrban.com` → بکاند
|
||||||
- `registry.abrban.com` → رجیستری داخلی (pull توسط kubelet)
|
- `registry.abrban.com` → Harbor (UI + proxy-cache) + registry قدیمی برای ایمیجهای platform — [`RUNBOOK-HARBOR.fa.md`](RUNBOOK-HARBOR.fa.md)
|
||||||
- `apps.abrban.com` → دامنهی پیشفرض اپهای کاربر
|
- `apps.abrban.com` → دامنهی پیشفرض اپهای کاربر
|
||||||
|
|
||||||
### ۱.۴ جریان احراز هویت
|
### ۱.۴ جریان احراز هویت
|
||||||
|
|||||||
@@ -0,0 +1,3 @@
|
|||||||
|
.rook-cluster-values-ref.yaml
|
||||||
|
*.md
|
||||||
|
scripts/
|
||||||
@@ -0,0 +1,758 @@
|
|||||||
|
# Default values for a single rook-ceph cluster
|
||||||
|
# This is a YAML-formatted file.
|
||||||
|
# Declare variables to be passed into your templates.
|
||||||
|
|
||||||
|
# -- Namespace of the main rook operator
|
||||||
|
operatorNamespace: rook-ceph
|
||||||
|
|
||||||
|
# -- The metadata.name of the CephCluster CR
|
||||||
|
# @default -- The same as the namespace
|
||||||
|
clusterName:
|
||||||
|
|
||||||
|
# -- Optional override of the target kubernetes version
|
||||||
|
kubeVersion:
|
||||||
|
|
||||||
|
# -- Cluster ceph.conf override
|
||||||
|
configOverride:
|
||||||
|
# configOverride: |
|
||||||
|
# [global]
|
||||||
|
# mon_allow_pool_delete = true
|
||||||
|
# osd_pool_default_size = 3
|
||||||
|
# osd_pool_default_min_size = 2
|
||||||
|
|
||||||
|
# Installs a debugging toolbox deployment
|
||||||
|
toolbox:
|
||||||
|
# -- Enable Ceph debugging pod deployment. See [toolbox](../Troubleshooting/ceph-toolbox.md)
|
||||||
|
enabled: false
|
||||||
|
# -- Toolbox image, defaults to the image used by the Ceph cluster
|
||||||
|
image: #quay.io/ceph/ceph:v20.2.1
|
||||||
|
# -- Toolbox tolerations
|
||||||
|
tolerations: []
|
||||||
|
# -- Toolbox affinity
|
||||||
|
affinity: {}
|
||||||
|
# -- Toolbox labels
|
||||||
|
labels: {}
|
||||||
|
# -- Toolbox container security context
|
||||||
|
containerSecurityContext:
|
||||||
|
runAsNonRoot: true
|
||||||
|
runAsUser: 2016
|
||||||
|
runAsGroup: 2016
|
||||||
|
capabilities:
|
||||||
|
drop: ["ALL"]
|
||||||
|
# -- Toolbox resources
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "100m"
|
||||||
|
memory: "128Mi"
|
||||||
|
# -- Set the priority class for the toolbox if desired
|
||||||
|
priorityClassName:
|
||||||
|
|
||||||
|
monitoring:
|
||||||
|
# -- Enable Prometheus integration, will also create necessary RBAC rules to allow Operator to create ServiceMonitors.
|
||||||
|
# Monitoring requires Prometheus to be pre-installed
|
||||||
|
enabled: false
|
||||||
|
# -- Whether to disable the metrics reported by Ceph. If false, the prometheus mgr module and Ceph exporter are enabled
|
||||||
|
metricsDisabled: false
|
||||||
|
# -- Whether to create the Prometheus rules for Ceph alerts
|
||||||
|
createPrometheusRules: false
|
||||||
|
# -- Edit Prometheus rules for Ceph alerts
|
||||||
|
prometheusRuleOverrides: {}
|
||||||
|
# CephHealthWarning:
|
||||||
|
# disabled: true
|
||||||
|
# NVMeoFHighWriteLatency:
|
||||||
|
# for: 3m
|
||||||
|
# labels:
|
||||||
|
# severity: critical
|
||||||
|
# -- The namespace in which to create the prometheus rules, if different from the rook cluster namespace.
|
||||||
|
# If you have multiple rook-ceph clusters in the same k8s cluster, choose the same namespace (ideally, namespace with prometheus
|
||||||
|
# deployed) to set rulesNamespaceOverride for all the clusters. Otherwise, you will get duplicate alerts with multiple alert definitions.
|
||||||
|
rulesNamespaceOverride:
|
||||||
|
# Monitoring settings for external clusters:
|
||||||
|
# externalMgrEndpoints: <list of endpoints>
|
||||||
|
# externalMgrPrometheusPort: <port>
|
||||||
|
# Scrape interval for prometheus
|
||||||
|
# interval: 10s
|
||||||
|
# allow adding custom labels and annotations to the prometheus rule
|
||||||
|
prometheusRule:
|
||||||
|
# -- Labels applied to PrometheusRule
|
||||||
|
labels: {}
|
||||||
|
# -- Annotations applied to PrometheusRule
|
||||||
|
annotations: {}
|
||||||
|
|
||||||
|
# imagePullSecrets option allow to pull docker images from private docker registry. Option will be passed to all service accounts.
|
||||||
|
# imagePullSecrets:
|
||||||
|
# - name: my-registry-secret
|
||||||
|
|
||||||
|
# Labels and annotations to add to the CephCluster CR
|
||||||
|
cephClusterMetadata:
|
||||||
|
annotations: {}
|
||||||
|
labels: {}
|
||||||
|
|
||||||
|
# Specify these values to override the Ceph image in the cephClusterSpec below.
|
||||||
|
# If specifying these values, do not include the cephVersion section in the cephClusterSpec.
|
||||||
|
cephImage:
|
||||||
|
# The repository from which to pull the ceph image
|
||||||
|
repository: quay.io/ceph/ceph
|
||||||
|
# In production, use a specific version tag instead of the general v20 flag, which pulls the latest release and could result in different
|
||||||
|
# versions running within the cluster. See tags available at https://hub.docker.com/r/ceph/ceph/tags/.
|
||||||
|
# To be more precise, you can always use a timestamp tag such as quay.io/ceph/ceph:v20.2.1-20260402
|
||||||
|
tag: v20.2.1
|
||||||
|
# Whether to allow unsupported versions of Ceph. Currently Squid and Tentacle are supported.
|
||||||
|
# Future versions would require this to be set to `true`.
|
||||||
|
# Do not set to true in production.
|
||||||
|
allowUnsupported: false
|
||||||
|
# The image pull policy for pulling the ceph image in the ceph daemon pods, defaults to IfNotPresent
|
||||||
|
# imagePullPolicy: IfNotPresent
|
||||||
|
|
||||||
|
# All values below are taken from the CephCluster CRD
|
||||||
|
# -- Cluster configuration.
|
||||||
|
# @default -- See [below](#ceph-cluster-spec)
|
||||||
|
cephClusterSpec:
|
||||||
|
# This cluster spec example is for a converged cluster where all the Ceph daemons are running locally,
|
||||||
|
# as in the host-based example (cluster.yaml). For a different configuration such as a
|
||||||
|
# PVC-based cluster (cluster-on-pvc.yaml), external cluster (cluster-external.yaml),
|
||||||
|
# or stretch cluster (cluster-stretched.yaml), replace this entire `cephClusterSpec`
|
||||||
|
# with the specs from those examples.
|
||||||
|
# For more details, check https://rook.io/docs/rook/v1.10/CRDs/Cluster/ceph-cluster-crd/
|
||||||
|
|
||||||
|
# The path on the host where configuration files will be persisted. Must be specified. If there are multiple clusters, the directory must be unique for each cluster.
|
||||||
|
# Important: if you reinstall the cluster, make sure you delete this directory from each host or else the mons will fail to start on the new cluster.
|
||||||
|
# In Minikube, the '/data' directory is configured to persist across reboots. Use "/data/rook" in Minikube environment.
|
||||||
|
dataDirHostPath: /var/lib/rook
|
||||||
|
|
||||||
|
# Whether or not upgrade should continue even if a check fails
|
||||||
|
# This means Ceph's status could be degraded and we don't recommend upgrading but you might decide otherwise
|
||||||
|
# Use at your OWN risk
|
||||||
|
# To understand Rook's upgrade process of Ceph, read https://rook.io/docs/rook/v1.10/Upgrade/ceph-upgrade/
|
||||||
|
skipUpgradeChecks: false
|
||||||
|
|
||||||
|
# Whether or not continue if PGs are not clean during an upgrade
|
||||||
|
continueUpgradeAfterChecksEvenIfNotHealthy: false
|
||||||
|
|
||||||
|
# WaitTimeoutForHealthyOSDInMinutes defines the time (in minutes) the operator would wait before an OSD can be stopped for upgrade or restart.
|
||||||
|
# If the timeout exceeds and OSD is not ok to stop, then the operator would skip upgrade for the current OSD and proceed with the next one
|
||||||
|
# if `continueUpgradeAfterChecksEvenIfNotHealthy` is `false`. If `continueUpgradeAfterChecksEvenIfNotHealthy` is `true`, then operator would
|
||||||
|
# continue with the upgrade of an OSD even if its not ok to stop after the timeout. This timeout won't be applied if `skipUpgradeChecks` is `true`.
|
||||||
|
# The default wait timeout is 10 minutes.
|
||||||
|
waitTimeoutForHealthyOSDInMinutes: 10
|
||||||
|
|
||||||
|
# Whether or not requires PGs are clean before an OSD upgrade. If set to `true` OSD upgrade process won't start until PGs are healthy.
|
||||||
|
# This configuration will be ignored if `skipUpgradeChecks` is `true`.
|
||||||
|
# Default is false.
|
||||||
|
upgradeOSDRequiresHealthyPGs: false
|
||||||
|
|
||||||
|
mon:
|
||||||
|
# Set the number of mons to be started. Generally recommended to be 3.
|
||||||
|
# For highest availability, an odd number of mons should be specified.
|
||||||
|
count: 3
|
||||||
|
# The mons should be on unique nodes. For production, at least 3 nodes are recommended for this reason.
|
||||||
|
# Mons should only be allowed on the same node for test environments where data loss is acceptable.
|
||||||
|
allowMultiplePerNode: false
|
||||||
|
|
||||||
|
mgr:
|
||||||
|
# When higher availability of the mgr is needed, increase the count to 2.
|
||||||
|
# In that case, one mgr will be active and one in standby. When Ceph updates which
|
||||||
|
# mgr is active, Rook will update the mgr services to match the active mgr.
|
||||||
|
count: 2
|
||||||
|
allowMultiplePerNode: false
|
||||||
|
modules:
|
||||||
|
# List of modules to optionally enable or disable.
|
||||||
|
# Note the "dashboard" and "monitoring" modules are already configured by other settings in the cluster CR.
|
||||||
|
# - name: rook
|
||||||
|
# enabled: true
|
||||||
|
|
||||||
|
# enable the ceph dashboard for viewing cluster status
|
||||||
|
dashboard:
|
||||||
|
enabled: true
|
||||||
|
# serve the dashboard under a subpath (useful when you are accessing the dashboard via a reverse proxy)
|
||||||
|
# urlPrefix: /ceph-dashboard
|
||||||
|
# serve the dashboard at the given port.
|
||||||
|
# port: 8443
|
||||||
|
# Serve the dashboard using SSL (if using ingress to expose the dashboard and `ssl: true` you need to set
|
||||||
|
# the corresponding "backend protocol" annotation(s) for your ingress controller of choice)
|
||||||
|
ssl: true
|
||||||
|
|
||||||
|
# Network configuration, see: https://github.com/rook/rook/blob/master/Documentation/CRDs/Cluster/ceph-cluster-crd.md#network-configuration-settings
|
||||||
|
network:
|
||||||
|
connections:
|
||||||
|
# Whether to encrypt the data in transit across the wire to prevent eavesdropping the data on the network.
|
||||||
|
# The default is false. When encryption is enabled, all communication between clients and Ceph daemons, or between Ceph daemons will be encrypted.
|
||||||
|
# When encryption is not enabled, clients still establish a strong initial authentication and data integrity is still validated with a crc check.
|
||||||
|
# IMPORTANT: Encryption requires the 5.11 kernel for the latest nbd and cephfs drivers. Alternatively for testing only,
|
||||||
|
# you can set the "mounter: rbd-nbd" in the rbd storage class, or "mounter: fuse" in the cephfs storage class.
|
||||||
|
# The nbd and fuse drivers are *not* recommended in production since restarting the csi driver pod will disconnect the volumes.
|
||||||
|
encryption:
|
||||||
|
enabled: false
|
||||||
|
# Whether to compress the data in transit across the wire. The default is false.
|
||||||
|
# The kernel requirements above for encryption also apply to compression.
|
||||||
|
compression:
|
||||||
|
enabled: false
|
||||||
|
# Whether to require communication over msgr2. If true, the msgr v1 port (6789) will be disabled
|
||||||
|
# and clients will be required to connect to the Ceph cluster with the v2 port (3300).
|
||||||
|
# Requires a kernel that supports msgr v2 (kernel 5.11 or CentOS 8.4 or newer).
|
||||||
|
requireMsgr2: false
|
||||||
|
# # enable host networking
|
||||||
|
# provider: host
|
||||||
|
# # EXPERIMENTAL: enable the Multus network provider
|
||||||
|
# provider: multus
|
||||||
|
# selectors:
|
||||||
|
# # The selector keys are required to be `public` and `cluster`.
|
||||||
|
# # Based on the configuration, the operator will do the following:
|
||||||
|
# # 1. if only the `public` selector key is specified both public_network and cluster_network Ceph settings will listen on that interface
|
||||||
|
# # 2. if both `public` and `cluster` selector keys are specified the first one will point to 'public_network' flag and the second one to 'cluster_network'
|
||||||
|
# #
|
||||||
|
# # In order to work, each selector value must match a NetworkAttachmentDefinition object in Multus
|
||||||
|
# #
|
||||||
|
# # public: public-conf --> NetworkAttachmentDefinition object name in Multus
|
||||||
|
# # cluster: cluster-conf --> NetworkAttachmentDefinition object name in Multus
|
||||||
|
# # Provide internet protocol version. IPv6, IPv4 or empty string are valid options. Empty string would mean IPv4
|
||||||
|
# ipFamily: "IPv6"
|
||||||
|
# # Ceph daemons to listen on both IPv4 and Ipv6 networks
|
||||||
|
# dualStack: false
|
||||||
|
|
||||||
|
# enable the crash collector for ceph daemon crash collection
|
||||||
|
crashCollector:
|
||||||
|
disable: false
|
||||||
|
# Uncomment daysToRetain to prune ceph crash entries older than the
|
||||||
|
# specified number of days.
|
||||||
|
# daysToRetain: 30
|
||||||
|
|
||||||
|
# enable log collector, daemons will log on files and rotate
|
||||||
|
logCollector:
|
||||||
|
enabled: true
|
||||||
|
periodicity: daily # one of: hourly, daily, weekly, monthly
|
||||||
|
maxLogSize: 500M # SUFFIX may be 'M' or 'G'. Must be at least 1M.
|
||||||
|
|
||||||
|
# automate [data cleanup process](https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/ceph-teardown.md#delete-the-data-on-hosts) in cluster destruction.
|
||||||
|
cleanupPolicy:
|
||||||
|
# Since cluster cleanup is destructive to data, confirmation is required.
|
||||||
|
# To destroy all Rook data on hosts during uninstall, confirmation must be set to "yes-really-destroy-data".
|
||||||
|
# This value should only be set when the cluster is about to be deleted. After the confirmation is set,
|
||||||
|
# Rook will immediately stop configuring the cluster and only wait for the delete command.
|
||||||
|
# If the empty string is set, Rook will not destroy any data on hosts during uninstall.
|
||||||
|
confirmation: ""
|
||||||
|
# sanitizeDisks represents settings for sanitizing OSD disks on cluster deletion
|
||||||
|
sanitizeDisks:
|
||||||
|
# method indicates if the entire disk should be sanitized or simply ceph's metadata
|
||||||
|
# in both case, re-install is possible
|
||||||
|
# possible choices are 'complete' or 'quick' (default)
|
||||||
|
method: quick
|
||||||
|
# dataSource indicate where to get random bytes from to write on the disk
|
||||||
|
# possible choices are 'zero' (default) or 'random'
|
||||||
|
# using random sources will consume entropy from the system and will take much more time then the zero source
|
||||||
|
dataSource: zero
|
||||||
|
# iteration overwrite N times instead of the default (1)
|
||||||
|
# takes an integer value
|
||||||
|
iteration: 1
|
||||||
|
# allowUninstallWithVolumes defines how the uninstall should be performed
|
||||||
|
# If set to true, cephCluster deletion does not wait for the PVs to be deleted.
|
||||||
|
allowUninstallWithVolumes: false
|
||||||
|
|
||||||
|
# To control where various services will be scheduled by kubernetes, use the placement configuration sections below.
|
||||||
|
# The example under 'all' would have all services scheduled on kubernetes nodes labeled with 'role=storage-node' and
|
||||||
|
# tolerate taints with a key of 'storage-node'.
|
||||||
|
# placement:
|
||||||
|
# all:
|
||||||
|
# nodeAffinity:
|
||||||
|
# requiredDuringSchedulingIgnoredDuringExecution:
|
||||||
|
# nodeSelectorTerms:
|
||||||
|
# - matchExpressions:
|
||||||
|
# - key: role
|
||||||
|
# operator: In
|
||||||
|
# values:
|
||||||
|
# - storage-node
|
||||||
|
# podAffinity:
|
||||||
|
# podAntiAffinity:
|
||||||
|
# topologySpreadConstraints:
|
||||||
|
# tolerations:
|
||||||
|
# - key: storage-node
|
||||||
|
# operator: Exists
|
||||||
|
# # The above placement information can also be specified for mon, osd, and mgr components
|
||||||
|
# mon:
|
||||||
|
# # Monitor deployments may contain an anti-affinity rule for avoiding monitor
|
||||||
|
# # collocation on the same node. This is a required rule when host network is used
|
||||||
|
# # or when AllowMultiplePerNode is false. Otherwise this anti-affinity rule is a
|
||||||
|
# # preferred rule with weight: 50.
|
||||||
|
# osd:
|
||||||
|
# mgr:
|
||||||
|
# cleanup:
|
||||||
|
|
||||||
|
# annotations:
|
||||||
|
# all:
|
||||||
|
# mon:
|
||||||
|
# osd:
|
||||||
|
# cleanup:
|
||||||
|
# prepareosd:
|
||||||
|
# # If no mgr annotations are set, prometheus scrape annotations will be set by default.
|
||||||
|
# mgr:
|
||||||
|
# dashboard:
|
||||||
|
|
||||||
|
# labels:
|
||||||
|
# all:
|
||||||
|
# mon:
|
||||||
|
# osd:
|
||||||
|
# cleanup:
|
||||||
|
# mgr:
|
||||||
|
# prepareosd:
|
||||||
|
# # monitoring is a list of key-value pairs. It is injected into all the monitoring resources created by operator.
|
||||||
|
# # These labels can be passed as LabelSelector to Prometheus
|
||||||
|
# monitoring:
|
||||||
|
# dashboard:
|
||||||
|
|
||||||
|
resources:
|
||||||
|
mgr:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "500m"
|
||||||
|
memory: "512Mi"
|
||||||
|
mon:
|
||||||
|
limits:
|
||||||
|
memory: "2Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "1000m"
|
||||||
|
memory: "1Gi"
|
||||||
|
osd:
|
||||||
|
limits:
|
||||||
|
memory: "4Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "1000m"
|
||||||
|
memory: "4Gi"
|
||||||
|
prepareosd:
|
||||||
|
# limits: It is not recommended to set limits on the OSD prepare job
|
||||||
|
# since it's a one-time burst for memory that must be allowed to
|
||||||
|
# complete without an OOM kill. Note however that if a k8s
|
||||||
|
# limitRange guardrail is defined external to Rook, the lack of
|
||||||
|
# a limit here may result in a sync failure, in which case a
|
||||||
|
# limit should be added. 1200Mi may suffice for up to 15Ti
|
||||||
|
# OSDs ; for larger devices 2Gi may be required.
|
||||||
|
# cf. https://github.com/rook/rook/pull/11103
|
||||||
|
requests:
|
||||||
|
cpu: "500m"
|
||||||
|
memory: "50Mi"
|
||||||
|
mgr-sidecar:
|
||||||
|
limits:
|
||||||
|
memory: "100Mi"
|
||||||
|
requests:
|
||||||
|
cpu: "100m"
|
||||||
|
memory: "40Mi"
|
||||||
|
crashcollector:
|
||||||
|
limits:
|
||||||
|
memory: "60Mi"
|
||||||
|
requests:
|
||||||
|
cpu: "100m"
|
||||||
|
memory: "60Mi"
|
||||||
|
logcollector:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "100m"
|
||||||
|
memory: "100Mi"
|
||||||
|
cleanup:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "500m"
|
||||||
|
memory: "100Mi"
|
||||||
|
exporter:
|
||||||
|
limits:
|
||||||
|
memory: "128Mi"
|
||||||
|
requests:
|
||||||
|
cpu: "50m"
|
||||||
|
memory: "50Mi"
|
||||||
|
cmd-reporter:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "500m"
|
||||||
|
memory: "100Mi"
|
||||||
|
|
||||||
|
# The option to automatically remove OSDs that are out and are safe to destroy.
|
||||||
|
removeOSDsIfOutAndSafeToRemove: false
|
||||||
|
|
||||||
|
# priority classes to apply to ceph resources
|
||||||
|
priorityClassNames:
|
||||||
|
mon: system-node-critical
|
||||||
|
osd: system-node-critical
|
||||||
|
mgr: system-cluster-critical
|
||||||
|
|
||||||
|
storage: # cluster level storage configuration and selection
|
||||||
|
useAllNodes: true
|
||||||
|
useAllDevices: true
|
||||||
|
# deviceFilter:
|
||||||
|
# config:
|
||||||
|
# crushRoot: "custom-root" # specify a non-default root label for the CRUSH map
|
||||||
|
# metadataDevice: "md0" # specify a non-rotational storage so ceph-volume will use it as block db device of bluestore.
|
||||||
|
# databaseSizeMB: "1024" # uncomment if the disks are smaller than 100 GB
|
||||||
|
# osdsPerDevice: "1" # this value can be overridden at the node or device level
|
||||||
|
# encryptedDevice: "true" # the default value for this option is "false"
|
||||||
|
# # Individual nodes and their config can be specified as well, but 'useAllNodes' above must be set to false. Then, only the named
|
||||||
|
# # nodes below will be used as storage resources. Each node's 'name' field should match their 'kubernetes.io/hostname' label.
|
||||||
|
# nodes:
|
||||||
|
# - name: "172.17.4.201"
|
||||||
|
# devices: # specific devices to use for storage can be specified for each node
|
||||||
|
# - name: "sdb"
|
||||||
|
# - name: "nvme01" # multiple osds can be created on high performance devices
|
||||||
|
# config:
|
||||||
|
# osdsPerDevice: "5"
|
||||||
|
# - name: "/dev/disk/by-id/ata-ST4000DM004-XXXX" # devices can be specified using full udev paths
|
||||||
|
# config: # configuration can be specified at the node level which overrides the cluster level config
|
||||||
|
# - name: "172.17.4.301"
|
||||||
|
# deviceFilter: "^sd."
|
||||||
|
|
||||||
|
# The section for configuring management of daemon disruptions during upgrade or fencing.
|
||||||
|
disruptionManagement:
|
||||||
|
# If true, the operator will create and manage PodDisruptionBudgets for OSD, Mon, RGW, and MDS daemons. OSD PDBs are managed dynamically
|
||||||
|
# via the strategy outlined in the [design](https://github.com/rook/rook/blob/master/design/ceph/ceph-managed-disruptionbudgets.md). The operator will
|
||||||
|
# block eviction of OSDs by default and unblock them safely when drains are detected.
|
||||||
|
managePodBudgets: true
|
||||||
|
# A duration in minutes that determines how long an entire failureDomain like `region/zone/host` will be held in `noout` (in addition to the
|
||||||
|
# default DOWN/OUT interval) when it is draining. This is only relevant when `managePodBudgets` is `true`. The default value is `30` minutes.
|
||||||
|
osdMaintenanceTimeout: 30
|
||||||
|
|
||||||
|
# Configure the healthcheck and liveness probes for ceph pods.
|
||||||
|
# Valid values for daemons are 'mon', 'osd', 'status'
|
||||||
|
healthCheck:
|
||||||
|
daemonHealth:
|
||||||
|
mon:
|
||||||
|
disabled: false
|
||||||
|
interval: 45s
|
||||||
|
osd:
|
||||||
|
disabled: false
|
||||||
|
interval: 60s
|
||||||
|
status:
|
||||||
|
disabled: false
|
||||||
|
interval: 60s
|
||||||
|
# Change pod liveness probe, it works for all mon, mgr, and osd pods.
|
||||||
|
livenessProbe:
|
||||||
|
mon:
|
||||||
|
disabled: false
|
||||||
|
mgr:
|
||||||
|
disabled: false
|
||||||
|
osd:
|
||||||
|
disabled: false
|
||||||
|
|
||||||
|
ingress:
|
||||||
|
# -- Enable an ingress for the ceph-dashboard
|
||||||
|
dashboard: {}
|
||||||
|
# labels:
|
||||||
|
# external-dns/private: "true"
|
||||||
|
# annotations:
|
||||||
|
# external-dns.alpha.kubernetes.io/hostname: dashboard.example.com
|
||||||
|
# nginx.ingress.kubernetes.io/rewrite-target: /ceph-dashboard/$2
|
||||||
|
# If the dashboard has ssl: true the following will make sure the NGINX Ingress controller can expose the dashboard correctly
|
||||||
|
# nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
|
||||||
|
# nginx.ingress.kubernetes.io/server-snippet: |
|
||||||
|
# proxy_ssl_verify off;
|
||||||
|
# host:
|
||||||
|
# name: dashboard.example.com
|
||||||
|
# path: "/ceph-dashboard(/|$)(.*)"
|
||||||
|
# pathType: Prefix
|
||||||
|
# tls:
|
||||||
|
# - hosts:
|
||||||
|
# - dashboard.example.com
|
||||||
|
# secretName: testsecret-tls
|
||||||
|
## Note: Only one of ingress class annotation or the `ingressClassName:` can be used at a time
|
||||||
|
## to set the ingress class
|
||||||
|
# ingressClassName: nginx
|
||||||
|
|
||||||
|
route:
|
||||||
|
# -- Enable an HTTPRoute for the ceph-dashboard
|
||||||
|
dashboard: {}
|
||||||
|
# labels:
|
||||||
|
# external-dns/private: "true"
|
||||||
|
# annotations:
|
||||||
|
# external-dns.alpha.kubernetes.io/hostname: dashboard.example.com
|
||||||
|
# nginx.ingress.kubernetes.io/rewrite-target: /ceph-dashboard/$2
|
||||||
|
# host:
|
||||||
|
# name: dashboard.example.com
|
||||||
|
# path: "/"
|
||||||
|
# pathType: PathPrefix
|
||||||
|
# parentRefs:
|
||||||
|
# - name: internal
|
||||||
|
# namespace: kube-system
|
||||||
|
# sectionName: https
|
||||||
|
|
||||||
|
# -- A list of CephBlockPool configurations to deploy
|
||||||
|
# @default -- See [below](#ceph-block-pools)
|
||||||
|
cephBlockPools:
|
||||||
|
- name: ceph-blockpool
|
||||||
|
# see https://github.com/rook/rook/blob/master/Documentation/CRDs/Block-Storage/ceph-block-pool-crd.md#spec for available configuration
|
||||||
|
spec:
|
||||||
|
failureDomain: host
|
||||||
|
replicated:
|
||||||
|
size: 3
|
||||||
|
# Enables collecting RBD per-image IO statistics by enabling dynamic OSD performance counters. Defaults to false.
|
||||||
|
# For reference: https://docs.ceph.com/docs/latest/mgr/prometheus/#rbd-io-statistics
|
||||||
|
# enableRBDStats: true
|
||||||
|
storageClass:
|
||||||
|
enabled: true
|
||||||
|
name: ceph-block
|
||||||
|
annotations: {}
|
||||||
|
labels: {}
|
||||||
|
isDefault: true
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
allowVolumeExpansion: true
|
||||||
|
volumeBindingMode: "Immediate"
|
||||||
|
mountOptions: []
|
||||||
|
# see https://kubernetes.io/docs/concepts/storage/storage-classes/#allowed-topologies
|
||||||
|
allowedTopologies: []
|
||||||
|
# - matchLabelExpressions:
|
||||||
|
# - key: rook-ceph-role
|
||||||
|
# values:
|
||||||
|
# - storage-node
|
||||||
|
# see https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/Block-Storage-RBD/block-storage.md#provision-storage for available configuration
|
||||||
|
parameters:
|
||||||
|
# (optional) mapOptions is a comma-separated list of map options.
|
||||||
|
# For krbd options refer
|
||||||
|
# https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
|
||||||
|
# For nbd options refer
|
||||||
|
# https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
|
||||||
|
# mapOptions: lock_on_read,queue_depth=1024
|
||||||
|
|
||||||
|
# (optional) unmapOptions is a comma-separated list of unmap options.
|
||||||
|
# For krbd options refer
|
||||||
|
# https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
|
||||||
|
# For nbd options refer
|
||||||
|
# https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
|
||||||
|
# unmapOptions: force
|
||||||
|
|
||||||
|
# RBD image format. Defaults to "2".
|
||||||
|
imageFormat: "2"
|
||||||
|
|
||||||
|
# RBD image features, equivalent to OR'd bitfield value: 63
|
||||||
|
# Available for imageFormat: "2". Older releases of CSI RBD
|
||||||
|
# support only the `layering` feature. The Linux kernel (KRBD) supports the
|
||||||
|
# full feature complement as of 5.4
|
||||||
|
imageFeatures: layering
|
||||||
|
|
||||||
|
# These secrets contain Ceph admin credentials.
|
||||||
|
csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
|
||||||
|
csi.storage.k8s.io/provisioner-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner
|
||||||
|
csi.storage.k8s.io/controller-expand-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
csi.storage.k8s.io/controller-publish-secret-name: rook-csi-rbd-provisioner
|
||||||
|
csi.storage.k8s.io/controller-publish-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
|
||||||
|
csi.storage.k8s.io/node-stage-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
# Specify the filesystem type of the volume. If not specified, csi-provisioner
|
||||||
|
# will set default as `ext4`. Note that `xfs` is not recommended due to potential deadlock
|
||||||
|
# in hyperconverged settings where the volume is mounted on the same node as the osds.
|
||||||
|
csi.storage.k8s.io/fstype: ext4
|
||||||
|
|
||||||
|
# -- A list of CephFileSystem configurations to deploy
|
||||||
|
# @default -- See [below](#ceph-file-systems)
|
||||||
|
cephFileSystems:
|
||||||
|
- name: ceph-filesystem
|
||||||
|
# see https://github.com/rook/rook/blob/master/Documentation/CRDs/Shared-Filesystem/ceph-filesystem-crd.md#filesystem-settings for available configuration
|
||||||
|
spec:
|
||||||
|
metadataPool:
|
||||||
|
replicated:
|
||||||
|
size: 3
|
||||||
|
dataPools:
|
||||||
|
- failureDomain: host
|
||||||
|
replicated:
|
||||||
|
size: 3
|
||||||
|
# Optional and highly recommended, 'data0' by default, see https://github.com/rook/rook/blob/master/Documentation/CRDs/Shared-Filesystem/ceph-filesystem-crd.md#pools
|
||||||
|
name: data0
|
||||||
|
metadataServer:
|
||||||
|
activeCount: 1
|
||||||
|
activeStandby: true
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
memory: "4Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "1000m"
|
||||||
|
memory: "4Gi"
|
||||||
|
priorityClassName: system-cluster-critical
|
||||||
|
storageClass:
|
||||||
|
enabled: true
|
||||||
|
isDefault: false
|
||||||
|
name: ceph-filesystem
|
||||||
|
# (Optional) specify a data pool to use, must be the name of one of the data pools above, 'data0' by default
|
||||||
|
pool: data0
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
allowVolumeExpansion: true
|
||||||
|
volumeBindingMode: "Immediate"
|
||||||
|
annotations: {}
|
||||||
|
labels: {}
|
||||||
|
mountOptions: []
|
||||||
|
# see https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/Shared-Filesystem-CephFS/filesystem-storage.md#provision-storage for available configuration
|
||||||
|
parameters:
|
||||||
|
# The secrets contain Ceph admin credentials.
|
||||||
|
csi.storage.k8s.io/provisioner-secret-name: rook-csi-cephfs-provisioner
|
||||||
|
csi.storage.k8s.io/provisioner-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
csi.storage.k8s.io/controller-expand-secret-name: rook-csi-cephfs-provisioner
|
||||||
|
csi.storage.k8s.io/controller-expand-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
csi.storage.k8s.io/controller-publish-secret-name: rook-csi-cephfs-provisioner
|
||||||
|
csi.storage.k8s.io/controller-publish-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
csi.storage.k8s.io/node-stage-secret-name: rook-csi-cephfs-node
|
||||||
|
csi.storage.k8s.io/node-stage-secret-namespace: "{{ .Release.Namespace }}"
|
||||||
|
# Specify the filesystem type of the volume. If not specified, csi-provisioner
|
||||||
|
# will set default as `ext4`. Note that `xfs` is not recommended due to potential deadlock
|
||||||
|
# in hyperconverged settings where the volume is mounted on the same node as the osds.
|
||||||
|
csi.storage.k8s.io/fstype: ext4
|
||||||
|
|
||||||
|
# -- Settings for the filesystem snapshot class
|
||||||
|
# @default -- See [CephFS Snapshots](../Storage-Configuration/Ceph-CSI/ceph-csi-snapshot.md#cephfs-snapshots)
|
||||||
|
cephFileSystemVolumeSnapshotClass:
|
||||||
|
enabled: false
|
||||||
|
name: ceph-filesystem
|
||||||
|
isDefault: true
|
||||||
|
deletionPolicy: Delete
|
||||||
|
annotations: {}
|
||||||
|
labels: {}
|
||||||
|
# see https://rook.io/docs/rook/v1.10/Storage-Configuration/Ceph-CSI/ceph-csi-snapshot/#cephfs-snapshots for available configuration
|
||||||
|
parameters: {}
|
||||||
|
|
||||||
|
# -- Settings for the block pool snapshot class
|
||||||
|
# @default -- See [RBD Snapshots](../Storage-Configuration/Ceph-CSI/ceph-csi-snapshot.md#rbd-snapshots)
|
||||||
|
cephBlockPoolsVolumeSnapshotClass:
|
||||||
|
enabled: false
|
||||||
|
name: ceph-block
|
||||||
|
isDefault: false
|
||||||
|
deletionPolicy: Delete
|
||||||
|
annotations: {}
|
||||||
|
labels: {}
|
||||||
|
# see https://rook.io/docs/rook/v1.10/Storage-Configuration/Ceph-CSI/ceph-csi-snapshot/#rbd-snapshots for available configuration
|
||||||
|
parameters: {}
|
||||||
|
|
||||||
|
# -- A list of CephObjectStore configurations to deploy
|
||||||
|
# @default -- See [below](#ceph-object-stores)
|
||||||
|
cephObjectStores:
|
||||||
|
- name: ceph-objectstore
|
||||||
|
# see https://github.com/rook/rook/blob/master/Documentation/CRDs/Object-Storage/ceph-object-store-crd.md#object-store-settings for available configuration
|
||||||
|
spec:
|
||||||
|
metadataPool:
|
||||||
|
failureDomain: host
|
||||||
|
replicated:
|
||||||
|
size: 3
|
||||||
|
dataPool:
|
||||||
|
failureDomain: host
|
||||||
|
erasureCoded:
|
||||||
|
dataChunks: 2
|
||||||
|
codingChunks: 1
|
||||||
|
parameters:
|
||||||
|
bulk: "true"
|
||||||
|
preservePoolsOnDelete: true
|
||||||
|
gateway:
|
||||||
|
port: 80
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
memory: "2Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "1000m"
|
||||||
|
memory: "1Gi"
|
||||||
|
# securePort: 443
|
||||||
|
# sslCertificateRef:
|
||||||
|
instances: 1
|
||||||
|
priorityClassName: system-cluster-critical
|
||||||
|
# opsLogSidecar:
|
||||||
|
# resources:
|
||||||
|
# limits:
|
||||||
|
# memory: "100Mi"
|
||||||
|
# requests:
|
||||||
|
# cpu: "100m"
|
||||||
|
# memory: "40Mi"
|
||||||
|
storageClass:
|
||||||
|
enabled: true
|
||||||
|
name: ceph-bucket
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
volumeBindingMode: "Immediate"
|
||||||
|
annotations: {}
|
||||||
|
labels: {}
|
||||||
|
# see https://github.com/rook/rook/blob/master/Documentation/Storage-Configuration/Object-Storage-RGW/ceph-object-bucket-claim.md#storageclass for available configuration
|
||||||
|
parameters:
|
||||||
|
# note: objectStoreNamespace and objectStoreName are configured by the chart
|
||||||
|
region: us-east-1
|
||||||
|
ingress:
|
||||||
|
# Enable an ingress for the ceph-objectstore
|
||||||
|
enabled: false
|
||||||
|
# The ingress port by default will be the object store's "securePort" (if set), or the gateway "port".
|
||||||
|
# To override those defaults, set this ingress port to the desired port.
|
||||||
|
# port: 80
|
||||||
|
# annotations: {}
|
||||||
|
# host:
|
||||||
|
# name: objectstore.example.com
|
||||||
|
# path: /
|
||||||
|
# pathType: Prefix
|
||||||
|
# tls:
|
||||||
|
# - hosts:
|
||||||
|
# - objectstore.example.com
|
||||||
|
# secretName: ceph-objectstore-tls
|
||||||
|
# ingressClassName: nginx
|
||||||
|
route:
|
||||||
|
# Enable an ingress for the ceph-objectstore
|
||||||
|
enabled: false
|
||||||
|
# The ingress port by default will be the object store's "securePort" (if set), or the gateway "port".
|
||||||
|
# To override those defaults, set this ingress port to the desired port.
|
||||||
|
# port: 80
|
||||||
|
# annotations: {}
|
||||||
|
# host:
|
||||||
|
# name: objectstore.example.com
|
||||||
|
# path: /
|
||||||
|
# pathType: PathPrefix
|
||||||
|
# parentRefs:
|
||||||
|
# - name: internal
|
||||||
|
# namespace: kube-system
|
||||||
|
# sectionName: https
|
||||||
|
## cephECBlockPools are disabled by default, please remove the comments and set desired values to enable it
|
||||||
|
## For erasure coded a replicated metadata pool is required.
|
||||||
|
## https://rook.io/docs/rook/latest/CRDs/Shared-Filesystem/ceph-filesystem-crd/#erasure-coded
|
||||||
|
#cephECBlockPools:
|
||||||
|
# - name: ec-pool
|
||||||
|
# spec:
|
||||||
|
# metadataPool:
|
||||||
|
# replicated:
|
||||||
|
# size: 2
|
||||||
|
# dataPool:
|
||||||
|
# failureDomain: osd
|
||||||
|
# erasureCoded:
|
||||||
|
# dataChunks: 2
|
||||||
|
# codingChunks: 1
|
||||||
|
# deviceClass: hdd
|
||||||
|
#
|
||||||
|
# parameters:
|
||||||
|
# # clusterID is the namespace where the rook cluster is running
|
||||||
|
# # If you change this namespace, also change the namespace below where the secret namespaces are defined
|
||||||
|
# clusterID: rook-ceph # namespace:cluster
|
||||||
|
# # (optional) mapOptions is a comma-separated list of map options.
|
||||||
|
# # For krbd options refer
|
||||||
|
# # https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
|
||||||
|
# # For nbd options refer
|
||||||
|
# # https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
|
||||||
|
# # mapOptions: lock_on_read,queue_depth=1024
|
||||||
|
#
|
||||||
|
# # (optional) unmapOptions is a comma-separated list of unmap options.
|
||||||
|
# # For krbd options refer
|
||||||
|
# # https://docs.ceph.com/docs/latest/man/8/rbd/#kernel-rbd-krbd-options
|
||||||
|
# # For nbd options refer
|
||||||
|
# # https://docs.ceph.com/docs/latest/man/8/rbd-nbd/#options
|
||||||
|
# # unmapOptions: force
|
||||||
|
#
|
||||||
|
# # RBD image format. Defaults to "2".
|
||||||
|
# imageFormat: "2"
|
||||||
|
#
|
||||||
|
# # RBD image features, equivalent to OR'd bitfield value: 63
|
||||||
|
# # Available for imageFormat: "2". Older releases of CSI RBD
|
||||||
|
# # support only the `layering` feature. The Linux kernel (KRBD) supports the
|
||||||
|
# # full feature complement as of 5.4
|
||||||
|
# # imageFeatures: layering,fast-diff,object-map,deep-flatten,exclusive-lock
|
||||||
|
# imageFeatures: layering
|
||||||
|
#
|
||||||
|
# storageClass:
|
||||||
|
# provisioner: rook-ceph.rbd.csi.ceph.com # csi-provisioner-name
|
||||||
|
# enabled: true
|
||||||
|
# name: rook-ceph-block
|
||||||
|
# isDefault: false
|
||||||
|
# annotations: { }
|
||||||
|
# labels: { }
|
||||||
|
# allowVolumeExpansion: true
|
||||||
|
# reclaimPolicy: Delete
|
||||||
|
|
||||||
|
# -- CSI driver name prefix for cephfs, rbd and nfs.
|
||||||
|
# @default -- `namespace name where rook-ceph operator is deployed`
|
||||||
|
csiDriverNamePrefix:
|
||||||
|
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
apiVersion: v2
|
||||||
|
name: cloudhost-ceph
|
||||||
|
description: CloudHost extras on top of Rook-Ceph (app source bucket, platform integration secrets)
|
||||||
|
type: application
|
||||||
|
version: 0.1.0
|
||||||
|
appVersion: "1.0.0"
|
||||||
|
keywords:
|
||||||
|
- ceph
|
||||||
|
- rook
|
||||||
|
- storage
|
||||||
|
- s3
|
||||||
|
maintainers:
|
||||||
|
- name: CloudHost
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
# CloudHost Ceph (Rook)
|
||||||
|
|
||||||
|
Helm chart and install scripts for **Rook-Ceph** on CloudHost clusters:
|
||||||
|
|
||||||
|
| Layer | Purpose |
|
||||||
|
|-------|---------|
|
||||||
|
| **rook-ceph-block** | Expandable PVCs for apps, databases, registry |
|
||||||
|
| **rook-ceph-bucket** | S3-compatible storage for uploaded source zip archives |
|
||||||
|
|
||||||
|
The chart does **not** vendor Rook itself — it installs the official [`rook-release`](https://charts.rook.io/release) charts and adds CloudHost-specific **ObjectBucketClaim** + credential sync.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick install
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend/helm/cloudhost-ceph
|
||||||
|
./scripts/install.sh single-node # one-node k3s (current abr cluster)
|
||||||
|
# or
|
||||||
|
./scripts/install.sh multi-node # production, 3+ nodes + raw disks
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/verify.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Profiles
|
||||||
|
|
||||||
|
### `single-node`
|
||||||
|
|
||||||
|
- OSD on **loop device** `/dev/loop6` (15Gi file at `/var/lib/rook/osd-loopback.img`) — no spare raw disk required
|
||||||
|
- Requires `ROOK_CEPH_ALLOW_LOOP_DEVICES=true` on the operator
|
||||||
|
- Replication **size: 1** (no HA)
|
||||||
|
- Suitable for **staging / single k3s node**
|
||||||
|
- Images must be pre-mirrored to `registry.abrban.com` (see `RUNBOOK-HARBOR.fa.md`)
|
||||||
|
|
||||||
|
### `multi-node`
|
||||||
|
|
||||||
|
- OSD on **raw devices** (`useAllDevices: true`)
|
||||||
|
- Replication **size: 3** for block + object metadata
|
||||||
|
- Erasure-coded object data pool
|
||||||
|
- Requires **3+ nodes** and dedicated disks
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What gets installed
|
||||||
|
|
||||||
|
| Step | Release | Namespace |
|
||||||
|
|------|---------|-----------|
|
||||||
|
| 1 | `rook-ceph` (operator) | `rook-ceph` |
|
||||||
|
| 2 | `rook-ceph-cluster` | `rook-ceph` |
|
||||||
|
| 3 | `cloudhost-ceph` (OBC + secrets) | `cloudhost-builds` |
|
||||||
|
|
||||||
|
### StorageClasses (from Rook)
|
||||||
|
|
||||||
|
| Name | Use |
|
||||||
|
|------|-----|
|
||||||
|
| `rook-ceph-block` | App PVC, DB PVC, Redis, registry, … |
|
||||||
|
| `rook-ceph-bucket` | `ObjectBucketClaim` → S3 bucket + credentials |
|
||||||
|
|
||||||
|
### CloudHost extras
|
||||||
|
|
||||||
|
| Resource | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| `ObjectBucketClaim/app-sources` | Bucket for user zip uploads |
|
||||||
|
| `Secret/ceph-app-sources-credentials` | Stable S3 credentials for backend |
|
||||||
|
| `ConfigMap/cloudhost-ceph-integration` | Suggested `PLATFORM_*` env values |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Platform integration
|
||||||
|
|
||||||
|
After install, configure the **backend**:
|
||||||
|
|
||||||
|
```env
|
||||||
|
PLATFORM_STORAGE_CLASS=rook-ceph-block
|
||||||
|
PLATFORM_CREATE_STORAGE_CLASS=false
|
||||||
|
PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com
|
||||||
|
```
|
||||||
|
|
||||||
|
Mount or env-from secret `cloudhost-builds/ceph-app-sources-credentials`:
|
||||||
|
|
||||||
|
```env
|
||||||
|
SOURCE_STORAGE_ENDPOINT=http://rook-ceph-rgw-ceph-objectstore.rook-ceph.svc.cluster.local:80
|
||||||
|
SOURCE_STORAGE_REGION=us-east-1
|
||||||
|
SOURCE_STORAGE_BUCKET=<from secret>
|
||||||
|
SOURCE_STORAGE_ACCESS_KEY=<from secret>
|
||||||
|
SOURCE_STORAGE_SECRET_KEY=<from secret>
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Note:** Existing PVCs on `local-path` / `cloudhost-expandable` are **not** migrated automatically. New apps use `rook-ceph-block` once the backend env is updated. Plan migration per workload (see `RUNBOOK-CEPH.fa.md`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Uninstall (destructive)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/uninstall.sh
|
||||||
|
# then on each node:
|
||||||
|
sudo rm -rf /var/lib/rook /var/lib/rook/osd
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cluster health
|
||||||
|
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph status
|
||||||
|
|
||||||
|
# OSD pods
|
||||||
|
kubectl -n rook-ceph get pods -l app=rook-ceph-osd
|
||||||
|
|
||||||
|
# RGW (object store)
|
||||||
|
kubectl -n rook-ceph get pods -l app=rook-ceph-rgw
|
||||||
|
|
||||||
|
# Bucket sync job
|
||||||
|
kubectl -n cloudhost-builds logs job -l job-name=cloudhost-ceph-bucket-sync --tail=50
|
||||||
|
```
|
||||||
|
|
||||||
|
Full operational guide (Persian): [`../../../RUNBOOK-CEPH.fa.md`](../../../RUNBOOK-CEPH.fa.md)
|
||||||
|
|
||||||
|
Registry / Harbor (Persian): [`../../../RUNBOOK-HARBOR.fa.md`](../../../RUNBOOK-HARBOR.fa.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| File | Role |
|
||||||
|
|------|------|
|
||||||
|
| `values-rook-cluster-single-node.yaml` | Rook cluster values (1 node) |
|
||||||
|
| `values-rook-cluster-multi-node.yaml` | Rook cluster values (production) |
|
||||||
|
| `values.yaml` | CloudHost OBC / secret sync |
|
||||||
|
| `scripts/install.sh` | Full install |
|
||||||
|
| `scripts/verify.sh` | Health check |
|
||||||
|
| `scripts/uninstall.sh` | Tear down |
|
||||||
+52
@@ -0,0 +1,52 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Install Rook-Ceph operator + cluster + CloudHost bucket extras.
|
||||||
|
# Usage: ./scripts/install.sh [single-node|multi-node]
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
PROFILE="${1:-single-node}"
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
CHART_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
ROOK_NS="rook-ceph"
|
||||||
|
EXTRAS_NS="cloudhost-builds"
|
||||||
|
CLUSTER_VALUES="${CHART_DIR}/values-rook-cluster-${PROFILE}.yaml"
|
||||||
|
|
||||||
|
if [[ ! -f "${CLUSTER_VALUES}" ]]; then
|
||||||
|
echo "Unknown profile: ${PROFILE} (missing ${CLUSTER_VALUES})" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "==> Profile: ${PROFILE}"
|
||||||
|
echo "==> Adding rook-release helm repo"
|
||||||
|
helm repo add rook-release https://charts.rook.io/release 2>/dev/null || true
|
||||||
|
helm repo update rook-release
|
||||||
|
|
||||||
|
echo "==> [1/4] Installing Rook operator in ${ROOK_NS}"
|
||||||
|
helm upgrade --install rook-ceph rook-release/rook-ceph \
|
||||||
|
--namespace "${ROOK_NS}" \
|
||||||
|
--create-namespace \
|
||||||
|
--wait \
|
||||||
|
--timeout 10m
|
||||||
|
|
||||||
|
echo "==> [2/4] Waiting for Rook operator deployment"
|
||||||
|
kubectl -n "${ROOK_NS}" rollout status deploy/rook-ceph-operator --timeout=300s
|
||||||
|
|
||||||
|
echo "==> [3/4] Installing Ceph cluster (${CLUSTER_VALUES})"
|
||||||
|
helm upgrade --install rook-ceph-cluster rook-release/rook-ceph-cluster \
|
||||||
|
--namespace "${ROOK_NS}" \
|
||||||
|
-f "${CLUSTER_VALUES}" \
|
||||||
|
--wait \
|
||||||
|
--timeout 25m
|
||||||
|
|
||||||
|
echo "==> Waiting for CephCluster phase = Ready (up to 20 min)"
|
||||||
|
"${SCRIPT_DIR}/wait-ceph-ready.sh" 1200
|
||||||
|
|
||||||
|
echo "==> [4/4] Installing CloudHost Ceph extras (ObjectBucketClaim) in ${EXTRAS_NS}"
|
||||||
|
kubectl create namespace "${EXTRAS_NS}" 2>/dev/null || true
|
||||||
|
helm upgrade --install cloudhost-ceph "${CHART_DIR}" \
|
||||||
|
--namespace "${EXTRAS_NS}" \
|
||||||
|
-f "${CHART_DIR}/values.yaml" \
|
||||||
|
--wait \
|
||||||
|
--timeout 15m
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "==> Done. Run ./scripts/verify.sh to confirm health and print integration hints."
|
||||||
+26
@@ -0,0 +1,26 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Remove CloudHost extras + Rook cluster + operator (DATA LOSS).
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
read -r -p "This deletes ALL Ceph data. Type 'delete-ceph' to continue: " CONFIRM
|
||||||
|
if [[ "${CONFIRM}" != "delete-ceph" ]]; then
|
||||||
|
echo "Aborted."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
helm uninstall cloudhost-ceph -n cloudhost-builds 2>/dev/null || true
|
||||||
|
helm uninstall rook-ceph-cluster -n rook-ceph 2>/dev/null || true
|
||||||
|
helm uninstall rook-ceph -n rook-ceph 2>/dev/null || true
|
||||||
|
|
||||||
|
echo "Waiting for Rook resources to terminate..."
|
||||||
|
sleep 15
|
||||||
|
kubectl -n rook-ceph get pods 2>/dev/null || true
|
||||||
|
|
||||||
|
cat <<'EOF'
|
||||||
|
|
||||||
|
IMPORTANT: On each node, wipe Rook state before reinstalling:
|
||||||
|
sudo rm -rf /var/lib/rook
|
||||||
|
sudo rm -rf /var/lib/rook/osd
|
||||||
|
|
||||||
|
For raw-disk OSDs also zap disks (see RUNBOOK-CEPH.fa.md).
|
||||||
|
EOF
|
||||||
+35
@@ -0,0 +1,35 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOK_NS="rook-ceph"
|
||||||
|
EXTRAS_NS="cloudhost-builds"
|
||||||
|
|
||||||
|
echo "=== StorageClasses ==="
|
||||||
|
kubectl get storageclass | grep -E 'NAME|rook-ceph' || true
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== Ceph status ==="
|
||||||
|
kubectl -n "${ROOK_NS}" exec deploy/rook-ceph-tools -- ceph status 2>/dev/null || echo "(tools pod not ready yet)"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== OSD / MON pods ==="
|
||||||
|
kubectl -n "${ROOK_NS}" get pods -l app=rook-ceph-osd 2>/dev/null || kubectl -n "${ROOK_NS}" get pods | grep -E 'osd|mon|mgr|rgw' || true
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== Object bucket claim ==="
|
||||||
|
kubectl -n "${EXTRAS_NS}" get obc,app-sources 2>/dev/null || kubectl -n "${EXTRAS_NS}" get obc 2>/dev/null || true
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== Platform credentials secret ==="
|
||||||
|
if kubectl -n "${EXTRAS_NS}" get secret ceph-app-sources-credentials >/dev/null 2>&1; then
|
||||||
|
echo "Secret ceph-app-sources-credentials exists"
|
||||||
|
kubectl -n "${EXTRAS_NS}" get secret ceph-app-sources-credentials -o jsonpath='{.data.SOURCE_STORAGE_BUCKET}' | base64 -d
|
||||||
|
echo ""
|
||||||
|
else
|
||||||
|
echo "Secret ceph-app-sources-credentials not ready — check bucket sync job:"
|
||||||
|
kubectl -n "${EXTRAS_NS}" get jobs,pods | grep bucket-sync || true
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "=== Suggested backend env ==="
|
||||||
|
kubectl -n "${EXTRAS_NS}" get configmap cloudhost-ceph-integration -o yaml 2>/dev/null | sed -n '/PLATFORM_/p;/SOURCE_STORAGE_ENDPOINT/p' || true
|
||||||
+39
@@ -0,0 +1,39 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Wait until Ceph reports HEALTH_OK or HEALTH_WARN (single-node often stays WARN).
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
TIMEOUT="${1:-900}"
|
||||||
|
ROOK_NS="rook-ceph"
|
||||||
|
START=$(date +%s)
|
||||||
|
|
||||||
|
echo "Waiting for rook-ceph-tools deployment..."
|
||||||
|
for _ in $(seq 1 60); do
|
||||||
|
if kubectl -n "${ROOK_NS}" get deploy rook-ceph-tools >/dev/null 2>&1; then
|
||||||
|
if kubectl -n "${ROOK_NS}" rollout status deploy/rook-ceph-tools --timeout=120s 2>/dev/null; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
sleep 5
|
||||||
|
done
|
||||||
|
|
||||||
|
while true; do
|
||||||
|
NOW=$(date +%s)
|
||||||
|
if (( NOW - START > TIMEOUT )); then
|
||||||
|
echo "Timed out after ${TIMEOUT}s waiting for Ceph health" >&2
|
||||||
|
kubectl -n "${ROOK_NS}" get cephcluster,pod -o wide || true
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if kubectl -n "${ROOK_NS}" get deploy rook-ceph-tools >/dev/null 2>&1; then
|
||||||
|
HEALTH=$(kubectl -n "${ROOK_NS}" exec deploy/rook-ceph-tools -- ceph health 2>/dev/null || echo "unknown")
|
||||||
|
echo "Ceph health: ${HEALTH}"
|
||||||
|
if [[ "${HEALTH}" == "HEALTH_OK" || "${HEALTH}" == HEALTH_WARN* ]]; then
|
||||||
|
PHASE=$(kubectl -n "${ROOK_NS}" get cephcluster rook-ceph -o jsonpath='{.status.phase}' 2>/dev/null || echo "")
|
||||||
|
echo "CephCluster phase: ${PHASE}"
|
||||||
|
if [[ "${PHASE}" == "Ready" ]]; then
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
sleep 15
|
||||||
|
done
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
CloudHost Ceph storage is ready (or still initializing).
|
||||||
|
|
||||||
|
Profiles
|
||||||
|
single-node Directory OSD on /var/lib/rook/osd — for one-node k3s (no HA)
|
||||||
|
multi-node Raw disk OSDs with replication=3 — production
|
||||||
|
|
||||||
|
StorageClasses created by Rook
|
||||||
|
rook-ceph-block Block volumes (app PVC, DB, registry, …)
|
||||||
|
rook-ceph-bucket S3-compatible buckets via ObjectBucketClaim
|
||||||
|
|
||||||
|
Verify cluster health
|
||||||
|
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph status
|
||||||
|
kubectl get storageclass | grep rook-ceph
|
||||||
|
kubectl -n cloudhost-builds get obc,secret | grep -E 'app-sources|ceph-app-sources'
|
||||||
|
|
||||||
|
Platform backend (after bucket sync Job completes)
|
||||||
|
PLATFORM_STORAGE_CLASS=rook-ceph-block
|
||||||
|
PLATFORM_CREATE_STORAGE_CLASS=false
|
||||||
|
PLATFORM_STORAGE_PROVISIONER=rook-ceph.rbd.csi.ceph.com
|
||||||
|
|
||||||
|
Mount secret cloudhost-builds/ceph-app-sources-credentials for zip upload S3 settings.
|
||||||
|
|
||||||
|
Full guide: backend/helm/cloudhost-ceph/README.md and RUNBOOK-CEPH.fa.md
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
{{/*
|
||||||
|
CloudHost Ceph chart helpers
|
||||||
|
*/}}
|
||||||
|
{{- define "cloudhost-ceph.name" -}}
|
||||||
|
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
|
||||||
|
{{- end }}
|
||||||
|
|
||||||
|
{{- define "cloudhost-ceph.fullname" -}}
|
||||||
|
{{- if .Values.fullnameOverride }}
|
||||||
|
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
|
||||||
|
{{- else }}
|
||||||
|
{{- $name := default .Chart.Name .Values.nameOverride }}
|
||||||
|
{{- if contains $name .Release.Name }}
|
||||||
|
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
|
||||||
|
{{- else }}
|
||||||
|
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
|
||||||
|
{{- end }}
|
||||||
|
{{- end }}
|
||||||
|
{{- end }}
|
||||||
|
|
||||||
|
{{- define "cloudhost-ceph.labels" -}}
|
||||||
|
helm.sh/chart: {{ include "cloudhost-ceph.name" . }}-{{ .Chart.Version }}
|
||||||
|
app.kubernetes.io/name: {{ include "cloudhost-ceph.name" . }}
|
||||||
|
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||||
|
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||||
|
app.kubernetes.io/part-of: cloudhost
|
||||||
|
{{- end }}
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
{{- if .Values.integration.createConfigMap }}
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: {{ .Values.integration.configMapName }}
|
||||||
|
namespace: {{ .Values.namespace }}
|
||||||
|
labels:
|
||||||
|
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
|
||||||
|
data:
|
||||||
|
PLATFORM_STORAGE_CLASS: rook-ceph-block
|
||||||
|
PLATFORM_CREATE_STORAGE_CLASS: "false"
|
||||||
|
PLATFORM_STORAGE_PROVISIONER: rook-ceph.rbd.csi.ceph.com
|
||||||
|
SOURCE_STORAGE_ENDPOINT: {{ .Values.platform.endpoint | quote }}
|
||||||
|
SOURCE_STORAGE_REGION: {{ .Values.platform.region | quote }}
|
||||||
|
SOURCE_STORAGE_CREDENTIALS_SECRET: {{ .Values.platform.credentialsSecretName | quote }}
|
||||||
|
README: |
|
||||||
|
Block PVCs: set PLATFORM_STORAGE_CLASS=rook-ceph-block on the backend.
|
||||||
|
New app PVCs use rook-ceph-block; existing local-path PVCs are NOT auto-migrated.
|
||||||
|
Object storage credentials: secret {{ .Values.platform.credentialsSecretName }} in {{ .Values.namespace }}.
|
||||||
|
{{- end }}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
{{- if .Values.objectStore.claimName }}
|
||||||
|
apiVersion: objectbucket.io/v1alpha1
|
||||||
|
kind: ObjectBucketClaim
|
||||||
|
metadata:
|
||||||
|
name: {{ .Values.objectStore.claimName }}
|
||||||
|
namespace: {{ .Values.namespace }}
|
||||||
|
labels:
|
||||||
|
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
|
||||||
|
annotations:
|
||||||
|
helm.sh/hook: post-install,post-upgrade
|
||||||
|
helm.sh/hook-weight: "5"
|
||||||
|
spec:
|
||||||
|
storageClassName: {{ .Values.objectStore.bucketStorageClass | quote }}
|
||||||
|
generateBucketName: {{ .Values.objectStore.generateBucketName | quote }}
|
||||||
|
{{- end }}
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
{{- if and .Values.platform.createCredentialsSecret .Values.objectStore.claimName }}
|
||||||
|
# Stable secret name for platform workers. Populated by a post-install Job once the OBC secret exists.
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ServiceAccount
|
||||||
|
metadata:
|
||||||
|
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
|
||||||
|
namespace: {{ .Values.namespace }}
|
||||||
|
labels:
|
||||||
|
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
|
||||||
|
annotations:
|
||||||
|
helm.sh/hook: post-install,post-upgrade
|
||||||
|
helm.sh/hook-weight: "1"
|
||||||
|
helm.sh/hook-delete-policy: before-hook-creation
|
||||||
|
---
|
||||||
|
apiVersion: rbac.authorization.k8s.io/v1
|
||||||
|
kind: Role
|
||||||
|
metadata:
|
||||||
|
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
|
||||||
|
namespace: {{ .Values.namespace }}
|
||||||
|
annotations:
|
||||||
|
helm.sh/hook: post-install,post-upgrade
|
||||||
|
helm.sh/hook-weight: "1"
|
||||||
|
helm.sh/hook-delete-policy: before-hook-creation
|
||||||
|
rules:
|
||||||
|
- apiGroups: [""]
|
||||||
|
resources: ["secrets"]
|
||||||
|
verbs: ["get", "list", "create", "patch", "update"]
|
||||||
|
---
|
||||||
|
apiVersion: rbac.authorization.k8s.io/v1
|
||||||
|
kind: RoleBinding
|
||||||
|
metadata:
|
||||||
|
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
|
||||||
|
namespace: {{ .Values.namespace }}
|
||||||
|
annotations:
|
||||||
|
helm.sh/hook: post-install,post-upgrade
|
||||||
|
helm.sh/hook-weight: "1"
|
||||||
|
helm.sh/hook-delete-policy: before-hook-creation
|
||||||
|
roleRef:
|
||||||
|
apiGroup: rbac.authorization.k8s.io
|
||||||
|
kind: Role
|
||||||
|
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
|
||||||
|
subjects:
|
||||||
|
- kind: ServiceAccount
|
||||||
|
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
|
||||||
|
namespace: {{ .Values.namespace }}
|
||||||
|
---
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
|
||||||
|
namespace: {{ .Values.namespace }}
|
||||||
|
labels:
|
||||||
|
{{- include "cloudhost-ceph.labels" . | nindent 4 }}
|
||||||
|
annotations:
|
||||||
|
helm.sh/hook: post-install,post-upgrade
|
||||||
|
helm.sh/hook-weight: "10"
|
||||||
|
helm.sh/hook-delete-policy: before-hook-creation
|
||||||
|
spec:
|
||||||
|
backoffLimit: 30
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
serviceAccountName: {{ include "cloudhost-ceph.fullname" . }}-bucket-sync
|
||||||
|
restartPolicy: OnFailure
|
||||||
|
containers:
|
||||||
|
- name: sync
|
||||||
|
image: registry.abrban.com/proxy-dockerhub/bitnami/kubectl:1.32
|
||||||
|
imagePullPolicy: IfNotPresent
|
||||||
|
env:
|
||||||
|
- name: OBC_SECRET
|
||||||
|
value: {{ printf "obc-%s-%s" .Values.namespace .Values.objectStore.claimName | quote }}
|
||||||
|
- name: TARGET_SECRET
|
||||||
|
value: {{ .Values.platform.credentialsSecretName | quote }}
|
||||||
|
- name: NAMESPACE
|
||||||
|
value: {{ .Values.namespace | quote }}
|
||||||
|
- name: ENDPOINT
|
||||||
|
value: {{ .Values.platform.endpoint | quote }}
|
||||||
|
- name: REGION
|
||||||
|
value: {{ .Values.platform.region | quote }}
|
||||||
|
command:
|
||||||
|
- /bin/bash
|
||||||
|
- -ec
|
||||||
|
- |
|
||||||
|
echo "Waiting for OBC secret ${OBC_SECRET} in ${NAMESPACE}..."
|
||||||
|
for i in $(seq 1 120); do
|
||||||
|
if kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" >/dev/null 2>&1; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 10
|
||||||
|
done
|
||||||
|
kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" >/dev/null
|
||||||
|
|
||||||
|
BUCKET=$(kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" -o jsonpath='{.data.BUCKET_NAME}' | base64 -d)
|
||||||
|
ACCESS=$(kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 -d)
|
||||||
|
SECRET=$(kubectl get secret -n "${NAMESPACE}" "${OBC_SECRET}" -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 -d)
|
||||||
|
|
||||||
|
cat <<EOF | kubectl apply -f -
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: ${TARGET_SECRET}
|
||||||
|
namespace: ${NAMESPACE}
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/part-of: cloudhost
|
||||||
|
type: Opaque
|
||||||
|
stringData:
|
||||||
|
SOURCE_STORAGE_ENDPOINT: "${ENDPOINT}"
|
||||||
|
SOURCE_STORAGE_REGION: "${REGION}"
|
||||||
|
SOURCE_STORAGE_BUCKET: "${BUCKET}"
|
||||||
|
SOURCE_STORAGE_ACCESS_KEY: "${ACCESS}"
|
||||||
|
SOURCE_STORAGE_SECRET_KEY: "${SECRET}"
|
||||||
|
EOF
|
||||||
|
echo "Synced bucket credentials to secret ${TARGET_SECRET} (bucket=${BUCKET})"
|
||||||
|
{{- end }}
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
# Rook-Ceph cluster values — MULTI NODE (production).
|
||||||
|
# Install: scripts/install.sh multi-node
|
||||||
|
#
|
||||||
|
# Prerequisites:
|
||||||
|
# - At least 3 worker nodes (odd mon count)
|
||||||
|
# - Raw disks available (useAllDevices) OR dedicated devices per node
|
||||||
|
# - Taint-free nodes labeled rook-ceph-role=storage-node (optional)
|
||||||
|
|
||||||
|
operatorNamespace: rook-ceph
|
||||||
|
|
||||||
|
toolbox:
|
||||||
|
enabled: true
|
||||||
|
|
||||||
|
cephClusterSpec:
|
||||||
|
dataDirHostPath: /var/lib/rook
|
||||||
|
|
||||||
|
mon:
|
||||||
|
count: 3
|
||||||
|
allowMultiplePerNode: false
|
||||||
|
|
||||||
|
mgr:
|
||||||
|
count: 2
|
||||||
|
allowMultiplePerNode: false
|
||||||
|
|
||||||
|
dashboard:
|
||||||
|
enabled: true
|
||||||
|
ssl: true
|
||||||
|
|
||||||
|
storage:
|
||||||
|
useAllNodes: false
|
||||||
|
useAllDevices: true
|
||||||
|
# Example: pin OSDs to storage nodes only
|
||||||
|
# nodes:
|
||||||
|
# - name: "node-1"
|
||||||
|
# - name: "node-2"
|
||||||
|
# - name: "node-3"
|
||||||
|
|
||||||
|
cephFileSystems: []
|
||||||
|
|
||||||
|
cephBlockPools:
|
||||||
|
- name: ceph-blockpool
|
||||||
|
spec:
|
||||||
|
failureDomain: host
|
||||||
|
replicated:
|
||||||
|
size: 3
|
||||||
|
storageClass:
|
||||||
|
enabled: true
|
||||||
|
name: rook-ceph-block
|
||||||
|
isDefault: false
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
allowVolumeExpansion: true
|
||||||
|
volumeBindingMode: WaitForFirstConsumer
|
||||||
|
|
||||||
|
cephObjectStores:
|
||||||
|
- name: ceph-objectstore
|
||||||
|
spec:
|
||||||
|
metadataPool:
|
||||||
|
failureDomain: host
|
||||||
|
replicated:
|
||||||
|
size: 3
|
||||||
|
dataPool:
|
||||||
|
failureDomain: host
|
||||||
|
erasureCoded:
|
||||||
|
dataChunks: 2
|
||||||
|
codingChunks: 1
|
||||||
|
preservePoolsOnDelete: true
|
||||||
|
gateway:
|
||||||
|
port: 80
|
||||||
|
instances: 2
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
memory: "2Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "500m"
|
||||||
|
memory: "1Gi"
|
||||||
|
storageClass:
|
||||||
|
enabled: true
|
||||||
|
name: rook-ceph-bucket
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
parameters:
|
||||||
|
region: us-east-1
|
||||||
|
ingress:
|
||||||
|
enabled: false
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
# Rook-Ceph cluster values — SINGLE NODE (k3s dev/staging).
|
||||||
|
# Install: scripts/install.sh single-node
|
||||||
|
#
|
||||||
|
# Uses loop device /dev/loop6 (15Gi) on single-node clusters without a spare raw disk.
|
||||||
|
# Replication factor = 1 (no HA). For production multi-node use values-rook-cluster-multi-node.yaml.
|
||||||
|
# See RUNBOOK-CEPH.fa.md for loop setup and image mirroring prerequisites.
|
||||||
|
|
||||||
|
operatorNamespace: rook-ceph
|
||||||
|
|
||||||
|
toolbox:
|
||||||
|
enabled: true
|
||||||
|
|
||||||
|
cephClusterSpec:
|
||||||
|
dataDirHostPath: /var/lib/rook
|
||||||
|
skipUpgradeChecks: true
|
||||||
|
continueUpgradeAfterChecksEvenIfNotHealthy: true
|
||||||
|
|
||||||
|
mon:
|
||||||
|
count: 1
|
||||||
|
allowMultiplePerNode: true
|
||||||
|
|
||||||
|
mgr:
|
||||||
|
count: 1
|
||||||
|
allowMultiplePerNode: true
|
||||||
|
|
||||||
|
dashboard:
|
||||||
|
enabled: true
|
||||||
|
ssl: false
|
||||||
|
|
||||||
|
resources:
|
||||||
|
mon:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "200m"
|
||||||
|
memory: "512Mi"
|
||||||
|
mgr:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "200m"
|
||||||
|
memory: "512Mi"
|
||||||
|
osd:
|
||||||
|
limits:
|
||||||
|
memory: "2Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "500m"
|
||||||
|
memory: "1Gi"
|
||||||
|
|
||||||
|
storage:
|
||||||
|
useAllNodes: true
|
||||||
|
useAllDevices: false
|
||||||
|
devices:
|
||||||
|
- name: "/dev/loop6"
|
||||||
|
|
||||||
|
# Disable CephFS to save RAM on single-node clusters.
|
||||||
|
cephFileSystems: []
|
||||||
|
|
||||||
|
cephBlockPools:
|
||||||
|
- name: ceph-blockpool
|
||||||
|
spec:
|
||||||
|
failureDomain: osd
|
||||||
|
replicated:
|
||||||
|
size: 1
|
||||||
|
storageClass:
|
||||||
|
enabled: true
|
||||||
|
name: rook-ceph-block
|
||||||
|
isDefault: false
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
allowVolumeExpansion: true
|
||||||
|
volumeBindingMode: WaitForFirstConsumer
|
||||||
|
|
||||||
|
cephObjectStores:
|
||||||
|
- name: ceph-objectstore
|
||||||
|
spec:
|
||||||
|
metadataPool:
|
||||||
|
failureDomain: osd
|
||||||
|
replicated:
|
||||||
|
size: 1
|
||||||
|
dataPool:
|
||||||
|
failureDomain: osd
|
||||||
|
replicated:
|
||||||
|
size: 1
|
||||||
|
preservePoolsOnDelete: true
|
||||||
|
gateway:
|
||||||
|
port: 80
|
||||||
|
instances: 1
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
requests:
|
||||||
|
cpu: "250m"
|
||||||
|
memory: "512Mi"
|
||||||
|
storageClass:
|
||||||
|
enabled: true
|
||||||
|
name: rook-ceph-bucket
|
||||||
|
reclaimPolicy: Delete
|
||||||
|
volumeBindingMode: Immediate
|
||||||
|
parameters:
|
||||||
|
region: us-east-1
|
||||||
|
ingress:
|
||||||
|
enabled: false
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
# CloudHost Ceph extras (ObjectBucketClaim for zip uploads).
|
||||||
|
# Rook operator + CephCluster are installed via scripts/install.sh using official rook-release charts.
|
||||||
|
|
||||||
|
namespace: cloudhost-builds
|
||||||
|
|
||||||
|
objectStore:
|
||||||
|
# Must match rook-ceph-cluster cephObjectStores[].storageClass.name
|
||||||
|
bucketStorageClass: rook-ceph-bucket
|
||||||
|
# Claim name; Rook generates bucket + credentials secret
|
||||||
|
claimName: app-sources
|
||||||
|
# Prefix for generated bucket name (Rook appends random suffix)
|
||||||
|
generateBucketName: cloudhost-app-sources
|
||||||
|
|
||||||
|
platform:
|
||||||
|
# Copy S3 credentials into a stable secret name for backend/workers
|
||||||
|
createCredentialsSecret: true
|
||||||
|
credentialsSecretName: ceph-app-sources-credentials
|
||||||
|
# In-cluster RGW endpoint (adjust if ingress is enabled on object store)
|
||||||
|
endpoint: http://rook-ceph-rgw-ceph-objectstore.rook-ceph.svc.cluster.local:80
|
||||||
|
region: us-east-1
|
||||||
|
|
||||||
|
integration:
|
||||||
|
# Emit a ConfigMap with suggested backend env vars (non-secret)
|
||||||
|
createConfigMap: true
|
||||||
|
configMapName: cloudhost-ceph-integration
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# Replaces registry.abrban.com docker distribution with Harbor.
|
||||||
|
# WARNING: This will delete the existing `Ingress/registry` routing. The old
|
||||||
|
# `Deployment/registry` and its PVC are left in place for rollback.
|
||||||
|
|
||||||
|
VALUES_FILE="${1:-/Users/keyhan/Documents/keyhan-project/cloud-host/backend/helm/cloudhost-harbor/values-registry.abrban.com.yaml}"
|
||||||
|
|
||||||
|
echo "==> Ensuring harbor repo"
|
||||||
|
helm repo add harbor https://helm.goharbor.io 2>/dev/null || true
|
||||||
|
helm repo update harbor
|
||||||
|
|
||||||
|
echo "==> [0/4] Preflight"
|
||||||
|
kubectl -n cloudhost get secret abrban-wildcard-tls >/dev/null
|
||||||
|
kubectl -n cloudhost get secret registry-egress-proxy >/dev/null
|
||||||
|
|
||||||
|
echo "==> [1/4] Disabling old registry ingress (host registry.abrban.com)"
|
||||||
|
kubectl -n cloudhost delete ingress registry --ignore-not-found
|
||||||
|
|
||||||
|
echo "==> [2/4] Scaling old registry deployment down (rollback-friendly)"
|
||||||
|
kubectl -n cloudhost scale deploy/registry --replicas=0 || true
|
||||||
|
|
||||||
|
echo "==> [3/4] Installing Harbor"
|
||||||
|
HTTP_PROXY="$(kubectl -n cloudhost get secret registry-egress-proxy -o jsonpath='{.data.HTTP_PROXY}' | base64 -d)"
|
||||||
|
HTTPS_PROXY="$(kubectl -n cloudhost get secret registry-egress-proxy -o jsonpath='{.data.HTTPS_PROXY}' | base64 -d)"
|
||||||
|
NO_PROXY="$(kubectl -n cloudhost get secret registry-egress-proxy -o jsonpath='{.data.NO_PROXY}' | base64 -d)"
|
||||||
|
|
||||||
|
TMP_PROXY_VALUES="$(mktemp)"
|
||||||
|
cat > "${TMP_PROXY_VALUES}" <<EOF
|
||||||
|
proxy:
|
||||||
|
httpProxy: "${HTTP_PROXY}"
|
||||||
|
httpsProxy: "${HTTPS_PROXY}"
|
||||||
|
noProxy: "${NO_PROXY}"
|
||||||
|
EOF
|
||||||
|
|
||||||
|
helm upgrade --install harbor harbor/harbor \
|
||||||
|
-n cloudhost \
|
||||||
|
-f "$VALUES_FILE" \
|
||||||
|
-f "${TMP_PROXY_VALUES}" \
|
||||||
|
--wait \
|
||||||
|
--timeout 20m
|
||||||
|
|
||||||
|
rm -f "${TMP_PROXY_VALUES}" || true
|
||||||
|
|
||||||
|
echo "==> [4/4] Done"
|
||||||
|
kubectl -n cloudhost get ingress | grep -n registry || true
|
||||||
|
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
## Harbor values to REPLACE registry.abrban.com
|
||||||
|
## Ingress controller on this cluster is Traefik (k3s).
|
||||||
|
##
|
||||||
|
## Install:
|
||||||
|
## helm upgrade --install harbor harbor/harbor -n cloudhost -f backend/helm/cloudhost-harbor/values-registry.abrban.com.yaml
|
||||||
|
##
|
||||||
|
externalURL: https://registry.abrban.com
|
||||||
|
|
||||||
|
proxy:
|
||||||
|
# Values are injected by install script from `cloudhost/registry-egress-proxy`.
|
||||||
|
httpProxy: ""
|
||||||
|
httpsProxy: ""
|
||||||
|
noProxy: ""
|
||||||
|
|
||||||
|
expose:
|
||||||
|
type: ingress
|
||||||
|
tls:
|
||||||
|
enabled: true
|
||||||
|
certSource: secret
|
||||||
|
secret:
|
||||||
|
secretName: abrban-wildcard-tls
|
||||||
|
ingress:
|
||||||
|
className: traefik
|
||||||
|
hosts:
|
||||||
|
core: registry.abrban.com
|
||||||
|
annotations:
|
||||||
|
traefik.ingress.kubernetes.io/router.entrypoints: websecure
|
||||||
|
# Increase timeouts for large pushes (skopeo/registry blobs)
|
||||||
|
traefik.ingress.kubernetes.io/router.tls: "true"
|
||||||
|
|
||||||
|
# Disable components we don't need for now to reduce resources
|
||||||
|
trivy:
|
||||||
|
enabled: false
|
||||||
|
notary:
|
||||||
|
enabled: false
|
||||||
|
chartmuseum:
|
||||||
|
enabled: false
|
||||||
|
|
||||||
|
# Single-node staging: keep resource usage modest
|
||||||
|
core:
|
||||||
|
replicas: 1
|
||||||
|
jobservice:
|
||||||
|
replicas: 1
|
||||||
|
registry:
|
||||||
|
replicas: 1
|
||||||
|
|
||||||
|
persistence:
|
||||||
|
enabled: true
|
||||||
|
persistentVolumeClaim:
|
||||||
|
# Use existing default storage (local-path) until Ceph is ready.
|
||||||
|
# After Ceph, switch to rook-ceph-block for Harbor's PVCs.
|
||||||
|
registry:
|
||||||
|
storageClass: local-path
|
||||||
|
size: 50Gi
|
||||||
|
jobservice:
|
||||||
|
storageClass: local-path
|
||||||
|
size: 5Gi
|
||||||
|
database:
|
||||||
|
storageClass: local-path
|
||||||
|
size: 10Gi
|
||||||
|
redis:
|
||||||
|
storageClass: local-path
|
||||||
|
size: 5Gi
|
||||||
|
|
||||||
|
database:
|
||||||
|
type: internal
|
||||||
|
|
||||||
|
redis:
|
||||||
|
type: internal
|
||||||
|
|
||||||
|
portal:
|
||||||
|
replicas: 1
|
||||||
|
|
||||||
|
# We will create proxy-cache projects after install (todo: configure-proxy-cache)
|
||||||
|
|
||||||
Reference in New Issue
Block a user