cerc-io
/
stack-orchestrator
mirror of https://github.com/cerc-io/stack-orchestrator
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity
stack-orchestrator/stack_orchestrator/constants.py

55 lines
2.0 KiB
Python
Raw Normal View History

Support external stack file (#650)
2023-11-15 03:59:48 +00:00
# Copyright © 2023 Vulcanize
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http:#www.gnu.org/licenses/>.
Generate a unique deployment id for each deployment (#680) * Move cluster name generation into a function * Generate a unique deployment id for each deployment
2023-12-06 05:56:58 +00:00
cluster_name_prefix = "laconic-"
Support external stack file (#650)
2023-11-15 03:59:48 +00:00
stack_file_name = "stack.yml"
Propagate env file for webapp deployment (#669)
2023-11-29 04:14:02 +00:00
spec_file_name = "spec.yml"
config_file_name = "config.env"
Generate a unique deployment id for each deployment (#680) * Move cluster name generation into a function * Generate a unique deployment id for each deployment
2023-12-06 05:56:58 +00:00
deployment_file_name = "deployment.yml"
compose_dir_name = "compose"
Changes for remote k8s (#655)
2023-11-20 16:12:57 +00:00
compose_deploy_type = "compose"
k8s_kind_deploy_type = "k8s-kind"
k8s_deploy_type = "k8s"
Generate a unique deployment id for each deployment (#680) * Move cluster name generation into a function * Generate a unique deployment id for each deployment
2023-12-06 05:56:58 +00:00
cluster_id_key = "cluster-id"
k8s: shared-cluster safety checks and deployment-id decoupling (#748) - **Kind extraMount compatibility**: fail fast at `deployment start` when a new deployment's mounts don't match the running cluster; warn when the first cluster is created without a `kind-mount-root` umbrella; replace the cryptic `ConfigException` with readable errors when the cluster is missing - **Auto-ConfigMap for file-level host-path compose volumes** (so-7fc): `../config/foo.sh:/opt/foo.sh`-style binds become per-namespace ConfigMaps at deploy start instead of aliasing via the kind extraMount chain. `deploy create` rejects `:rw`, subdirs, and over-budget sources. Deployment-dir layout unchanged - **Namespace ownership**: stamp the namespace with `laconic.com/deployment-dir` on create; fail loudly if another deployment tries to land in the same namespace. Pre-existing namespaces adopt ownership on next start - **deployment-id / cluster-id decoupling**: split the two roles (kube context vs resource-name prefix) into separate `deployment.yml` fields. Backward-compat fallback keeps existing resource names stable - Close stale pebbles `so-n1n` and `so-ad7`
2026-04-21 06:47:28 +00:00
deployment_id_key = "deployment-id"
Changes for remote k8s (#655)
2023-11-20 16:12:57 +00:00
kube_config_key = "kube-config"
Add image push command (#656)
2023-11-21 03:23:55 +00:00
deploy_to_key = "deploy-to"
Support for k8s ingress and tls (#659)
2023-11-21 23:04:36 +00:00
network_key = "network"
http_proxy_key = "http-proxy"
For k8s, use provisioner-managed volumes when an absolute host path is not specified. (#741) In kind, when we bind-mount a host directory it is first mounted into the kind container at /mnt, then into the pod at the desired location. We accidentally picked this up for full-blown k8s, and were creating volumes at /mnt. This changes the behavior for both kind and regular k8s so that bind mounts are only allowed if a fully-qualified path is specified. If no path is specified at all, a default storageClass is assumed to be present, and the volume managed by a provisioner. Eg, for kind, the default provisioner is: https://github.com/rancher/local-path-provisioner ``` stack: test deploy-to: k8s-kind config: test-variable-1: test-value-1 network: ports: test: - '80' volumes: # this will be bind-mounted to a host-path test-data-bind: /srv/data # this will be managed by the k8s node test-data-auto: configmaps: test-config: ./configmap/test-config ``` Reviewed-on: https://git.vdb.to/cerc-io/stack-orchestrator/pulls/741 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-14 21:45:01 +00:00
image_registry_key = "image-registry"
configmaps_key = "configmaps"
Add Job and secrets support for k8s-kind deployments (#995) Part of https://plan.wireit.in/deepstack/browse/VUL-315 Reviewed-on: https://git.vdb.to/cerc-io/stack-orchestrator/pulls/995 Co-authored-by: Prathamesh Musale <prathamesh.musale0@gmail.com> Co-committed-by: Prathamesh Musale <prathamesh.musale0@gmail.com>
2026-03-11 03:56:21 +00:00
secrets_key = "secrets"
For k8s, use provisioner-managed volumes when an absolute host path is not specified. (#741) In kind, when we bind-mount a host directory it is first mounted into the kind container at /mnt, then into the pod at the desired location. We accidentally picked this up for full-blown k8s, and were creating volumes at /mnt. This changes the behavior for both kind and regular k8s so that bind mounts are only allowed if a fully-qualified path is specified. If no path is specified at all, a default storageClass is assumed to be present, and the volume managed by a provisioner. Eg, for kind, the default provisioner is: https://github.com/rancher/local-path-provisioner ``` stack: test deploy-to: k8s-kind config: test-variable-1: test-value-1 network: ports: test: - '80' volumes: # this will be bind-mounted to a host-path test-data-bind: /srv/data # this will be managed by the k8s node test-data-auto: configmaps: test-config: ./configmap/test-config ``` Reviewed-on: https://git.vdb.to/cerc-io/stack-orchestrator/pulls/741 Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-02-14 21:45:01 +00:00
resources_key = "resources"
volumes_key = "volumes"
security_key = "security"
annotations_key = "annotations"
labels_key = "labels"
Support multiple NodePorts, static NodePort mapping, and add 'replicas' spec option (#913) NodePort example: ``` network: ports: caddy: - 1234 - 32020:2020 ``` Replicas example: ``` replicas: 2 ``` This also adds an optimization for k8s where if a directory matching the name of a configmap exists in beneath config/ in the stack, its contents will be copied into the corresponding configmap. For example: ``` # Config files in the stack ❯ ls stack-orchestrator/config/caddyconfig Caddyfile Caddyfile.one-req-per-upstream-example # ConfigMap in the spec ❯ cat foo.yml | grep config ... configmaps: caddyconfig: ./configmaps/caddyconfig # Create the deployment ❯ laconic-so --stack ~/cerc/caddy-ethcache/stack-orchestrator/stacks/caddy-ethcache deploy create --spec-file foo.yml # The files from beneath config/<config_map_name> have been copied to the ConfigMap directory from the spec. ❯ ls deployment-001/configmaps/caddyconfig Caddyfile Caddyfile.one-req-per-upstream-example ``` Reviewed-on: https://git.vdb.to/cerc-io/stack-orchestrator/pulls/913 Reviewed-by: David Boreham <dboreham@noreply.git.vdb.to> Co-authored-by: Thomas E Lackey <telackey@bozemanpass.com> Co-committed-by: Thomas E Lackey <telackey@bozemanpass.com>
2024-08-09 02:32:06 +00:00
replicas_key = "replicas"
Add support for k8s pod to node affinity and taint toleration (#917) Reviewed-on: https://git.vdb.to/cerc-io/stack-orchestrator/pulls/917 Reviewed-by: Thomas E Lackey <telackey@noreply.git.vdb.to> Co-authored-by: David Boreham <david@bozemanpass.com> Co-committed-by: David Boreham <david@bozemanpass.com>
2024-08-15 20:32:58 +00:00
node_affinities_key = "node-affinities"
node_tolerations_key = "node-tolerations"
Changes for remote k8s (#655)
2023-11-20 16:12:57 +00:00
kind_config_filename = "kind-config.yml"
kube_config_filename = "kubeconfig.yml"
Add unlimited-memlock support for Kind clusters Add spec.yml option `security.unlimited-memlock` that configures RLIMIT_MEMLOCK to unlimited for Kind cluster pods. This is needed for workloads like Solana validators that require large amounts of locked memory for memory-mapped files during snapshot decompression. When enabled, generates a cri-base.json file with rlimits and mounts it into the Kind node to override the default containerd runtime spec. Also includes flake8 line-length fixes for affected files. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 01:20:19 +00:00
cri_base_filename = "cri-base.json"
unlimited_memlock_key = "unlimited-memlock"
Add RuntimeClass support for unlimited RLIMIT_MEMLOCK The previous approach of mounting cri-base.json into kind nodes failed because we didn't tell containerd to use it via containerdConfigPatches. RuntimeClass allows different stacks to have different rlimit profiles, which is essential since kind only supports one cluster per host and multiple stacks share the same cluster. Changes: - Add containerdConfigPatches to kind-config.yml to define runtime handlers - Create RuntimeClass resources after cluster creation - Add runtimeClassName to pod specs based on stack's security settings - Rename cri-base.json to high-memlock-spec.json for clarity - Add get_runtime_class() method to Spec that auto-derives from unlimited-memlock setting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 06:58:38 +00:00
runtime_class_key = "runtime-class"
high_memlock_runtime = "high-memlock"
high_memlock_spec_filename = "high-memlock-spec.json"
Fix Caddy ingress ACME email and RBAC issues - Add acme_email_key constant for spec.yml parsing - Add get_acme_email() method to Spec class - Modify install_ingress_for_kind() to patch ConfigMap with email - Pass acme-email from spec to ingress installation - Add 'delete' verb to leases RBAC for certificate lock cleanup The acme-email field in spec.yml was previously ignored, causing Let's Encrypt to fail with "unable to parse email address". The missing delete permission on leases caused lock cleanup failures. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 00:13:10 +00:00
acme_email_key = "acme-email"
feat: add kind-mount-root for unified Kind extraMount When kind-mount-root is set in spec.yml, emit a single extraMount mapping the root to /mnt instead of per-volume mounts. This allows adding new volumes without recreating the Kind cluster. Volumes whose host path is under the root skip individual extraMounts and their PV paths resolve to /mnt/{relative_path}. Volumes outside the root keep individual extraMounts as before. Cherry-picked from branch enya-ac868cc4-kind-mount-propagation-fix (commits b6d6ad81, 929bdab8) and adapted for current main. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 21:28:40 +00:00
kind_mount_root_key = "kind-mount-root"
feat(k8s): manage Caddy ingress image lifecycle via spec (so-p3p) The Caddy ingress image was hardcoded in the component manifest and had no update path shy of cluster recreate or manual kubectl patch. That forced woodburn to run an out-of-band ansible playbook to bump Caddy, and broke the "spec.yml is source of truth" model. Changes: - spec.yml: new `caddy-ingress-image` key (default `ghcr.io/laconicnetwork/caddy-ingress:latest`). - Deployment manifest: `strategy: Recreate` on the Caddy Deployment — required because the pod binds hostPort 80/443, which prevents any rolling update from completing (new pod hangs Pending forever waiting for old pod to release the ports). - install_ingress_for_kind: accepts caddy_image and templates the manifest before applying, same pattern as the existing acme-email templating. - update_caddy_ingress_image: patches the running Caddy Deployment when the spec image differs from the live image. No-op if they match. Returns True if a patch was applied so the caller can wait for the rollout. - deploy_k8s._setup_cluster: on cluster reuse (ingress already up), reconcile the running image against the spec. Installs path unchanged; only the "already running, maybe needs update" branch is new. Cluster-scoped caveat: caddy-system is shared by every deployment on the cluster, so the spec value in any one deployment rolls Caddy for all of them — last `deployment start` wins. Documented in deployment_patterns.md alongside the other cluster-scoped concerns (kind-mount-root, namespace ownership). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 06:51:53 +00:00
caddy_ingress_image_key = "caddy-ingress-image"
default_caddy_ingress_image = "ghcr.io/laconicnetwork/caddy-ingress:latest"
Add external-services and ca-certificates spec keys New spec.yml features for routing external service dependencies: external-services: s3: host: example.com # ExternalName Service (production) port: 443 s3: selector: {app: mock} # headless Service + Endpoints (testing) namespace: mock-ns port: 443 ca-certificates: - ~/.local/share/mkcert/rootCA.pem # testing only laconic-so creates the appropriate k8s Service type per mode: - host mode: ExternalName (DNS CNAME to external provider) - selector mode: headless Service + Endpoints with pod IPs discovered from the target namespace at deploy time ca-certificates mounts CA files into all containers at /etc/ssl/certs/ and sets NODE_EXTRA_CA_CERTS for Node/Bun. Also includes the previously committed PV Released state fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 15:25:47 +00:00
external_services_key = "external-services"
ca_certificates_key = "ca-certificates"