Initial commit: COSMOS - turn your old PC to workstation
This commit is contained in:
@@ -0,0 +1,94 @@
|
||||
# 🌌 COSMOS Architecture & Technical Specifications
|
||||
|
||||
Cosmos is an open-source, self-hosted cloud platform engineered to transform ordinary consumer hardware or retired office computers into an enterprise-class personal workstation and private cloud.
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ High-Level Topology
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ Client / Web │
|
||||
└──────────────────────┬───────────────────────┘
|
||||
│ HTTPS / SSL (:443)
|
||||
▼
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ Nginx Reverse Proxy │
|
||||
└──────┬────────────────┬───────────────┬──────┘
|
||||
│ │ │
|
||||
HTTP / │ HTTP / │ HTTP / │
|
||||
Portals │ Gitea │ MinIO │
|
||||
(:8080) │ (:3000) │ (:9000) │
|
||||
▼ ▼ ▼
|
||||
┌───────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ Cosmos Core │ │ Gitea Forge │ │ MinIO S3 │
|
||||
│ (Go App) │ │ (Git Engine)│ │(Object Store│
|
||||
└───────┬───────┘ └─────────────┘ └──────┬──────┘
|
||||
│ │
|
||||
│ CQL (:9042) │ Local Storage
|
||||
▼ ▼
|
||||
┌───────────────┐ ┌─────────────┐
|
||||
│ ScyllaDB │ │ Local Disks │
|
||||
│ (NoSQL Store) │ │(NVMe / HDD) │
|
||||
└───────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧩 Core Architectural Components
|
||||
|
||||
### 1. Unified Identity & Application Core (`portals-go`)
|
||||
* **Language:** Go (Golang) 1.22+
|
||||
* **Serving:** Native `net/http` server with embedded HTML/CSS templates via `embed.FS`.
|
||||
* **Single Sign-On (SSO):** A centralized identity provider managing accounts across S3 Storage, Git Source Control, and CDN.
|
||||
* **Security Model:**
|
||||
* Passwords hashed with `golang.org/x/crypto/bcrypt`.
|
||||
* Temporary registration passwords stored in transient, self-expiring memory structures (`sync.Map`) only during automated provisioning—never stored plain-text in database storage.
|
||||
* Session identification using cryptographically random UUIDs stored in HTTP-only cookies and ScyllaDB secondary index lookups.
|
||||
|
||||
### 2. Distributed Persistence Layer (`ScyllaDB`)
|
||||
* **Database Engine:** ScyllaDB 5.x / Apache Cassandra compatibility mode.
|
||||
* **Protocol:** CQL native binary protocol over port `9042`.
|
||||
* **Keyspace Model:**
|
||||
* `cosmos.users`: Master user directory holding bcrypt hashes, S3 credentials, display metadata, and active session tokens.
|
||||
* `cosmos.pending`: Ephemeral registration tokens and OTP codes.
|
||||
* `cosmos.password_reset_tokens`: Time-expiring password reset tokens.
|
||||
* `cosmos.buckets`: User bucket metadata, ownership tracking, and friendly aliases.
|
||||
* `cosmos.bucket_domains`: Vanity domain and CNAME mapping for edge serving.
|
||||
* `cosmos.quota_requests`: Administrative quota elevation queue.
|
||||
* `cosmos.s3_analytics` & `cosmos.s3_analytics_hourly`: Telemetry tracking counters.
|
||||
|
||||
### 3. High-Performance Object Storage (`MinIO S3`)
|
||||
* **Engine:** MinIO S3-compatible object storage daemon.
|
||||
* **Automated Tenant Provisioning:**
|
||||
* Upon account activation, the Cosmos core automatically invokes the MinIO Client (`mc`) to provision dedicated IAM users.
|
||||
* Each user is constrained by an isolated AWS IAM policy restricting `s3:*` actions solely to their assigned bucket ARN (`arn:aws:s3:::cosmos-<id>/*`).
|
||||
* Automatic object versioning is enabled by default to prevent accidental data loss.
|
||||
* Storage quotas (default 10GB, expandable via admin approval) are enforced at the bucket level.
|
||||
|
||||
### 4. Git Engine & Source Control (`Gitea`)
|
||||
* **Engine:** Gitea self-hosted Git service running on port `3000`.
|
||||
* **Authentication Sync:**
|
||||
* Cosmos manages registration and credential synchronization via CLI/API hooks (`gitea admin user create`, `gitea admin user change-password`).
|
||||
* Auto-healing authentication logic: if credentials fall out of sync during login, Cosmos automatically re-synchronizes the bcrypt hash with Gitea's internal database and retries transparently.
|
||||
|
||||
### 5. Edge Caching & CDN Simulation
|
||||
* **Caching Engine:** In-memory high-concurrency cache (`sync.Map`) with TTL expiration.
|
||||
* **Headers:** Standard HTTP cache control headers (`Cache-Control: public, max-age=...`), ETag generation, and conditional request handling (`If-None-Match`).
|
||||
* **Cache Bypass:** Diagnostic and instant-purge queries (`?nocache=1` or `Cache-Control: no-cache`).
|
||||
|
||||
### 6. Real-Time Telemetry & Observability
|
||||
* **In-Memory Accumulator:** High-throughput `atomic.Int64` counters for request counts, bytes transferred, cache hits/misses, and status code categories (2xx, 4xx, 5xx).
|
||||
* **Periodic Flushing:** Background goroutine flushes aggregated metrics to ScyllaDB hourly tables (`cosmos.s3_analytics_hourly`).
|
||||
* **Visual Telemetry:** Interactive 3D rotating orthographic globe rendered on HTML5 Canvas using vector latitude/longitude projections, simulating real-time global edge node activity.
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Hardening
|
||||
|
||||
1. **Least-Privilege Storage Isolation:**
|
||||
Users can never view or manipulate storage belonging to other accounts. Every S3 access request is validated against ScyllaDB session keys and verified via MinIO IAM policies.
|
||||
2. **Environment Variable Decoupling:**
|
||||
All sensitive endpoints, administrative emails, domain names, and database credentials are fully externalized via `.env` configurations.
|
||||
3. **Transient Memory Buffering:**
|
||||
Credentials required for downstream provisioning (such as Gitea account creation) exist in memory for a maximum 15-minute window before automatic garbage collection.
|
||||
@@ -0,0 +1,138 @@
|
||||
# 🛠️ COSMOS Setup & Deployment Guide
|
||||
|
||||
This guide details how to turn an old desktop PC, home server, or cloud instance into a fully functional Cosmos workstation.
|
||||
|
||||
---
|
||||
|
||||
## 💻 Hardware Prerequisites
|
||||
|
||||
| Requirement | Minimum | Recommended |
|
||||
|---|---|---|
|
||||
| **CPU** | 2 cores (x86_64) | 4+ cores |
|
||||
| **RAM** | 4 GB | 8 GB or more |
|
||||
| **Storage** | 30 GB available | SSD / NVMe drive for ScyllaDB + HDD for bulk S3 storage |
|
||||
| **OS** | Debian 12 / Ubuntu 22.04 LTS | Debian 12 Minimal |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Option 1: Quick Start with Docker Compose (Recommended)
|
||||
|
||||
The fastest method to deploy the entire stack on an old PC is using Docker and Docker Compose.
|
||||
|
||||
### Step 1: Install Docker & Docker Compose
|
||||
```bash
|
||||
sudo apt update && sudo apt install -y curl git docker.io docker-compose
|
||||
sudo systemctl enable --now docker
|
||||
```
|
||||
|
||||
### Step 2: Clone & Configure
|
||||
```bash
|
||||
git clone https://github.com/vaishnosingh/COSMOS.git cosmos
|
||||
cd cosmos
|
||||
|
||||
# Copy and configure environment variables
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
```
|
||||
|
||||
### Step 3: Launch Stack
|
||||
```bash
|
||||
cd deploy
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Step 4: Initialize ScyllaDB Schema
|
||||
Wait 15-20 seconds for ScyllaDB to start, then execute:
|
||||
```bash
|
||||
docker exec -i cosmos-scylla cqlsh < schema.cql
|
||||
```
|
||||
|
||||
Access your workstation portal at `http://<your-pc-ip>:8080`.
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Option 2: Bare-Metal Native Setup
|
||||
|
||||
For maximum performance on hardware with limited RAM, install the native binaries directly on the host system.
|
||||
|
||||
### 1. Install Prerequisites
|
||||
```bash
|
||||
sudo apt update && sudo apt install -y golang git nginx curl msmtp ca-certificates
|
||||
```
|
||||
|
||||
### 2. Install MinIO & MinIO Client (mc)
|
||||
```bash
|
||||
# MinIO Server
|
||||
wget https://dl.min.io/server/minio/release/linux-amd64/minio
|
||||
chmod +x minio && sudo mv minio /usr/local/bin/
|
||||
|
||||
# MinIO Client
|
||||
wget https://dl.min.io/client/mc/release/linux-amd64/mc
|
||||
chmod +x mc && sudo mv mc /usr/local/bin/
|
||||
```
|
||||
|
||||
Start MinIO as a systemd service or background daemon:
|
||||
```bash
|
||||
export MINIO_ROOT_USER=minioadmin
|
||||
export MINIO_ROOT_PASSWORD=your_secure_password
|
||||
minio server /var/lib/minio/data --console-address ":9001" &
|
||||
```
|
||||
|
||||
### 3. Install ScyllaDB or Apache Cassandra
|
||||
```bash
|
||||
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5e08f18826677713
|
||||
sudo curl -s https://repositories.scylladb.com/scylla/repo/deb/debian/scylla-5.4.list -o /etc/apt/sources.list.d/scylla.list
|
||||
sudo apt update && sudo apt install -y scylla-server
|
||||
sudo systemctl enable --now scylla-server
|
||||
```
|
||||
|
||||
Apply the database schema:
|
||||
```bash
|
||||
cqlsh -f deploy/schema.cql
|
||||
```
|
||||
|
||||
### 4. Install Gitea
|
||||
```bash
|
||||
sudo wget -O /usr/local/bin/gitea https://dl.gitea.com/gitea/1.21.11/gitea-1.21.11-linux-amd64
|
||||
sudo chmod +x /usr/local/bin/gitea
|
||||
```
|
||||
|
||||
### 5. Build and Launch Cosmos
|
||||
```bash
|
||||
cd /opt/cosmos
|
||||
go build -o portals .
|
||||
sudo cp deploy/cosmos.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now cosmos
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🌐 Production Domain & SSL (Nginx + Certbot)
|
||||
|
||||
To access your workstation securely across the internet:
|
||||
|
||||
### 1. Configure Nginx
|
||||
```bash
|
||||
sudo cp deploy/nginx.conf /etc/nginx/sites-available/cosmos.conf
|
||||
sudo ln -s /etc/nginx/sites-available/cosmos.conf /etc/nginx/sites-enabled/
|
||||
# Edit server_name to match your registered domain:
|
||||
sudo nano /etc/nginx/sites-available/cosmos.conf
|
||||
```
|
||||
|
||||
### 2. Generate Free SSL Certificate
|
||||
```bash
|
||||
sudo apt install -y certbot python3-certbot-nginx
|
||||
sudo certbot --nginx -d yourdomain.com -d *.yourdomain.com
|
||||
sudo systemctl restart nginx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Verification Checklist
|
||||
|
||||
- [ ] **Portal Home:** Visit `https://yourdomain.com/` (or `http://localhost:8080/`).
|
||||
- [ ] **Account Creation:** Sign up for an account and check OTP verification.
|
||||
- [ ] **Storage Console:** Open `/oort/console` and test uploading/downloading a file.
|
||||
- [ ] **Git Engine:** Open `/source/` to inspect repositories and commit via SSH/HTTPS.
|
||||
- [ ] **Live Telemetry:** Open the Analytics tab to view real-time request tracking and the 3D globe.
|
||||
Reference in New Issue
Block a user