Arca Storage

Software-Defined Storage system with Storage Virtual Machine (SVM) functionality, built using Linux standard technologies.

Overview

Arca Storage is a Software-Defined Storage system that provides NetApp ONTAP-like SVM functionality using Linux standard technologies:

Multi-protocol: NFS v4.1 / v4.2 (default), with optional NFSv3 support
Multi-tenancy: Per-SVM Ganesha bind addresses, with optional Network Namespace/VLAN isolation
High Availability: Pacemaker-based Active/Active failover
Data Efficiency: LVM Thin Provisioning with overcommit
Client Integration: Kubernetes (CSI) (docs) and OpenStack (Cinder NFS Driver, Manila) support

Architecture

The system combines:

Pacemaker + Corosync: HA clustering and resource management
NFS-Ganesha: User-space NFS server (one process per SVM)
Network Namespace: Optional tenant network isolation when VLAN-backed SVMs are used
XFS: NVMe-optimized filesystem
LVM Thin Provisioning: Virtual volume management and snapshots
DRBD: Node-to-node synchronous data mirroring

Internal Design

The Python codebase follows a declarative reconciliation architecture:

Resource Models (models/): Pydantic models with Spec (desired state) and Status (actual state) for SVM, Volume, Snapshot, and Export resources.
Reconcilers (reconcilers/): Idempotent reconciliation loops that drive resources from desired to actual state, one step at a time. Each step is persisted so that retries resume from the last successful point.
Adapters (adapters/): Protocol-based abstractions for system operations (LVM, XFS, Network Namespace, Pacemaker, NFS-Ganesha, systemd). Production implementations call real commands; Fake implementations enable in-memory testing without root privileges.
State Store (db/): SQLite WAL-backed state database with ACID transactions, replacing the previous JSON file-based state.
Structured Errors (errors.py): Machine-readable error codes (e.g., NOT_FOUND, ALREADY_EXISTS) that map to HTTP status codes and are consumed by the Go CSI driver.
Unified Config (config.py): TOML-based configuration validated through Pydantic, replacing the dual INI boot/runtime config.
Application Context (context.py): Dependency wiring — a single AppContext provides the DB, adapters, and reconcilers to both CLI and API.

Quick Start

Prerequisites

RHEL/Alma/Rocky Linux 8/9, Debian, or Ubuntu
Pacemaker/Corosync/pcs, NFS-Ganesha, LVM2, DRBD installed
2-node cluster configuration

Installation

Install OS dependencies (example):

# EL9 (RHEL/Alma/Rocky 9)
sudo dnf install -y pacemaker corosync pcs resource-agents \
  nfs-ganesha nfs-ganesha-utils \
  lvm2 xfsprogs \
  drbd-utils drbd-kmod

# Debian/Ubuntu (package names may vary)
sudo apt-get update
sudo apt-get install -y pacemaker corosync pcs resource-agents \
  nfs-ganesha \
  lvm2 xfsprogs \
  drbd-utils

Install arca-storage package (rpm/deb):

Download the latest package from GitHub Releases and install it.

# EL9 (rpm)
sudo dnf install -y ./arca-storage-*.rpm

# Debian/Ubuntu (deb)
sudo apt-get install -y ./arca-storage_*.deb

Follow MVP setup guide:

See docs/mvp-setup.md for detailed setup instructions.

Configuration

NFSv3 Support (Optional)

By default, Arca Storage uses NFSv4 only. To enable NFSv3 support:

Edit config:

In /etc/arca-storage/config.toml:

[ganesha]
# Enable NFSv3 (use both v3 and v4)
protocols = [3, 4]

# Fixed ports (recommended when using NFSv3)
mountd_port = 20048
nlm_port = 32768

Re-render configs and reload services:

# Keep env file in sync (optional but recommended after config edits)
sudo arca bootstrap render-env

# Re-render per-SVM ganesha.conf and reload
sudo arca export sync --all

Required firewall ports when NFSv3 is enabled:

111/tcp,udp   (rpcbind/portmapper)
2049/tcp,udp  (NFS)
20048/tcp,udp (mountd)
32768/tcp,udp (NLM)

Client mount examples:

# NFSv4 (default)
mount -t nfs4 server:/101 /mnt

# NFSv3 (when enabled)
mount -t nfs -o vers=3 server:/exports /mnt

Note: When using NFSv3, ensure rpcbind is installed and running. Both NFSv3 and NFSv4 protocols will be available simultaneously.

Usage

CLI Tool (arca)

# Bootstrap (without Ansible)
sudo arca bootstrap install

# (Optional) edit config
sudo vi /etc/arca-storage/config.toml

# Re-generate /etc/arca-storage/arca-storage.env after editing config
sudo arca bootstrap render-env

# Create an SVM
# --vlan is optional. Without it, Ganesha binds directly to the SVM VIP in the host namespace.
# --gateway is optional for VLAN-backed SVMs; if omitted, it is inferred from --ip (except /31,/32)
arca svm create tenant_a --vlan 100 --ip 192.168.10.5/24
# or without a VLAN:
arca svm create tenant_b --ip 192.168.20.5/32

# Create a volume
arca volume create vol1 --svm tenant_a --size 100

# Add an export
arca export add --volume vol1 --svm tenant_a --client 10.0.0.0/24 --access rw

# List SVMs
arca svm list

REST API

Start the API server with bearer-token authentication:

export ARCA_API_TOKEN="$(openssl rand -hex 32)"
arca-storage-api --host 127.0.0.1 --port 8080

When binding outside loopback, provide TLS certificates so bearer tokens are not sent over plain HTTP:

export ARCA_API_TOKEN="$(openssl rand -hex 32)"
arca-storage-api --host 0.0.0.0 --port 8443 \
  --ssl-certfile /etc/arca-storage/tls/api.crt \
  --ssl-keyfile /etc/arca-storage/tls/api.key

Use the same token for client requests:

curl -H "Authorization: Bearer <token>" http://localhost:8080/v1/svms

Or run it as a systemd service (when installed via package):

API_TOKEN="$(openssl rand -hex 32)"
sudo install -m 0600 /dev/null /etc/arca-storage/api.env
printf 'ARCA_API_TOKEN=%s\n' "$API_TOKEN" | sudo tee /etc/arca-storage/api.env >/dev/null
unset API_TOKEN
sudo systemctl enable --now arca-storage-api

For loopback-only development, you may explicitly allow unauthenticated access:

unset ARCA_API_TOKEN ARCA_AUTH_TOKEN
ARCA_ALLOW_UNAUTHENTICATED_LOOPBACK=true arca-storage-api --host 127.0.0.1 --port 8080

API endpoints:

POST /v1/svms - Create SVM
GET /v1/svms - List SVMs
GET /v1/svms/{name} - Get SVM details
DELETE /v1/svms/{name} - Delete SVM
POST /v1/directories - Create a directory for CSI-managed volumes
DELETE /v1/directories/{svm_name} - Delete a directory
POST /v1/quotas - Set a directory quota
PATCH /v1/quotas - Expand a directory quota
GET /v1/quotas/{svm_name} - Get a directory quota
POST /v1/volumes - Create volume
GET /v1/volumes - List volumes
PATCH /v1/volumes/{name} - Resize volume
DELETE /v1/volumes/{name} - Delete volume
POST /v1/volumes/{name}/clone - Clone a volume from a snapshot
PATCH /v1/volumes/{name}/qos - Apply QoS limits
GET /v1/volumes/{name}/qos - Get QoS settings
DELETE /v1/volumes/{name}/qos - Remove QoS limits
POST /v1/exports - Add export
GET /v1/exports - List exports
DELETE /v1/exports - Remove export
POST /v1/snapshots - Create snapshot
GET /v1/snapshots - List snapshots
DELETE /v1/snapshots/{name} - Delete snapshot

The interactive API documentation is available at http://localhost:8080/docs. When token authentication is enabled, this endpoint is protected like the API; use the explicit loopback-only unauthenticated development mode for browser access to Swagger UI.

OpenStack (Cinder)

See docs/openstack-cinder.md.

OpenStack (Manila)

See docs/openstack-manila.md.

Project Structure

arca-storage/
├── arca_storage/               # Python package
│   ├── arca_storage/           # Package source code
│   │   ├── api/                # FastAPI REST API
│   │   │   ├── main.py         # API application & error handlers
│   │   │   ├── models.py       # Request/Response Pydantic models
│   │   │   └── services/       # Service layer (delegates to reconcilers)
│   │   ├── cli/                # CLI tool (Typer)
│   │   │   ├── cli.py          # Main CLI entry
│   │   │   ├── commands/       # Command implementations
│   │   │   └── lib/            # Validators, helpers
│   │   ├── models/             # Resource models (Spec/Status)
│   │   │   ├── svm.py          # SVM resource
│   │   │   ├── volume.py       # Volume resource
│   │   │   ├── snapshot.py     # Snapshot resource
│   │   │   └── export.py       # Export resource
│   │   ├── reconcilers/        # Reconciliation loops
│   │   │   ├── svm.py          # SVM reconciler
│   │   │   ├── volume.py       # Volume reconciler
│   │   │   ├── snapshot.py     # Snapshot reconciler
│   │   │   └── export.py       # Export reconciler
│   │   ├── adapters/           # System operation adapters
│   │   │   ├── lvm.py          # LVM adapter (Protocol + Subprocess + Fake)
│   │   │   ├── xfs.py          # XFS adapter
│   │   │   ├── netns.py        # Network Namespace adapter
│   │   │   ├── pacemaker.py    # Pacemaker adapter
│   │   │   ├── ganesha.py      # NFS-Ganesha adapter
│   │   │   └── systemd.py      # systemd adapter
│   │   ├── db/                 # SQLite WAL state store
│   │   ├── errors.py           # Structured error codes
│   │   ├── config.py           # TOML config (Pydantic)
│   │   ├── context.py          # Dependency wiring (AppContext)
│   │   ├── openstack/          # OpenStack drivers (Cinder, Manila)
│   │   ├── resources/          # Pacemaker RA, systemd units
│   │   └── templates/          # Configuration templates
│   ├── tests/                  # Test suite
│   │   ├── unit/               # Unit tests (models, errors, db, reconcilers)
│   │   └── integration/        # Integration tests (CLI, API, scenarios)
│   ├── pyproject.toml          # Package configuration
│   └── pytest.ini              # Test configuration
├── csi-arca-storage/           # Go CSI driver
│   ├── pkg/arca/               # ARCA API client & structured errors
│   ├── cmd/                    # CLI entry point
│   └── deploy/                 # Kubernetes manifests
├── ansible/                    # Ansible playbooks
│   ├── roles/                  # Ansible roles
│   └── site.yml                # Main playbook
├── docs/                       # Project documentation
│   └── mvp-setup.md            # MVP setup guide
└── README.md                   # This file

Development

Setup Development Environment

# Clone repository
git clone https://github.com/akam1o/arca-storage.git
cd arca-storage/arca_storage

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install in development mode with dev dependencies
pip install -e ".[dev]"

Running Tests

cd arca_storage

# Run all tests
pytest

# Run with coverage
pytest --cov=arca_storage --cov-report=html

# Run specific tests
pytest tests/unit/
pytest tests/integration/

Code Style

Follow PEP 8 for Python code.

Documentation

docs/mvp-setup.md - MVP setup guide
csi-arca-storage/docs/quickstart.md - CSI quick start
csi-arca-storage/docs/deployment.md - CSI deployment guide
csi-arca-storage/docs/deployment-checklist.md - CSI deployment checklist
arca_storage/arca_storage/resources/pacemaker/ - Pacemaker RA documentation
arca_storage/arca_storage/resources/systemd/ - systemd unit files
arca_storage/arca_storage/templates/ - Template documentation

License

Apache License 2.0

Contributing

Contributions are welcome! See CONTRIBUTING.md.

Contact

For inquiries, open an issue on GitHub Issues. For security reports, use GitHub Security Advisories.

Status

This project is in active development. The MVP implementation is complete, but additional features and optimizations are planned.

Name		Name	Last commit message	Last commit date
Latest commit History 519 Commits
.github		.github
ansible		ansible
arca_storage		arca_storage
csi-arca-storage		csi-arca-storage
docs		docs
packaging		packaging
.gitignore		.gitignore
CONTRIBUTING.ja.md		CONTRIBUTING.ja.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.ja.md		README.ja.md
README.md		README.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Arca Storage

Overview

Architecture

Internal Design

Quick Start

Prerequisites

Installation

Configuration

NFSv3 Support (Optional)

Usage

CLI Tool (arca)

REST API

OpenStack (Cinder)

OpenStack (Manila)

Project Structure

Development

Setup Development Environment

Running Tests

Code Style

Documentation

License

Contributing

Contact

Status

About

Releases

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

Arca Storage

Overview

Architecture

Internal Design

Quick Start

Prerequisites

Installation

Configuration

NFSv3 Support (Optional)

Usage

CLI Tool (arca)

REST API

OpenStack (Cinder)

OpenStack (Manila)

Project Structure

Development

Setup Development Environment

Running Tests

Code Style

Documentation

License

Contributing

Contact

Status

About

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Contributors

Languages