Documentation - MatsushibaDB

Getting Started

What is MatsushibaDB?

MatsushibaDB is a next-generation SQL database designed for modern applications. It provides high performance, advanced security features, and multi-protocol support for enterprise deployments.

Key Features

High Performance: Optimized SQL engine with connection pooling
Multi-Protocol: HTTP, WebSocket, and TCP support
Advanced Security: RBAC, audit logging, and encryption
Real-time Monitoring: Health checks and metrics
High Availability: Clustering and failover support
Easy Integration: Client libraries for multiple languages

Basic Concepts

Database

A single SQLite database file that contains all your data, tables, and indexes.

Connection

A network connection to the database server supporting multiple protocols.

Session

An authenticated user session with specific permissions and roles.

Transaction

ACID-compliant database transactions ensuring data consistency.

Quick Example

// Node.js Example
const { MatsushibaDBClient } = require('matsushibadb');

const client = new MatsushibaDBClient({
  host: 'localhost',
  port: 8000,
  username: 'admin',
  password: 'password'
});

async function example() {
  await client.connect();
  
  // Create table
  await client.query(`
    CREATE TABLE users (
      id INTEGER PRIMARY KEY,
      name TEXT NOT NULL,
      email TEXT UNIQUE
    )
  `);
  
  // Insert data
  await client.query(
    'INSERT INTO users (name, email) VALUES (?, ?)',
    ['John Doe', 'john@example.com']
  );
  
  // Query data
  const users = await client.query('SELECT * FROM users');
  console.log(users);
  
  await client.disconnect();
}

Architecture

System Overview

MatsushibaDB follows a modular architecture with clear separation of concerns:

Client Layer

NPM, PyPI, and Docker clients with language-specific APIs

Protocol Layer

HTTP, WebSocket, and TCP protocol handlers

Security Layer

Authentication, authorization, and encryption

Query Engine

SQL parser, optimizer, and executor

Storage Engine

SQLite with WAL mode and connection pooling

Components

Server

The main database server handling connections and queries.

Multi-protocol support
Connection pooling
Query optimization
Transaction management

Client Libraries

Language-specific client libraries for easy integration.

Node.js client
Python client
Async support
Connection pooling

Security Manager

Handles authentication, authorization, and encryption.

JWT authentication
Role-based access control
Audit logging
Data encryption

Monitoring

Health checks, metrics collection, and logging.

Health endpoints
Performance metrics
Structured logging
Alert system

Data Flow

1. Client connects via HTTP/WebSocket/TCP
2. Authentication and authorization
3. Query parsing and optimization
4. Transaction management
5. SQLite execution
6. Result formatting and return
7. Audit logging

Security

Authentication

MatsushibaDB uses JWT (JSON Web Tokens) for authentication:

// Login
const response = await fetch('http://localhost:8000/auth/login', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    username: 'admin',
    password: 'password'
  })
});

const { token } = await response.json();

// Use token in requests
const client = new MatsushibaDBClient({
  host: 'localhost',
  port: 8000,
  token: token
});

Role-Based Access Control (RBAC)

MatsushibaDB supports four built-in roles:

Admin

Full access to all operations

Create/Delete databases
Manage users and roles
System configuration
Audit logs access

User

Standard database operations

Read/Write data
Create/Modify tables
Execute queries
Manage own sessions

ReadOnly

Read-only access

SELECT queries only
View table structure
Read metadata
No modifications

Guest

Limited access

Basic queries
Limited tables
No admin functions
Session restrictions

Encryption

MatsushibaDB provides encryption at rest using AES-256-CBC:

// Enable encryption
const client = new MatsushibaDBClient({
  host: 'localhost',
  port: 8000,
  encryptionKey: 'your-encryption-key'
});

// Encrypted database operations
await client.query('CREATE TABLE encrypted_data (id INTEGER, data TEXT)');
await client.query('INSERT INTO encrypted_data VALUES (1, ?)', ['sensitive data']);

Audit Logging

All database operations are logged for security and compliance:

// Audit log entry example
{
  "timestamp": "2025-01-24T10:30:00Z",
  "user": "admin",
  "action": "SELECT",
  "table": "users",
  "query": "SELECT * FROM users WHERE id = ?",
  "parameters": [123],
  "ip": "192.168.1.100",
  "success": true,
  "execution_time": 15
}

Performance

Optimization Guidelines

Connection Pooling

MatsushibaDB uses connection pooling to improve performance:

// Configure connection pool
const client = new MatsushibaDBClient({
  host: 'localhost',
  port: 8000,
  poolSize: 10,        // Maximum connections
  poolTimeout: 30000,  // Connection timeout
  poolRetries: 3       // Retry attempts
});

Query Optimization

Follow these guidelines for optimal query performance:

Use prepared statements for repeated queries
Create appropriate indexes
Limit result sets with WHERE clauses
Use transactions for multiple operations

// Good: Prepared statement
const stmt = await client.prepare('SELECT * FROM users WHERE id = ?');
const user = await stmt.get(123);

// Good: Proper indexing
await client.query('CREATE INDEX idx_users_email ON users(email)');

// Good: Transaction for multiple operations
await client.transaction(async (tx) => {
  await tx.query('INSERT INTO users (name) VALUES (?)', ['John']);
  await tx.query('INSERT INTO profiles (user_id) VALUES (?)', [lastInsertId]);
});

Configuration Tuning

# Performance configuration
PRAGMA journal_mode=WAL;          # Write-Ahead Logging
PRAGMA synchronous=NORMAL;        # Balance safety/speed
PRAGMA cache_size=10000;          # 10MB cache
PRAGMA mmap_size=268435456;       # 256MB memory mapping
PRAGMA wal_autocheckpoint=1000;   # Checkpoint every 1000 pages
PRAGMA busy_timeout=30000;        # 30 second timeout

Benchmarks

Read Performance

10,000+ queries/sec

Simple SELECT queries on indexed columns

Write Performance

5,000+ inserts/sec

Bulk INSERT operations in transactions

Concurrent Users

1,000+ connections

Simultaneous connections with connection pooling

Memory Usage

50MB base

Minimal memory footprint with efficient caching

Deployment

Production Deployment

Docker Deployment

# docker-compose.yml
version: '3.8'
services:
  matsushibadb:
    image: matsushiba/matsushibadb:latest
    container_name: matsushibadb
    restart: unless-stopped
    ports:
      - "8000:8000"
      - "8001:8001"
      - "8002:8002"
    environment:
      - MATSUSHIBA_MODE=production
      - DATABASE_PATH=/app/data/matsushiba.db
      - LOG_LEVEL=info
      - JWT_SECRET=your-production-secret
    volumes:
      - matsushiba_data:/app/data
      - matsushiba_logs:/app/logs
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8005/health"]
      interval: 30s
      timeout: 10s
      retries: 3

volumes:
  matsushiba_data:
  matsushiba_logs:

Systemd Service

# /etc/systemd/system/matsushibadb.service
[Unit]
Description=MatsushibaDB Server
After=network.target

[Service]
Type=simple
User=matsushiba
Group=matsushiba
WorkingDirectory=/opt/matsushibadb
ExecStart=/usr/local/bin/matsushiba-db start
Restart=always
RestartSec=5
Environment=MATSUSHIBA_MODE=production
Environment=DATABASE_PATH=/opt/matsushibadb/data/matsushiba.db

[Install]
WantedBy=multi-user.target

High Availability

For high availability deployments, consider:

Load balancer with multiple instances
Database replication (read replicas)
Health checks and automatic failover
Backup and recovery procedures

Scaling

Vertical Scaling

Increase server resources
Adjust connection pool size
Optimize cache settings
Enable WAL mode

Horizontal Scaling

Multiple read replicas
Load balancer distribution
Database sharding
Microservices architecture

Monitoring

Health Checks

MatsushibaDB provides comprehensive health monitoring:

# Health check endpoint
curl http://localhost:8005/health

# Response
{
  "status": "healthy",
  "timestamp": "2025-01-24T10:30:00Z",
  "uptime": 86400,
  "version": "1.0.8",
  "database": {
    "status": "connected",
    "size": "50MB",
    "connections": 5
  },
  "memory": {
    "used": "25MB",
    "free": "75MB"
  },
  "performance": {
    "queries_per_second": 150,
    "average_response_time": 5
  }
}

Metrics

Prometheus-compatible metrics are available at /metrics:

# Prometheus metrics
matsushibadb_queries_total{type="select"} 1500
matsushibadb_queries_total{type="insert"} 500
matsushibadb_queries_total{type="update"} 200
matsushibadb_queries_total{type="delete"} 50

matsushibadb_connections_active 5
matsushibadb_connections_total 100

matsushibadb_response_time_seconds{quantile="0.5"} 0.005
matsushibadb_response_time_seconds{quantile="0.95"} 0.020
matsushibadb_response_time_seconds{quantile="0.99"} 0.050

Logging

Structured logging with configurable levels:

# Log configuration
LOG_LEVEL=info
LOG_FILE=/app/logs/matsushiba.log
LOG_FORMAT=json
LOG_ROTATION=daily
LOG_RETENTION=30

# Log entry example
{
  "timestamp": "2025-01-24T10:30:00Z",
  "level": "info",
  "message": "Query executed successfully",
  "query": "SELECT * FROM users",
  "execution_time": 5,
  "user": "admin",
  "ip": "192.168.1.100"
}

Alerting

Configure alerts for critical events:

High error rate (>5%)
Slow queries (>1 second)
Connection pool exhaustion
Disk space low (<10%)
Memory usage high (>90%)