資料分析平台部署

Redash、ClickHouse 與 Redpanda 容器化部署指南。

📊 Redash 商業智能

部署指南

• Redash

核心內容：

• Redash 安裝部署
• 資料源配置
• 查詢與視覺化
• 儀表板設計
• 使用者管理

🗄️ ClickHouse 列式資料庫

基礎配置

• ClickHouse

核心內容：

• ClickHouse 簡介
• 資料模型
• 查詢語法
• 效能優化

部署設置

• ClickHouse Setup

核心內容：

• Docker 部署
• 配置文件
• 資料導入
• 叢集設置
• 監控與維護

🔄 Redpanda 事件流平台

部署指南

• Redpanda

核心內容：

• Redpanda 簡介（Kafka 相容）
• Docker 部署
• 主題與分區
• 生產者與消費者
• 效能調優

💡 Redash 部署實戰

docker-compose.yml

version: '3.8'

services:
  server:
    image: redash/redash:latest
    command: server
    depends_on:
      - postgres
      - redis
    ports:
      - "5000:5000"
    environment:
      REDASH_WEB_WORKERS: 4
      REDASH_DATABASE_URL: postgresql://postgres:password@postgres/redash
      REDASH_REDIS_URL: redis://redis:6379/0
      REDASH_SECRET_KEY: supersecretkey

  scheduler:
    image: redash/redash:latest
    command: scheduler
    depends_on:
      - server

  worker:
    image: redash/redash:latest
    command: worker
    depends_on:
      - server

  postgres:
    image: postgres:13-alpine
    environment:
      POSTGRES_DB: redash
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    volumes:
      - postgres_data:/var/lib/postgresql/data

  redis:
    image: redis:6-alpine

volumes:
  postgres_data:

🗄️ ClickHouse 部署

docker-compose.yml

version: '3.8'

services:
  clickhouse:
    image: clickhouse/clickhouse-server:latest
    ports:
      - "8123:8123"  # HTTP
      - "9000:9000"  # Native
    volumes:
      - clickhouse_data:/var/lib/clickhouse
      - ./config.xml:/etc/clickhouse-server/config.xml
    ulimits:
      nofile:
        soft: 262144
        hard: 262144

volumes:
  clickhouse_data:

基本使用

# 進入 ClickHouse 客戶端
docker exec -it clickhouse clickhouse-client

# 建立資料庫
CREATE DATABASE test;

# 建立表
CREATE TABLE test.events (
    date Date,
    user_id UInt32,
    event_type String
) ENGINE = MergeTree()
PARTITION BY toYYYYMM(date)
ORDER BY (date, user_id);

# 插入資料
INSERT INTO test.events VALUES
    ('2024-01-01', 1, 'click'),
    ('2024-01-01', 2, 'view');

# 查詢
SELECT * FROM test.events;

🔄 Redpanda 部署

docker-compose.yml

version: '3.8'

services:
  redpanda:
    image: docker.redpanda.com/redpandadata/redpanda:latest
    command:
      - redpanda
      - start
      - --smp 1
      - --memory 1G
      - --reserve-memory 0M
      - --overprovisioned
      - --node-id 0
      - --kafka-addr PLAINTEXT://0.0.0.0:29092,EXTERNAL://0.0.0.0:9092
      - --advertise-kafka-addr PLAINTEXT://redpanda:29092,EXTERNAL://localhost:9092
    ports:
      - "9092:9092"
      - "9644:9644"  # Admin API
    volumes:
      - redpanda_data:/var/lib/redpanda/data

  console:
    image: docker.redpanda.com/redpandadata/console:latest
    ports:
      - "8080:8080"
    environment:
      KAFKA_BROKERS: redpanda:29092
    depends_on:
      - redpanda

volumes:
  redpanda_data:

Python 生產者/消費者

from kafka import KafkaProducer, KafkaConsumer
import json

# 生產者
producer = KafkaProducer(
    bootstrap_servers=['localhost:9092'],
    value_serializer=lambda v: json.dumps(v).encode('utf-8')
)

producer.send('events', {'user_id': 1, 'action': 'click'})
producer.flush()

# 消費者
consumer = KafkaConsumer(
    'events',
    bootstrap_servers=['localhost:9092'],
    value_deserializer=lambda m: json.loads(m.decode('utf-8'))
)

for message in consumer:
    print(message.value)

📊 整合應用範例

資料流架構

version: '3.8'

services:
  # 事件流
  redpanda:
    image: docker.redpanda.com/redpandadata/redpanda:latest
    # ... 配置 ...

  # 資料倉儲
  clickhouse:
    image: clickhouse/clickhouse-server:latest
    # ... 配置 ...

  # 資料視覺化
  redash:
    image: redash/redash:latest
    # ... 配置 ...

  # ETL 應用
  etl:
    build: ./etl
    depends_on:
      - redpanda
      - clickhouse
    environment:
      KAFKA_BROKERS: redpanda:29092
      CLICKHOUSE_HOST: clickhouse

最後更新: 2025-12-01