480 lines
22 KiB
Markdown
480 lines
22 KiB
Markdown
Project 11b - Hardware Specification v0.1
|
||
|
||
## 1. Requirements
|
||
|
||
Annual amount of data: 11 billion QR codes, each 50kb - 200kb.
|
||
- **Total storage:** 550 TB - 2.2 PB per year
|
||
- **Import window:** 1/20 to 1/10 of year (18-36 days)
|
||
- **Write throughput:** 3,480 - 6,960 QR codes/sec sustained
|
||
- **Data rate during import:** 348 MB/s - 1.39 GB/s sustained
|
||
- **Peak burst (3x):** up to 21K QR codes/sec, 4 GB/s
|
||
|
||
**Solution: 13 storage + 3 control/compute + 1 import node (JBOD + 3-replica, SINGLE RACK)**
|
||
- **Usable capacity:** 2.29 PB (covers ALL 11B QR codes at maximum 200KB size)
|
||
- **Raw capacity:** 6.86 PB (24 × 22TB × 13 storage nodes)
|
||
- **Aggregate IOPS:** ~1.25M (96K per node × 13, adequate for sequential QR storage)
|
||
- **Aggregate throughput:** 39 GB/s read, 19.5 GB/s write (exceeds 1.39 GB/s requirement)
|
||
- **Redundancy:** 3x replication (JBOD, software-managed, can tolerate 10 node failures)
|
||
- **Hardware (Lenovo + Huawei, RECOMMENDED):**
|
||
- 13x Lenovo ThinkSystem SR650 V2 storage nodes (24x 22TB SAS each)
|
||
- 3x Lenovo ThinkSystem SR630 V2 control/compute nodes (multi-purpose)
|
||
- 1x Lenovo ThinkSystem SR650 V2 import node (front-panel hot-plug: USB/NVMe/SATA/SAS)
|
||
- 2x Huawei CloudEngine 6800 switches (48-port 25GbE)
|
||
- **Import capability:** 10-32 GB/s physical media import (no internet bottleneck)
|
||
- **Cost (China, Lenovo+Huawei):** ¥1,973,000 CapEx (~$274K USD)
|
||
- **Note:** UPS excluded (out of project scope). Pricing based on Lenovo China and Huawei public pricing.
|
||
- **Fits in:** Single 42U rack (35U used, 7U for expansion)
|
||
|
||
## 2. Hardware Configuration
|
||
|
||
**Vendor: Lenovo (RECOMMENDED for China deployment)**
|
||
- **Servers:** Lenovo ThinkSystem SR650 V2 and SR630 V2
|
||
- Global brand with excellent China presence
|
||
- 20-25% cheaper than Dell in China market
|
||
- Same Intel Xeon CPUs, enterprise-grade quality
|
||
- Public pricing available: ~¥95,000-120,000 per configured node
|
||
- **Networking:** Huawei CloudEngine 6800
|
||
- Industry-leading in China market
|
||
- Public pricing: ~¥90,000 per 48-port 25GbE switch
|
||
- Better integration with China networks (China Telecom/Unicom/Mobile)
|
||
- **Management:** Lenovo XClarity Controller (IPMI/Redfish standard)
|
||
- **Support:** Lenovo China 4-hour onsite response
|
||
- **Drives:** Seagate Exos or WD Ultrastar (assembled in China, no import VAT)
|
||
|
||
### 2.1 Rack Layout (SINGLE 42U Standard Rack)
|
||
|
||
**Complete Deployment in One Rack (13 storage + 3 control/compute + 1 import node)**
|
||
|
||
- **U1-U2:** 2x Huawei CloudEngine 6800-48S-EI switches (48-port 25GbE, redundant, CSS)
|
||
- **U3-U4:** 2x Domestic PDUs (20x C13, 4x C19 outlets, 208V 3-phase, A+B feeds)
|
||
- **U5-U30:** 13x Storage nodes (2U each, Lenovo ThinkSystem SR650 V2 with 24x 22TB SAS)
|
||
- 2.29 PB usable (covers all 11B QR codes at max 200KB size)
|
||
- **U31-U33:** 3x Control/Compute nodes (1U each, Lenovo ThinkSystem SR630 V2, multi-purpose)
|
||
- Runs: Ingestion workers, API servers, load balancers, control plane
|
||
- All services on all 3 nodes for redundancy
|
||
- **U34-U35:** 1x Import node (2U, Lenovo ThinkSystem SR650 V2, front-panel hot-plug for physical media)
|
||
- USB 3.2 / NVMe / SATA / SAS hot-plug bays
|
||
- Direct physical media import (no network bottleneck)
|
||
- **U36-U42:** Reserved for future expansion (7U available)
|
||
|
||
**Total rack space used: 35U of 42U**
|
||
|
||
|
||
### 2.2 Network Design
|
||
|
||
**Top-of-Rack Switches:**
|
||
- Model: Huawei CloudEngine 6800-48S-EI (RECOMMENDED)
|
||
- 48x 25GbE SFP28 ports + 6x 100GbE QSFP28 uplink ports
|
||
- 2x redundant PSU (AC/DC)
|
||
- 2x redundant fans
|
||
- VRP OS (Huawei Versatile Routing Platform)
|
||
- Public pricing: ~¥90,000 per switch in China
|
||
- Industry-leading in China market, 25% cheaper than Dell
|
||
- Configuration:
|
||
- CSS (Cluster Switch System) pair for redundancy
|
||
- LACP bonding for all server connections (2x25GbE per node)
|
||
- VLANs: Management (VLAN10), Storage (VLAN20), API (VLAN30)
|
||
- Jumbo frames enabled (MTU 9000)
|
||
|
||
**Uplink:**
|
||
- 2x 100GbE fiber to core/distribution switches
|
||
- ECMP routing for load distribution
|
||
|
||
**Cables:**
|
||
- DAC (Direct Attach Copper) Twinax for intra-rack (< 5m)
|
||
- OM4 MMF or SMF for inter-rack connections
|
||
|
||
### 2.3 Power Design
|
||
|
||
**Power Requirements (Single Rack - With Import Node: 13 storage + 3 compute + 1 import):**
|
||
- Storage nodes: 13 nodes × 500W = 6,500W (HDDs use much less power than NVMe)
|
||
- Each HDD: ~10W idle, ~12W active
|
||
- 24 HDDs per node: ~300W
|
||
- CPU + RAM + fans: ~200W
|
||
- Control/Compute nodes: 3 nodes × 600W = 1,800W
|
||
- Multi-purpose nodes running all services
|
||
- Higher CPU utilization but consolidated
|
||
- Import node: 1 node × 700W = 700W
|
||
- Higher power due to NVMe staging drives + multiple controllers
|
||
- Peak during import operations
|
||
- Network switches: 2 switches × 400W = 800W
|
||
- **Total: ~9,800W = 9.8 kW**
|
||
- **With 30% headroom: 12.7 kW single rack**
|
||
|
||
**Power Summary:**
|
||
- Storage nodes: 6.5 kW (13 nodes × 500W)
|
||
- Control/Compute nodes: 1.8 kW (3 nodes × 600W)
|
||
- Import node: 0.7 kW (1 node × 700W)
|
||
- Network switches: 0.8 kW (2 switches × 400W)
|
||
- **Total: 9.8 kW (12.7 kW with 30% headroom)**
|
||
|
||
**PDU Configuration (Single Rack - 13 Storage Nodes):**
|
||
- 2x Domestic Metered Rack PDU (redundant feeds, A+B power)
|
||
- Input: 208V 3-phase, 30A per PDU
|
||
- Capacity: 208V × 30A × 1.732 = 10.8 kVA per PDU (3-phase)
|
||
- Total: 21.6 kVA for rack (sufficient for 12.7 kW load with headroom)
|
||
- Each server: Dual PSU connected to separate PDUs (A+B feeds)
|
||
|
||
|
||
**Cooling (Single Rack - With Import Node, 13 Storage Nodes):**
|
||
- Heat dissipation: 12.7 kW × 3.41 = 43,307 BTU/hr (single rack)
|
||
- Requirement: Cooling capacity for 43,400 BTU/hr (~3.6 tons)
|
||
- Options:
|
||
- Use existing IT room CRAC/HVAC (if capacity available)
|
||
- Add 1x 20kW in-row cooling unit (¥40,000-50,000)
|
||
- Standard precision AC unit (4-5 ton capacity)
|
||
- No special containment needed for single rack
|
||
|
||
### 2.4 Storage Node Specifications
|
||
|
||
**Storage Capacity Planning:**
|
||
- Maximum requirement: 2.2 PB per year
|
||
- Replication strategy affects raw capacity needs:
|
||
- RAID 10 + 2x replication: 4.4 PB raw (complex, dual-layer redundancy)
|
||
- JBOD + 3x replication: 6.6 PB raw (simpler, software-only redundancy)
|
||
|
||
**Architecture Choice: JBOD vs RAID**
|
||
|
||
**JBOD + 3-replica (RECOMMENDED for software simplicity):**
|
||
- No RAID controller complexity
|
||
- Software handles all redundancy (Ceph, MinIO, etc.)
|
||
- Each drive independent, easier failure recovery
|
||
- Better performance observability
|
||
- Standard approach for distributed storage
|
||
- Easier rebalancing and maintenance
|
||
|
||
**RAID 10 + 2-replica (traditional approach):**
|
||
- Hardware RAID provides local redundancy
|
||
- Lower network replication traffic
|
||
- Faster local reads (RAID striping)
|
||
- More complex failure scenarios (RAID + cluster)
|
||
- RAID rebuild overhead on drive failure
|
||
|
||
**Storage Node: 13 nodes with 22TB SAS HDD + JBOD + 3-replica (covers all 11B QR codes)**
|
||
|
||
**Model: Lenovo ThinkSystem SR650 V2 (2U, 24x 2.5" drive bays)**
|
||
**Public pricing: ~¥95,000-105,000 per configured node in China**
|
||
|
||
**Per Storage Node:**
|
||
- **CPU:** 2x Intel Xeon Silver 4316 (20 cores, 2.3 GHz, 30MB cache)
|
||
- Total: 40 cores, 80 threads per node
|
||
- Part#: 4XG7A42589
|
||
- **RAM:** 256 GB DDR4-3200 ECC RDIMM (8x 32GB)
|
||
- Part#: 4X77A08633 or equivalent
|
||
- **Boot:** 2x 480GB SATA SSD (RAID 1, OS)
|
||
- Lenovo 2.5" 6Gb SATA SSD
|
||
- **Storage:** 24x 22TB SAS HDD 12Gbps 7.2K RPM
|
||
- Seagate Exos X22 (ST22000NM00 series) or WD Ultrastar
|
||
- Public pricing: ~¥2,800-3,200 per drive (~$390-445)
|
||
- JBOD configuration (no RAID, all drives independent)
|
||
- Per node: 528 TB raw (24 × 22TB)
|
||
- Total cluster: 6.86 PB raw (13 nodes × 528 TB)
|
||
- With 3x replication: 2.29 PB usable
|
||
- Performance: 4K IOPS per drive, ~96K IOPS per node, 1.5 GB/s per node
|
||
- **Network:** 2x Lenovo ThinkSystem Broadcom 57504 25GbE (4-port, bonded)
|
||
- **HBA:** Lenovo ThinkSystem 430-8i SAS/SATA 12Gb HBA (pass-through mode)
|
||
- **PSU:** 2x 1100W Platinum (redundant, 208V)
|
||
- **Rack space:** 13 nodes × 2U = 26U
|
||
|
||
### 2.5 Import Node Specifications (1 node, dedicated for physical media import)
|
||
|
||
**Model: Lenovo ThinkSystem SR650 V2 (2U, front-accessible hot-plug)**
|
||
**Public pricing: ~¥140,000 per configured node**
|
||
|
||
**Per Node:**
|
||
- **CPU:** 2x Intel Xeon Gold 6338 (32 cores, 2.0 GHz, 48MB cache)
|
||
- Total: 64 cores, 128 threads
|
||
- High core count for parallel import processing
|
||
- **RAM:** 256 GB DDR4-3200 ECC
|
||
- Large buffer for staging imported data
|
||
- **Boot:** 2x 480GB SATA SSD (RAID 1, OS)
|
||
- **Import staging storage:** 4x 7.68TB NVMe (JBOD)
|
||
- Total: ~31TB staging buffer
|
||
- High-speed local cache before copying to storage cluster
|
||
- **Hot-plug bays (front panel):** 12 bays supporting:
|
||
- **USB 3.2 Gen 2x2:** 4x front-panel USB-C ports (2.5 GB/s each)
|
||
- **NVMe U.2:** 4x hot-swap NVMe bays (up to 8 GB/s per drive)
|
||
- **SATA/SAS:** 4x hot-swap 3.5" bays (12 Gbps SAS)
|
||
- **Controllers:**
|
||
- Lenovo ThinkSystem 430-8i SAS/SATA HBA (pass-through mode)
|
||
- PCIe Gen4 NVMe switch
|
||
- USB 3.2 Gen 2x2 controller
|
||
- **Network:** 2x 25GbE (bonded, LACP)
|
||
- **PSU:** 2x 1100W Platinum
|
||
|
||
**Import Workflow:**
|
||
1. Operator inserts physical media (USB, NVMe, SATA, SAS) into front panel
|
||
2. Drive auto-mounts or hot-detects
|
||
3. Import software reads data from media → staging NVMe (fast local)
|
||
4. Processes/validates QR codes in parallel (32-48 cores)
|
||
5. Streams validated data to storage cluster over 25GbE (or 100GbE)
|
||
6. Operator removes media when import complete (LED indicator)
|
||
|
||
**Performance:**
|
||
- **USB 3.2:** 4 ports × 2.5 GB/s = 10 GB/s aggregate
|
||
- **NVMe:** 4 drives × 8 GB/s = 32 GB/s aggregate
|
||
- **SATA/SAS:** 4 drives × 1.5 GB/s = 6 GB/s aggregate
|
||
- **Network:** 2x 25GbE = 6.25 GB/s to storage cluster
|
||
- Staging NVMe: 31TB buffer allows offline processing before network transfer
|
||
|
||
**Benefits:**
|
||
- **No internet/WiFi bottleneck:** Direct physical media import at full speed
|
||
- **Parallel import:** Accept up to 12 drives simultaneously
|
||
- **Flexible media:** USB, NVMe, SATA, SAS all supported
|
||
- **High throughput:** Local staging eliminates network bottleneck during read
|
||
- **Operator-friendly:** Front-panel access, LED indicators, hot-plug safe
|
||
|
||
### 2.6 Control/Compute Node Specifications (3 nodes, consolidated)
|
||
|
||
**Model: Lenovo ThinkSystem SR630 V2 (1U, multi-purpose)**
|
||
**Public pricing: ~¥102,000 per configured node**
|
||
|
||
**Per Node (runs ALL services):**
|
||
- **CPU:** 2x Intel Xeon Gold 6338 (32 cores, 2.0 GHz)
|
||
- Total: 64 cores, 128 threads per node
|
||
- Sufficient for ingestion + API + control plane + load balancing
|
||
- **RAM:** 256 GB DDR4-3200 ECC (8x 32GB)
|
||
- Split: 128GB for ingestion, 64GB for API, 64GB for system/control
|
||
- **Boot:** 2x 480GB SATA SSD (RAID 1, OS)
|
||
- **Local storage:** 2x 3.84TB NVMe
|
||
- Ingestion write buffer + API read cache + monitoring data
|
||
- **Network:** 2x Lenovo ThinkSystem Broadcom 57504 25GbE (bonded, LACP)
|
||
- **PSU:** 2x 800W Platinum
|
||
|
||
## 3. Network Topology
|
||
```
|
||
Core Switch (100GbE uplink)
|
||
↓
|
||
Rack ToR Switches (Dell VLT pair, 25GbE to servers)
|
||
↓
|
||
Dell Servers (dual-homed, LACP bonded)
|
||
```
|
||
|
||
## 4. Cost Estimate (Approximate)
|
||
|
||
**Hardware (Single Rack - 13 storage + 3 control/compute + 1 import for FULL 2.29 PB):**
|
||
- Storage nodes (13x): $195K
|
||
- Control/Compute nodes (3x): $48K
|
||
- Import node (1x): $22K
|
||
- Network switches (2x): $30K
|
||
- Other hardware & infrastructure: $13K
|
||
- **Total: $308K CapEx**
|
||
- **Note:** UPS excluded (out of project scope)
|
||
|
||
|
||
**Power Requirements:**
|
||
- **Total rack power:** 9.5kW maximum
|
||
- 13x storage nodes @ 400W each = 5.2kW
|
||
- 3x control/compute nodes @ 300W each = 0.9kW
|
||
- 1x import node @ 500W = 0.5kW
|
||
- 2x switches @ 400W each = 0.8kW
|
||
- Margin: ~1.6kW reserve
|
||
- **PDU requirements:** 2x 5kW PDUs (A+B feeds)
|
||
- **Cooling requirement:** ~12kW thermal (9.5kW × 1.3 PUE)
|
||
|
||
## 5. Redundancy & High Availability (Single Rack, 13 Storage Nodes)
|
||
|
||
- **Storage:** JBOD + 3x replication distributed across 13 Lenovo ThinkSystem SR650 V2 nodes
|
||
- **Any 10 storage nodes can fail simultaneously without data loss (77% redundancy)**
|
||
- 312 independent drives (24 per node × 13), software-managed
|
||
- Single drive failure: automatic rebalancing to other nodes
|
||
- No RAID rebuild overhead - software handles recovery
|
||
- Drive recovery: ~22TB over 25GbE = ~2 hours (vs days for RAID rebuild)
|
||
- Data striped and replicated for parallel I/O
|
||
- Minimum 4 nodes needed to maintain data availability (with 3x replication)
|
||
- **Control/Compute:** 3 multi-purpose nodes (active-active-active)
|
||
- All 3 nodes run: ingestion workers, API servers, load balancers, control plane
|
||
- Any 2 of 3 nodes can handle full workload (N+1 redundancy)
|
||
- Can lose 1 control/compute node without service interruption
|
||
- Load balanced via DNS round-robin or floating VIP (Keepalived)
|
||
- **Network:** Huawei VLT/CSS switches, dual-homed servers (2x25GbE bonded)
|
||
- **Power:** Dual PSU per server, redundant PDUs (A+B feeds)
|
||
|
||
**Expansion Path (if future requirements increase):**
|
||
- Year 1: 13 storage + 3 control/compute + 1 import = 2.29 PB usable (covers all 11B QR codes)
|
||
- Future expansion: 7U available in rack = up to 3 more 2U storage nodes
|
||
- With 16 storage nodes: 2.82 PB usable (23% more capacity)
|
||
- Beyond 16 nodes: Would require second rack or higher capacity drives
|
||
|
||
## 6. Performance Validation (HDD-based, Single Rack)
|
||
|
||
**Target metrics (13 storage nodes):**
|
||
- Write throughput: 6,960 QR codes/sec sustained (1.39 GB/s)
|
||
- With 3x replication: 4.17 GB/s network write traffic
|
||
- HDD capability: 19.5 GB/s aggregate write (13 nodes × 1.5 GB/s) ✓ **EXCELLENT**
|
||
- Peak burst: 21,000 QR codes/sec (4.2 GB/s)
|
||
- With 3x replication: 12.6 GB/s network write traffic
|
||
- HDD capability: 19.5 GB/s aggregate write (peak) ✓ **EXCELLENT** (55% headroom)
|
||
- Read latency: < 50ms P99 (with cache) ✓
|
||
- HDD seek: 4-8ms, sequential: fast
|
||
- Storage IOPS: 1.25M aggregate (96K per node × 13, 4K per drive)
|
||
- Adequate for large sequential QR code writes (50-200 KB each)
|
||
- QR codes are large blobs, not small random I/O
|
||
- Network throughput: 650 Gbps aggregate (50 Gbps per node × 13)
|
||
- Network utilization: < 8% during sustained writes, < 24% during peak burst
|
||
- Storage capacity: 2.29 PB usable (6.86 PB raw with 3x replication)
|
||
- **Covers ALL 11 billion QR codes at maximum 200KB size**
|
||
- 4% buffer above 2.2 PB maximum requirement
|
||
- Drive failure recovery: Distributed across all healthy nodes
|
||
- Redundancy: Can tolerate up to 10 storage node failures (77% redundancy)
|
||
|
||
**HDD Performance Characteristics:**
|
||
- Sequential throughput: Excellent (150-250 MB/s per drive)
|
||
- Random IOPS: Lower than NVMe (4K IOPS vs 100K+ IOPS)
|
||
- QR code workload: Mostly sequential large-blob writes (50-200 KB)
|
||
- **Verdict:** HDD is well-suited for this workload (large sequential I/O)
|
||
|
||
## 7. Recommended Configuration Summary (13 Storage Nodes for Full Capacity)
|
||
|
||
**Configuration: 13 storage + 3 control/compute + 1 import (JBOD + 3-replica, SINGLE RACK)**
|
||
|
||
| Metric | Specification |
|
||
|--------|---------------|
|
||
| **Storage Servers** | 13x Lenovo ThinkSystem SR650 V2 (2U each) |
|
||
| **Control/Compute** | 3x Lenovo ThinkSystem SR630 V2 (1U each) |
|
||
| **Import Node** | 1x Lenovo ThinkSystem SR650 V2 (2U) |
|
||
| **Switches** | 2x Huawei CE6800 (25GbE) |
|
||
| **Drives per Storage Node** | 24x 22TB SAS (12Gbps, 7.2K RPM) |
|
||
| **Usable Capacity** | **2.29 PB** (covers all 11B QR codes at 200KB max) |
|
||
| **Raw Capacity** | 6.86 PB (with 3x replication) |
|
||
| **Rack Space** | 35U of 42U (7U expansion available) |
|
||
| **Aggregate IOPS** | 1.25M (96K per storage node) |
|
||
| **Aggregate Throughput** | 39 GB/s read, 19.5 GB/s write |
|
||
| **Power** | 12.7 kW (9.8 kW actual, 30% headroom) |
|
||
| **CapEx (China, Lenovo+Huawei)** | ¥1,973,000 (~$274K USD) |
|
||
| **Storage Redundancy** | Can lose 10 storage nodes (77% redundancy) |
|
||
| **Compute Redundancy** | N+1 (any 2 of 3 nodes) |
|
||
| **Import Speed** | 10-32 GB/s (physical media) |
|
||
| **Drive Reliability** | SAS enterprise-grade |
|
||
| **Expansion** | 7U available = up to 3 more storage nodes |
|
||
|
||
**Key Features:**
|
||
- ✅ **Full capacity:** 2.29 PB covers ALL 11 billion QR codes at maximum 200KB size
|
||
- ✅ **High redundancy:** Can tolerate 10 simultaneous storage node failures
|
||
- ✅ **Single rack:** All equipment in one 42U rack (35U used)
|
||
- ✅ **Domestic brands:** 20% cheaper than Dell, excellent local support
|
||
- ✅ **Physical import:** 10-32 GB/s via USB/NVMe/SATA/SAS hot-plug
|
||
- ✅ **Industry-standard:** JBOD + 3-replica architecture
|
||
|
||
## 10. Cost Estimate for China Private Datacenter (CNY)
|
||
|
||
**Exchange Rate: 1 USD = 7.2 CNY (approximate, January 2026)**
|
||
|
||
**Summary for China (13 Storage Nodes, Full Capacity, IT Room):**
|
||
- **Recommended:** Lenovo ThinkSystem + Huawei CE6800
|
||
- **CapEx:** ¥1,973,000 (~$274K USD, based on public pricing)
|
||
- **Capacity:** 2.29 PB usable (covers ALL 11B QR codes at max 200KB size)
|
||
- **Power requirement:** 12.7 kW total (9.8 kW actual + 30% headroom)
|
||
- **Import node benefit:** 10-32 GB/s physical media import (USB/NVMe/SATA/SAS)
|
||
- Eliminates internet/WiFi bottleneck
|
||
- Front-panel hot-plug for operator convenience
|
||
- 31TB staging buffer for offline processing
|
||
|
||
### China Pricing - Lenovo ThinkSystem + Huawei Configuration
|
||
|
||
**Hardware (Lenovo + Huawei - 13 Storage Nodes for 2.29 PB):**
|
||
- Storage nodes (13x Lenovo ThinkSystem SR650 V2 with 24x 22TB SAS): ¥1,235,000
|
||
- Base server: ~¥35,000 per node
|
||
- 24x 22TB SAS drives @ ¥3,000 each: ¥72,000 per node
|
||
- CPU, RAM, HBA upgrades: ~¥23,000 per node
|
||
- Total per node: ¥95,000
|
||
- Public pricing reference: Lenovo China website
|
||
- Control/Compute nodes (3x Lenovo ThinkSystem SR630 V2): ¥306,000
|
||
- Each node: ¥102,000 (~$14,200)
|
||
- Multi-purpose: ingestion + API + load balancing + control
|
||
- Import node (1x Lenovo ThinkSystem SR650 V2 with hot-plug): ¥140,000
|
||
- Front-panel USB-C, NVMe U.2, SATA/SAS hot-swap
|
||
- Network switches (2x Huawei CloudEngine 6800-48S-EI): ¥180,000
|
||
- Public pricing: ¥90,000 per switch
|
||
- PDUs, cabling, misc: ¥64,000
|
||
- **Hardware Subtotal (Lenovo + Huawei): ¥1,925,000**
|
||
|
||
**Infrastructure (if not existing in IT room):**
|
||
- Rack (42U, domestic): ¥8,000
|
||
- Cooling (1x 20kW in-row, if IT room HVAC insufficient): ¥40,000
|
||
- **Infrastructure Subtotal: ¥48,000**
|
||
- **Note:** May be partially/fully available in existing IT room (reduce CapEx accordingly)
|
||
|
||
**Total CapEx (13 Storage Nodes for Full 2.29 PB):**
|
||
- **With Lenovo + Huawei: ¥1,973,000** (~$274K USD, RECOMMENDED)
|
||
- **Note:** UPS excluded (out of project scope). Pricing based on Lenovo China public pricing and Huawei enterprise quotes.
|
||
|
||
**Power Requirements (13 Storage Nodes):**
|
||
- **Total power:** 12.7 kW
|
||
- 13x storage nodes @ 400W each = 5.2 kW
|
||
- 3x control/compute nodes @ 500W each = 1.5 kW
|
||
- 1x import node @ 600W = 0.6 kW
|
||
- 2x switches @ 400W each = 0.8 kW
|
||
- Overhead and margin = 4.6 kW
|
||
- **Cooling requirement:** 16.5 kW thermal (12.7 kW × 1.3 PUE)
|
||
- **Circuit requirement:** 2x 32A 3-phase 208V circuits minimum
|
||
|
||
### HDD Sourcing in China
|
||
|
||
**Recommended: Locally assembled enterprise drives**
|
||
- Seagate Exos X22 22TB (Suzhou facility): ¥2,800-3,200 per drive
|
||
- Western Digital Ultrastar DC HC570 22TB (Shanghai/Shenzhen): ¥2,900-3,300 per drive
|
||
- Local assembly = no import VAT (save 13%)
|
||
- Comes with VAT invoice (增值税发票) for tax deduction
|
||
- Same warranty as international versions
|
||
- Faster replacement (local stock)
|
||
|
||
### Quick Reference: China Pricing Summary (13 Storage Nodes, 2.29 PB)
|
||
|
||
| Item | Value |
|
||
|------|-------|
|
||
| **CapEx** | ¥1,973,000 (~$274K USD) |
|
||
| **Capacity** | 2.29 PB usable |
|
||
| **Power** | 12.7 kW total |
|
||
| **Rack space** | 35U of 42U |
|
||
|
||
**Lenovo ThinkSystem SR650 V2 + SR630 V2 + Huawei CE6800**
|
||
- Covers ALL 11 billion QR codes at maximum 200KB size
|
||
- Import node: 10-32 GB/s physical media import (no network bottleneck)
|
||
- UPS excluded (out of project scope)
|
||
|
||
**Recommendations for China Deployment (IT Room, 13 Storage Nodes):**
|
||
1. **Use Lenovo ThinkSystem + Huawei** - public pricing available, 20% savings vs Dell
|
||
2. **IT room requirements (13 storage nodes for full 2.29 PB):**
|
||
- Power: 12.7 kW capacity needed (208V 3-phase, 30A × 2 PDUs)
|
||
- Cooling: 43,400 BTU/hr heat dissipation (~3.6 tons, 12.7 kW × 3,412 BTU/kW)
|
||
- Floor loading: ~1,100 kg total weight (17 servers + switches + rack)
|
||
- Operator access for import node (front-panel hot-plug)
|
||
- Rack space: 35U of 42U used (7U for future expansion)
|
||
3. **Local support contracts** essential (Lenovo + Huawei have excellent China support with 4-hour response)
|
||
5. **Bulk purchase discount** - negotiate 10-15% off for large orders
|
||
6. **Payment terms:** Net 30-90 common, some vendors offer 6-12 month financing
|
||
7. **VAT invoice (增值税发票):** Essential for tax deduction, ensure all vendors provide
|
||
8. **Domestic drives:** Buy Seagate/WD from China facilities to avoid import VAT
|
||
9. **Network integration:** Plan uplink to existing corporate network
|
||
|
||
**Vendor Contact (China):**
|
||
- Lenovo: DCG (Data Center Group) - https://www.lenovo.com/cn/zh/data-center (联想数据中心)
|
||
- Public pricing on website for ThinkSystem servers
|
||
- Contact: 400-100-6000 (China hotline)
|
||
- Huawei: Enterprise network division - https://e.huawei.com (华为企业)
|
||
- CloudEngine switches require enterprise quote
|
||
- Contact local Huawei account manager
|
||
- Local integrators: Often 5-10% cheaper than direct (e.g., 神州数码 Digital China)
|
||
|
||
## 11. Final Recommendation
|
||
|
||
**RECOMMENDED CONFIGURATION: Lenovo ThinkSystem + Huawei in Tier B City**
|
||
|
||
**Hardware (13 Storage Nodes for Full 2.29 PB Capacity):**
|
||
- **13x Lenovo ThinkSystem SR650 V2** storage nodes (24x 22TB SAS each) = **2.29 PB usable**
|
||
- **3x Lenovo ThinkSystem SR630 V2** control/compute nodes (multi-purpose)
|
||
- **1x Lenovo ThinkSystem SR650 V2** import node (front-panel hot-plug)
|
||
- **2x Huawei CloudEngine 6800-48S-EI** switches (48-port 25GbE)
|
||
|
||
**Capacity:** 2.29 PB usable - covers ALL 11 billion QR codes at maximum 200KB size
|
||
|
||
**Cost:**
|
||
- **CapEx:** ¥1,973,000 (~$274K USD)
|
||
- **Note:** UPS excluded (out of project scope). Based on Lenovo China and Huawei public pricing.
|
||
|
||
**Power Requirements:**
|
||
- **Total:** 12.7 kW (9.8 kW actual + 30% headroom)
|
||
- **Cooling:** 43,400 BTU/hr (~3.6 tons)
|
||
|