EdgeXpert DGX Spark에 Ollama 올려서 LLM 서버로 쓰려고 설정.

하드웨어 확인.

nvidia-smi

NVIDIA GB10
CUDA Version: 13.0
Total Memory: 119.6 GiB
Available Memory: 115.7 GiB

lscpu

Architecture: aarch64
CPU: Cortex-X925 + Cortex-A725
CPU cores: 20

lsblk

NVMe: 931.5G
OS: Ubuntu 22.04 LTS

Docker 권한 설정.

sudo usermod -aG docker ioniere
newgrp docker

docker ps

Ollama 설치.

curl -fsSL https://ollama.ai/install.sh | sh

설치 로그.

>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-arm64.tar.zst
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> NVIDIA GPU installed.
>>> The Ollama API is now available at 127.0.0.1:11434

처음 실행.

nohup ollama serve &

로그.

time=2026-02-10T19:12:42.439+09:00 level=INFO msg="Listening on 127.0.0.1:11434"
time=2026-02-10T19:12:43.294+09:00 level=INFO msg="inference compute" 
  id=GPU-4b3a0335-7002-d8d7-4b25-e60076f87725 
  description="NVIDIA GB10" 
  total="119.6 GiB" 
  available="115.5 GiB"

모델 다운로드.

ollama pull mistral

pulling manifest
pulling f5074b1221da: 100% ▕█████████████████████████▏ 4.4 GB
verifying sha256 digest
writing manifest
success

로컬 테스트.

ollama run mistral

API 테스트.

curl http://127.0.0.1:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "Hello, how are you?",
  "stream": false
}'

응답.

{
  "model": "mistral",
  "response": "I'm an artificial intelligence and don't have feelings, but I'm here to help you!",
  "total_duration": 7128211507,
  "load_duration": 13630859,
  "prompt_eval_count": 10,
  "eval_count": 65
}

기본 상태에서는 127.0.0.1에만 붙어서 외부 접근이 안 됨.

0.0.0.0으로 다시 실행.

kill %1

nohup env OLLAMA_HOST=0.0.0.0:11434 ollama serve > nohup.out 2>&1 &

로그.

time=2026-02-10T19:24:44.865+09:00 level=INFO msg="Listening on [::]:11434"

내부망 테스트.

curl http://192.168.200.100:11434/api/generate \
  -d '{"model":"mistral","prompt":"test","stream":false}'

모니터링.

watch nvidia-smi

ps aux | grep ollama

netstat -tlnp | grep 11434

일단 Mistral 7B부터 올림.

119.6GB VRAM이면 훨씬 큰 모델도 가능해 보임.