Santa Rally Local LLM API

내부 서비스와 에이전트가 바로 연결할 수 있는 OpenAI-compatible Gemma 4 API입니다.

OpenAI-compatible Streaming SSE Vision input Internal use

Connection

Docs URL
https://server5.tailb3d7cf.ts.net/docs/
Base URL
https://server5.tailb3d7cf.ts.net/v1
Model
gemma-4-26b-a4b-it
OpenAPI
https://server5.tailb3d7cf.ts.net/openapi.json
Agent notes
https://server5.tailb3d7cf.ts.net/llms.txt
Monitor
https://server5.tailb3d7cf.ts.net/monitor/

현재 이 엔드포인트는 내부 테스트/서비스 연동 편의를 위해 no-key로 열려 있고, nginx rate limit만 적용되어 있습니다. 공개 제품 API처럼 배포하지 말고 내부 서비스 서버에서만 호출하세요.

Supported

  • GET /v1/models
  • POST /v1/chat/completions
  • stream: true SSE streaming
  • OpenAI-style image_url content for image input
  • POST /v1/completions
  • POST /tokenize
  • GET /health
  • GET /monitor/ recent request/response samples
  • GET /monitor/stats request latency and server CPU/RAM metrics

Limits

  • Public proxy rate limit: 24r/m, burst 12
  • Request body limit: 8m
  • Recommended app timeout: 60-120s
  • Recommended concurrency: start with 1-2
  • Image generation and embeddings are not supported by this model server.

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://server5.tailb3d7cf.ts.net/v1",
  apiKey: "internal-no-key"
});

const response = await client.chat.completions.create({
  model: "gemma-4-26b-a4b-it",
  messages: [
    { role: "system", content: "한국어로 간결하게 답한다." },
    { role: "user", content: "산타랠리 로컬 LLM API가 무엇인지 설명해줘." }
  ],
  temperature: 0.2,
  max_tokens: 256
});

console.log(response.choices[0].message.content);

Streaming

curl -N https://server5.tailb3d7cf.ts.net/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-26b-a4b-it",
    "stream": true,
    "messages": [
      {"role": "user", "content": "한 문단으로 답해줘."}
    ],
    "max_tokens": 256,
    "temperature": 0.2
  }'

Vision Input

{
  "model": "gemma-4-26b-a4b-it",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "이 이미지를 한국어로 요약해줘." },
        { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,..." } }
      ]
    }
  ],
  "max_tokens": 300,
  "temperature": 0.2
}

Operational Contract