Beyond Art

By the numbers

Baselines from the first playable slice (local melee forge + slow vision score) versus the shipped Mode B build on itch.io.

Distribution measured as public Windows builds on itch.io (versions 0.1.0 and 0.1.1). Swap in dashboard download / view / play counts when you want an exact player metric.

The hook

Most sandboxes give you a menu of items. Beyond Art gives you a blank canvas. Want to smash a building? Draw a sword, axe, or boxing gloves. Want to run faster? Draw legs. Want a ranged loadout? Draw a pistol, rifle, or bow — then draw the projectile art.

Three ideas stacked into one fantasy:

• Creative expression — invent solutions like Scribblenauts, not a fixed shop.
• Skill expression — better, clearer drawings map to stronger stats (crafting-quality fantasy).
• Emergent gameplay — players invent tools; the systems react with validation, stats, and equip.

“The better you draw, the better it works” — the entire product promise in one sentence.

Player loop

1. Explore a stylized third-person city sandbox (low-poly / modular plaza).
2. Open the forge (draw UI) for melee weapons, ranged weapons, or body slots.
3. Sketch on a 1024×1024 render-target canvas with brush, color wheel, clear/confirm.
4. Confirm → export PNG → send to the Judge Bridge → validate (+ reject with a clear reason) or score stats.
5. Equip the drawing as a flat textured weapon / projectile / body customization and play.

Invalid drawings are hard-blocked with a friendly message (empty canvas, obvious rectangles, wrong subject). Valid drawings always produce something usable, with stats clamped so balance doesn’t explode.

Gameplay systems shipped

Recognized / scored subjects

• Sword
• Axe
• Spear
• Dagger
• Pistol
• Rifle
• Bow
• Bazooka
• Legs
• Arms
• Generic weapon

AI judge architecture

The game never embeds API keys. Unreal talks only to a small Python service — judge_bridge.py — over HTTP/HTTPS JSON.

UE game  →  HTTPS POST (image_base64 + metadata)
         →  Judge Bridge (Render / local)
         →  Gemini Flash  (or Ollama / local ink)
         →  JSON stats back to Blueprints

Backends (env-selected)

• local — instant ink heuristics (fast Blueprint testing).
• gemini — Google Gemini Flash vision (primary cloud path).
• ollama — local moondream / vision models for offline AI.
• openai — optional ChatGPT vision.

Why a bridge?

• Blueprint HTTP is awkward for Base64 + long vision latency.
• Secrets stay on the server (Render env), never in the Shipping package.
• Game updates ≠ judge updates: repackage the game vs redeploy the bridge.
• Hybrid scoring: AI for recognition, deterministic ink metrics for fair, repeatable stats.

Validate → score pipeline

Confirm runs two async HTTP calls (not one giant request):

Example validate body

{
  "image_base64": "<png bytes as base64>",
  "image_path": "C:/.../weapon.png",
  "expected_subject": "sword",
  "description": "sword",
  "context": "weapon"
}

GET /judge/ping gates online vs offline fallback so the game stays playable if the bridge is down.

Mode B — cloud deploy

Local Mode A sent a disk image_path to a Python process on the same PC. Packaged builds can’t do that for strangers. Mode B:

1. UE exports PNG → reads bytes → Base64 string.
2. Game POSTs JSON to https://artgame-judge.onrender.com.
3. Bridge decodes Base64 to a temp PNG, runs validate/score, returns JSON.
4. GEMINI_API_KEY lives only in Render environment variables.

Cloud vs home network reality

Free-tier Gemini often works from a residential IP but fails from cloud datacenter IPs (403 / empty candidates). The bridge therefore:

• Soft-accepts clear ink doodles when Gemini is noisy.
• Falls back to local ink validate when Gemini throws — so itch players still pass swords/pistols/legs.
• Uses instant ink scoring for Gemini score (avoids long hangs on “Judging…”).

Deployed as a Docker web service on Render (Dockerfile + render.yaml), health check on /judge/ping, auto-deploy from GitHub (Saarth09/ArtGame_Judge).

Unreal engineering notes

• Async HTTP in Blueprints — Http Request + bind On Request Complete; orchestration via custom events.
• JSON — Json Blueprint Utilities; string bools ("true"/"false") for reliable field reads.
• Long timeouts — DefaultEngine.ini [HTTP] activity timeouts raised to 180s for cold starts + vision latency.
• Plugins — HttpBlueprint, JsonBlueprintUtilities, BlueprintFileUtils, FileSDK, Low Entry Extended Standard Library.
• Contexts — weapon / ranged / body_legs / body_arms route finish/fallback equip functions.
• PNG alpha trap — UE export often writes alpha=0; bridge flattens RGB for visible debug previews.

Hard problems solved

Shipping & update model

Live download: saarth09.itch.io/beyondart. Steam is feasible later with the same Shipping build; Mode B judge stays on Render.

Stack & what I owned

Solo project end-to-end: design, UE gameplay/UI, Python judge service, cloud deploy, packaging, and distribution.

• Unreal Engine 5.8
• Blueprints
• C++ plugins
• Python 3
• Pillow
• Gemini Flash
• Ollama / moondream
• Docker
• Render
• itch.io / butler
• HTTP JSON APIs
• Computer vision
• Prompt engineering

Status

Playable Windows build public on itch.io. Core loop live: draw → validate → score → equip for melee, ranged, and body. Ongoing: balance tuning, more object classes, grip polish, and optional Steam release.