RAM DEVELOPMENT LAB

Local Intelligence Feasibility Spike v3

Second-pass Safari-only benchmark. It never reads RAM data. It uses only synthetic RAM-style text. Internet is used only to download test runtimes/models; inference runs on this iPhone.

0. Run order

  1. Run Check device.
  2. Run the three Encoder Challenge tests.
  3. Run deep candidates in order: Qwen2.5 1.5B stable-runtime control → Gemma 3 1B → Qwen3 1.7B → Qwen3.5 0.8B.
  4. Only if those are stable, optionally run Qwen3.5 2B.
  5. Tap Copy diagnostics and paste the entire result into the RAM development chat.

Do not retry a model repeatedly if Safari reloads during it. Reopen the page; v3 records the last phase reached. Completed results survive reloads.

1. Device capability

Not run.

2. Harder semantic encoder challenge

All three encoders already passed the easy v2 screen. This uses 15 harder RAM-style retrieval probes with closer distractors and reports Top-1 accuracy, mean reciprocal rank, and average winner margin.

MiniLM-L6

Not run.

BGE-small v1.5

Not run.

mxbai-embed-xsmall

Not run.

3. Structured local parser — realistic one-Capture-at-a-time test

Unlike v2, each model stays loaded but receives one Capture per inference, which matches how RAM would actually use a deep interpreter. Grounding is stricter: every extracted span must be an exact substring of the Capture, and temporal/condition fields must also be grounded.

Qwen2.5 1.5B · stable runtime control

Same model class that reached v2 output, but WebLLM is pinned to 0.2.82 and the harness is corrected.

Not run.

Gemma 3 1B

Very low stated WebLLM VRAM requirement (~711 MB); interesting quality-per-memory candidate.

Not run.

Qwen3 1.7B

Stronger modern candidate. Pinned to WebLLM 0.2.82 for the stability comparison.

Not run.

Qwen3.5 0.8B

Newer compact candidate; requires current WebLLM model support.

Not run.

Optional: Qwen3.5 2B

~2.25 GB stated WebLLM VRAM. Run only if the earlier candidates complete without Safari instability.

Not run.

Deep-parser cases

  1. call mom tomorrow buy milk tonight finish report now
  2. don't buy milk I already got it but grab eggs tomorrow
  3. I'm exhausted but I still need to finish UWorld questions tonight
  4. after Sarah sends the file review it then email John
  5. find the glue gun and find my Apple Watch
  6. finish 10 questions from set 6 then do set 8

Diagnostics

No results yet.