LLM on Margrop Blog

Why Does AI Lie So Confidently? — It Is Not Thinking, It Is Playing Supercharged Word Association

Sun, 19 Jul 2026 20:30:00 +0800

The short answer

AI hallucination is not simply an occasional bug. It is a predictable risk of a system whose basic job is to generate the next likely token from context. The model can produce a fluent, confident, highly structured answer without having verified that every claim is supported by evidence.

The easiest analogy is this: a large language model is a supercharged word-association player. It is exceptionally good at continuing a sentence. It is not automatically a fact-checker.

Why Does AI Keep Saying ‘I Forgot’? Tokens, Context Windows, and the Sticky Notes It Must Throw Away

Sun, 19 Jul 2026 18:30:00 +0800

The short version

AI often appears to have a bad memory because only a limited amount of material can be carried into the model for the current request. Think of a context window as a stack of sticky notes with finite capacity. New questions and tool outputs keep adding notes. Once the stack is full, the system must summarize, truncate, filter, or remove older notes. If a note is removed from the current context, the model may no longer be able to use it.

Have you ever told an AI, “Use Python for every code example,” only to receive JavaScript fifteen turns later? Or supplied a long document that the model can summarize accurately at the beginning and end, but cannot answer about the important restriction buried in the middle?

It is tempting to say that the AI “forgot.” A more precise explanation is this: the model can only see what fits inside the current context window, and seeing a piece of text does not guarantee that every part of it receives equal attention.

This article explains tokens and context windows without turning them into intimidating mathematics. The main analogy is a stack of sticky notes. We will look at what tokens are, how a request is assembled, why long conversations become unreliable, why agents fill their context quickly, and how to make AI forget less through simple information management.

Stop Sprinkling API Keys Everywhere: My Self-Hosted NewAPI Relay Station

Fri, 10 Jul 2026 15:30:00 +0800

TL;DR

NewAPI is not a free-model trick and it is not a mysterious proxy. In my setup, it is the front desk for AI usage: every app talks to one endpoint, while the gateway routes requests to authorized upstream providers and handles tokens, quotas, groups, model limits, logs, and usage accounting. For a personal deployment, the big win is not “one more dashboard”; it is no longer scattering upstream keys across every client, script, and agent.

This post explains why I wanted a self-hosted model relay, what people usually misunderstand, how to deploy a minimal Docker Compose version, why I add persistent storage, and how to use one-click scripts for Windows 11, Ubuntu 26.04, and macOS 26. It also includes an agent-driven setup prompt with strict boundaries. No private address, complete machine name, private domain, or real secret is included.

I dockerized AnythingLLM and the container turned into a wound-up squirrel — restarting forever until I handed it that magic 1000:1000

Wed, 17 Jun 2026 20:30:00 +0800

TL;DR

AnythingLLM, Mintplex Labs’ “stuff-anything-in-a-LLM” desktop, ships an official Docker image. In the happy path one docker run is all you need. But the in-container user anythingllm is picky: the host directory you bind-mount has to be owned by 1000:1000, otherwise it can’t write its own SQLite file, the Prisma migration step on startup crashes, and the container enters a restart-crash-restart loop. The fix takes under a minute: chown -R 1000:1000 /your/data/dir, then docker compose up -d again, and it dutifully listens on 3001.

I’ll also explain why Prisma’s file:../storage/anythingllm.db relative path is a footgun, give you a Portainer-ready stack file, and finish with a Q&A on the most common gotchas.

The Markdown Throne Falls: When AI Agents Step Out of the Chat Box, HTML Becomes the New Answer

Wed, 10 Jun 2026 10:00:00 +0800

If 2023 was the “chatbox era” of ChatGPT and Markdown, then 2026 is heralding the arrival of a new phase: AI no longer merely “answers” you—it is set to “take over” your interface. As Agents begin directly controlling browsers, operating SaaS, and building interactive application prototypes, Markdown—long the de facto standard for LLM output, that elegant format prized for its simplicity, readability, and token-friendliness—is rapidly revealing its fatal shortcoming as the “last mile.

Stop Worshipping Agent Frameworks: 12-Factor Agents as an Engineering Baseline

Sat, 23 May 2026 14:50:00 +0800

The short version

12-Factor Agents is not an installable agent framework, and it is not a Skill for Codex, Claude, Cursor, or any other agent runtime. It is better understood as a set of engineering principles for the AI era. When a system made from a large model, a tool list, and a while loop reaches the familiar 70% or 80% quality bar but refuses to become production-grade, these principles tell you where to look: prompts, context, structured outputs, state, human approval, error compaction, control flow, and recovery.

The best part is that it does not romanticize agents. It says, in effect, that reliable agents are mostly software. The LLM is powerful, but it should live inside a system that is observable, interruptible, recoverable, auditable, and testable.

All examples in this article use public projects, public links, and placeholders. No real server addresses, accounts, tokens, business configuration, or private network details are included. The figures are taken from the official humanlayer/12-factor-agents repository and are referenced under the project author’s CC BY-SA 4.0 content license statement. Code license details should be checked in the upstream repository.