2024
React
Python
Vision models

AI PROTOTYPING / SELF-HOSTED APPLICATION

Screenshot-to-Code

Visual references turned into editable product code

4
AI providers
React
Frontend
Python
Backend
Docker
Deployment
A practical environment for reconstructing interfaces

Screenshot-to-Code accepts screenshots, mock-ups and interface references, then asks a multimodal model to produce an editable implementation. I maintain a self-hosted fork so the complete workflow—from model selection to preview and deployment—can be controlled and adapted.

I was developing it before the current era of design-tool integrations with Claude Code and Codex. I used it to prepare several landing pages, but the more important lesson was about context: the model needs to know which elements to build and which to place as graphics.

The important result is not a screenshot-perfect static artifact. It is a useful first codebase that a developer or coding agent can inspect, change and connect to the rest of a product.

THE PROBLEM

Visual references carry intent, but not structure

Rebuilding an existing interface by hand means repeatedly measuring spacing, identifying patterns and translating visual decisions into components. A model can accelerate that mechanical first pass, provided the output remains understandable and editable.

A screenshot alone does not tell the model clearly how many graphic elements are on the page or which should remain images. AI may turn a graphic into an arbitrary HTML structure or miss an asset that should have been placed as an image.

  • Support screenshots and existing pages
  • Compare several vision-capable models
  • Keep the generated result available as ordinary source code

INPUT CONTEXT

More accurate HTML starts with a deliberate split of the materials

I got the best results when I supplied the full-page screenshot together with each important graphic as separate supporting material. The full view showed the model the layout, while the individual graphics helped it recognize what should be placed instead of rebuilt as HTML.

This is not a trick that finishes the work. Responsiveness, accessibility, content, interactions and component boundaries still need review. It is a better input than expecting the model to guess the split between code and graphics.

  • The full-page screenshot as layout context
  • Separate graphics as supporting material
  • An explicit decision about what becomes code and what remains an asset

THE IMPLEMENTATION

Model output is only one part of the application

The fork combines a React editing and preview interface with a Python backend, provider adapters, prompt control and containerized deployment. Running it on my own infrastructure also exposed the operational side of model-heavy applications: secrets, timeouts, streaming responses and browser-safe previews.

THE USE

A faster way to begin, not an automatic finish

I use the system to create a credible first version from a visual direction. It helped several landing pages move faster from reference to editable code, after which normal product work continued: responsive behavior, accessibility, data, interactions and maintainable component boundaries.

What the workflow provides

Multiple model providers

Compare model behavior without binding the application to one vendor.

Editable output

Generated interfaces remain source code rather than flattened images.

Supporting materials

A full screenshot and separate graphics help the model split code from assets.

Live preview

Review the result immediately and iterate from the same environment.

Self-hosted deployment

Control credentials, runtime limits and the complete application stack.

Does Screenshot-to-Code replace a frontend developer?

No. It accelerates reconstruction and prototyping, while responsive behavior, accessibility, interactions and production architecture still require engineering work.

Why is a screenshot alone not always enough?

It does not tell the model directly which elements are graphics, how many there are, or how they should be used. A full view together with separate graphic materials gives better context for splitting HTML from assets.

Why self-host the application?

Self-hosting gives control over providers, credentials, limits, prompts and deployment while keeping the workflow available to other local tools.

KNOWLEDGE BASE

Related articles

FROM REFERENCE TO PRODUCT

Have an interface that needs to become working software?

Bring the reference and the product constraints. We can use AI for the mechanical first pass and spend the real effort where the product needs judgment.

Discuss a project