Do you have a collection of printed or handwritten recipes you’d like to digitize? You’ve come to the right place!

This tool makes it easy to convert photos of your recipes into a machine-readable format.

The output format is a subset of the schema.org Recipe and can be expanded (merge request are welcome).

How it works

The Recipe Scanner is a web application built with the Nuxt framework. Through its user interface, you can upload photos of recipes, which are stored on the server. The server then sends the images, along with a prompt, to a vision Large Language Model (LLM). The LLM extracts the relevant recipe information and formats it as specified. Finally, the server validates the structure of the returned data and sends it back to the client.

Technical considerations

This application was created for personal use (I wanted to digitize many old and messy recipes from my school days) and for experimentation so it is not designed to be fully “production-ready.” This influenced some of my technical choices.

For instance, I wanted to try out running a vision LLM on my own machine instead of using a hosted service. I tried out Ollama and LM Studio and opted for the latter because its interface made managing connections easier. In principle, Ollama should also work.

I also tried out several models. I began with the smalles vision-enabled Gemma 3 model (4B parameters, 3.3 GB). Unfortunately, it made many errors when extracting recipe data. Hosted models such as ChatGPT-5 and Mistral (Large?) performed much better, proving the task was feasible for LLMs. Among open-weight models, Mistral Small 3.2 (24B parameters, 15 GB) was the first that consistently produced accurate results. It’s possible that bigger Gemma 3 models (e.g., 12B) or specialized models like LLaVA, could also perform well or even better. If you test them, I’d be interested in hearing your results.

Another unusual design choice is how persistence is handled. Since this is a personal project, I opted for simplicity. The server does not use a database. There are no user accounts and images and parsed recipe data are saved directly to the server’s file system. To simulate a multi-user experience, each user only sees their own recipes. This is managed on the client side by storing image IDs in the browser’s IndexedDB. IndexedDB is not the ideal tool for this, but I wanted to experiment with it.

Next steps

My immediate goal is to digitize my collection of printed and handwritten recipes. Once that data is gathered, I plan to use both the images and extracted recipe information to fine-tune a smaller, more efficient model specifically optimized for this task.

Link to the repository: https://github.com/realJanWeiss/recipe-scanner