Prepasto

December 3, 2024


I made Prepasto, an automated calorie tracker. It takes any text or photo as an input and returns complete nutritional data to the user. Prepasto does this by using large language models (LLMs) to match user inputs to a list of foods in the USDA food database, it then uses LLMs and USDA portion size data to estimate the exact quantity of the food. The calorie tracker runs over WhatsApp. Anyone can WhatsApp +1 (415) 347-6103 to use it.

The problem: tedious nutrition tracking

Over the past eight years, I’ve tried on and off to plan and track the nutritional content of my food. In my experience, understanding my nutritional data has been instrumental to achieve my fitness goals.


To gain muscle or lose weight, one needs to be in a caloric surplus or deficit. My activity level (calorie burn) is generally consistent week to week. Hence, my calorie balance is mostly a function of what I eat (calorie intake). To gain or lose weight at a steady and sustainable pace, I find it very important to set and meet daily calorie targets.


In the past, I’ve used apps like MyFitnessPal and Cronometer to track my calorie intake. However, I’ve always been frustrated by how much time it takes to search for each food in one of these app’s nutritional databases. Furthermore, I find the quality of the data in these databases to be inconsistent. For example, sometimes food items in MyFitnessPal (e.g., ‘pizza’) have radically different, and sometimes incomplete, nutritional values. This adds another layer of complexity to food searching, since I have to identify the ‘correct’ food item.

The solution: Prepasto

Project highlight

Prepasto aims to make it as simple as possible for you to track what you eat. Logging food is as simple as texting a friend.

You can log a food by either:

  • Texting Prepasto
  • Narrating a message to Prepasto with speech-to-text on your keyboard
  • Asking Siri to text Prepasto
  • Sending a picture of your food to Prepasto
  • Project highlight

    You can text “/stats” to Prepasto to ask it for a visualization of your nutrition over the past 14 days, a spreadsheet of all of your meals, and a spreadsheet of your daily nutrient totals. I added a weekly visualization because I find that many calorie trackers fixate too much on daily nutrition goals. In my experience, day-to-day fluctuations are less important than roughly achieving my goals over the week. I also provide users with their complete meal data because I feel strongly that people should be able to easily explore their complete nutrition data.

    How it works

    Prepasto uses an LLM (GPT-4o at the time of writing) to search the USDA food database for the food items which most closely match the user’s message.

    Project highlight

    The USDA has the world’s most comprehensive public food database. Above is what an entry in the database looks like (a great video explaining this dataset here). By design, the ~7k foods and drinks in the USDA database are comprehensive enough to describe the nutritional content of every food eaten by a random sample of ~5k Americans.


    After estimating the portion size of your food, Prepasto multiplies the grams of food by the nutrient densities from the USDA database to determine calorie and macronutrient values. Prepasto then keeps track of your total calories and macronutrients over time, which helps you monitor your progress towards your goals


    Prepasto will also detect if a food you mention is a “branded food”. For example,“Trader Joe's spaghetti” is branded but “spaghetti” is not branded. If Prepasto detects a branded food item, it will directly ask an LLM for the nutritional content of that food. This tends to enhance the accuracy, since many specific branded foods are not listed in the USDA nutritional database.

    How I made this

    I used both a Django server and AWS Lambda functions to build this application. I wanted to separate the basic server operations of reading and replying to user WhatsApp messages from the slow and complex LLM API calls which determine the nutritional content of user meals. By offloading the text to nutrition data processing to AWS Lambda, I made my application easily scalable.


    Django and PostgreSQL on Railway

    The backbone of this project is a Django server with a PostgreSQL database, hosted on Railway. The server listens for user messages, which come as webhooks from the WhatsApp Cloud API. The server creates and sends simple reply messages to users. It invokes Lambda functions to do the heavy lifting of the application and listens for webhooks which return the outputs of these Lambda functions.


    Text to nutrition data AWS Lambda function

    The workhorse of this application is a Lambda function which uses a series of LLM calls to match each food item in a user’s nutrition log to a respective entry in the USDA food database. This Lambda function uses pandas to search and manipulate a local copy of the USDA database. It also uses asyncio to run the matching operations for each food item in a user’s meal in parallel. This parallelization is crucial, cutting backend meal processing time by ~70% (for a typical meal with four foods).

    Next steps

    There are a few more features I would love to add. I outline them here.


    Duplicate a meal

    It would be great to have a feature where users can press a button to duplicate meals rather than having to type them in again / copy paste an old text message.


    Edit a meal

    It would be great to at least allow users to scale meals up or down. An even more powerful edit function would allow users to simply explain the required meal modifications, with an LLM executing the modifications.


    Internet search for branded foods

    It would be ideal if the branded food nutritional data came directly from the internet so that it was more accurate (e.g., simply search the Kraft website for the nutritional value of Kraft Mac & Cheese). I think LLMs do a good job estimating the nutritional content of the most popular branded foods (e.g., a Cliff Bar). However, it sometimes hallucinates, especially with niche foods (e.g., Trader Joe’s Gingerbread Non-Dairy Oat Creamer).


    More sophisticated food photo to nutrition data

    Currently, the food photo to nutrition data feature first asks an LLM to describe the food photo and estimate portion sizes. Second, it takes this text summary and plugs it into the text to nutrition function. This method works, but it could be improved to more accurately estimate portion sizes and identify foods in step one.

    Conclusion

    Thanks for having a look at Prepasto. I hope other people get some use out of it too. If you have any questions, ideas for how this project could be better, or just thought it was cool, feel free to drop me an email!