Sony AI and Korea University
ingredient pairing with FlavorGraph
April 12, 2021
Among the partnerships we’re setting up to drive our Gastronomy Flagship Project, we’re excited that our work with Korea University (KU) is already pushing our understanding of how AI can meaningfully pair ingredients. A project looking to advance flavor pairing knowledge has resulted in an ingredient mapping tool called FlavorGraph.
Generally speaking, FlavorGraph predicts the pairing fit of two ingredients by combining information on the molecules in a given ingredient with the way people have used that ingredient in the past: as such, FlavorGraph sits on the data of 1,561 flavor molecules and of a million recipes.
The outcome is pairing suggestions that achieve better results than ever before. These suggestions can be used to predict relationships between compounds and foods, hinting at new and exciting recipe techniques and driving new perspectives on food science in general.
In this blog, we thought we’d go a little way into the context of the project and of the ingredient pairing science in general!
The origins of FlavorGraph lie in the work of Donghyeon Park, Keonwoo Kim and Professor Jaewoo Kang. From November 2019 to March 2020, Donghyeon joined Sony AI as an intern, and during his stay worked on AI for recipe creation. The aim was to develop an intelligent deep learning model that recommends complementary and novel ingredient pairings to help chefs with new creations.
After returning to Korea University, Donghyeon and his team’s research culminated in January 2021 with the publishing of a paper in Scientific Reports, a sister journal of Nature, titled, FlavorGraph: a large-scale food-chemical graph for generating food representations and recommending food pairings.
Our COO, Michael Spranger, described the achievement as follows:
"Ingredient pairing is an excellent challenge for AI systems. It requires AI systems to build models of how people pair ingredients from recipes as well as through other strategies. This work is a great step in AI for creativity within our Gastronomy project."
Chefs traditionally mastered ingredient pairing through their intuition, rather than through scientific knowledge or statistical learning. And we can probably thank the gradual progress of cultural evolution for many of the pairings we see as standard today – cheese and tomato, garlic and ginger, pork and apple.
More recently, science was used to explain why such pairings work. Even novel pairings, like Heston Blumenthal’s caviar and chocolate, were discovered first by him, then later explained using his molecular science. Blumenthal then used this approach to explain and explore how ingredients sharing dominant flavor molecules paired well together.
A 2011 study led by Yong-Yeol Ahn questioned whether this was the only method of reaching ingredient pairing success. Based on analyses of recipes across the world, there was a suggestion that some cultures (especially North American and European) favored recipes that paired ingredients whose flavors matched, but that others (especially East Asian) tended towards recipes whose ingredients’ flavors didn’t match. What has become clear is that ingredient pairing is no simple feat!
The challenge for the team
Based on the complex nature of ingredient pairing and its need to understand food through various scientific attributes, Donghyeon and the team identified two main challenges to overcome in their project:
- How can we utilize available recipe data and meaningful descriptive features (e.g., flavor molecules) to obtain better food representations?
- How can we apply these improved food representations to food pairing?
The solution was FlavorGraph.
FlavorGraph extracts relationships between food items from a million food recipes, analyzing how chefs and recipe creators have brought ingredients together; and it combines these with information for 1,561 flavor molecules.
Such a graph enables analysis of the chemical and statistical relationships between ingredients, allowing the team to better represent the foods.
Ingredient pairing in the future
As well as relationships between food ingredients and flavor compounds that have not been explored before, the FlavorGraph research will allow greater flexibility for matching single or multiple ingredients to many others.
As the science develops and we get ever better representations of food, we should discover more and more intriguing pairings of ingredients, as well as new substitutes for ingredients that are either unhealthy or unsustainable.
So we hope that projects like this will continue to complement both the complex ingredient systems fossilized over time through cultural evolution, as well as the electric ingenuity of modern innovators and chefs.
And in addition to individual ingredient pairings, analysis like FlavorGraph will help us to discover new recipes, changing the way we eat forever.
Sony AI is always looking to develop partnerships with academic institutions and food companies to develop knowledge in AI and gastronomy. And we are always looking to work with talented researchers like Donghyeon! Please get in touch if you’d like to discuss more.
More information on FlavorGraph, and the paper, can be found here.
Donghyeon (Lamy) Park
Post-doctoral Researcher at Korea University
Born and raised in Seoul, Donghyeon (Lamy) has studied in Korea University for his B.S., M.S., and Ph.D. His major originates in Computer Science, specifically in Data Mining with AI. During his Ph.D. years, he developed an interest in utilizing AI techniques in the food industry and, in his role at Korea University, he continues research on supporting the industry with AI. His future interests are in automated cooking recipe generation and the personalization of diets and recipes.
Strategy & Partnerships Manager
Born in London but raised in Tokyo, Fred has worked across the world in roles related to product innovation and corporate and business strategy, and he completed his MBA at ESADE in Barcelona in 2019. His work in AI has crossed city simulation, biopharmaceutical drug development and skyscraper energy management. In his role at Sony AI, he is helping set the strategy for the gastronomy project and build relationships with companies across the food industry.
October 6, 2023 | PPML
Advancements in Federating Learning Highlighted in Papers Presented at ICCV 2023
As the field of machine learning continues to evolve, Sony AI researchers are constantly exploring innovative solutions to address the pressing issues faced by the industry. Two re…
September 21, 2023 | AI Ethics
Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color
-->Advancing Fairness in Computer Vision: A Multi-Dimensional Approach to Skin Color Analysis In the ever-evolving landscape of artificial intelligence (AI) and computer vision, fa…
September 15, 2023
Sony AI Showcases Innovation and Creativity at Sony Creators Conference and Gran…
At the Intersection of Creativity and Technology: Sony AI Shines at the Sony Creators ConferenceOn August 8-9, Sony held its first-ever technology conference for the creative commu…