Home RecipesDesserts The Chef, the Computer, and the Baltic Apple Pie

The Chef, the Computer, and the Baltic Apple Pie

by Florian

I usually don’t speak about my day job on this blog, mostly because it has nothing whatsoever to do with adventures in Eastern Bloc cuisine. Or rather, it didn’t until recently…

About two years ago, a small team of researchers at IBM (including yours truly) started working on computational creativity. By winning on Jeopardy, IBM has shown that computers can make inferences about the world as it is. But could they also be creative, and produce quality artifacts that have never been seen before? To investigate, we built a cognitive cooking system.

Cognitive Cooking - Baltic Apple PieI won’t get into too many details (you can check out our short project page here), but our application allows users to engage in a dialog with the machine, input a key ingredient, cuisine, and dish type, and create billions or more of never-before-seen ingredient combinations. These results are then filtered using predicted surprise and pleasantness factors as well as personal preferences. Once users have picked the recipe they want to make, the system generates ingredient proportions and basic preparation instructions. We’ve partnered with my culinary alma mater, the Institute of Culinary Education to pick their chefs’ brains, have them try some of those computer-generated recipes, and give us some feedback.

This week, we finally started sharing the results with a wider public audience. IBM sent a cognitive cooking food truck to the Pulse Conference in Vegas. Every day, people can vote, via on a dish they’d like to try, and the ICE chefs use the cognitive cooking system to create a recipe that’s served the next day. You can take a look at what’s been served so far on the web site.

Which brings us to the Eastern Bloc connection: on the opening day, the plat du jour was a Baltic apple pie. Since I don’t often have the chance to see an Eastern European recipe created by the collaboration of a computer and three big names like Michael Laiskonis, James Briscione, and Michael Garrett, I wanted to share it here.

Let’s take a look at how the Baltic apple pie was born. The screenshots that follow are taken from our old, “straight from Research” user interface (a more user-friendly app is in the works). On the first day of the conference, the most tweeted dish was pie. We therefore entered pie as the dish we wanted to make. The system first told us what ingredients are frequently used in pies, and what cuisines make pies the most often, inviting us to choose at least one key ingredient and one cuisine. Chef Michael Laiskonis picked apple as the key ingredient, thus reimagining the classic apple pie. Maybe because of his Lithuanian ancestry (check out his takes on Lithuanian desserts: rye bread ice cream with poached apples and caramel, and poppyseed spurgos), he opted for a Baltic recipe (Estonian, Lithuanian, Latvian and Swedish cuisines selected below), and I was very glad he did.

Cognitive Cooking - Baltic Apple PieNext, the system looked at its database of existing recipes to figure out what is needed to make a pie. It came back with what I call a “bill of ingredients” — the ingredient categories found in the typical pie — and suggestions as to how many ingredients of each type one is most likely to require. Users are can accept the recommended bill of ingredients, or tweak the specifications. Michael wanted to make a pie with both sweet and savory ingredients. You’ll notice that we have over 80 quintillion possible recipes for our Baltic apple pie — this is 6 trillion times as many results as you’ll get on Google.

Cognitive Cooking - Baltic Apple PieNow we can really get into the creative part of the process. The system took our inputs, tried all the combinations of ingredients that satisfied our specifications, excluded the ingredients that aren’t found in Baltic cuisines or don’t pair well with apple, and presented the top 5,000 results. To score these results, we designed three evaluators that predict the novelty of a given ingredient combination, its pleasantness based on psychophysical theories of people’s likes and dislikes, and its flavor pairing count (inspired by the flavor pairing hypothesis). Please note that the result I’m showing in the following image isn’t exactly the recipe that was made at the food truck — there are so many possible combinations that we introduced an element of randomness when displaying the results, and thus I could not reproduce the exact same screenshot.

Cognitive Cooking - Baltic Apple PieFor each result, the system also explained why it chose this particular combination from amongst the quintillion other possibilities. You can click on the image below to expand it. Dill is used prominently in Lithuanian, Swedish and Latvian cuisines, hence the thick connecting lines. Similarly, apple is often paired with onion, raisins, vanilla, and (like nearly every other food on earth) salt.

Cognitive Cooking - Baltic Apple PieAfter being the Executive Pasty Chef at Le Bernardin for nearly a decade, Michael didn’t need a computer to figure out the ingredient proportions or recipe instructions 🙂 Here are his comments about the recipe:

As a pastry chef, I was motivated to put a traditional ‘dessert’ forms into the system, but with the added challenge of savory and sweet elements, and the cold-weather cuisines of the Baltic and Scandinavia offer some great inspiration. What I think is interesting here is how the balance can easily shift from ‘sweet pie with savory elements’ to ‘savory pie with sweet elements’ simply with the ratio and treatment of the ingredients. The approach here will be to find that elusive middle that highlights the qualities of both elements – to play with the guest’s expectations through presentation.

Finally, the pie was served at the IBM food truck to the 10,000 or so attendees of the 2014 IBM Pulse Conference. In the short video below, James Briscione shows the making of the pie behind the scenes:

Oh, and if you missed us in Vegas (what, you don’t travel the country to attend conferences on cloud computing???), the truck’s going to South by Southwest next week! And so am I!

The recipe below yields about 6 servings.

Pork and apple filling

1 lb pork tenderloin, trimmed and cut into 1/4 inch dice
2 cups water
3 tablespoons smoked salt
1 Granny Smith apple, peeled, cored and diced
2 tablespoons onion, diced
margarine, as needed
light brown sugar, as needed
fine sea salt, as needed

  • Brine the pork in the water and smoked salt for 30 minutes, then drain and reserve for assembly.
  • Meanwhile, gently sweat the apple and onion in a small amount of margarine until softened, allowing some of the excess moisture from the apples to evaporate. Season lightly with brown sugar and salt; bear in mind that the pork will be slightly salty from brining.
  • Allow the apple mixture to cool, and combine thoroughly with the drained diced pork; reserve for assembly.

Blueberry apricot preserves

1/4 cup blueberries
2 tablespoons dried apricot, diced
1/2 cup water
1 tablespoon light brown sugar
1 teaspoon fresh ginger, peeled and fine