Best Chocolate Chip Cookie Bake Off

Best Chocolate Chip Cookie Bake OffIn this post, we set out to answer the eternal question: what is the best chocolate chip cookie recipe on the internet?

Well, at least based on the first three pages of Google.

Background
In which you find out how this came to be

Well over a year ago, my friend Skyler came up with an idea to bake a bunch of chocolate chip cookies and visualize the data of the different recipes. Before we were able to do so, Skyler ended up moving to NYC…but then came back to Houston for a visit–just in time to get stranded by Hurricane Harvey. For all the bad that Harvey wrought, our silver lining was that we finally had time to bring our cookie project to fruition.

nestle2

The Project
In which we bake 200+ cookies and Skyler builds magnificent Tableaus to display our data

The project was two-pronged: the first part was to gather and analyze the top chocolate chip recipes from the web. The second was to bake the top 12 recipes and hold a taste test to see which recipes ranked the highest.

We scraped almost all the chocolate chip cookie recipes off the first three pages of Google (as of September 2017) and then narrowed the selection of cookies to bake to 12, trying to pick some of the most popular recipes and eliminating any duplicate or similar recipes. We omitted recipes that used ingredients outside of the basic butter, flour, sugar, etc. except for a few exceptions that used very small quantities of ingredients like molasses, cornstarch, baking powder or turbinado sugar. Although we excluded oatmeal chocolate chip cookies from this experiment, we included the Neiman Marcus recipe which uses oat flour out of curiosity.

Anatomy of a Chocolate Chip Cookie
Explore the ingredient composition and techniques of the cookies we baked

Skyler built the below chart on Tableau and it’s interactive, meaning you can hover over almost any element of the chart to see more data! Here, you can see the 12 cookies we tested and each’s percentage of ingredients. One of my favorite parts of this chart is hovering over the various types of “Butter Treatment” so you can visualize what cookies look like with cold, softened, browned and melted butter. {the chart may need a minute to load}

In exploring the chart, you might notice the outliers in the ingredient categories–Smitten Kitchen, for example, far outdid the others with 33.5% chocolate whereas Mrs. Field’s had way more flour than the rest, with 32%. These are interesting observations to note in light of the overall rankings in the next section.

Final Rankings
In which 30 friends eat cookies for research

You can see the rankings in the below Tableau! Hover over any of the cookies to see its average rating and rank, or click on a cookie to see taster comments, recipe, and more.

Findings and Discussion
Stop above if you’ve had enough cookie talk; keep going if you want more…

    1. Audience matters: Interestingly, during the ranking process, we encountered an unexpected phenomenon: drastically different ratings between groups of tasters. We held the original cookie tasting (sampled in a blind tasting) with 15 attendees, and then I sent a bunch of cookies back to college with my brother so that he could hold another tasting to gather more data points. When we got those results back, the data completely changed: some of my favorite cookies had been drastically demoted and some of the cookies reviled by the original tasters were favored by the college students.The cookies we considered “fancy” and took the most work to make (browning butter, chilling, etc. ) that had scored well in the first group trended towards the bottom with the college students. Overall trends showed that they hated “salty” cookies and loved the more classic cookies. Our conclusion here? Your audience matters when it comes to baking. Not everyone will appreciate the artisanal salt-sprinkled cookies (and some are very sensitive to salt, period)–sometimes, a simple classic will be the crowd-pleaser.
    2. Salt matters: Unfortunately, we started the baking process after carefully thinking out all the ingredients–except one. This article is a great explanation about the difference in kosher salt brands (essentially, Morton’s is saltier by weight vs. Diamond Crystal kosher salt). We used Morton’s kosher salt in all the cookie recipes, which could have made some cookies more salty than they were originally meant to be–most recipes just list “salt” or “kosher salt,” and don’t specify which kind. To be safe, in the future I’ll stick to using Diamond Crystal kosher salt in case I come across more salt-hating college kids 😉
    3. Butter matters…but not that much: One might consider butter to be the key ingredient to a chocolate chip cookie–but in the final rankings, the outliers on both ends of the butter spectrum took top honors. Cook’s Illustrated and Joy the Baker tied for the highest percentage of butter (21%) and took first and second place, but Not Without Salt–which had the least butter with only 13% butter–came in a close third. So there’s really no hard and fast rule like “more butter = better chocolate chip cookies” that you can apply–it’s the combination of all the ingredients and the technique used that makes a difference.
    4. Egg matters: The cookie with the lowest percentage of egg consistently performed the worst–poor Betty Crocker with her lone egg to moisturize a huge batch of dough resulted in a crumbly dough that baked up into storebought-esque, ultra-crispy cookies. Any good baker worth his/her salt knows that more egg = more moisture. An extra egg yolk (as seen in Joy the Baker) will add more chew to the cookie.
    5. Flour matters: Surprisingly, some of the thickest cookies (Cake Over Steak and Serious Eats) had only moderate percentages of flour and some of the thinnest (Nestle and All Recipes) didn’t necessarily have the smallest percentages of flour. I thought that more flour would automatically equal thicker cookies, but it didn’t have quite that effect–other factors like the amount of moisture from eggs and the treatment of butter (melted vs. softened vs. browned–which reduces a lot of the moisture and results in a thicker cookie) played greatly into the ultimate thickness.

cooksillustrated6

Rankings: Post-Grad Millennials vs. The College Kids
What’s up with those rankings, though!!

Although the point of this exercise is to show the crowd-pleasing cookies rankings as an aggregate, the final rankings from all 30 tasters shown in the above Tableaus don’t accurately line up with the opinion of my tastebuds.

I thought it would be interesting to break out the original rankings from the 15 tasters in Group A (~24+ years old with a generally high interest in food) vs. the rankings from the 15 tasters in Group B (the college students).

cookierankings

 

 

You can see from the chart that there is a drastic difference between Group A and Group B’s rankings. As I mentioned previously, I think the main cause behind these discrepancies is a difference in a taste for salty-sweet cookies. I think most people reading this blog (like myself) may have more similar tastes to Group A, but if you consider yourself more of a classic cookie connoisseur, maybe pay more attention to Group B.

And now…

notwithoutsalt3

In-Depth Individual Cookie Analysis
If you want more than what the Tableaus tell you, let me tell you all my thoughts…

My ideal chocolate chip cookie is a thick cookie with a caramelized, crunchy edge, chewy interior and gooey center. Funnily enough, my favorite cookie out of this bunch turned out to be one of the flattest–read on for my take on each of the cookies.

CI vs. Joy the Baker

Cook’s Illustrated: Cook’s Illustrated dominated over both Group A and B with an intensely brown butter flavor, lingering chocolate notes and a rich and chewy, but not doughy texture. Although it was a clearly a quality cookie (hefty, shiny, with beautiful bronzed ridges), I still favored the flavor of Joy the Baker of this cookie. Cook’s Illustrated also had one of the most involved cookie processes (using browned and softened butter, a time-consuming rest-and whisk procedure, etc.) but it apparently paid off across both groups A and B with a consistently high ranking.
Would make again for a fancy occasion if I had a leisurely amount of time.

Joy the Baker: Like Cook’s Illustrated, Joy also uses a mixture of brown butter and softened regular butter that results in a cookie with a prominently toasty butter flavor with notes of toffee and vanilla, incredibly crispy edges and a wide swath of chewy center. Despite its rather thin appearance, this was my #1 favorite cookie in terms of taste. I thought it had a perfect amount of salt and a luscious juiciness. We experimented with several variations after the bake off to combine a thicker cookie texture with the incredible flavor from Joy’s, but we couldn’t quite replicate it (browning all the butter makes for a thicker, tasty, but less “juicy” cookie and omitting an egg yolk results in slightly thicker, paler, slightly less tasty cookies). Of course, the college kids thought this cookie was too salty and flat. (Note: we also looked at Martha Stewart’s chocolate chip cookie and it’s similar to Joy’s–except Martha uses all softened butter, slightly less baking soda, and a whole egg instead of an egg + yolk.
Would definitely make again.

Not Without Salt vs. All RecipesNot Without SaltThis had been my go-to cookie for more than a couple batches due to the ease of the recipe and option to bake straight without chilling. They’re perfectly-sized with a delightfully toothsome texture and a LOT of chocolate. Thicker in the center, these cookies had crispy edges and a chewy, dense, rich center. You can dress these cookies up or down in terms of the effort you want to put in–i.e. you can chill them or not, and they’ll still taste great. I think these cookies ultimately give you the best flavor for the least amount of work.
Would definitely make again–best work : taste ratio.

All Recipes: These notably flat specimens had a wrinkly dough with a delicate, almost lacy texture. Although chewy in the center, they were crisper around the edges compared to a similar cookie like Nestle, which was basically just chewy. Overall, it was an inoffensive cookie to most, though it didn’t really stand out from the pack. It likely got a boost in rankings due to Rice students who liked the classic, nostalgic quality of the cookie. The recipe is actually very similar to Cake Over Steak except that Cake Over Steak uses browned butter (which reduces moisture in the cookie for a thicker texture instead of spreading out like All Recipes) and a much higher ratio of brown sugar.
Probably wouldn’t make again.

Smitten Kitchen vs. SBA

Smitten Kitchen: Given the number of successful SK recipes I’ve made, I was surprised at the blandness of the dough. It was chewy with crunchy edges, not as buttery as others, and reminded me of a thicker version of the Nestle cookie–my overall impression was that it would be a solid bake sale cookie. To be fair, this post is entitled “Our Favorite Chocolate Chip Cookie”…and dated 2008. I think the other CCC cookies on her site are much fairer game (like her version of Not Without Salt’s cookie).
Would not make again.

Sally’s Baking Addiction: These chewy, tender cookies looked positively petite in size compared to some of the monstrosities we baked. Overall, they had an even consistency and good flavor, though they weren’t quite as thick as I expected based on the pictures. Although some tasters called the flavor “average,” I think these would be a great option if you’re making cookies for a crowd or a party.  A few tasters noted a weird taste, as if there was too much baking soda. (If you’re wondering why we omitted the “Most-Pinned” recipe from Kelsey’s Apple a Day, it’s because the recipe is the same, except with slightly more chocolate chips.)
Might make again for a group.
Nestle vs. Pinch of YumNestle Tollhouse: Similar to All Recipes, these cookies had a very light, delicate structure that was very chewy with pops of salt. It’s the kind of cookie that you could probably eat five of because it’s so light. But it’s also kind of limp–I think it’s a less good version of All Recipes, although there was a good balance of chocolate. Overall, kind of a mediocre cookie.
Would not make again.

Pinch of Yum: These large, thick, puffy cookies had one of my favorite textures: they were best fresh out of the oven. Even baked, they’re kind of cookie-dough-like with a chewy–verging on fudgey in some areas–though the dough lacks flavor. Called “Pillsbury or sugar-cookie esque” by some, it is admittedly so pale it looks like a sugar cookie with chocolate–and the flavor is correspondingly a bit bland. I liked this cookie, but would try it again with some more brown sugar or something to up the flavor.
Would make again with modifications.

Cake Over Steak vs. Serious EatsCake over SteakThis was another of my favorite cookies, appearance-wise. Sara had conducted her own extensive chocolate chip cookie research and used her results to create this “frankenstein” ideal cookie recipe. Incredibly thick in the middle with luscious ripples of dough that flowed around mounds of chocolate, this cookie reminded me most of luxury bakery cookies (a la Levain). The texture is the dictionary definition of a fudgy cookie–the interior resembles cookie dough with its thick, doughy, rich softness with toffee, butterscotch and even coffee-like notes. One taster called it “too rare,” which is now one of my favorite cookie adjectives ever. It’s quite salty, but I personally liked the balance of salty and sweet.
Would make again when I have some time on my hands (and for an audience that likes salty-sweet).

Serious Eats: Though less thick in appearance than Cake Over Steak, these cookies also had a thick, fudgy texture. This was perhaps the most involved cookie recipe, and I was shocked to see it fall to the bottom of the overall rankings. I think it deserves at least fourth place, as it ranked with Group A. I will shoulder some blame for a heavy hand with the salt sprinkled on top, but I’m still upset that the toasty, caramelized flavors in the rich dough with strong undertones of salt didn’t carry it farther in the overall rankings. I’ve made this cookie before, and I think the resulting cookie is worth the effort.
Would make again when I have some time on my hands (and for an audience that likes salty-sweet).

Neiman Marcus vs. Betty CrockerNeiman MarcusThis is actually the cookie that Skyler and I grew up on, which we didn’t realize until we were researching cookies (he called it the Mrs. Field’s cookie growing up; I called it Neiman Marcus). This was the only cookie that used oats (ground up into flour), and we included it for kicks. However, the oat flour made it stand out from the other chocolate chip cookies in most a negative way–although it was hearty and toothsome with good flavor, it was a little dry. As an oatmeal chocolate chip cookie, it’s pretty solid–and a good cookie if you don’t like really rich-tasting cookies–it just didn’t fare well against more classic CCCs.
Probably wouldn’t make again, but they’re good for those who like thick, hearty cookies that aren’t too rich.

Betty Crocker: This is actually the same recipe as Nestle Tollhouse except with one egg instead of two–and what a difference it makes. It was the cunchiest of the lot with a slight buttery flavor that reminded me of those Famous Amos cookies. I didn’t get much flavor aside from chocolate and a little salt. The texture is sandy, gritty, granular and just distractingly crunchy with no chew. We ended up with extra dough, so we chilled it and baked up some later at 350 degrees instead of 375 and liked the results much better.
Wouldn’t make again, but they’re good for those who like really crunchy cookies.

Phew. Now, while Jacqueline (baking director and resident cookie monster) and I test out iterations of our favorite chocolate chip cookie, I’d love to hear about yours! What favorite recipes did we miss in this testing? What are your favorite techniques? Any ingredients I must try?

allrecipes2

Addendum
Methodology notes in case you’re interested in the nuts and bolts of the baking

  • Ingredients used include: Gold Medal flour, Spice Island vanilla, Domino light, brown and granulated sugar, Kroger butter and Morton kosher salt (with fleur de sel for garnishing).
  • When a recipe did not specify the type of salt, we defaulted to Morton kosher salt.
  • When a recipe did not specify the type of brown sugar, we defaulted to light brown sugar.
  • All five recipes that required some kind of chill time were made 2 nights before the day of the tasting and baked the day of.
  • All seven recipes that did not require a chill time were made and baked the day of the tasting.

Fun fact: this bake off happens to fall on the 2nd anniversary of this Houston-centric cookie taste off!

For more awesome bake-offs and articles about the science of chocolate chip cookie baking:

Related Posts Plugin for WordPress, Blogger...

20 thoughts on “Best Chocolate Chip Cookie Bake Off

  1. Jeremy

    Erika, This is a great bakeoff and presentation. I did a similar bakeoff in 2011 with my family. We baked 11 recipes and added a few more in from local bakeries and some packaged cookies from the store, with 40 people tasting. The adults’ favorite recipe was the same one you found, from Cook’s Illustrated! A very close second was Shirley O. Corriher’s Roasted Pecan recipe out of her book Bakewise. Interestingly, the kids all liked Chips Ahoy, so I guess Nabisco’s R&D department knows what they’re doing. A local website did a little story on it: http://alloveralbany.com/archive/2011/11/22/jeremy-snyders-search-for-the-perfect-chocolate-ch

    Reply
    1. erika Post author

      Jeremy! That article is SO COOL–thanks for sharing!!! I love that you included storebought cookies…and that you guys also founds Cook’s Illustrated to be the winner!! Although I’m very interested to know why our CI cookie turned out such an interesting shade (yours looks more typical of a chocolate chip cookie whereas ours turned a rather unusual light brown color–maybe due to more browned butter?) I also love that you found the same kind of phenomenon with kids liking the simpler storebought cookies–I guess that trend lasts up through college based on my results!

      Wow! Still so impressed by your experiment–definitely going to give that roasted pecan cookie recipe a try at some point. Thanks again for sharing!

      Reply
  2. Gina Benson

    I love this so much. Our family did a cc cookie bake off at our last reunion. It was so much fun but a lot of work. We talk about cookies all the time in our family. It is a major text thread at least once a month. What kind of chocolate chips do you use? Is there a specific brand you used for your testing? In our family milk chocolate chips are a huge no-no! We like semi-sweet or bittersweet. I would think your tastes are similar to what I would like rather than the second group. Thank you for all this awesome data. 🍪🍪🍪

    Reply
    1. erika Post author

      Ahhh yesss a family reunion cookie bake off sounds like the BEST THING EVER!!! Need to bookmark this idea for our next family get together…

      For this tasting, we used all Ghiradelli semi-sweet chips. Milk chocolate chips are also a general no-no for me (and a huge no-no for our other baker friend) unless the recipe specifically calls for them for a good reason. Semi-sweet are def my fav for chocolate chip cookies!

      And thank YOU for reading, Gina! :)

      Reply
  3. Karen

    Thank you for your analysis! So fascinating. One thing that is important to me is the work to taste ratio. I love the brown butter cookies from Cook’s illustrated but the process is so so long and tedious. This is my recipe which I appreciate so much because it has the right salt ratio (so important stupid college kids) and it takes no chilling. I measure all my ingredients for perfection every time. I’d love it when you get a desire for sweets again for you to try it out and see what you think!! https://www.365daysofcrockpot.com/chocolate-chip-cookies-video/

    Reply
    1. erika Post author

      So so true! That’s why I love the Not Without Salt cookies–minimal fuss, but GREAT results! Thanks so much for sharing–I will bookmark your recipe to try soon!!

      Reply
  4. Nathalie Walker

    Thank you so much for this informative post, Erika! Chocolate chip cookies are my favorite dessert and I am always interested in new recipes. You obviously tried these recipes with an American crowd. I find it interesting how cultural differences seem to account for a change in our taste buds – I live in France and my parents in Switzerland and honestly, nearly every single one of these recipes is almost disgustingly sweet for my friends in both countries. I usually make Sally’s Baking Addiction cookies, cut the granulated sugar to 1/3 of a cup and the dark brown to 1/2 a cup and omit a large tablespoon of flour. They’re absolutely delicious and tolerably sweet enough for my friends. It’s interesting because they can all scarf down buckets of buttery croissant, but cannot eat more than 2-3 smallish cookies due to the increased sensitivity to sugar. I’m still hunting for a recipe that will yield less sweet cookies – if you ever find one, please let me know :) Greetings from the other side of the pond!

    Reply
    1. erika Post author

      Nathalie, thanks so much for the comment! That’s SO interesting–you’re definitely right in that all the tasters have very American tastebuds (I personally have a disgustingly high tolerance for sweet things)–and now I’m interested in trying your modified version of Sally’s cookies to see if my tastebuds would tolerate them! I will definitely keep an eye out for a less sweet cookie–have you tried using just bittersweet chocolate in your cookies (or maybe a mix of bittersweet and semisweet) and sprinkling salt on top? I wonder if that would help cut the sweetness for your European friends, or if it’s truly the dough that’s too sweet…

      Reply
  5. Abby

    This is incredibly helpful!!
    For years I have been struggling to find my (as your study has found, we each have our own) perfect chocolate chip cookie and because of this in-depth analysis (love the hover feature), I think I’ll finally get my answer!
    Yippee!
    Thanks again.

    Reply
    1. erika Post author

      Thanks for the lovely comment Abby!! The “perfect” chocolate chip cookie for each person varies widely, but best of luck in finding yours!! :)

      Reply
  6. Sara @ Cake Over Steak

    Wow, Erika – this is absolutely INCREDIBLE and I am so honored that my cookie made it into your bake-off. What is up with those college kids? Lol. Although I have to say that I did have to slowly groom my coworkers over the course of my long experiment to appreciate the salty-sweet chocolate chip cookies, but they’re officially on board now. I think it’s interesting how your group tried them all at once and how that might have affected their rankings, compared to trying them separately. Since I did mine so slowly, I was able to live with each cookie for a few days and really think about the process of making them, how well they kept, etc. I can see how if you’re trying a bunch of cookies at once that mine might come across as too intense (for wimps … hehe jk!). Also, what I looked for in my ultimate cookie definitely changed over the course of the two years I was searching for it. To each their own! I’m glad you enjoyed the richness and “rareness” of mine. (That is a really good cookie adjective.) Also, THE SALT THING! I think it was during this experiment where I really learned how the different kinds of salt vary so wildly in their saltiness. Now I get really upset when a recipe doesn’t specify what kind of salt they used, especially if it’s a baking recipe. (You used the same kind of salt that I do, so my recipe should have been pretty accurately salty for you.) These charts are also totally bonkers and amazing. Thank you so much for including me!!! xoxo (PS I hope you’re doing okay after the hurricane! Thinking of all of you in that area <3)

    Reply
    1. erika Post author

      Sara!! Thank you for your awesome comment! I am TOTALLY on board with you about the salt–I never cared when people didn’t specify before (and just defaulted to whatever kosher I had on hand) but now I’ve come to realize how crucial it is in baking! Deffo stocking up on the Diamond Crystal.

      Haha and such a good point about cookie fatigue–on one hand, it’s nice to do it side by side because after you taste all the cookies, you can go back and re-taste any that you’re unsure about or compare a specific pair side by side. On the other hand, the fullness factor is REAL. Luckily, I had trained most of my friends on that front…but in any case! I guess it’s up to us to lead the salty-sweet cookie movement…20 friends/coworkers at a time 😉 Cannot wait to make your cookies again!

      Reply
  7. Katie

    Erika, this is honestly some amazing work. Hats off to you and Skyler for the comprehensive analysis, detailed methodology, and all of the recipe data. Seriously impressive. Not Without Salt has been my go-to chocolate chip cookie for some time now, so I’m pleased you ranked it as having the best flavour for the effort. I’ll have to try out the other top choices and see how they compare. And those college kids don’t know anything – a sprinkle of salt is where it’s at.

    Reply
    1. erika Post author

      Thank you so much Katie!! This took a ton of work, so I really appreciate your kind words *heart eyes emoji*

      UGH YES THOSE COLLEGE KIDS KNOW NOTHING!!! I’m so curious how my college self would have ranked the cookies…oh how the palate can change!

      Also, I JUST discovered this bake off by Buzzfeed and am putting Tara/Seven Spoons on my list to test next (it got rave reviews in their bake off)!!

      Reply
  8. Courtney

    Holy crap. YES TO ALL OF THIS. This is so detailed and informational and can I also mention that I didn’t realize this till I had a cookie taste test of my own but audience is SO key! I loved certain cookies and found some friends agreed with me but others like their cookies doughier/chewy and picked the ones I ranked lowest. ALSO, how long the cookie has sat between baking and consuming is an interesting factor too! For some cookies, they tasted best within an hour after baking and others tasted better the next day… so much so a lot of people’s answers for best cookie would flip around depending on if I had let the cookies sit out for a day before giving it to them or had put them in the freezer to preserve their “out of the oven” texture as best I could. This is so super neat and I love love love posts like this! Fantastic job, Erika and Skyler!

    Reply
    1. erika Post author

      Thank you SO MUCH Courtney!!! YES I feel like it’s not until you have a cookie tasting that you realize how VASTLY different people’s tastebuds are–to me, there are certain cookies are are just indisputably perfect (or awful), but when your friend hates/loves the same cookie, it turns your entire perspective upside down.

      And so true on the time factor–like the Betty Crocker cookies immediately went stale-tasting several hours after the tasting whereas the Serious Eats, Cake Over Steak and Cook’s Illustrated preserved their flavor and moisture pretty well the next day. I’m now an even bigger proponent of freezing balls of cookie dough and baking them fresh! (As I’m sure you know from your amazing cookie experiments!) <33333

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *