Best Chocolate Chip Cookie Bake Off

In this post, we set out to answer the eternal question: what is the best chocolate chip cookie recipe on the internet?

Well, at least based on the first three pages of Google.

Background

In which you find out how this came to be

Over a year ago, my friend Skyler came up with an idea to bake a bunch of chocolate chip cookies and visualize the data of the different recipes. Before we were able to do so, Skyler ended up moving to NYC…but then came back to Houston for a visit–just in time to get stranded by Hurricane Harvey. For all the bad that Harvey wrought, our silver lining was that we finally had time to bring our cookie project to fruition.

The Project

In which we bake 200+ cookies and Skyler builds magnificent Tableaus to display our data

The project was two-pronged: the first part was to gather and analyze the top chocolate chip recipes from the web. The second was to bake the top 12 recipes and hold a taste test to see which recipes ranked the highest.

We scraped almost all the chocolate chip cookie recipes off the first three pages of Google (as of September 2017) and then narrowed the selection of cookies to bake to 12, trying to pick some of the most popular recipes and eliminating any duplicate or similar recipes. We omitted recipes that used ingredients outside of the basic butter, flour, sugar, etc. except for a few exceptions that used very small quantities of ingredients like molasses, cornstarch, baking powder or turbinado sugar. Although we excluded oatmeal chocolate chip cookies from this experiment, we included the Neiman Marcus recipe which uses oat flour out of curiosity.

Anatomy of a Chocolate Chip Cookie

Explore the ingredient composition and techniques of the cookies we baked

Skyler built the below chart on Tableau and it’s interactive, meaning you can hover over almost any element of the chart to see more data! Here, you can see the 12 cookies we tested and each’s percentage of ingredients. One of my favorite parts of this chart is hovering over the various types of “Butter Treatment” so you can visualize what cookies look like with cold, softened, browned and melted butter. {the chart may need a minute to load}

In exploring the chart, you might notice the outliers in the ingredient categories–Smitten Kitchen, for example, far outdid the others with 33.5% chocolate whereas Mrs. Field’s had way more flour than the rest, with 32%. These are interesting observations to note in light of the overall rankings in the next section.

Final Rankings

In which 30 friends eat cookies for research

You can see the rankings in the below Tableau! Hover over any of the cookies to see its average rating and rank, or click on a cookie to see taster comments, recipe, and more.

Findings and Discussion

Stop above if you’ve had enough cookie talk; keep going if you want more…

Audience matters: Interestingly, during the ranking process, we encountered an unexpected phenomenon: drastically different ratings between groups of tasters. We held the original cookie tasting (sampled in a blind tasting) with 15 attendees, and then I sent a bunch of cookies back to college with my brother so that he could hold another tasting to gather more data points. When we got those results back, the data completely changed: some of my favorite cookies had been drastically demoted and some of the cookies reviled by the original tasters were favored by the college students.The cookies we considered “fancy” and took the most work to make (browning butter, chilling, etc. ) that had scored well in the first group trended towards the bottom with the college students. Overall trends showed that they hated “salty” cookies and loved the more classic cookies. Our conclusion here? Your audience matters when it comes to baking. Not everyone will appreciate the artisanal salt-sprinkled cookies (and some are very sensitive to salt, period)–sometimes, a simple classic will be the crowd-pleaser.
Salt matters: Unfortunately, we started the baking process after carefully thinking out all the ingredients–except one. This article is a great explanation about the difference in kosher salt brands (essentially, Morton’s is saltier by weight vs. Diamond Crystal kosher salt). We used Morton’s kosher salt in all the cookie recipes, which could have made some cookies more salty than they were originally meant to be–most recipes just list “salt” or “kosher salt,” and don’t specify which kind. To be safe, in the future I’ll stick to using Diamond Crystal kosher salt in case I come across more salt-hating college kids 😉
Butter matters…but not that much: One might consider butter to be the key ingredient to a chocolate chip cookie–but in the final rankings, the outliers on both ends of the butter spectrum took top honors. Cook’s Illustrated and Joy the Baker tied for the highest percentage of butter (21%) and took first and second place, but Not Without Salt–which had the least butter with only 13% butter–came in a close third. So there’s really no hard and fast rule like “more butter = better chocolate chip cookies” that you can apply–it’s the combination of all the ingredients and the technique used that makes a difference.
Egg matters: The cookie with the lowest percentage of egg consistently performed the worst–poor Betty Crocker with her lone egg to moisturize a huge batch of dough resulted in a crumbly dough that baked up into storebought-esque, ultra-crispy cookies. Any good baker worth his/her salt knows that more egg = more moisture. An extra egg yolk (as seen in Joy the Baker) will add more chew to the cookie.
Flour matters: Surprisingly, some of the thickest cookies (Cake Over Steak and Serious Eats) had only moderate percentages of flour and some of the thinnest (Nestle and All Recipes) didn’t necessarily have the smallest percentages of flour. I thought that more flour would automatically equal thicker cookies, but it didn’t have quite that effect–other factors like the amount of moisture from eggs and the treatment of butter (melted vs. softened vs. browned–which reduces a lot of the moisture and results in a thicker cookie) played greatly into the ultimate thickness.

Rankings: Post-Grad Millennials vs. The College Kids

What’s up with those rankings, though!!

Although the point of this exercise is to show the crowd-pleasing cookies rankings as an aggregate, the final rankings from all 30 tasters shown in the above Tableaus don’t accurately line up with the opinion of my tastebuds.

I thought it would be interesting to break out the original rankings from the 15 tasters in Group A (~24+ years old with a generally high interest in food) vs. the rankings from the 15 tasters in Group B (the college students).

You can see from the chart that there is a drastic difference between Group A and Group B’s rankings. As I mentioned previously, I think the main cause behind these discrepancies is a difference in a taste for salty-sweet cookies. I think most people reading this blog (like myself) may have more similar tastes to Group A, but if you consider yourself more of a classic cookie connoisseur, maybe pay more attention to Group B.

And now…

In-Depth Individual Cookie Analysis

If you want more than what the Tableaus tell you, let me tell you all my thoughts…

My ideal chocolate chip cookie is a thick cookie with a caramelized, crunchy edge, chewy interior and gooey center. Funnily enough, my favorite cookie out of this bunch turned out to be one of the flattest–read on for my take on each of the cookies.

Cook’s Illustrated

Cook’s Illustrated dominated over both Group A and B with an intensely brown butter flavor, lingering chocolate notes and a rich and chewy, but not doughy texture. Although it was a clearly a quality cookie (hefty, shiny, with beautiful bronzed ridges), I still favored the flavor of Joy the Baker over this cookie. Cook’s Illustrated also had one of the most involved cookie processes (using browned and softened butter, a time-consuming rest-and whisk procedure, etc.) but it apparently paid off across both groups A and B with a consistently high ranking.

Would make again for a fancy occasion if I had a leisurely amount of time.

Joy the Baker

Like Cook’s Illustrated, Joy also uses a mixture of brown butter and softened regular butter that results in a cookie with a prominently toasty butter flavor with notes of toffee and vanilla, incredibly crispy edges and a wide swath of chewy center. Despite its rather thin appearance, this was my #1 favorite cookie in terms of taste. I thought it had a perfect amount of salt and a luscious juiciness.

We experimented with several variations after the bake off to combine a thicker cookie texture with the incredible flavor from Joy’s, but we couldn’t quite replicate it (browning all the butter makes for a thicker, tasty, but less “juicy” cookie and omitting an egg yolk results in slightly thicker, paler, slightly less tasty cookies). Of course, the college kids thought this cookie was too salty and flat. (Note: we also looked at Martha Stewart’s chocolate chip cookie and it’s similar to Joy’s–except Martha uses all softened butter, slightly less baking soda, and a whole egg instead of an egg + yolk.

Would definitely make again.

Not Without Salt

This had been my go-to cookie for more than a couple batches due to the ease of the recipe and option to bake straight without chilling. They’re perfectly-sized with a delightfully toothsome texture and a LOT of chocolate. Thicker in the center, these cookies had crispy edges and a chewy, dense, rich center. You can dress these cookies up or down in terms of the effort you want to put in–i.e. you can chill them or not, and they’ll still taste great. I think these cookies ultimately give you the best flavor for the least amount of work.

Would definitely make again–best work to taste ratio.

All Recipes

These notably flat specimens had a wrinkly dough with a delicate, almost lacy texture. Although chewy in the center, they were crisper around the edges compared to a similar cookie like Nestle, which was basically just chewy. Overall, it was an inoffensive cookie to most, though it didn’t really stand out from the pack. It likely got a boost in rankings due to Rice students who liked the classic, nostalgic quality of the cookie. The recipe is actually very similar to Cake Over Steak except that Cake Over Steak uses browned butter (which reduces moisture in the cookie for a thicker texture instead of spreading out like All Recipes) and a much higher ratio of brown sugar.

Probably wouldn’t make again.

Smitten Kitchen

Given the number of successful SK recipes I’ve made, I was surprised at the blandness of the dough. It was chewy with crunchy edges, not as buttery as others, and reminded me of a thicker version of the Nestle cookie–my overall impression was that it would be a solid bake sale cookie. To be fair, this post is entitled “Our Favorite Chocolate Chip Cookie”…and dated 2008. I think the other CCC cookies on her site are much fairer game (like her version of Not Without Salt’s cookie).

Would not make again.

Sally’s Baking Addiction

These chewy, tender cookies looked positively petite in size compared to some of the monstrosities we baked. Overall, they had an even consistency and good flavor, though they weren’t quite as thick as I expected based on the pictures. Although some tasters called the flavor “average,” I think these would be a great option if you’re making cookies for a crowd or a party. A few tasters noted a weird taste, as if there was too much baking soda. (If you’re wondering why we omitted the “Most-Pinned” recipe from Kelsey’s Apple a Day, it’s because the recipe is the same, except with slightly more chocolate chips.)

Might make again for a group.

Nestle Tollhouse

Similar to All Recipes, these cookies had a very light, delicate structure that was very chewy with pops of salt. It’s the kind of cookie that you could probably eat five of because it’s so light. But it’s also kind of limp–I think it’s a less good version of All Recipes, although there was a good balance of chocolate. Overall, kind of a mediocre cookie.

Would not make again.

Pinch of Yum

These large, thick, puffy cookies had one of my favorite textures: they were best fresh out of the oven. Even baked, they’re kind of cookie-dough-like with a chewy–verging on fudgey in some areas–though the dough lacks flavor. Called “Pillsbury or sugar-cookie esque” by some, it is admittedly so pale it looks like a sugar cookie with chocolate–and the flavor is correspondingly a bit bland. I liked this cookie, but would try it again with some more brown sugar or something to up the flavor.

Would make again with modifications.

Cake over Steak

This was another of my favorite cookies, appearance-wise. Sara had conducted her own extensive chocolate chip cookie research and used her results to create this “frankenstein” ideal cookie recipe. Incredibly thick in the middle with luscious ripples of dough that flowed around mounds of chocolate, this cookie reminded me most of luxury bakery cookies (a la Levain). The texture is the dictionary definition of a fudgy cookie–the interior resembles cookie dough with its thick, doughy, rich softness with toffee, butterscotch and even coffee-like notes. One taster called it “too rare,” which is now one of my favorite cookie adjectives ever. It’s quite salty, but I personally liked the balance of salty and sweet.

Would make again when I have some time on my hands (and for an audience that likes salty-sweet).

Serious Eats

Though less thick in appearance than Cake Over Steak, these cookies also had a thick, fudgy texture. This was perhaps the most involved cookie recipe, and I was shocked to see it fall to the bottom of the overall rankings. I think it deserves at least fourth place, as it ranked with Group A. I will shoulder some blame for a heavy hand with the salt sprinkled on top, but I’m still upset that the toasty, caramelized flavors in the rich dough with strong undertones of salt didn’t carry it farther in the overall rankings. I’ve made this cookie before, and I think the resulting cookie is worth the effort.

Would make again when I have some time on my hands (and for an audience that likes salty-sweet).

Neiman Marcus

This is actually the cookie that Skyler and I grew up on, which we didn’t realize until we were researching cookies (he called it the Mrs. Field’s cookie growing up; I called it Neiman Marcus). This was the only cookie that used oats (ground up into flour), and we included it for kicks. However, the oat flour made it stand out from the other chocolate chip cookies in most a negative way–although it was hearty and toothsome with good flavor, it was a little dry. As an oatmeal chocolate chip cookie, it’s pretty solid–and a good cookie if you don’t like really rich-tasting cookies–it just didn’t fare well against more classic CCCs.

Probably wouldn’t make again, but they’re good for those who like thick and hearty cookies that aren’t too rich.

Betty Crocker

This is actually the same recipe as Nestle Tollhouse except with one egg instead of two–and what a difference it makes. It was the crunchiest of the lot with a slight buttery flavor that reminded me of those Famous Amos cookies. I didn’t get much flavor aside from chocolate and a little salt. The texture is sandy, gritty, granular and just distractingly crunchy with no chew. We ended up with extra dough, so we chilled it and baked up some later at 350 degrees instead of 375 and liked the results much better.

Wouldn’t make again, but they’re good for those who like really crunchy cookies.

Phew. Now, while Jacqueline (baking director and resident cookie monster) and I test out iterations of our favorite chocolate chip cookie, I’d love to hear about yours! What favorite recipes did we miss in this testing? What are your favorite techniques? Any ingredients I must try?

Addendum
Methodology notes in case you’re interested in the nuts and bolts of the baking

Ingredients used include: Gold Medal flour, Spice Island vanilla, Domino light, brown and granulated sugar, Kroger butter and Morton kosher salt (with fleur de sel for garnishing).
When a recipe did not specify the type of salt, we defaulted to Morton kosher salt.
When a recipe did not specify the type of brown sugar, we defaulted to light brown sugar.
All five recipes that required some kind of chill time were made 2 nights before the day of the tasting and baked the day of.
All seven recipes that did not require a chill time were made and baked the day of the tasting.