Chill the ice cream bowl for at least 24 hours.
Preheat the oven to 350˚F. Place the walnuts in a single layer on a small baking sheet. Toast in the oven for 5-10 minutes, until lightly browned and fragrant. Let cool.
Add whole milk and brown sugar to a large saucepan and heat over medium. Add ½ cup of the black walnuts, reserving the rest for later. Stir to combine.
Bring the milk mixture to a simmer. Once simmering, remove from heat and cover with a lid. Allow to steep for 1 hour.
Strain the walnuts out of the mixture, pressing down as you strain to release as much flavor as possible.
Return the flavored milk to the saucepan and again bring to a simmer.
In a medium bowl, whisk together the egg yolks and granulated sugar until smooth and pale yellow in color.
Add about 1/2 cup of the hot milk to the egg and sugar mixture and whisk constantly until combined.
Then, add the tempered egg mixture back into the saucepan with the remaining hot milk.
Heat the saucepan on low. Stirring constantly, heat until the base has thickened enough to coat the back of a spoon. This should take a few minutes.
Strain the mixture into a bowl to remove any pieces of egg that may have cooked.
Add the heavy cream and vanilla extract. Stir to combine.
Cover the bowl with plastic wrap making sure to have the plastic wrap touching the surface of the ice cream base. This will help keep it from forming a skin while chilling. Refrigerate for at least 3 hours, or until cold.
Churn the ice cream mixture in the bowl of your ice cream machine according to your manufacturer’s instructions (mine took about 20 minutes), until it is thick and the consistency of soft-serve ice cream.
In the last two minutes, add the remaining black walnut pieces and churn until they are evenly distributed.
Spoon the ice cream into a shallow container or a loaf pan. Press plastic wrap or wax paper to the surface of the ice cream. Freeze for at least 4 hours, or until solid. Let sit at room temperature for about 5 minutes before serving.