In a heavy sauce pan, combine the milk and 1 cup of the cream. Cook over medium-low heat, stirring occasionally, until the mixture reaches a temperature of 170°F and lightly starts to bubble. Turn off the heat, whisk in the cocoa powder and 4 oz chocolate until completely melted and the mixture is smooth.
In a separate bowl, whisk together the egg yolks, sugar, and remaining 1/2 cup of cream.
Temper the egg yolks by slowly pouring 3/4 cup of the hot milk mixture into the egg mixture while whisking the mixture fast to keep the eggs from cooking. Stirring quickly, pour this mixture slowly back into the saucepan. cook over low heat until the mixture reaches a temperature of 185°F and coats the back of a spoon.
Pour this mixture through a fine mesh strainer and into a clean bowl. Set the bowl over an ice bath, and stir constantly, until the mixture is cool. Cover and refrigerate at least 4 hours or overnight.
Remove the mixture from the refrigerator and whisk in the vanilla extract. Pour into an ice cream machine and churn according to your ice cream makers directions.
While ice cream is churning, place the remaining 4oz. of chocolate in a microwave-safe bowl and microwave in 20 second intervals, stirring after each time, until it's fully melted.
Transfer a third of the ice cream to an airtight container. Drop some marshmallow creme in dollops, drizzle melted chocolate and add some graham crackers. Repeat process 2 more times. Freeze until solid, about 2-3 hours.