Cut bread into half-inch cubes. Place bread in a baking dish in 300 degree oven for 10 - 15 minutes until bread is slightly toasted.
In a medium sized bowl, combine half and half, white sugar, brown sugar, pumpkin puree, eggs, nutmeg, ginger, cinnamon and vanilla extract. Combine well, ensure egg yolks are broken up.
When bread cubes are finished toasting, remove them from the oven. Pour 3/4 of the custard over the bread, ensuring that the custard soaks into all of the bread cubes. Using a spoon or spatula, gently press the bread cubes down to help them soak up the custard. Pour the remaining 1/4 of custard as needed to ensure all of the bread cubes are moist.
Let the soaked bread cubes sit for approximately 10 minutes for bread cubes to soak.
Bake in preheated 375 degree oven for 1 hour, or until a knife inserted comes out clean.
In a saucepan over medium heat, melt the butter and brown sugar. Combine well.
Increase heat to medium-high, and slowly pour in heavy cream, and incorporate well into butter and brown sugar.
Add whiskey, if using, and continue stirring until alcohol burns off. It will bubble up and then settle down.
Remove from heat and add vanilla.
Keep warm until bread pudding comes out of the oven.
When the bread pudding comes out of the oven, slowly pour warm caramel cream over the bread pudding.