Mix Dough: In a stand mixer bowl, combine ½ cup warm water, ¼ cup warm whole milk, 2 Tablespoons granulated sugar, 2 ¼ teaspoons instant yeast, and 4 Tablespoons melted salted butter. Mix with a fork. Attach dough hook, add 2 cups flour and ½ teaspoon salt, and mix on low until combined. Increase speed slightly and knead 4-5 minutes until dough wraps around the hook. Add a few additional Tablespoons of flour if dough is too sticky — it should be soft and tacky, not wet.
First Rise: Spray bowl with cooking spray, return dough, cover with plastic wrap, and let rise until doubled, about 45 minutes.
Make Cinnamon Sugar: In a shallow dish or medium bowl combine ⅓ cup packed light brown sugar, 3 Tablespoons granulated sugar, and 1 Tablespoon ground cinnamon. Transfer 4 Tablespoons of this mixture to a separate bowl, then add 2 Tablespoons melted salted butter. Mix together and set aside (cinnamon-butter mixture is needed for after bread has baked).
Prep Pan: Grease an 8×4-inch loaf pan with cooking spray. Set aside.
Prep Butter: In a shallow bowl, melt 5 Tablespoons salted butter. Set aside.
Shape Dough: Pat dough into about a 9x5-inch rectangle (it will be thick, no need to use a rolling pin), and then use a pizza cutter to cut rectangle into 6 strips. Dip each strip in butter followed by dry cinnamon sugar mixture. Lay each strip in the loaf pan, side by side, so they fill the length of the pan.
Second Rise: Cover and let rise until almost doubled, about 20 minutes. The dough should look puffy and reach near the rim.
Bake: Preheat oven to 350 °F. Bake 28-32 minutes until golden and center reaches at least 190°F. Avoid overbaking to keep the bread soft.
Finish and Glaze: Brush top with reserved cinnamon-butter mixture. Let cool 15 minutes, then mix ⅔ cup powdered sugar, 1 ½ Tablespoons whole milk, and ½ teaspoon vanilla extract, and drizzle over the top of the bread. Serve warm.