Cook Rice: In a saucepan over medium-high heat, bring 4 cups of broth and 2 cups of water to a boil. Add rice, reduce stovetop heat to low, cover saucepan with lid, and cook until rice is tender, about 40-45 minutes. (After cooking this, do not drain broth.)
Sauté Onions and Garlic: While rice cooks, heat olive oil in a large pot over medium heat. Add onions and garlic and sauté until translucent, about 3-5 minutes.
Stir in Mushrooms and Herbs: Stir in mushrooms, thyme, rosemary, salt, and pepper and cook until mushrooms are soft and their liquid has mostly evaporated, about 8-10 minutes, stirring occasionally.
Stir in Carrots and Celery: Stir in carrots and celery, and cook until timer for rice has about 7-10 minutes left. (Allow at least 5 minutes for carrots and celery to cook, but it can be longer.)
Prepare Instant Roux: Add butter to pot and let it melt. Sprinkle flour over sautéed vegetables and stir well to combine. Cook for 1 minute.
Add Milk: Slowly add milk to pot, stirring continuously to avoid lumps. Continue stirring as mixture thickens; simmer until rice has finished cooking.
Add Rice Then Simmer: Once rice is done cooking, carefully pour or ladle scoops of cooked rice and broth into pot and stir to combine. Bring to a gentle simmer and let cook for additional 5 minutes to allow flavors to meld together. Remove pot from stovetop heat, then carefully taste and add salt and pepper, based on taste preferences. Optionally, stir in some lemon juice to brighten up the flavors.
Garnish and Serve: Ladle soup into bowls, garnish with parsley and grated Parmesan cheese, then serve.