Preheat oven to 250°C (482°F). Rinse duck inside and outside, pat dry, rub with salt and pepper. Place duck in a roasting pan without added fat and roast for 30 minutes, turning once, until skin is brown and crispy. Meanwhile, zest the oranges into the finest strips using a zester, then squeeze the juice from all oranges.
Bring sherry vinegar to a boil over medium-high heat. Add brown sugar and caramelize over high heat for 2-3 minutes, watching carefully to avoid burning black. Reduce oven temperature to 180°C (356°F), or 150°C (302°F) if using fan circulation.
Remove duck from roasting pan and transfer to a plate. Pour the pan drippings into a small bowl. Pour a small amount of orange juice into the roasting pan bottom. Return duck to pan, breast-side up. Scatter the orange zest strips over the duck and pour the caramel sauce over it. Roast for 1 hour at 180°C (356°F), basting alternately with orange juice and reserved pan drippings every 10 minutes, until the thickest part of the thigh reaches 74°C (165°F) on a meat thermometer.
Transfer duck to a warm platter and carve. Heat Grand Marnier in a saucepan over medium heat, carefully light it on fire, and pour the flaming Grand Marnier over the carved duck. Keep warm. Pour the sauce from the roasting pan into a saucepan, bring to a boil over medium-high heat, add port wine, and simmer for 5 minutes. Whisk the sauce until smooth. Serve duck portions on plates with sauce generously poured over.
Serve with small potato pancakes or French fries duchesse.