In a large pot, bring water to a boil and add the baby potatoes. Boil for 10-12 minutes or until they are fork-tender but not falling apart. Drain and let them cool slightly.
In a separate saucepan, combine white vinegar, water, sugar, and salt. Stir and heat over medium heat until the sugar and salt dissolve completely. Remove from heat and let the mixture cool slightly.
Place the boiled baby potatoes into a large, sterilized jar, layering with garlic cloves, mustard seeds, black peppercorns, and red pepper flakes if using. Add fresh dill sprigs for extra flavor if desired.
Pour the warm vinegar mixture over the potatoes, making sure they are fully submerged. Seal the jar tightly with a lid.
Let the jar cool to room temperature, then refrigerate for at least 24 hours to allow flavors to develop. For best taste, allow the potatoes to pickle for 3-5 days before eating.
Store in the refrigerator and enjoy as a side dish or snack.