Fill a large pot with water and bring to a boil over high heat. Add the peeled and cut potatoes, then reduce to a simmer. Cook for 12-15 minutes until the potatoes are easily pierced with a fork and just tender. Drain and set aside to cool slightly.
In a mixing bowl, whisk together mayonnaise, mustard, vinegar, salt, and pepper until smooth and well combined. This will be the dressing for the salad.
Dice the cooled potatoes into bite-sized chunks, maintaining a uniform size for even texture. Add the diced potatoes to the bowl with the dressing.
Chop the fresh dill and finely dice the celery stalks. Gently fold them into the potato mixture, ensuring even distribution without breaking the potatoes.
Mix everything thoroughly until the potatoes are fully coated and the ingredients are combined. Taste and adjust seasoning if needed. Refrigerate for at least 15 minutes before serving to allow flavors to meld.
Serve the potato salad chilled, garnished with additional dill if desired, with a fresh cracker or as a side dish at your meal.
