Mix the flour with the yeast, rub in the butter, then mix in 3 tablespoons of the sugar. Make a well in the center then beat in the egg and milk until you have a soft, smooth dough. Mix well in the bowl and knead for about 4 minutes.
Grease a 10 inch shallow, round, springform cake pan. Roll or pat out the dough to fit the pan. Dip your fingers in flour if the dough sticks. Press the dough in lightly, taking it right to the edge. Cover and leave to rise until doubled in size, about 1/2 to 1 hour.
Arrange the plum quarters skin-side-down in circles on top. Sprinkle over the remaining sugar.
Heat the oven to 350 degrees F. Bake the cake for about 35 to 40 minutes until it is risen and the plums are soft.
Cool in the pan and scatter over the almonds and lemon zest. Remove from the pan and cool completely on a wire rack.