Lightly grease a 9-inch springform pan.
In a medium bowl, whisk together the flour, baking powder, baking soda, and salt; set aside.
In a small bowl, mix together the buttermilk, lemon juice, and orange juice; set aside.
In a large mixing bowl, beat the eggs and granulated sugar at high speed until pale and fluffy, about 5 minutes.
Lower the speed to medium-high, and drizzle in the olive oil slowly, beating until incorporated.
Mix in the lemon and orange zests (of if using extracts, add them to the buttermilk mixture).
With the mixer on low, add half the flour mixture until incorporated.
Add the buttermilk mixture next, then the remaining flour mixture, until just blended.
Fold in the almonds.
Scrape batter into prepared pan.
Bake at 350 for 40-45 minutes, or until the top is golden and a toothpick inserted near the center comes out clean.
Let the cake cool in pan on a wire rack for 20 minutes, then carefully remove the pan sides and allow cake to cool completely.
Dust with powdered sugar just before serving.