Preheat the oven to 350°F (175°C). Grease an 8-inch square or 9-inch round baking pan. Line with parchment paper and grease the paper.*
Combine the flour, brown sugar, cinnamon, and salt in a small bowl.
Add the butter, and mix with a fork or your fingers until the mixture forms crumbs.
Refrigerate while you make the cake batter.
To make the cake:
Whisk together the flour, almond flour, baking powder, and salt. Set aside.
Using an electric mixer on medium speed, beat the butter and sugar until light and fluffy.
Add the eggs, one at a time, mixing well after each addition. Mix in the almond extract.
Reduce the mixer speed to low. Add the flour mixture in three portions, alternating with first the sour cream and then the milk. Mix just until combined.
Transfer the batter to the prepared pan, and spread it evenly. Sprinkle the crumb topping over the batter. Sprinkle the sliced almonds over the topping, pressing them down lightly.
Bake 40 to 45 minutes, or until the edges are browned and a pick inserted into the center comes out clean.
Completely cool the cake in the pan on a wire rack before serving.
Notes
*If you make a "sling" out of the parchment paper, you can lift the cooled cake out of the pan after it has cooled. Otherwise, you may lose some crumb topping when trying to flip the cake out of the pan. Of course, you can also skip the lining if you plan to serve directly from the pan.
Store in an airtight container at room temperature up to 3 days.