Pour oat milk into a saucepan and heat over medium-low heat.
Combine egg yolks and sugar in a bowl and whisk together.
Slowly pour the warmed oat milk into the yolk mixture, stirring constantly.
Transfer the mixture back to the saucepan and cook over low heat, stirring until it thickens.
Remove from heat and mix in vanilla extract, nutmeg, and cinnamon thoroughly.
Chill in the refrigerator until cold.
Pour into cups and, if desired, top with whipped cream or extra nutmeg before serving.