Iirc, the problem with Bayesian neural networks is that they're significantly more difficult to train. Using stuff like SVI reduces a lot of the representational ability of the distribution over weights. It's also questionable how useful the uncertainty over weights is.

I suppose in the tradition of Bayesian influence, VAEs and the like are still common though.