Does breast size really matter?

my breast size is a 34 B but a small B and i get very paraniod about taking my bra off when im with my boyfriend i havent been with him too long and dont want to put him off, is this normal? does it really matter if they are smaller to ones that he might have seen before, like will he care or put him off? please give an honest opinion, thanks :)