Transparent materials in V-Ray are, frankly, a pain in the bum. It won’t recognise the transparency value of a png; and despite, the straightforward explanation in the manual, I’ve never yet managed to create a successful linked material with transparency that will simply replace all instances of the SU material.
In fact, it often seems to refuse to work properly with png files at all. It will recognise the diffuse map ok, but simply refuses to apply any transparency to it…as you’ve probably found out.
My method is to go to the original png image of the leaves and create two jpeg images from it…one a straight diffuse map (you might want to create another background layer in the png and colour it leaf green, before flattening the image and saving as jpg…just in case of any slight mismatch between the diffuse and alpha maps)
The second image needs to be an alpha map. Copy one of the original image channels…the blue is usually the strongest. Make a separate image of this, then pull the levels sliders together, or turn the contrast right up, so it’s entirely black and white. Save this as the alpha image.
In V-Ray, create a new material, specify the two new jpgs as the diffuse and transparency maps and check that they work ok on the preview sphere. You may have to invert the alpha image if you have it the usual white=opaque, black=transparent.
Once it’s looking ok in the V-Ray material editor, apply it to both faces of the leaf masses on the model. If they are all separate you are going to have your work cut out. If your leaf masses are 1 or 2 repeated components, it takes no time at all.
Here’s a quick V-Ray render of a 3D Ghost Gum that I’ve just applied the above method to.
gg_render.jpg