Bleh
Try not using the anti-aliasing, that may be the problem. If not then something in your PC doesn't want to work with high-detail.

Can't think straight since it's 2 am here lol, but try all the usual things like virus scanning and de-frag (even though they probably aren't the culprits)
You should try low detail and check to see if its high-detail specifically, that your PC doesn't like, or just runescape/java programs in general before finding how to solve it. I would say... that... you need to look for odd signs in your computer too, things that are happening but shouldn't happen (physically mainly).