Sexual violence is a well-recognised global health problem, but there has been remarkably little research on men as perpetrators. The objectives of this paper are to describe the prevalence, patterns and factors associated with rape of an intimate partner and a woman who was not a partner with men aged 15-26 years in rural South Africa. The analysis presented here is of data collected during a baseline survey of participants in a cluster randomised controlled trial of an HIV behavioural intervention. A total of 1370 male volunteers were recruited from 70 rural South African villages. They completed a questionnaire asking about background, sexual practices and perpetration of rape and intimate partner violence. Among these men 16.3% had raped a non-partner, or participated in a form of gang rape; 8.4% had been sexually violent towards an intimate partner; and 79.1% had done neither. The mean age of first rape was 17 years. There was overlap between rape of a non-partner and partner, in that 44.3% of men who raped an intimate partner had also raped a non-partner, but overall the great majority of men who raped did not disclose both types of rape. The factors associated with rape of an intimate partner and non-partner had similarities and differences. After adjusting for the other variables, both forms of rape were strongly associated with ever having been physically violent to a partner, having had transactional sex with a casual partner and more sexual partners. Non-partner rape was also associated with peer-related variables, including gang membership and peer pressure to have sex, and also drug use. Non-partner rape was more common among wealthier and relatively more socially advantaged men. Both types of rape were associated with having more adverse childhood experiences. There was considerable overlap between rape-associated factors and known HIV risk factors, suggesting a need for further research on the interface of rape and HIV, and integrated prevention programming.