The second IBIS/ISGRI survey has produced a catalogue containing 209 hard X-ray sources visible down to a flux limit of around 1 milliCrab. The point source location accuracy of typically 1-3 arcminutes has allowed the counterparts for most of these sources to be found at other wavelengths. In order to help identify the remaining objects, we have used the cross-correlation recently found between the ISGRI catalogue and the ROSAT All Sky Survey Bright Source Catalogue. In this way, for ISGRI sources which have a counterpart in soft X-rays, we can use the much smaller ROSAT error box to search for identifications. For this second survey, we find 114 associations with the number expected by chance to be ~2. Of these sources, 8 are in the list of unidentified objects and, using the smaller ROSAT error boxes, we can find tentative counterparts for five of them. We have performed the same analysis for the ROSAT Faint Source Catalogue, finding a further nine associations with ISGRI unidentified sources from a total of 29 correlations, and, notwithstanding the poorer location accuracy of these sources and higher chance coincidence possibility, we have managed to find a counterpart for another source. Finally, we have used the ROSAT HRI catalogue to search the ISGRI error boxes and find 5 more X-ray objects, of which two are neither in the bright or faint source catalogues, and for which we have managed to find optical/near infrared associations. This makes a total of 19 objects with X-ray counterparts for which we have found possible identifications for nine, most of which are extragalactic.