In system reliability analysis, the random variables entering the system performance function often include multi-state or binary variables. When solving this type of reliability problems, the standard subset simulation algorithm with fixed intermediate conditional probabilities and number of samples per level may lead to significant errors, because of the discontinuous system performance function. This is related to the adaptive definition of the intermediate failure domains in terms of a prefixed percentile of the samples from the performance function. A discontinuous performance function can lead to an ambiguous definition of the sought percentile and, hence, of the intermediate domains. In order to address this problem and extend the applicability of subset simulation to system problems with discontinuous performance functions, we propose a new algorithm that chooses the intermediate conditional probabilities and number of samples per level adaptively. We demonstrate the efficiency and accuracy of the proposed approach through numerical examples.