Quality control of welded joints is an important step before commissioning of various types of metal structures. The main obstacles to the commissioning of such facilities are the areas where the welded joint deviates from acceptable defective standards. The defects of welded joints include non-welded, foreign inclusions, cracks, pores, etc. The article describes an approach to the detection of the main types of defects of welded joints using a combination of convolutional neural networks and support vector machine methods. Convolutional neural networks are used for primary classification. The support vector machine is used to accurately define defect boundaries. As a preprocessing in our work, we use the methods of morphological filtration. A series of experiments confirms the high efficiency of the proposed method in comparison with pure CNN method for detecting defects.