The spread of video sharing services has made available an enormous number of videos. Even when searching video sharing services, too many videos are returned to view. Viewers want to classify videos to easily grasp results, because videos may include varied topics or differing viewpoints. Video clustering is one solution to addressing this problem with many related approaches to research. However, existing approaches have problems: text information in metadata tends to be of low quality, visual information is difficult to analyze, and some information on user viewing behavior includes noise. This paper focuses on playlist information, which is a type of user viewing behavior. A playlist is useful because it is based on the viewers' knowledge or intuitiveness; beyond that, it is not noisy. We propose the playlist-based video clustering method (PVClustering), a novel framework that can form new clusters independent of text or visual similarities. The proposed method is computationally inexpensive and language-independent. By our method, users can grasp the outline of search result videos in a new light. Our experiments show good result clusters generated by PVClustering and prove that it can capture relativity or proximity among videos, which is not coded in text information. They also present the characteristics of PVClustering.