Matt Schultz, the frontman of Grammy-winning rock band Cage the Elephant, was arrested in New York on Thursday night for felony criminal possession of a firearm, police said. The night before the arrest, an employee at the Bowery Hotel said they saw Schultz, 39, carrying a handgun into a public restroom on the ground floor, the Daily Mail reports. When police knocked on the singer’s hotel door the following morning, he allegedly admitted to having two .45 calibre guns. A judge granted a search warrant allowing police to investigate his room, where they found two loaded handguns, authorities said. Schultz reportedly spent the night at the 9th Precinct station in Lower Manhattan. A rep for Cage the Elephant did not immediately respond to The Daily Beast’s request for comment.