A judge somewhere has to declare you an imminent danger to yourself or others, or you have to sign over your rights as a condition of bail being set. We don't remove people's constitutional rights without due process, and due process has always involved a judge.
I find it hilarious that the NYT ran an editorial last year saying that the no fly list was unconstitutional because it removed a person's non existent right to fly without due process, but last week they ran an editorial saying that they want to use the no fly list to remove your second amendment rights without due process.
The flip side of that coin are the people who use that same pretzel logic to bar people from entering the country based entirely upon religion. Not even an accusation of terrorist connections beyond the name of their imaginary friend.
The Free Exercise Clause is well rooted in tort. It all boils down to two sentences in Reynolds vs. United States: "Congress cannot pass a law for the government of the Territory which shall prohibit the free exercise of religion. The first amendment to the Constitution expressly forbids such legislation. Laws are made for the government of actions, and while they cannot interfere with mere religious beliefs and opinions, they may with practices." In other words, they can regulate an activity, say blowing people up, but they cannot discriminate against one Muslim for the actions of another just because they have a prayer mat in their trunk.