It's largely the fault of the justice responsible for the decision:
Since high school, John Knox had been star-struck by the Supreme Court Justices, attempting to strike up correspondences with them, sending them birthday greetings, and so on. After graduating from Harvard Law School, Knox landed a clerkship with Justice James McReynolds for the 1936–37 term. McReynolds preferred to work out of his D.C. apartment, rather than in the Supreme Court’s then-new building. Knox’s role was secretarial. Knox later wrote: “I appreciated his anti-New Deal view and agreed with it, but that was the only thing I could possibly agree with him on. He was selfish to an extreme, vindictive, almost sadistically inclined at times, inconceivably narrow, temperamental, and heaven knows what. All of his employees lived in a reign of terror and were crushed under foot without any hesitation on his part.”
More relevantly for Miller, McReynolds “found great difficulty in expressing himself in writing and, sadly enough, was genuinely lazy.” In the September of the clerkship, Knox had dinner at the home of Mr. and Mrs. Edward Everett Gann. The Ganns were well-connected in Washington; Mrs. Dolly Gann was the sister of Herbert Hoover’s Vice-President, Charles Curtis (1929–33). Mr. Gann was a friend of McReynolds, and accidentally caught McReynolds in a tryst with a woman. Knox recalled Gann’s words: “I concluded finally that he is not really interested in the work of the Court any more. He’s old, evidently bored with life and would probably retire now if he could do so without letting other conservatives on the Court ‘down.’”
Source:
http://volokh.com/2010/02/27/united-states-v-miller/Fascinating. I'd always wondered about that opinion.
Keep in mind that this is one conservative's opinion of another conservative: “I appreciated his anti-New Deal view and agreed with it...." but he was a sorry character.